Designing Facebook photo storage

27072 2019-01-04 12:11

Motivation & Assumptions

  • PB-level Blob storage
  • Traditional NFS based desgin (Each image stored as a file) has metadata bottleneck: large metadata size severely limits the metadata hit ratio.
    • Explain more about the metadata overhead

For the Photos application most of this metadata, such as permissions, is unused and thereby wastes storage capacity. Yet the more significant cost is that the file’s metadata must be read from disk into memory in order to find the file itself. While insignificant on a small scale, multiplied over billions of photos and petabytes of data, accessing metadata is the throughput bottleneck.


Eliminates the metadata overhead by aggregating hundreds of thousands of images in a single haystack store file.


Facebook Photo Storage Architecture

Data Layout

index file (for quick memory load) + haystack store file containing needles.

index file layout index file layout 1

index file layout 2

haystack store file

haystack store file

CRUD Operations

  • Create: write to store file and then async write index file, because index is not critical
  • Read: read(offset, key, alternate_key, cookie, data_size)
  • Update: Append only. If the app meets duplicate keys, then it can choose one with largest offset to update.
  • Delete: soft delete by marking the deleted bit in the flag field. Hard delete is executed by the compact operation.



Photo Storage Upload


Photo Storage Download


在 Twitter 上 Follow 我 :)

下载 硅谷io App

关注互联网创业,用移动 App ,随时随地收藏和复习好文章