Skip to content

docs(research): extract Magic Pocket architecture from sources - #44

Merged
VyasGuru merged 1 commit into
mainfrom
research/8-architecture
Oct 21, 2026
Merged

VyasGuru merged 1 commit into
mainfrom
research/8-architecture

Conversation

@frectrium

Copy link
Copy Markdown
Owner

What

Adds docs/research/architecture.md: how Magic Pocket is built, component by component, with what each part does, how it works and the reason Dropbox gave, all cited to the blog (B) and talk (T) sources.

Closes #8

Why

The architecture diagram (#11), the solution write-up (#13) and the exercise plan (#14) all need one cited reference for Magic Pocket's design.

Covers every item in #8:

  • Blocks: up to 4 MB, SHA-256.
  • Buckets, volumes and extents.
  • Zones and cells.
  • Frontends.
  • Block Index and Replication Table.
  • Master and Volume Managers.
  • OSDs.
  • Replication (4 copies per zone before coding).
  • Erasure coding: RS(6,3) or LRC(12,2,2) by 2024.
  • Repair and verification.
  • Deletes and compaction.
  • SMR hardware.
  • Go to Rust.

It also includes the put, get and repair paths, scale from 2012 to 2025, and the design principles Dropbox stated.

The last section lists open questions and the decisions Satchel has to make for #14. The biggest gap: no source says how long data stays replicated before it's erasure coded.

How to check it

Read docs/research/architecture.md. Quotes from the blog posts were checked word for word against the posts. The T8 figures were checked against the QCon 2022 transcript.

Checklist

  • cargo fmt, cargo clippy and cargo test pass locally (code PRs)
  • Tests added or updated for the change (code PRs)
  • New sources are added to docs/sources/README.md (research PRs)
  • Design decisions are recorded in the Design Decision Log
  • Docs and README are updated if behaviour or setup changed

@frectrium frectrium self-assigned this Oct 21, 2026

@VyasGuru VyasGuru left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked every issue #8 component against the cited blog and talk notes. The dated replication and erasure-coding claims are scoped correctly, the architecture paths are internally consistent, the open questions cover the undisclosed choices Satchel must make, and CI is green.

@VyasGuru
VyasGuru merged commit 04cbada into main Oct 21, 2026
2 checks passed
@VyasGuru
VyasGuru deleted the research/8-architecture branch October 21, 2026 06:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Research] Extract Magic Pocket's architecture from sources

2 participants