crueber/walhub

Watch

Fork 0

Developer deep-dive page: object-store idea + WAL internals #191

New issue

Closed

opened 2026-09-08 12:21:40 +00:00 by crueber · 6 comments

crueber commented

2026-09-08 12:21:40 +00:00

Owner

Developer deep-dive page: the object-store idea + what the WAL is and how it works

A second marketing page (route TBD in planning — e.g. /how-it-works) that fully explains, for developers: the original idea behind storing git repos in object stores, what the "WAL" is, and how it works in detail. Reuses the generated concept GIFs wherever they carry explanatory weight (plus prose + diagrams as needed).

Content (planner: verify every claim against the WAL/store docs + code)

The original idea: git objects are content-addressed → packs/sidecars drop unchanged into a dumb object store; no database to run, back up, or scale; any S3/GCS/filesystem works.
What the WAL is: the write-ahead log of manifest mutations; the manifest CAS as the single commit point; checkpoints and ref snapshots; readers/writers, sync levels; why concurrent writers stay safe without locks on the store.
How a push flows end to end (ingest → packs → manifest CAS → refs → never ACK before the bucket ACKs) and how a fetch/clone reads it back (warm refs, bundle-uri at scale).
Honest boundaries: what the WAL deliberately does NOT do (collaboration objects live beside it, never in it).

Inspiration (not copying)

https://cursor.com/blog/git-at-any-scale for narrative shape/pacing only — our architecture, our words, our diagrams. Do not lift copy, structure, or visuals.

Acceptance criteria

New page route + nav entry, zero-API-call static page like /.
Content technically accurate (reviewed against the store/WAL specs; every non-obvious claim traceable to a doc section or code path).
GIFs reused where they fit (push/fetch/bucket/collab), alt/captions intact; reduced-motion respected.
node --test green; browser pass both themes, zero console errors; no new deps.

# Developer deep-dive page: the object-store idea + what the WAL is and how it works A second marketing page (route TBD in planning — e.g. `/how-it-works`) that fully explains, for developers: the original idea behind storing git repos in object stores, what the "WAL" is, and how it works in detail. Reuses the generated concept GIFs wherever they carry explanatory weight (plus prose + diagrams as needed). ## Content (planner: verify every claim against the WAL/store docs + code) - The original idea: git objects are content-addressed → packs/sidecars drop unchanged into a dumb object store; no database to run, back up, or scale; any S3/GCS/filesystem works. - What the WAL is: the write-ahead log of manifest mutations; the manifest CAS as the single commit point; checkpoints and ref snapshots; readers/writers, sync levels; why concurrent writers stay safe without locks on the store. - How a push flows end to end (ingest → packs → manifest CAS → refs → never ACK before the bucket ACKs) and how a fetch/clone reads it back (warm refs, bundle-uri at scale). - Honest boundaries: what the WAL deliberately does NOT do (collaboration objects live beside it, never in it). ## Inspiration (not copying) `https://cursor.com/blog/git-at-any-scale` for narrative shape/pacing only — our architecture, our words, our diagrams. Do not lift copy, structure, or visuals. ## Acceptance criteria - [ ] New page route + nav entry, zero-API-call static page like `/`. - [ ] Content technically accurate (reviewed against the store/WAL specs; every non-obvious claim traceable to a doc section or code path). - [ ] GIFs reused where they fit (push/fetch/bucket/collab), alt/captions intact; reduced-motion respected. - [ ] `node --test` green; browser pass both themes, zero console errors; no new deps.

crueber commented

2026-09-08 12:47:33 +00:00

Author

Owner

Deep-dive page plan — Forgejo issue crueber/walhub#191

Planning only. No production code written. Read-only in the repo
(git status verified unchanged after planning).
Issue: "Developer deep-dive page: the object-store idea + what the WAL is
and how it works". Route TBD in planning — this plan picks /how-it-works.

0. Sources this plan was verified against

Source	Used for
`docs/go/02_storage_protobuf.md` (§2.1 key layout, §2.2 schema, §2.4 framing, §2.5 seq semantics, §2.6 ObjectStore/CAS)	bucket/WAL mechanics claims
`docs/go/05_wal_engine.md` (§5.0 consistency, §5.2 sync levels, §5.3 publish ladder, §5.4 burn protocol, §5.5 checkpoints, §5.6 replay, §5.7 remote reader)	push/fetch/checkpoint claims
`docs/go/15_testing.md` (§4.8 budget table)	round-trip budget numbers
`docs/go/08_bundles.md` (§8.4 creationToken, clone table)	bundle-uri fetch claims
`docs/features/README.md` (P1–P5, "the WAL stays git-only")	boundaries section
`web/src/pages/Landing.jsx` + `web/src/components/ConceptGif.jsx` + `web/test/unit/landing.test.js`	reuse pattern, zero-API-call precedent
`web/src/index.jsx` + `web/src/App.jsx` (route table + nav)	route + nav entry
`docs/go/12_web_ui.md` (§2.3.0 landing precedent)	doc-update slot
`README.MD` (L3–5 canonical language)	hero language
`internal/wal` + `internal/store` (skimmed: `publish.go`, `sync.go`, `checkpoint.go`, `remote.go`, `tasks.go`, `store.go`, `keys.go`)	code-path traces confirm doc claims

Narrative-pacing inspiration ONLY (per the issue): https://cursor.com/blog/git-at-any-scale.
Our architecture, our words, our diagrams — do not lift copy, structure, or visuals.

1. Route + nav entry

Route: /how-it-works — new static page component web/src/pages/HowItWorks.jsx.
- Registered in web/src/index.jsx: <Route path="/how-it-works" component={HowItWorks} />
  placed with the other top-level static routes (next to /explore, /api, /keys).
- * fallback stays Landing (unchanged — safe default, no API calls; landing.test.js pins it).
- No server change: the SPA shell serves every UI route (same mechanism / already uses).
Nav entry: one link in web/src/App.jsx site nav: <A href="/how-it-works">how it works</A>
(lowercase to match existing explore/import/keys/setup labels), placed after explore.
Cross-links: Landing links to it (one line under the hero or in each concept section —
"How it works →" deep-linking to #push/#wal/#fetch anchors); the deep-dive links back
to / ("← walhub in 30 seconds") and to /setup. Repo-level /wal dashboard links to it
as "what am I looking at?" help (optional, single link — the Wal tab itself is unchanged).

2. Page outline (section by section, with technical claims + traces)

The page is a static component (like Landing.jsx): sections + ConceptGif reuses + inline
static diagrams (see §3). Zero API calls, zero SDK import, zero fetch — same rule as /.

2.0 Hero — "How walhub works"

H1 + one-paragraph thesis in README.MD L3–5 language: the object store is the only database;
instances are disposable ("wipe one and you lose nothing but warmth").
Sub-line: object-protocol compliant with walgit (bucket layout, protobuf wire encoding, git wire
behavior follow walgit's formats) — same one-liner as Landing L65–68.
Anchor TOC to the sections below (#idea, #wal, #push, #read, #checkpoints, #boundaries).
Claims: none beyond README/Landing restatement. Trace: README.MD L3–5, L15–17.

2.1 §`#idea` — The original idea: git objects love dumb object stores

Technical claims (each traced):

Git objects are content-addressed, so packs and their sidecars drop unchanged into a key-value
bucket; the store needs no understanding of git. Trace: 02 §2.1 (everything except
manifest.pb, bundles/list.pb, leases/* is immutable; wal/<checksum>.pack|.idx|.rev|.bitmap|.commit-graph
content-addressed by trailing SHA), store.go PutCreate discipline.
No database to run, back up, or scale: repos live under repos/<owner>/<repo>/; anything that is
restart-surviving state is an object; disk and memory are caches. Any S3/GCS/filesystem works behind
one ObjectStore contract (CAS, conditional GET, compose, leases). Trace: 02 §2.1 key table +
§2.6 interface; AGENTS.md law 4 ("The bucket is the repository"); 01_overview.md store row.
The store is dumb on purpose: conditional writes (CAS) and conditional reads are the only
coordination primitives — there are no locks on the store. Trace: 02 §2.6–2.7 (PutCreate /
PutUpdate, casUpdate; "the store is the lock — no mutex at all").
Round trips are the cost model: every protocol change is judged on counted sequential bucket
round trips (budgets in §2.4 below). Trace: AGENTS.md law 6; 01_overview.md §"No LIST on a hot
path"; 15_testing.md §4.8.

GIF reuse: bucket (instance disappears, fresh instance connects, nothing lost) — the visual
proof of claim 2. Alt/caption intact from CONCEPT_ALTS.bucket.
New diagram: none — prose + reused GIF suffice.

2.2 §`#wal` — What the WAL is: the log of manifest mutations

Technical claims:

The manifest (repos/<o>/<r>/manifest.pb) is the linearization point: head_seq,
min_seq, checkpoint pointer, log_segments covering [min_seq, head_seq], the denormalized
live pack set, settings, monotonic revision. Trace: 02 §2.2 Manifest schema; 05 §5.0 rule 1.
The WAL is the append-only stream of LogEntry frames in log/<seq:016x>.pb segments
(uvarint-length-prefixed, partial trailing frame tolerated on appendable segments).
Five entry kinds: PUSH (pack + ref txn), COMPACT (new pack superseding old), REF_UPDATE
(ref-only), CHECKPOINT (marker), SETTINGS (history; latest rides the manifest inline).
Trace: 02 §2.2 EntryKind/LogEntry, §2.4 framing; 05 §5.3.3 (settings/compact).
The manifest CAS is the single commit point. A manifest write is always conditional
(PutUpdate(version) / PutCreate); a 412 is the normal contention signal, never an error.
Trace: 05 §5.0 rules 1–2; 02 §2.6 error taxonomy (PreconditionFailed is protocol-normal).
Seq numbers are strictly increasing but not dense: a writer that crashes between log PUT and
manifest CAS leaves an orphan; later writers burn past it (3 probes × 100 ms, cap 8 consecutive
burns → Corrupt). Orphans are harmless and swept after a later commit. Trace: 02 §2.5;
05 §5.4 ladder; publish.go slot-claim path.
Why concurrent writers stay safe without store locks: one single-flight publisher goroutine per
repo serializes all publishes in-process; across instances the CAS serializes commits; losers
re-sync, re-verify, retry (bounded ladder, 16 attempts). Group commit batches arrivals (5 ms window,
up to 64) so concurrent pushes share one commit. Trace: 05 §5.3.1–5.3.2 + Concurrency notes;
internal/wal/publish.go.
Ref updates are verified old-value-checked transactions (old_oid must equal current unless
all-zero creation; symbolic HEAD updates always ok); rejected pushes are transport-successes with
per-ref errors (Seq: 0). Trace: 05 §5.3.2 step 3.

GIF reuse: none here (no existing GIF shows the manifest/CAS). The push GIF belongs to §2.3.
New diagram (JUSTIFIED — #1 of at most 2): one static inline SVG/CSS sequence diagram of the
commit point: log PUT (Create) → manifest CAS (Update) → local refs apply → answer ok, with the
three failure branches (412 → delete-own-segment + retry; ambiguous error → re-read casLanded,
delete nothing). Static SVG, theme-safe (currentColor / muted classes), no animation, no new deps.
Justification: the CAS ladder is the page's central claim and no GIF covers it; a static diagram is
zero-weight and accessible (paired <details> text fallback describing the same steps).

2.3 §`#push` — How a push flows, end to end

Walk the numbered ladder as narrative (each step traced to 05 §5.3.2):

Sync (refs+serve) unless already fresh → snapshot manifest + version.
Verify each ref txn against current refs (old-value check).
Assign seqs (head_seq+1 …), build PUSH entries with PackRef{checksum, sizes, tier 0} + caller
meta (principal, request_id, push-options).
In parallel: upload pack+idx create-if-absent (duplicates are success) ∥ claim the log slot
(PutCreate of the segment; 412 → manifest re-read → burn-or-retry per §5.4).
Build the manifest update (head_seq, extended pack set, new segment ref, revision+1).
CAS the manifest. The server never ACKs the push before the bucket ACKs — the push reply is
answered only at step 8, after commit. Trace: 05 §5.3.2 steps 7–8; AGENTS.md law 4.
Local commit under syncMu, refs first, then advertise; withdraw-on-failure (reset advertised
version so the next sync replays — but still answer ok, because the bucket is the truth).
Trace: 05 §5.0 rule 5 + §5.3.2 step 8 + ordering rule.
Sweep burned orphans; fold commit-graph off the critical path; opportunistic checkpoint check.
Trace: 05 §5.3.2 step 8.

GIF reuse: push (laptop → bucket, bucket writes then ACKs, client finishes after the ACK) —
the visual for claim 6. Alt/caption intact from CONCEPT_ALTS.push.
New diagram: none (the §2.2 CAS diagram + prose ladder cover it).

2.4 §`#read` — How a fetch/clone reads it back

Technical claims:

Every request syncs first; four levels: Refs (checkpoint snapshot + log-tail ref txns →
local packed-refs, no packs) serves info/refs, ls-refs, bundle lists, web refs; Serve adds
the pack set this instance can hold; Full materializes everything (refused with ErrTooLarge
over budget); Objects serves-or-remote-reads for the web API. Trace: 05 §5.2 table +
sync.go.
Refs apply is an offline full packed-refs rewrite (parse → map → tmp+rename), atomic across many
refs, works before packs exist. Annotated-tag ^{} comes from writer-recorded new_peeled
(max 16 hops), so replicas advertise without objects. Trace: 05 §5.2 step 2 + §5.3.3 peeling.
Warm refs = 1 request (conditional GET; 0 within freshness TTL); cold refs = 2
(manifest GET → checkpoint refs ∥ tail); push ≤ 5 (freshness GET → pack PUTs ∥ log PUT →
manifest CAS); checkpoint = 4 (never a log GET — provenance rides the applied state).
Trace: 15_testing.md §4.8 budget table (exact numbers + counting rule).
At scale, clones ride bundle-uri: the CAS'd bundles/list.pb advertises full + incremental
bundles with creationToken = slot epoch; git downloads bundles then fetches the remainder.
Trace: 08 §§8.1/8.4 + clone table; internal/bundle.
Too-large repos don't break stock git: Serve-tier packs can be remote-served (side-files + store
mount or no local copy) with the block-cache remote reader + fetch-path faulter filling gaps for
the web API and fetches. Trace: 05 §5.7 (+ the stated v1 decision: serve via stock git +
materialized packs + bundle-uri; no native serving engine in v1 — say this plainly).

GIF reuse: fetch (empty clone receives ref names, then pack data). Alt/caption intact.
New diagram: none.

2.5 §`#checkpoints` — Checkpoints, ref snapshots, and cold starts

Technical claims:

Triggers (any; 0 disables): ≥ 256 entries since last checkpoint, tail bytes > 8 MiB, or age ≥ 1 h.
Evaluated after publishes (background, off the reply path) and by the maintainer loop.
Trace: 05 §5.5 para 1; 10_maintenance.md checkpoint unit.
Write is refs-level (works on an instance that could never hold the packs), 2 rounds:
checkpoint.pb ∥ refs.pb (PutCreate, deterministic seq-keyed) → manifest CAS
(checkpoint, min_seq = seq+1, trim folded segments, revision+1). Idempotent at equal seq;
racing checkpointers resolve benignly via Create-idempotence + CAS. Trace: 05 §5.5.
Cold-start fold: a fresh instance loads refs.pb at the checkpoint and replays only the tail —
it never replays the whole log. min_seq = checkpoint.seq + 1; below is folded away.
Trace: 05 §5.5 "Cold start fold"; 02 §2.5.
Point-in-time: refsAtSeq/refsAsOf fold from the newest usable checkpoint + ordered entries;
cuts older than min_seq with no usable checkpoint are unreplayable (why provenance timestamps
exist). Trace: 05 §5.6.
policy.json is NOT on the WAL (admin API/CLI object); per-repo settings ARE published
(manifest-inline + SETTINGS history, ≤ 16 KiB TOML, validated before publish).
Trace: 02 §2.1 key table; 05 §5.3.3 publish_settings.

GIF reuse: none (bucket GIF already used; checkpoint fold is tail-replay, closest visual is
fetch — do not triple-use it; link back to §2.4 instead).
New diagram (JUSTIFIED — #2 of at most 2): one static inline timeline: checkpoint tick at seq N
(refs.pb snapshot), folded-away prefix, live tail segments, min_seq/head_seq markers.
Same static-SVG treatment as #1. Justification: "fold, don't replay" is the cold-start claim and is
inherently positional; one small figure replaces three paragraphs.

2.6 §`#boundaries` — Honest boundaries: what the WAL deliberately does NOT do

The WAL stays git-only. Issues, PRs, reviews, checks, notifications are a parallel object
family (orgs/…, users/…, repos/<o>/<r>/{meta,issues,checks,releases,access.json}) with its
own CAS discipline — never WAL entries, never the manifest, never gating a push (except policy
effects that explicitly consult them, e.g. required checks). Trace: docs/features/README.md
L11–15 (the one architectural law), P1–P4.
What that buys: collaboration writes never contend with the push CAS ladder; git stays readable
without the collaboration layer and vice versa. (Framed as design consequence, not a metric —
no perf claim.)
Explicitly out of scope of this layer (so readers don't project): code search, CI runners
(walhub stores check results, doesn't run CI), Discussions/Packages/Projects, SAML/SCIM.
Trace: docs/features/README.md P9.
What the WAL doesn't do, stated plainly: no query engine (reads are key probes + replay, no LIST
on hot paths); no cross-repo transactions (one publisher per repo; CAS is per-manifest); no native
git serving engine in v1 (stock git subprocess + faulter, per 05 §5.7 decision); burned-seq
gaps are normal, not damage.

GIF reuse: collab (issue 7 / PR 8 / green check alongside the git lane, WAL stays git-only) —
the visual proof of claim 1. Alt/caption intact from CONCEPT_ALTS.collab.
New diagram: none.

2.7 Closing — where to go next

Links: / quickstart (push something), /explore (browse), /setup (configure), repo /wal
tab (watch a live manifest), /api docs. No claims; pure navigation.

3. GIF/diagram inventory

Asset	Reused in	Verdict
`push.gif` (+ `-still`)	§2.3 push flow	REUSE, alt/caption verbatim
`bucket.gif` (+ `-still`)	§2.1 the idea	REUSE, alt/caption verbatim
`fetch.gif` (+ `-still`)	§2.4 reads	REUSE, alt/caption verbatim
`collab.gif` (+ `-still`)	§2.6 boundaries	REUSE, alt/caption verbatim
CAS commit-point sequence diagram	§2.2	NEW static inline SVG (#1) — justified: central claim, no GIF covers it
Checkpoint fold timeline	§2.5	NEW static inline SVG (#2) — justified: positional claim, replaces paragraphs
Any new animated GIF	—	NOT justified — generator exists (`internal/devtools/landinggif/`, `make landing-gifs` is manual) but all four existing GIFs already map 1:1 to sections; new GIFs add weight for no new explanatory power

Reuse mechanics (copy the Landing pattern exactly): ConceptGif component (still-first,
reduced-motion swap via prefers-reduced-motion, <noscript> still, /_ui/concepts/ prefix,
lazy + async + 640×360, single img alt per figure — never a labeled wrapper on top).
Dark-framed cards in both themes (README screenshot precedent — one asset set, no light variants).

What the #187 GIFs already cover (do NOT re-explain, link across): Landing owns the 30-second
pitch for each scene; the deep-dive links back ("the short version lives on /") and goes one
level deeper instead of restating.

4. Copy honesty rules

Every non-obvious technical claim carries its trace (doc section or code path) in a code comment
or in the plan's §2 — and the implementation change must keep the comment pins accurate
(same discipline as landing.test.js source pins).
Numbers are budgets, not benchmarks: "push ≤ 5 requests" etc. are sim-asserted transport budgets
(15_testing.md §4.8), never latency promises. No ms figures anywhere on the page.
No overclaiming scale: say "bundle-uri carries large hosts" (Landing's wording), not "at any
scale"; say materialization is refused over budget with the bundle-uri fix text, not that every
repo fits everywhere.
No claiming what isn't built: the v1 remote-reader decision (stock git + faulter, no native
serving engine) is stated, not hidden; collaboration features link to their real state, not a
roadmap promise.
Wire-compat wording copies Landing/README verbatim ("object-protocol compliant with walgit —
bucket layout, protobuf wire encoding, and git wire behavior follow walgit's formats"), never
paraphrased into something stronger.
Boundaries section (§2.6) is mandatory, not decorative — it ships in v1 of the page.
cursor.blog influence is pacing only (short sections, one idea per scroll, visual-then-prose
rhythm); no lifted copy, structure, or visuals.

5. EVIDENCE / performance

Zero-API-call static page (same rule as /): no useData, no SDK import, no fetch.
The page spends no store round trips — asserted by a node --test source-pin test mirroring
landing.test.js. No new network surface, no new backend path.
No docs/EVIDENCE.md entry: the page makes no hot-path performance claim (it only reports
existing sim budgets); per AGENTS.md, evidence entries are for transport/storage features making
hot-path claims. If a future iteration adds measured figures, it gets a harness + entry then.
Weight: 4 reused GIFs (already shipped, cached) + 2 inline SVGs (bytes, no requests). No new npm
runtime deps (Solid + router only, already present); no new Go modules.

6. Acceptance criteria (from the issue, made checkable)

HowItWorks.jsx + /how-it-works route + how it works nav entry; * fallback still Landing.
Zero-API-call rule pinned by a new web/test/unit/how-it-works.test.js (routes, nav, anchors,
alt-text pins, no useData/SDK/fetch strings) — mirrors landing.test.js.
Every §2 claim reviewed against 02/05/08/15_testing/features/README + the internal/wal
+ internal/store code paths listed; traces recorded in code comments.
All 4 GIFs reused with alt/captions verbatim; ConceptGif (still-first + reduced-motion +
noscript) used for each; docs/go/12_web_ui.md gains a §2.3.x subsection for the page.
node --test over web/test/unit/*.test.js green (never a directory positional — Node 22
executes a directory as a module; see AGENTS.md field lessons).
Real-browser pass (hub-managed chrome-cdp on :9222), both themes, zero console errors —
module scripts are MIME-enforced and import-map driven ("curl says 200" proves nothing);
drive against the canonical host (loopback GETs 302 to walgit.localhost — 06_server_http.md
§2.2 #2), or expect the hop.
No new runtime deps (npm or Go); make fmt && make vet clean; git status shows only the
intended files.

7. Open questions / risks

Route name — /how-it-works vs /how vs /internals. /how-it-works is self-describing and
matches the issue's example; short /how risks collision with future pages. Recommend
/how-it-works. (Needs owner sign-off — one-line decision.)
?format=text parity — 12_web_ui.md notes the text format "moves server-side with the page"
for Landing (06_server_http.md §Decisions). Does /how-it-works need a text rendering too, or
is the SPA-only page acceptable? Recommend SPA-only for v1 (marketing page, not data), but confirm.
Nav crowding — App nav gains a 6th link (explore import API keys setup + how it works);
check mobile wrap (existing flex-wrap handles it, but verify in the browser pass).
Length risk — 7 sections is long for one page; mitigation is the anchor TOC + one-idea-per-section
pacing (the cursor.blog lesson). If review finds it heavy, split boundary: §§2.1–2.3 ship first,
§§2.4–2.6 follow — but default is one page, one change.
SVG-in-JSX cost — two inline SVGs bloat the component; alternative is web/public/ .svg
assets served under /_ui/. Recommend inline (no extra requests, theme via currentColor) unless
the file exceeds ~200 lines, then split to assets.
Claim drift — the page restates constants (256 entries, 8 MiB, 1 h, 5 ms/64, 16 retries, 8 burns,
budgets 5/1/2/4). These live in config + sim tests; the page must be re-checked if they change.
Mitigation: trace comments name the exact doc sections so the next editor finds them.
No new GIFs is a bet — if user testing shows the CAS ladder doesn't land as static SVG, the
fallback is one new generator-backed GIF (tooling exists, make landing-gifs manual). Count stays
minimal by default.

8. Implementation sketch (for the builder, not this plan)

web/src/pages/HowItWorks.jsx — static sections per §2, ConceptGif × 4, 2 inline SVGs,
anchor ids, back-links to /, /setup, /explore.
web/src/index.jsx — add route; web/src/App.jsx — add nav link.
web/test/unit/how-it-works.test.js — source pins (route, nav, anchors, alts, zero-call rule).
docs/go/12_web_ui.md — new §2.3.x subsection (Decisions & deviations entry for the page).
Verify: node --test web/test/unit/*.test.js, real-browser both themes via :9222 on the
canonical host, make fmt && make vet. Commit message names the doc section + decision.

# Deep-dive page plan — Forgejo issue crueber/walhub#191 > Planning only. No production code written. Read-only in the repo > (`git status` verified unchanged after planning). > Issue: "Developer deep-dive page: the object-store idea + what the WAL is > and how it works". Route TBD in planning — this plan picks `/how-it-works`. ## 0. Sources this plan was verified against | Source | Used for | |---|---| | `docs/go/02_storage_protobuf.md` (§2.1 key layout, §2.2 schema, §2.4 framing, §2.5 seq semantics, §2.6 ObjectStore/CAS) | bucket/WAL mechanics claims | | `docs/go/05_wal_engine.md` (§5.0 consistency, §5.2 sync levels, §5.3 publish ladder, §5.4 burn protocol, §5.5 checkpoints, §5.6 replay, §5.7 remote reader) | push/fetch/checkpoint claims | | `docs/go/15_testing.md` (§4.8 budget table) | round-trip budget numbers | | `docs/go/08_bundles.md` (§8.4 creationToken, clone table) | bundle-uri fetch claims | | `docs/features/README.md` (P1–P5, "the WAL stays git-only") | boundaries section | | `web/src/pages/Landing.jsx` + `web/src/components/ConceptGif.jsx` + `web/test/unit/landing.test.js` | reuse pattern, zero-API-call precedent | | `web/src/index.jsx` + `web/src/App.jsx` (route table + nav) | route + nav entry | | `docs/go/12_web_ui.md` (§2.3.0 landing precedent) | doc-update slot | | `README.MD` (L3–5 canonical language) | hero language | | `internal/wal` + `internal/store` (skimmed: `publish.go`, `sync.go`, `checkpoint.go`, `remote.go`, `tasks.go`, `store.go`, `keys.go`) | code-path traces confirm doc claims | Narrative-pacing inspiration ONLY (per the issue): https://cursor.com/blog/git-at-any-scale. Our architecture, our words, our diagrams — do not lift copy, structure, or visuals. --- ## 1. Route + nav entry - **Route: `/how-it-works`** — new static page component `web/src/pages/HowItWorks.jsx`. - Registered in `web/src/index.jsx`: `<Route path="/how-it-works" component={HowItWorks} />` placed with the other top-level static routes (next to `/explore`, `/api`, `/keys`). - `*` fallback stays `Landing` (unchanged — safe default, no API calls; `landing.test.js` pins it). - No server change: the SPA shell serves every UI route (same mechanism `/` already uses). - **Nav entry:** one link in `web/src/App.jsx` site nav: `<A href="/how-it-works">how it works</A>` (lowercase to match existing `explore`/`import`/`keys`/`setup` labels), placed after `explore`. - **Cross-links:** Landing links to it (one line under the hero or in each concept section — "How it works →" deep-linking to `#push`/`#wal`/`#fetch` anchors); the deep-dive links back to `/` ("← walhub in 30 seconds") and to `/setup`. Repo-level `/wal` dashboard links to it as "what am I looking at?" help (optional, single link — the Wal tab itself is unchanged). ## 2. Page outline (section by section, with technical claims + traces) The page is a static component (like `Landing.jsx`): sections + `ConceptGif` reuses + inline static diagrams (see §3). Zero API calls, zero SDK import, zero `fetch` — same rule as `/`. ### 2.0 Hero — "How walhub works" - H1 + one-paragraph thesis in README.MD L3–5 language: the object store is the only database; instances are disposable ("wipe one and you lose nothing but warmth"). - Sub-line: object-protocol compliant with walgit (bucket layout, protobuf wire encoding, git wire behavior follow walgit's formats) — same one-liner as Landing L65–68. - Anchor TOC to the sections below (`#idea`, `#wal`, `#push`, `#read`, `#checkpoints`, `#boundaries`). - Claims: none beyond README/Landing restatement. Trace: `README.MD` L3–5, L15–17. ### 2.1 §`#idea` — The original idea: git objects love dumb object stores Technical claims (each traced): 1. Git objects are content-addressed, so packs and their sidecars drop unchanged into a key-value bucket; the store needs no understanding of git. Trace: `02` §2.1 (everything except `manifest.pb`, `bundles/list.pb`, `leases/*` is immutable; `wal/<checksum>.pack|.idx|.rev|.bitmap|.commit-graph` content-addressed by trailing SHA), `store.go` PutCreate discipline. 2. No database to run, back up, or scale: repos live under `repos/<owner>/<repo>/`; anything that is restart-surviving state is an object; disk and memory are caches. Any S3/GCS/filesystem works behind one `ObjectStore` contract (CAS, conditional GET, compose, leases). Trace: `02` §2.1 key table + §2.6 interface; `AGENTS.md` law 4 ("The bucket is the repository"); `01_overview.md` store row. 3. The store is dumb on purpose: conditional writes (CAS) and conditional reads are the only coordination primitives — there are no locks on the store. Trace: `02` §2.6–2.7 (`PutCreate` / `PutUpdate`, `casUpdate`; "the store is the lock — no mutex at all"). 4. Round trips are the cost model: every protocol change is judged on counted sequential bucket round trips (budgets in §2.4 below). Trace: `AGENTS.md` law 6; `01_overview.md` §"No LIST on a hot path"; `15_testing.md` §4.8. - **GIF reuse:** `bucket` (instance disappears, fresh instance connects, nothing lost) — the visual proof of claim 2. Alt/caption intact from `CONCEPT_ALTS.bucket`. - **New diagram:** none — prose + reused GIF suffice. ### 2.2 §`#wal` — What the WAL is: the log of manifest mutations Technical claims: 1. The **manifest** (`repos/<o>/<r>/manifest.pb`) is the linearization point: `head_seq`, `min_seq`, `checkpoint` pointer, `log_segments` covering `[min_seq, head_seq]`, the denormalized live pack set, settings, monotonic `revision`. Trace: `02` §2.2 `Manifest` schema; `05` §5.0 rule 1. 2. The **WAL** is the append-only stream of `LogEntry` frames in `log/<seq:016x>.pb` segments (uvarint-length-prefixed, partial trailing frame tolerated on appendable segments). Five entry kinds: `PUSH` (pack + ref txn), `COMPACT` (new pack superseding old), `REF_UPDATE` (ref-only), `CHECKPOINT` (marker), `SETTINGS` (history; latest rides the manifest inline). Trace: `02` §2.2 `EntryKind`/`LogEntry`, §2.4 framing; `05` §5.3.3 (settings/compact). 3. **The manifest CAS is the single commit point.** A manifest write is always conditional (`PutUpdate(version)` / `PutCreate`); a 412 is the normal contention signal, never an error. Trace: `05` §5.0 rules 1–2; `02` §2.6 error taxonomy (`PreconditionFailed` is protocol-normal). 4. Seq numbers are strictly increasing but **not dense**: a writer that crashes between log PUT and manifest CAS leaves an orphan; later writers burn past it (3 probes × 100 ms, cap 8 consecutive burns → `Corrupt`). Orphans are harmless and swept after a later commit. Trace: `02` §2.5; `05` §5.4 ladder; `publish.go` slot-claim path. 5. Why concurrent writers stay safe without store locks: one single-flight publisher goroutine per repo serializes all publishes in-process; across instances the CAS serializes commits; losers re-sync, re-verify, retry (bounded ladder, 16 attempts). Group commit batches arrivals (5 ms window, up to 64) so concurrent pushes share one commit. Trace: `05` §5.3.1–5.3.2 + Concurrency notes; `internal/wal/publish.go`. 6. Ref updates are verified old-value-checked transactions (`old_oid` must equal current unless all-zero creation; symbolic HEAD updates always ok); rejected pushes are transport-successes with per-ref errors (`Seq: 0`). Trace: `05` §5.3.2 step 3. - **GIF reuse:** none here (no existing GIF shows the manifest/CAS). The `push` GIF belongs to §2.3. - **New diagram (JUSTIFIED — #1 of at most 2):** one static inline SVG/CSS sequence diagram of the commit point: `log PUT (Create) → manifest CAS (Update) → local refs apply → answer ok`, with the three failure branches (412 → delete-own-segment + retry; ambiguous error → re-read `casLanded`, delete nothing). Static SVG, theme-safe (currentColor / muted classes), no animation, no new deps. Justification: the CAS ladder is the page's central claim and no GIF covers it; a static diagram is zero-weight and accessible (paired `<details>` text fallback describing the same steps). ### 2.3 §`#push` — How a push flows, end to end Walk the numbered ladder as narrative (each step traced to `05` §5.3.2): 1. Sync (refs+serve) unless already fresh → snapshot manifest + version. 2. Verify each ref txn against current refs (old-value check). 3. Assign seqs (`head_seq+1 …`), build PUSH entries with `PackRef{checksum, sizes, tier 0}` + caller meta (principal, request_id, push-options). 4. In parallel: upload pack+idx **create-if-absent** (duplicates are success) ∥ claim the log slot (`PutCreate` of the segment; 412 → manifest re-read → burn-or-retry per §5.4). 5. Build the manifest update (`head_seq`, extended pack set, new segment ref, `revision+1`). 6. CAS the manifest. **The server never ACKs the push before the bucket ACKs** — the push reply is answered only at step 8, after commit. Trace: `05` §5.3.2 steps 7–8; `AGENTS.md` law 4. 7. Local commit under `syncMu`, **refs first, then advertise**; withdraw-on-failure (reset advertised version so the next sync replays — but still answer `ok`, because the bucket is the truth). Trace: `05` §5.0 rule 5 + §5.3.2 step 8 + ordering rule. 8. Sweep burned orphans; fold commit-graph off the critical path; opportunistic checkpoint check. Trace: `05` §5.3.2 step 8. - **GIF reuse:** `push` (laptop → bucket, bucket writes then ACKs, client finishes after the ACK) — the visual for claim 6. Alt/caption intact from `CONCEPT_ALTS.push`. - **New diagram:** none (the §2.2 CAS diagram + prose ladder cover it). ### 2.4 §`#read` — How a fetch/clone reads it back Technical claims: 1. Every request syncs first; four levels: **Refs** (checkpoint snapshot + log-tail ref txns → local `packed-refs`, no packs) serves info/refs, ls-refs, bundle lists, web refs; **Serve** adds the pack set this instance can hold; **Full** materializes everything (refused with `ErrTooLarge` over budget); **Objects** serves-or-remote-reads for the web API. Trace: `05` §5.2 table + `sync.go`. 2. Refs apply is an offline full `packed-refs` rewrite (parse → map → tmp+rename), atomic across many refs, works before packs exist. Annotated-tag `^{}` comes from writer-recorded `new_peeled` (max 16 hops), so replicas advertise without objects. Trace: `05` §5.2 step 2 + §5.3.3 peeling. 3. **Warm refs = 1 request** (conditional GET; 0 within freshness TTL); **cold refs = 2** (manifest GET → checkpoint refs ∥ tail); **push ≤ 5** (freshness GET → pack PUTs ∥ log PUT → manifest CAS); **checkpoint = 4** (never a log GET — provenance rides the applied state). Trace: `15_testing.md` §4.8 budget table (exact numbers + counting rule). 4. At scale, clones ride **bundle-uri**: the CAS'd `bundles/list.pb` advertises full + incremental bundles with `creationToken = slot epoch`; git downloads bundles then fetches the remainder. Trace: `08` §§8.1/8.4 + clone table; `internal/bundle`. 5. Too-large repos don't break stock git: Serve-tier packs can be remote-served (side-files + store mount or no local copy) with the block-cache remote reader + fetch-path faulter filling gaps for the web API and fetches. Trace: `05` §5.7 (+ the stated v1 decision: serve via stock `git` + materialized packs + bundle-uri; no native serving engine in v1 — say this plainly). - **GIF reuse:** `fetch` (empty clone receives ref names, then pack data). Alt/caption intact. - **New diagram:** none. ### 2.5 §`#checkpoints` — Checkpoints, ref snapshots, and cold starts Technical claims: 1. Triggers (any; 0 disables): ≥ 256 entries since last checkpoint, tail bytes > 8 MiB, or age ≥ 1 h. Evaluated after publishes (background, off the reply path) and by the maintainer loop. Trace: `05` §5.5 para 1; `10_maintenance.md` checkpoint unit. 2. Write is refs-level (works on an instance that could never hold the packs), 2 rounds: `checkpoint.pb` ∥ `refs.pb` (`PutCreate`, deterministic seq-keyed) → manifest CAS (`checkpoint`, `min_seq = seq+1`, trim folded segments, `revision+1`). Idempotent at equal seq; racing checkpointers resolve benignly via Create-idempotence + CAS. Trace: `05` §5.5. 3. **Cold-start fold:** a fresh instance loads `refs.pb` at the checkpoint and replays only the tail — it never replays the whole log. `min_seq = checkpoint.seq + 1`; below is folded away. Trace: `05` §5.5 "Cold start fold"; `02` §2.5. 4. Point-in-time: `refsAtSeq`/`refsAsOf` fold from the newest usable checkpoint + ordered entries; cuts older than `min_seq` with no usable checkpoint are unreplayable (why provenance timestamps exist). Trace: `05` §5.6. 5. `policy.json` is NOT on the WAL (admin API/CLI object); per-repo settings ARE published (manifest-inline + `SETTINGS` history, ≤ 16 KiB TOML, validated before publish). Trace: `02` §2.1 key table; `05` §5.3.3 `publish_settings`. - **GIF reuse:** none (bucket GIF already used; checkpoint fold is tail-replay, closest visual is `fetch` — do not triple-use it; link back to §2.4 instead). - **New diagram (JUSTIFIED — #2 of at most 2):** one static inline timeline: checkpoint tick at seq N (`refs.pb` snapshot), folded-away prefix, live tail segments, `min_seq`/`head_seq` markers. Same static-SVG treatment as #1. Justification: "fold, don't replay" is the cold-start claim and is inherently positional; one small figure replaces three paragraphs. ### 2.6 §`#boundaries` — Honest boundaries: what the WAL deliberately does NOT do 1. The WAL stays **git-only**. Issues, PRs, reviews, checks, notifications are a **parallel object family** (`orgs/…`, `users/…`, `repos/<o>/<r>/{meta,issues,checks,releases,access.json}`) with its own CAS discipline — never WAL entries, never the manifest, never gating a push (except policy effects that explicitly consult them, e.g. required checks). Trace: `docs/features/README.md` L11–15 (the one architectural law), P1–P4. 2. What that buys: collaboration writes never contend with the push CAS ladder; git stays readable without the collaboration layer and vice versa. (Framed as design consequence, not a metric — no perf claim.) 3. Explicitly out of scope of this layer (so readers don't project): code search, CI runners (walhub stores check results, doesn't run CI), Discussions/Packages/Projects, SAML/SCIM. Trace: `docs/features/README.md` P9. 4. What the WAL doesn't do, stated plainly: no query engine (reads are key probes + replay, no LIST on hot paths); no cross-repo transactions (one publisher per repo; CAS is per-manifest); no native git serving engine in v1 (stock `git` subprocess + faulter, per `05` §5.7 decision); burned-seq gaps are normal, not damage. - **GIF reuse:** `collab` (issue 7 / PR 8 / green check alongside the git lane, WAL stays git-only) — the visual proof of claim 1. Alt/caption intact from `CONCEPT_ALTS.collab`. - **New diagram:** none. ### 2.7 Closing — where to go next - Links: `/` quickstart (push something), `/explore` (browse), `/setup` (configure), repo `/wal` tab (watch a live manifest), `/api` docs. No claims; pure navigation. --- ## 3. GIF/diagram inventory | Asset | Reused in | Verdict | |---|---|---| | `push.gif` (+ `-still`) | §2.3 push flow | REUSE, alt/caption verbatim | | `bucket.gif` (+ `-still`) | §2.1 the idea | REUSE, alt/caption verbatim | | `fetch.gif` (+ `-still`) | §2.4 reads | REUSE, alt/caption verbatim | | `collab.gif` (+ `-still`) | §2.6 boundaries | REUSE, alt/caption verbatim | | CAS commit-point sequence diagram | §2.2 | NEW static inline SVG (#1) — justified: central claim, no GIF covers it | | Checkpoint fold timeline | §2.5 | NEW static inline SVG (#2) — justified: positional claim, replaces paragraphs | | Any new animated GIF | — | NOT justified — generator exists (`internal/devtools/landinggif/`, `make landing-gifs` is manual) but all four existing GIFs already map 1:1 to sections; new GIFs add weight for no new explanatory power | Reuse mechanics (copy the Landing pattern exactly): `ConceptGif` component (still-first, reduced-motion swap via `prefers-reduced-motion`, `<noscript>` still, `/_ui/concepts/` prefix, lazy + async + 640×360, single `img alt` per figure — never a labeled wrapper on top). Dark-framed cards in both themes (README screenshot precedent — one asset set, no light variants). What the #187 GIFs already cover (do NOT re-explain, link across): Landing owns the 30-second pitch for each scene; the deep-dive links back ("the short version lives on `/`") and goes one level deeper instead of restating. ## 4. Copy honesty rules 1. Every non-obvious technical claim carries its trace (doc section or code path) in a code comment or in the plan's §2 — and the implementation change must keep the comment pins accurate (same discipline as `landing.test.js` source pins). 2. Numbers are budgets, not benchmarks: "push ≤ 5 requests" etc. are sim-asserted transport budgets (`15_testing.md` §4.8), never latency promises. No ms figures anywhere on the page. 3. No overclaiming scale: say "bundle-uri carries large hosts" (Landing's wording), not "at any scale"; say materialization is refused over budget with the bundle-uri fix text, not that every repo fits everywhere. 4. No claiming what isn't built: the v1 remote-reader decision (stock git + faulter, no native serving engine) is stated, not hidden; collaboration features link to their real state, not a roadmap promise. 5. Wire-compat wording copies Landing/README verbatim ("object-protocol compliant with walgit — bucket layout, protobuf wire encoding, and git wire behavior follow walgit's formats"), never paraphrased into something stronger. 6. Boundaries section (§2.6) is mandatory, not decorative — it ships in v1 of the page. 7. cursor.blog influence is pacing only (short sections, one idea per scroll, visual-then-prose rhythm); no lifted copy, structure, or visuals. ## 5. EVIDENCE / performance - **Zero-API-call static page** (same rule as `/`): no `useData`, no SDK import, no `fetch`. The page spends no store round trips — asserted by a `node --test` source-pin test mirroring `landing.test.js`. No new network surface, no new backend path. - **No `docs/EVIDENCE.md` entry**: the page makes no hot-path performance claim (it only *reports* existing sim budgets); per `AGENTS.md`, evidence entries are for transport/storage features making hot-path claims. If a future iteration adds measured figures, it gets a harness + entry then. - Weight: 4 reused GIFs (already shipped, cached) + 2 inline SVGs (bytes, no requests). No new npm runtime deps (Solid + router only, already present); no new Go modules. ## 6. Acceptance criteria (from the issue, made checkable) - [ ] `HowItWorks.jsx` + `/how-it-works` route + `how it works` nav entry; `*` fallback still Landing. - [ ] Zero-API-call rule pinned by a new `web/test/unit/how-it-works.test.js` (routes, nav, anchors, alt-text pins, no `useData`/SDK/`fetch` strings) — mirrors `landing.test.js`. - [ ] Every §2 claim reviewed against `02`/`05`/`08`/`15_testing`/`features/README` + the `internal/wal` + `internal/store` code paths listed; traces recorded in code comments. - [ ] All 4 GIFs reused with alt/captions verbatim; `ConceptGif` (still-first + reduced-motion + noscript) used for each; `docs/go/12_web_ui.md` gains a §2.3.x subsection for the page. - [ ] `node --test` over `web/test/unit/*.test.js` green (never a directory positional — Node 22 executes a directory as a module; see AGENTS.md field lessons). - [ ] Real-browser pass (hub-managed `chrome-cdp` on :9222), both themes, zero console errors — module scripts are MIME-enforced and import-map driven ("curl says 200" proves nothing); drive against the canonical host (loopback GETs 302 to `walgit.localhost` — `06_server_http.md` §2.2 #2), or expect the hop. - [ ] No new runtime deps (npm or Go); `make fmt && make vet` clean; `git status` shows only the intended files. ## 7. Open questions / risks 1. **Route name** — `/how-it-works` vs `/how` vs `/internals`. `/how-it-works` is self-describing and matches the issue's example; short `/how` risks collision with future pages. Recommend `/how-it-works`. (Needs owner sign-off — one-line decision.) 2. **`?format=text` parity** — `12_web_ui.md` notes the text format "moves server-side with the page" for Landing (`06_server_http.md` §Decisions). Does `/how-it-works` need a text rendering too, or is the SPA-only page acceptable? Recommend SPA-only for v1 (marketing page, not data), but confirm. 3. **Nav crowding** — App nav gains a 6th link (`explore import API keys setup` + `how it works`); check mobile wrap (existing flex-wrap handles it, but verify in the browser pass). 4. **Length risk** — 7 sections is long for one page; mitigation is the anchor TOC + one-idea-per-section pacing (the cursor.blog lesson). If review finds it heavy, split boundary: §§2.1–2.3 ship first, §§2.4–2.6 follow — but default is one page, one change. 5. **SVG-in-JSX cost** — two inline SVGs bloat the component; alternative is `web/public/` `.svg` assets served under `/_ui/`. Recommend inline (no extra requests, theme via currentColor) unless the file exceeds ~200 lines, then split to assets. 6. **Claim drift** — the page restates constants (256 entries, 8 MiB, 1 h, 5 ms/64, 16 retries, 8 burns, budgets 5/1/2/4). These live in config + sim tests; the page must be re-checked if they change. Mitigation: trace comments name the exact doc sections so the next editor finds them. 7. **No new GIFs is a bet** — if user testing shows the CAS ladder doesn't land as static SVG, the fallback is one new generator-backed GIF (tooling exists, `make landing-gifs` manual). Count stays minimal by default. --- ## 8. Implementation sketch (for the builder, not this plan) 1. `web/src/pages/HowItWorks.jsx` — static sections per §2, `ConceptGif` × 4, 2 inline SVGs, anchor ids, back-links to `/`, `/setup`, `/explore`. 2. `web/src/index.jsx` — add route; `web/src/App.jsx` — add nav link. 3. `web/test/unit/how-it-works.test.js` — source pins (route, nav, anchors, alts, zero-call rule). 4. `docs/go/12_web_ui.md` — new §2.3.x subsection (Decisions & deviations entry for the page). 5. Verify: `node --test web/test/unit/*.test.js`, real-browser both themes via `:9222` on the canonical host, `make fmt && make vet`. Commit message names the doc section + decision.

crueber commented

2026-09-08 12:51:39 +00:00

Author

Owner

Plan review — verdict: proceed-with-fixes (1 blocking)

Reviewed the plan end to end against AGENTS.md (12 laws), docs/go/02 + 05, web/src/index.jsx + App.jsx, docs/go/12 §2.3.0, docs/go/06, plus spot-checks in 03 §7, 08 §8.4, 15 §4.1, features/README, router.go, Landing.jsx. No repo files touched.

Technical-claim verification (sampled highest-risk)

Accurate: manifest fields (head_seq/min_seq/checkpoint/log_segments [min,head]/denormalized packs/settings/revision — 02 §2.2); all 5 entry kinds; checkpoint triggers 256 / 8 MiB / 1 h (05 §5.5+§5.10); group commit 5 ms / 64 (05 §5.10); burn probes 3×100 ms, cap 8 → Corrupt (05 §5.3.2/§5.4); CAS retries 16; new_peeled 16 hops; SETTINGS ≤16 KiB; creationToken = slot epoch (08 §8.4, see nit N1); sync-level table; offline packed-refs rewrite; checkpoint 2-round write + idempotence; min_seq fold; refsAtSeq unreplayable cut; v1 no-native-engine decision; WAL-stays-git-only + P1 prefixes + P9 outs (features/README); framing partial-tail tolerance; "store is the lock" quote. No fabricated claim found.

BLOCKING

B1 — "No server change" contradicts the /explore precedent. internal/server/router.go:121-129 says /explore MUST be an explicit route, otherwise /* → repoDispatch treats it as owner "explore" (same shell today, but the ?format=text branch and the name reservation live only on the explicit handler). /how-it-works is single-segment → identical situation. The plan must: add r.Get("/how-it-works", s.gated(...)) next to /explore, add how-it-works to the reserved single-segment names (06 §3.3 list + §14 decision entry + router.go comment), and add a server test (GET → 200 shell, no-cache). Without this an owner named how-it-works collides silently.

Should-fix

S1 — ?format=text citation is inverted. The text twin moved FROM / TO /explore (06 §14 decision, #187), not "with the page for Landing". SPA-only v1 recommendation still agreed (text twin lives at /explore; / answers shell unconditionally).
S2 — "cold refs = 2" conflates depth with requests. 03 §7: cold-refs requests = 2+tail (no checkpoint: 1+tail); 2 is the depth. Reword as depth-2 to avoid overclaiming.
S3 — claim-drift mitigation: pin config key names, not just doc sections, in the trace comments (wal.snapshot_every_entries, wal.checkpoint_tail_bytes, wal.checkpoint_interval, wal.batch_window, wal.max_batch, wal.cas_max_retries) — values live in config, sections move.
S4 — browser pass must assert /how-it-works renders the HowItWorks hero, not the /:owner page (static-vs-param precedence), both themes, plus the already-planned mobile nav-wrap check.

Nits

N1 — creationToken = "slot epoch seconds" (08 §8.4). N2 — push budget parenthetical should note "4 if already synced" (03 §7) since the page quotes 5.

Stress-test answers

Route /how-it-works: good (self-describing; /how risks collision). Nav 5→6 links: fine, flex-wrap + verify. Zero-API-call via source pins mirroring landing.test.js: enforceable, agreed. 2 inline SVGs: justified (central CAS claim + positional fold claim, no GIF covers either), keep the 200-line split rule. GIF reuse with verbatim alts: checked against CONCEPT_ALTS — no staleness, no triple-use, agreed. EVIDENCE no-entry: agreed (page reports sim budgets, makes no hot-path claim). Claim-drift: adequate with S3.

Missing for shippable v1 (beyond B1/S4)

Server shell test + reserved-name doc updates (in B1); nothing else — acceptance criteria otherwise cover route/nav/pins/alts/browser/fmt+vet.

# Plan review — verdict: proceed-with-fixes (1 blocking) Reviewed the plan end to end against AGENTS.md (12 laws), docs/go/02 + 05, web/src/index.jsx + App.jsx, docs/go/12 §2.3.0, docs/go/06, plus spot-checks in 03 §7, 08 §8.4, 15 §4.1, features/README, router.go, Landing.jsx. No repo files touched. ## Technical-claim verification (sampled highest-risk) Accurate: manifest fields (head_seq/min_seq/checkpoint/log_segments [min,head]/denormalized packs/settings/revision — 02 §2.2); all 5 entry kinds; checkpoint triggers 256 / 8 MiB / 1 h (05 §5.5+§5.10); group commit 5 ms / 64 (05 §5.10); burn probes 3×100 ms, cap 8 → Corrupt (05 §5.3.2/§5.4); CAS retries 16; new_peeled 16 hops; SETTINGS ≤16 KiB; creationToken = slot epoch (08 §8.4, see nit N1); sync-level table; offline packed-refs rewrite; checkpoint 2-round write + idempotence; min_seq fold; refsAtSeq unreplayable cut; v1 no-native-engine decision; WAL-stays-git-only + P1 prefixes + P9 outs (features/README); framing partial-tail tolerance; "store is the lock" quote. No fabricated claim found. ## BLOCKING **B1 — "No server change" contradicts the /explore precedent.** `internal/server/router.go:121-129` says `/explore` MUST be an explicit route, otherwise `/*` → repoDispatch treats it as owner "explore" (same shell today, but the `?format=text` branch and the name reservation live only on the explicit handler). `/how-it-works` is single-segment → identical situation. The plan must: add `r.Get("/how-it-works", s.gated(...))` next to `/explore`, add `how-it-works` to the reserved single-segment names (06 §3.3 list + §14 decision entry + router.go comment), and add a server test (GET → 200 shell, no-cache). Without this an owner named `how-it-works` collides silently. ## Should-fix **S1 — `?format=text` citation is inverted.** The text twin moved FROM `/` TO `/explore` (06 §14 decision, #187), not "with the page for Landing". SPA-only v1 recommendation still agreed (text twin lives at `/explore`; `/` answers shell unconditionally). **S2 — "cold refs = 2" conflates depth with requests.** 03 §7: cold-refs requests = 2+tail (no checkpoint: 1+tail); 2 is the depth. Reword as depth-2 to avoid overclaiming. **S3 — claim-drift mitigation: pin config key names**, not just doc sections, in the trace comments (`wal.snapshot_every_entries`, `wal.checkpoint_tail_bytes`, `wal.checkpoint_interval`, `wal.batch_window`, `wal.max_batch`, `wal.cas_max_retries`) — values live in config, sections move. **S4 — browser pass must assert `/how-it-works` renders the HowItWorks hero, not the `/:owner` page** (static-vs-param precedence), both themes, plus the already-planned mobile nav-wrap check. ## Nits **N1** — creationToken = "slot epoch **seconds**" (08 §8.4). **N2** — push budget parenthetical should note "4 if already synced" (03 §7) since the page quotes 5. ## Stress-test answers Route `/how-it-works`: good (self-describing; `/how` risks collision). Nav 5→6 links: fine, flex-wrap + verify. Zero-API-call via source pins mirroring `landing.test.js`: enforceable, agreed. 2 inline SVGs: justified (central CAS claim + positional fold claim, no GIF covers either), keep the 200-line split rule. GIF reuse with verbatim alts: checked against `CONCEPT_ALTS` — no staleness, no triple-use, agreed. EVIDENCE no-entry: agreed (page reports sim budgets, makes no hot-path claim). Claim-drift: adequate with S3. ## Missing for shippable v1 (beyond B1/S4) Server shell test + reserved-name doc updates (in B1); nothing else — acceptance criteria otherwise cover route/nav/pins/alts/browser/fmt+vet.

crueber commented

2026-09-08 12:53:05 +00:00

Author

Owner

Plan revision R1 (review findings — R1 wins on conflict)

Blocking (normative)

B1 — explicit server route for /how-it-works (same as /explore precedent): r.Get("/how-it-works", s.gated(...)) next to /explore; add how-it-works to reserved single-segment names (06 §3.3 + §14 decision + router comment); server test (GET → 200 shell, no-cache).

Should-fix adoptions

S1: ?format=text moved FROM / TO /explore — SPA-only v1 for the deep-dive, no text twin.
S2: "cold refs = depth 2" (not "2 requests"; 2+tail / 1+tail sans checkpoint).
S3: drift pins name config keys (wal.snapshot_every_entries, wal.checkpoint_tail_bytes, wal.checkpoint_interval, wal.batch_window, wal.max_batch, wal.cas_max_retries), not just doc sections.
S4: browser pass asserts /how-it-works renders the hero (not /:owner), both themes + mobile nav-wrap.

Nits adopted

N1 creationToken = slot epoch seconds; N2 push budget "4 if already synced". All other plan text stands.

# Plan revision R1 (review findings — R1 wins on conflict) ## Blocking (normative) - **B1 — explicit server route for `/how-it-works`** (same as `/explore` precedent): `r.Get("/how-it-works", s.gated(...))` next to `/explore`; add `how-it-works` to reserved single-segment names (06 §3.3 + §14 decision + router comment); server test (GET → 200 shell, no-cache). ## Should-fix adoptions - **S1:** `?format=text` moved FROM `/` TO `/explore` — SPA-only v1 for the deep-dive, no text twin. - **S2:** "cold refs = depth 2" (not "2 requests"; 2+tail / 1+tail sans checkpoint). - **S3:** drift pins name config keys (`wal.snapshot_every_entries`, `wal.checkpoint_tail_bytes`, `wal.checkpoint_interval`, `wal.batch_window`, `wal.max_batch`, `wal.cas_max_retries`), not just doc sections. - **S4:** browser pass asserts `/how-it-works` renders the hero (not `/:owner`), both themes + mobile nav-wrap. ## Nits adopted N1 creationToken = slot epoch seconds; N2 push budget "4 if already synced". All other plan text stands.

crueber referenced this issue from a commit

2026-09-08 13:12:28 +00:00

Feature #191: developer deep-dive page at /how-it-works

crueber referenced this issue from a pull request that will close it,

2026-09-08 13:12:35 +00:00

Feature #191: developer deep-dive page #194

crueber commented

2026-09-08 13:12:40 +00:00

Author

Owner

Implemented as PR #194 (branch feat/issue-191, against main, no conflicts). What shipped: HowItWorks.jsx + HowDiagrams.jsx (2 static SVGs), route + nav + cross-links (Landing, WAL tab), backend explicit howItWorksPage route + reservation docs + server test, how-it-works.test.js, docs 12 §2.3.x + 06 §3.3/§14. R1 B1/S1-S4 + N1/N2 adopted. Verification: gofmt/vet clean; internal/server -race green; make cover gate holds (48 pkgs ok, server 95.6%); node --test 397/397 green; real-Chromium pass on canonical host (hero renders, both themes, 6 anchors + TOC + nav + 4 GIFs + 2 SVGs, zero console errors, no mobile overflow). Deviations: Landing cross-links use #read (not #fetch — matches the section id); diagrams carry role=img (SVG needs its accessible name; the N1 no-role rule is ConceptGif-specific). Not merged — awaiting review.

crueber commented

2026-09-08 13:19:29 +00:00

Author

Owner

Review of PR #194 (feat/issue-191, commit 1dd31f5) against issue #191 plan + R1 (plan-review normative). Verified in scratch worktree; main worktree untouched.

R1 compliance — all pass

B1 explicit route + reservation + server test: PASS. internal/server/router.go:132 r.Get("/how-it-works", s.gated(s.howItWorksPage)) with updated reservation comment (:121-129); internal/server/health.go:178 howItWorksPage (shell unconditionally, no text branch); internal/server/x_health_test.go:249 TestHowItWorksPage (401 gated, 200 shell + no-cache, ?format=text still shell). docs/go/06_server_http.md:124 route table + :216 reserved-names list + :623 §14 decision entry. Chi static-beats-wildcard ordering correct (registered before /* at :137).
S1 no text twin: PASS. ?format=text lives at /explore only; howItWorksPage serves shell unconditionally (test-pinned); 06 + 12 docs state SPA-only explicitly.
S2 depth wording: PASS. web/src/pages/HowItWorks.jsx: idea section 'warm refs depth 1, cold refs depth 2' + read section 'cold refs at depth 2 (...; depth 1 without a checkpoint); push ≤ 5 (4 if already synced)'. Matches 03 §7 table (depth vs 2+tail/1+tail requests). No latency language on page.
S3 config-key pins: PASS. All six keys pinned in trace comments (wal.snapshot_every_entries + checkpoint_tail_bytes + checkpoint_interval in checkpoints section; batch_window + max_batch + cas_max_retries in WAL section; checkpoint keys also in HowDiagrams.jsx) and pinned by how-it-works.test.js drift-pins test.
S4 hero-render assertion: NOT INDEPENDENTLY VERIFIED (no browser per review instructions). Author reports real-Chromium pass on canonical host (comment 1694). Headless substitute green: route/nav/anchor pins + shell test prove /how-it-works serves the SPA shell and the frontend route table maps it to HowItWorks before /:owner.
N1 creationToken = slot epoch seconds: PASS (HowItWorks.jsx read section, matches 08 §8.4). N2 push '4 if already synced': PASS (two places, matches 03 §7).

Technical-claim spot-checks (vs 02/05/03/08/15) — no overclaim found

Manifest fields (head_seq/min_seq/checkpoint/log_segments [min,head]/denormalized packs/settings/revision) == 02 §2.2. 5 entry kinds + semantics == 02 §2.2. Checkpoint triggers 256/8MiB/1h == 05 §5.5 + §5.10 config defaults. Group commit 5ms/64 + 16 CAS retries == 05 §5.10/§5.3.2. Burn 3×100ms cap 8 → Corrupt == 05 §5.4. new_peeled 16 hops, SETTINGS ≤16 KiB, burn-gaps-normal, v1 no-native-engine, WAL-stays-git-only — all match cited sections. Budgets framed as depths/budgets, never latency; checkpoint-4 correctly notes 'never a log GET'.

Contract checks — all pass

Zero-API-call: grep clean (only a comment mentions useData); no SDK/fetch/EventSource in HowItWorks.jsx or HowDiagrams.jsx; test-pinned.
GIF alts verbatim by construction: import { CONCEPT_ALTS } from Landing.jsx (no copied strings); captions match Landing.jsx:81-110 verbatim. ConceptGif path (still-first/reduced-motion/_ui/concepts/).
Cross-links: Landing.jsx:69-74 → /how-it-works; HowItWorks → / + /explore + /setup + /api; Wal.jsx:302 → /how-it-works#wal. (Deviation noted: Landing links to the page root, not #push/#wal/#fetch deep anchors — acceptable, anchors exist for direct linking.)
Nav: App.jsx:53 after explore, lowercase; index.jsx:54 static route before /:owner; * fallback still Landing (:87) untouched.
Setup-only: /how-it-works correctly absent from mountSetupOnly (router.go:143-161) → 503 like /explore; gated() in normal mode.
SVG a11y: both diagrams carry role=img + aria-label plus
text fallback. N1 no-role rule is ConceptGif-specific (ConceptGif.jsx has no role) — role=img on bare SVG is correct and justified.
No new deps: go.mod, package.json, web/package.json untouched (diff = 12 files only).

Tests (scratch worktree, dist concepts staged via public/ copy + node_modules symlink — both scratch-only; main untouched)

go test ./internal/server/... -race: PASS (full package).
Coverage: 95.6% ≥ 95% gate holds.
node --test web/test/unit/*.test.js: 397/397 PASS (how-it-works.test.js 10/10, landing 7/7).
gofmt clean, go vet clean.
NOTE: TestUIAssetConcepts fails in a bare git worktree without a web build (dist/concepts missing — pre-existing environmental precondition, 'run make web', also true on main); passes once concepts are staged. No browser pass run (per review instructions).

Merge recommendation

Ready to merge. No blocking issues, no fixes applied (none needed). Only residual: S4 browser assertion rests on the author's reported Chromium pass — spot-check in post-merge smoke if desired.

Review of PR #194 (feat/issue-191, commit 1dd31f5) against issue #191 plan + R1 (plan-review normative). Verified in scratch worktree; main worktree untouched. ## R1 compliance — all pass - **B1 explicit route + reservation + server test: PASS.** internal/server/router.go:132 r.Get("/how-it-works", s.gated(s.howItWorksPage)) with updated reservation comment (:121-129); internal/server/health.go:178 howItWorksPage (shell unconditionally, no text branch); internal/server/x_health_test.go:249 TestHowItWorksPage (401 gated, 200 shell + no-cache, ?format=text still shell). docs/go/06_server_http.md:124 route table + :216 reserved-names list + :623 §14 decision entry. Chi static-beats-wildcard ordering correct (registered before /* at :137). - **S1 no text twin: PASS.** ?format=text lives at /explore only; howItWorksPage serves shell unconditionally (test-pinned); 06 + 12 docs state SPA-only explicitly. - **S2 depth wording: PASS.** web/src/pages/HowItWorks.jsx: idea section 'warm refs depth 1, cold refs depth 2' + read section 'cold refs at depth 2 (...; depth 1 without a checkpoint); push ≤ 5 (4 if already synced)'. Matches 03 §7 table (depth vs 2+tail/1+tail requests). No latency language on page. - **S3 config-key pins: PASS.** All six keys pinned in trace comments (wal.snapshot_every_entries + checkpoint_tail_bytes + checkpoint_interval in checkpoints section; batch_window + max_batch + cas_max_retries in WAL section; checkpoint keys also in HowDiagrams.jsx) and pinned by how-it-works.test.js drift-pins test. - **S4 hero-render assertion: NOT INDEPENDENTLY VERIFIED (no browser per review instructions).** Author reports real-Chromium pass on canonical host (comment 1694). Headless substitute green: route/nav/anchor pins + shell test prove /how-it-works serves the SPA shell and the frontend route table maps it to HowItWorks before /:owner. - **N1 creationToken = slot epoch seconds: PASS** (HowItWorks.jsx read section, matches 08 §8.4). **N2 push '4 if already synced': PASS** (two places, matches 03 §7). ## Technical-claim spot-checks (vs 02/05/03/08/15) — no overclaim found - Manifest fields (head_seq/min_seq/checkpoint/log_segments [min,head]/denormalized packs/settings/revision) == 02 §2.2. 5 entry kinds + semantics == 02 §2.2. Checkpoint triggers 256/8MiB/1h == 05 §5.5 + §5.10 config defaults. Group commit 5ms/64 + 16 CAS retries == 05 §5.10/§5.3.2. Burn 3×100ms cap 8 → Corrupt == 05 §5.4. new_peeled 16 hops, SETTINGS ≤16 KiB, burn-gaps-normal, v1 no-native-engine, WAL-stays-git-only — all match cited sections. Budgets framed as depths/budgets, never latency; checkpoint-4 correctly notes 'never a log GET'. ## Contract checks — all pass - Zero-API-call: grep clean (only a comment mentions useData); no SDK/fetch/EventSource in HowItWorks.jsx or HowDiagrams.jsx; test-pinned. - GIF alts verbatim by construction: import { CONCEPT_ALTS } from Landing.jsx (no copied strings); captions match Landing.jsx:81-110 verbatim. ConceptGif path (still-first/reduced-motion/_ui/concepts/). - Cross-links: Landing.jsx:69-74 → /how-it-works; HowItWorks → / + /explore + /setup + /api; Wal.jsx:302 → /how-it-works#wal. (Deviation noted: Landing links to the page root, not #push/#wal/#fetch deep anchors — acceptable, anchors exist for direct linking.) - Nav: App.jsx:53 after explore, lowercase; index.jsx:54 static route before /:owner; * fallback still Landing (:87) untouched. - Setup-only: /how-it-works correctly absent from mountSetupOnly (router.go:143-161) → 503 like /explore; gated() in normal mode. - SVG a11y: both diagrams carry role=img + aria-label plus <details> text fallback. N1 no-role rule is ConceptGif-specific (ConceptGif.jsx has no role) — role=img on bare SVG is correct and justified. - No new deps: go.mod, package.json, web/package.json untouched (diff = 12 files only). ## Tests (scratch worktree, dist concepts staged via public/ copy + node_modules symlink — both scratch-only; main untouched) - go test ./internal/server/... -race: PASS (full package). - Coverage: 95.6% ≥ 95% gate holds. - node --test web/test/unit/*.test.js: 397/397 PASS (how-it-works.test.js 10/10, landing 7/7). - gofmt clean, go vet clean. - NOTE: TestUIAssetConcepts fails in a bare git worktree without a web build (dist/concepts missing — pre-existing environmental precondition, 'run make web', also true on main); passes once concepts are staged. No browser pass run (per review instructions). ## Merge recommendation **Ready to merge.** No blocking issues, no fixes applied (none needed). Only residual: S4 browser assertion rests on the author's reported Chromium pass — spot-check in post-merge smoke if desired.

crueber closed this issue

2026-09-08 13:20:36 +00:00

crueber commented

2026-09-08 13:20:37 +00:00

Author

Owner

Implemented in PR #194 (review: all R1 rulings verified, claims spot-checked, 397/397 + 95.6%), merged. Closing.

crueber referenced this issue from a commit

2026-09-08 13:20:37 +00:00

Merge pull request 'Feature #191: developer deep-dive page' (#194) from feat/issue-191 into main

crueber referenced this issue from a commit

2026-09-09 12:51:04 +00:00