crueber/walhub
Watch
1
0
Fork
You've already forked walhub
 0
Code Issues 2 Pull requests Projects Releases Packages Wiki Activity

Feature #248: server-side repo size tracking #258

Merged
crueber merged 3 commits from feat/issue-248 into main 2026-09-09 22:06:15 +00:00
Conversation 0 Commits 3 Files changed 35 +2865 -16
crueber commented 2026-09-09 21:34:54 +00:00
Owner
Copy link

Implements Forgejo #248 per plan R1 (lands first; shared sidecar + catalog rails #247 builds on).

What lands here:

  • Proto: RepoCatalog field-3 entries (RepoCatalogEntry) + hand codec + catalog_entry/catalog_size golden fixtures; legacy field-1/2 readers interoperate (nil entries, never zero). wal.proto schema text updated in the same change.
  • Per-repo sidecar repos///meta/stats.json (CAS'd JSON v1) written ONLY by the maintainer sweep; push contributes pure arithmetic on the in-hand live pack set (+0 trips, no I/O).
  • Maintainer fold: sizecatalog.Sweep (bounded 256/pass, in-memory cursor resume, PUT-if-changed, per-repo isolation, one catalog CAS/pass) wired as pass-end best-effort foldSizeCatalog.
  • NEW endpoint GET /api/v1/owners/{owner}/repos/detailed (+ api-browser + services twins, discovery endpoints[], SDK owners.detailed): object rows, sort=name|size, order, min/max_bytes; one catalog GET per query; absent catalog degrades to null rows. v1 string lists untouched.
  • Size semantic: stored-object size = PackSize+IdxSize (canonical); overview LiveBytes stays PackSize-only (documented delta); AddPack approximation, .rev/.bitmap/.commit-graph + bundles + LFS exclusions, null-vs-0 all documented.
  • Docs: Decisions appended in 02/07/10/14; endpoint docs in 07; EVIDENCE E16 (fold linear 2 GETs + <=1 PUT per repo + 1 CAS; listing flat 1 GET; push +0; 0 LIST).

Tests: sizecatalog 97.8%, api 95.1%, maintain 96.3%, proto 98.5%, wal 95.4%, store 95.0% (all -race); node --test 500/500 (incl. 2 new SDK surface rows). No new Go modules, no new npm deps. UI display out of scope (queryable state only; owner page can consume owners.detailed next).

Closes #248

Implements Forgejo #248 per plan R1 (lands first; shared sidecar + catalog rails #247 builds on). What lands here: - Proto: RepoCatalog field-3 entries (RepoCatalogEntry) + hand codec + catalog_entry/catalog_size golden fixtures; legacy field-1/2 readers interoperate (nil entries, never zero). wal.proto schema text updated in the same change. - Per-repo sidecar repos/<o>/<r>/meta/stats.json (CAS'd JSON v1) written ONLY by the maintainer sweep; push contributes pure arithmetic on the in-hand live pack set (+0 trips, no I/O). - Maintainer fold: sizecatalog.Sweep (bounded 256/pass, in-memory cursor resume, PUT-if-changed, per-repo isolation, one catalog CAS/pass) wired as pass-end best-effort foldSizeCatalog. - NEW endpoint GET /api/v1/owners/{owner}/repos/detailed (+ api-browser + services twins, discovery endpoints[], SDK owners.detailed): object rows, sort=name|size, order, min/max_bytes; one catalog GET per query; absent catalog degrades to null rows. v1 string lists untouched. - Size semantic: stored-object size = PackSize+IdxSize (canonical); overview LiveBytes stays PackSize-only (documented delta); AddPack approximation, .rev/.bitmap/.commit-graph + bundles + LFS exclusions, null-vs-0 all documented. - Docs: Decisions appended in 02/07/10/14; endpoint docs in 07; EVIDENCE E16 (fold linear 2 GETs + <=1 PUT per repo + 1 CAS; listing flat 1 GET; push +0; 0 LIST). Tests: sizecatalog 97.8%, api 95.1%, maintain 96.3%, proto 98.5%, wal 95.4%, store 95.0% (all -race); node --test 500/500 (incl. 2 new SDK surface rows). No new Go modules, no new npm deps. UI display out of scope (queryable state only; owner page can consume owners.detailed next). Closes #248
Per-repo size sidecar (meta/stats.json) + aggregate catalog rows
(RepoCatalog field-3 entries) folded by the maintainer sweep; NEW
detailed listing endpoint alongside v1 (triple twins + discovery +
SDK owners.detailed). Push stays +0 trips (pure arithmetic on the
in-hand live pack set). Size = PackSize+IdxSize (canonical;
overview LiveBytes documented as PackSize-only).

Decisions appended in 02/07/10/14; EVIDENCE E16.
The publish ladder now PUTs repos/<o>/<r>/meta/stats.json (same v1 shape)
in parallel with the manifest CAS whenever a batch holds PUSH/COMPACT
entries (ref-only/settings skip; annotate bypasses the ladder): +1 total
op, +0 sequential trips. Best-effort (WARN-only; manifest CAS stays the
only commit point); the sweep remains as backfill/repair. Durability test
asserts sidecar bytes after push and compact without running the sweep.

Corrects the 02/10/14 decision entries (they claimed R1 B1 authority for
sweep-only writes — the opposite of what R1 ordered); updates E16, the
15_testing push budget (<=6 with R1 justification), and the push-budget
fence (second sanctioned touch, bounded). Endpoint, catalog fold, codec,
fixtures untouched.
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
actions

   
bug
Something is not working

  
cli

   
duplicate
This issue or pull request already exists

  
enhancement
New feature

  
fork

   
forum

   
git storage

   
help wanted
Need some help

  
insights

   
invalid
Something is wrong

  
issues

   
moderation

   
oidc

   
ownership transfer

   
packages

   
pr/merge protection rules

   
projects

   
pull requests

   
question
More information is needed

  
releases

   
sponsorships

   
tags

   
webhooks

   
wiki

   
wontfix
This won't be fixed

No labels
actions
bug
cli
duplicate
enhancement
fork
forum
git storage
help wanted
insights
invalid
issues
moderation
oidc
ownership transfer
packages
pr/merge protection rules
projects
pull requests
question
releases
sponsorships
tags
webhooks
wiki
wontfix
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
crueber/walhub!258
No description provided.