Paged tag lists load and sort every repository tag

perfloop-oss/harbor · Late Filtering

https://perfloop.ai/t/oss/case_6vtk8g6m68

Verdict

VERIFIED · settled 2026-11-14 · pull request opened as goharbor/harbor#24343

What happened: The paired measurements met the required improvement.

Hypothesis

A paged tag-list request loads every tag row in the repository before applying the cursor and count. It then copies and sorts all names and encodes only one page. This work grows with tag count rather than page size; real database and ORM costs remain unmeasured.

In tagHandler.ServeHTTP, tag.List receives only a repository filter, with no name cursor, name ordering or limit. The query uses the tag model's default push-time/ID order, then builds a full name slice. SendListTagsResponse runs sortedAndUniqueItems over all names before pickItems selects the page. The query layer already supports filtered queries, sorting and PageSize limits. The tag table's unique constraint on repository_id and name also exposes the needed key; its query plan was not measured.

profileWorkload receipt call_Tai5Q1EgGYWoBFGeON2SgduM ran the unchanged handler against a fixed in-memory repository. That fixture held 10,000 shuffled distinct names, requested n=100, and checked the first page plus continuation. sortedAndUniqueItems frames accounted for 3.27 s of 3.89 s sampled CPU in that receipt. Fixture reads exclude database, ORM hydration, authorization, network and production request mix, so this does not establish a deployment saving.

The benefit applies only to requests with a positive n on the local handler path. Unpaginated requests still need every name. TagsListMiddleware separately merges local and remote proxy tags; that path should keep whole-collection normalization unless it gets its own design. A database-side change must use byte-equivalent ordering rather than default collation, because pickItems uses Go string comparison.

The Case proof target is a differential test against the real database layer showing identical pages and continuations across mixed case, digits, punctuation, cursor positions and page boundaries. It should also measure rows fetched and handler cost before and after across tag counts. The benchmark and checker layouts here are provisional. The admitted overflow fix, hyp_4t83s9qyws, remains a separate correctness repair. No matching upstream change appeared in the inspected open-PR pages. https://github.com/goharbor/harbor/pull/23981 touches tag creation only. Accepted history after the shallow boundary showed no related change. Development-list and file-history web reads were refused, leaving upstream evidence gaps.

Change to test: For local repositories with a positive count, query only names after the cursor in byte order, limited to one more than the page size, so the database narrows the result before Go copies or sorts it. Preserve Go string ordering, uniqueness, strict cursor behavior, terminal continuation rules and the unchanged unpaginated and proxy merge paths.

Where it lives

perfloop-oss/harbor · src/server/registry/tag.go

Evidence

Local tag list with 2 tags · 87 sample pairs

metric baseline candidate paired median change confidence range required result
B/op 13071 9175 −29.8% (−3896) −3896 to −3895 gain > 0; gain > 0.05 × |baseline median|; arm medians separated PASSED
ns/op 401623 385151 −4.6% (−18476) −47464 to −6676 loss ≤ 0.05 × |baseline median| PASSED
allocs/op 329 217 −34% (−112) −112 to −112 loss ≤ 0.05 × |baseline median| PASSED
tag_rows/op 2 2 0 0 to 0 loss ≤ 0.05 × |baseline median| PASSED
db_ns/op 33106 33751 +743 −1019 to +1561 loss ≤ 0.05 × |baseline median| PASSED

Local tag list with 1500 tags · 16 sample pairs

metric baseline candidate paired median change confidence range required result
B/op 651206 114940 −82.4% (−536282) −537656 to −535335 gain > 0; gain > 0.05 × |baseline median|; arm medians separated PASSED
ns/op 4079197 1530553 −62.1% (−2533572) −2637931 to −2337894 loss ≤ 0.05 × |baseline median| PASSED
allocs/op 23728 4233 −82.2% (−19495) −19496 to −19494 loss ≤ 0.05 × |baseline median| PASSED
tag_rows/op 1500 1001 −33.3% (−499) −499 to −499 loss ≤ 0.05 × |baseline median| PASSED
db_ns/op 1057471 657692 −37.9% (−400642) −446871 to −326619 loss ≤ 0.05 × |baseline median| PASSED

Local tag list with 10000 tags · 16 sample pairs

metric baseline candidate paired median change confidence range required result
B/op 4501482 115723 −97.4% (−4385849) −4386571 to −4381765 gain > 0; gain > 0.05 × |baseline median|; arm medians separated PASSED
ns/op 22071357 3121612 −86.2% (−19033510) −20084972 to −16870971 loss ≤ 0.05 × |baseline median| PASSED
allocs/op 159760 4234 −97.3% (−155525) −155527 to −155523 loss ≤ 0.05 × |baseline median| PASSED
tag_rows/op 10000 1001 −90% (−8999) −8999 to −8999 loss ≤ 0.05 × |baseline median| PASSED
db_ns/op 5230750 2193850 −58% (−3035672) −3445275 to −2447397 loss ≤ 0.05 × |baseline median| PASSED

Timeline