variant.Marshal builds a throwaway Value tree then re-serializes it
perfloop/parquet-go · REDUNDANT SERIALIZATION
https://perfloop.ai/t/oss/case_krz077z3t7
Verdict
VERIFIED · settled 2026-07-02 · merged as parquet-go/parquet-go#550
What happened: The candidate successfully modernized the codebase by converting benchmark loops in `variant/benchmark_test.go` to use `b.Loop()`, removing `b.ResetTimer()`, and converting simple integer loops in `variant/encoding.go` to the range-over-int form (`range n`). This completely resolves the format CI failure on the upstream PR. Crucially, all performance wins are preserved: allocations are held at 18 allocs/op (99.1% reduction from 2006) and execution speed remains highly optimized.
Hypothesis
On the Parquet Write Workload, variant.Marshal serializes in two passes: goToVariantReflect first materializes the whole structure as heap Value nodes (one allocation per scalar, array element, and object field), then Value.Encode re-walks that tree appending to a freshly-grown buffer per node. Every element is allocated and visited twice purely to be discarded. Encoding straight from the reflect walk into one reused buffer is a measured 1006->12 allocs/op (84x) with ns/op -76% on a []int64 of length 1000; the in-situ marshal is 86-88% of the full variant-column write, so the end-to-end write improves -20..-27% on object-heavy and -66..-72% on array-heavy data.
Change to test: Encode directly from the reflect.Value into one reused output buffer, deleting the intermediate Value node tree entirely so each element is visited and emitted exactly once. Make this the single encoder: remove goToVariant/goToVariantReflect and route the exported Encode through the same encoder rather than leaving a second divergent path, and use the package's existing makeHeader plus Primitive*/Basic* header constants instead of hardcoded type-header bytes.
Where it lives
perfloop/parquet-go · parquet.go