Benchmarks

GoFastr ships a tiered set of Go benchmarks. Each tier maps to a category of claim or hot path; together they exercise the parts of the framework the README makes claims about, against both SQLite and Postgres.

Tiers

Running

make bench               # everything, BENCHTIME=1s BENCH_COUNT=3
make bench-tier1         # claims-defending end-to-end
make bench-tier2         # hot-path microbenchmarks
make bench-tier3         # concurrency & background
make bench-tier4         # startup & infra
make bench-tier5         # TechEmpower-style endpoints
make bench-tier6         # latency percentiles + concurrency
make bench-tier7         # stdlib baselines
make bench-tier8         # operational (cold start, heap, goroutines)
make bench-tier9         # UI runtime (streaming, SSE, islands, page render)

make bench-techempower   # alias for tier 5
make bench-overhead      # alias for tier 7 (framework vs hand-rolled)

make bench-sqlite        # everything, BENCH_SKIP_PG=1
make bench-pg            # only the /postgres/ sub-benchmarks
make bench-smoke         # every benchmark in the affected packages, once

Tunable via env:

• BENCHTIME: -benchtime value (default 1s).
• BENCH_COUNT: -count value (default 3). Pair with benchstat.
• BENCH_TIMEOUT: -timeout value (default 30m).
• BENCH_PKGS: which packages to scan (default ./framework/... ./core/router/... ./battery/search/...).

Output captures land under dist/bench/<scope>.txt so they survive across runs and can be compared with benchstat dist/bench/old.txt dist/bench/new.txt.

Postgres availability

Postgres benchmarks resolve a connection via the same path as the test suite:

1. TEST_POSTGRES_DSN env var (make postgres-up starts one via docker-compose and prints the DSN), or
2. skip: Postgres sub-benchmarks are silently skipped, not failed.

make bench-pg fails fast if neither path is available; make bench and the tier targets will just skip the Postgres half.

Use make bench-pg-evidence for CI or release notes. It runs the Postgres-sensitive witnesses (StreamingVsBuffered_RealVolume, SchemaDiff, and idempotent AutoMigrate), writes dist/bench/postgres-evidence.txt, and fails if no /postgres/ sub-benchmark actually ran.

Reading the output

Standard Go benchmark line:

BenchmarkTier1_IncludesVsN1/sqlite3/eager-include/limit=100-14  607  1834105 ns/op  ...
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                            sub-benchmark path: dialect / variant / size

For benchmarks that report custom metrics (b.ReportMetric), look for named columns like drop_rate, delivered, dropped, response_bytes.

Tier 1: what to look for

These exist to either back up the README or expose a hole.

1.1 Includes vs N+1

Eager (?include=author,comments) must outscale naive per-row fetches. The win factor is ≈ N * round-trip cost; on Postgres the gap should be 1–2 orders of magnitude.

1.2 Cursor pagination at depth

offset/page=180 should degrade vs offset/page=1 as table size grows. cursor/page=180 should be flat: same cost as page 1. If cursor degrades, the keyset query lost its index or the cursor field is wrong.

1.3 Batch vs N

50-item batch should beat 50 individual POSTs by both per-tx overhead and per-request middleware overhead. The gap is widest on Postgres where the network round-trips dominate.

1.4 Streaming vs buffered

Known caveat: parsePagination clamps ?limit= to ≤100, so the workload streaming is meant for (limit ≥ streamListThreshold = 1000) can't be reached over HTTP. The benchmark therefore exercises streaming at limit=100, which measures the per-row encode/write overhead but does not show the bounded-memory advantage. Worth fixing.

Tier 2: what to look for

• Router lookup: should be near-flat across N=1..1000 routes (tree match, not linear scan). Compare against http.ServeMux.
• Middleware chain: measure the cost of Recovery → RequestID → Logging → SecurityHeaders → Timeout. The Logging middleware writes stdout; benchmarks that don't redirect it will see I/O overhead.
• JSON casing: every request runs mapToCamelCase once (write paths run mapToSnakeCase too). 4 allocations per 10-key row is the 2026-10-18 baseline.
• DSL parse: agents may invoke this per prompt. Should stay well under 100µs for a complex query.

Tier 3: what to look for

• Event bus fan-out: synchronous is dominated by handler slice copy + handler calls; per-emit allocations should scale linearly with N (the snapshot copy).
• SSE backpressure: a bounded subscriber buffer paired with a slow consumer should drop the oldest surplus in default mode; the drop_rate metric records this. ?slow=block is the opt-in stronger-delivery mode and intentionally trades emitter latency for delivery.
• Cron tick: scanning N jobs at minute boundaries. ~175µs for N=1000 baseline; budget for in-process schedulers running many jobs.

Tier 4: what to look for

• AutoMigrate re-run: the "safe to run on every boot" claim. Should stay under ~10ms even at 50 entities. Postgres is much slower than SQLite because the live-schema read (one bulk information_schema.columns query feeding both existence and column-add detection) is a network round-trip.
• Schema diff: same shape, but pays for the full information_schema lookup. Acceptable as a one-shot CLI command, not as a hot path.
• In-memory search: confirms O(corpus) scan. 10k docs ≈ 3ms per query is fine for tests/demos; production needs a real backend.

Tier 5: TechEmpower-style endpoints

The six canonical comparable workloads. Numbers here can be cross- referenced with the published TechEmpower Framework Benchmarks (techempower.com/benchmarks) to see roughly where GoFastr sits next to Gin/Echo/Fiber/Actix/etc.

Throughput in req/s = 1e9 / ns_per_op. Compare with caution: the TechEmpower harness uses real network listeners + multiple client connections, while these run through httptest's in-memory ResponseRecorder, which is faster than the wire. The relative shape (Plaintext > JSON > SingleQuery > MultiQuery > Updates) holds; absolute numbers don't translate one-to-one.

Tier 6: latency percentiles + concurrency

BenchmarkT6_* benchmarks record per-operation latencies and emit p50_ns, p90_ns, p99_ns, p999_ns, and max_ns via b.ReportMetric. Mean ns/op hides the tail; the percentiles surface it.

Parallelism is exercised via b.RunParallel + b.SetParallelism(N), where N is the multiplier over GOMAXPROCS. So parallelism=8 on an 8-core machine gives 64 worker goroutines.

What to look for:

• p99 ÷ p50 ratio: under load, anything above 3× means significant tail growth. SQLite write-heavy workloads will hit this immediately because writes serialise.
• List concurrency slope: read-only list endpoints should scale near-linearly on Postgres (no write lock) but stay flat (or get worse) on SQLite (single connection).
• Mixed RW reads vs writes ratio: should match the 9:1 mix the benchmark drives. If reads drastically outnumber that, writes are starving on the lock.

Tier 7: stdlib baselines (the overhead tax)

Hand-rolled net/http + database/sql implementations of plaintext, JSON, single-query, and filtered list endpoints, paired with the same endpoints expressed through the framework. The delta is what the declare-once style actually costs.

What to look for (2026-10-18, SQLite, darwin/arm64):

• Plaintext and JSON go through the router and not much else. Both run within 10% of http.ServeMux (Plaintext 445 ns against 410 ns, JSON 542 ns against 494 ns, 10 allocs on each side). A bigger gap points at router work on the request path.
• SingleQuery exposes the entity and CRUD machinery: filter parsing, case conversion, projection, the access check. It runs about 30% above the hand-rolled handler (7.6 µs against 5.8 µs).
• FilteredList is the widest path. The hand-rolled baseline runs the same COUNT query, selects the same eight columns and writes the same data/total/page/perPage/totalPages envelope as the generated list, so the pair compares equal work. The framework side allocates less (2,381 allocs against 3,027) and runs about 4% slower (167 µs against 161 µs).

Benchmarks measure the production posture

The fixtures build requests the way a deployed app sees them:

• benchRequest (in bench_helpers_test.go) creates a request whose context switches off the two wrappers a test binary arms for every response: the retired-markup scan (internal/retired.ProductionContext) and the render diagnostics (internal/renderdiag.ProductionContext). Production never runs either, so a benchmark that keeps them on times dev instrumentation.
• benchAuthed does the same and stamps a user into the context with handler.SetUser, the shape production auth middleware produces. Generated CRUD requires a session by default, so without it the setupBlogDomain entities answer 401 before the handler body runs.
• benchMountCrud mounts CRUD routes through App.CrudHandler, which wires the registry, JSON casing, hooks and events. A bare NewCrudHandler has no registry, so ?include= answers 400.

The *_NetHTTP baselines keep httptest.NewRequest: they never pass through either wrapper.

Every benchmark runs once in CI

go test never runs benchmarks, so one that breaks fails nobody's build. make bench-smoke (scripts/bench-smoke.sh) runs every benchmark in the affected packages once (-benchtime=1x, BENCH_SKIP_PG=1), and CI runs it after the test step. It proves each benchmark still runs; it measures nothing.

Tier 9: UI runtime streams, islands, and full SSR

The framework does more than JSON CRUD: it also has an SSR-with-hydration runtime, SSE/island plumbing, and a UI host. These benchmarks measure those paths.

What to look for

• Streaming wins over buffered-paginated at any volume that pays network round-trips. SQLite shows a small win (12.2 ms vs 13.4 ms on 2026-10-18); Postgres showed 12ms vs 46ms, a 3.9× win, because one query beats 50 round-trips.
• SSE delivery_ratio ≪ 1.0 under bursty emit. Default subscribers drop oldest events on overflow; this is documented behaviour. A regression means the ratio drops further at a fixed emit rate.
• Island RPC should stay sub-100µs for a small (~10-row) fragment. This is the response-time floor for "click → see new content."
• UIHostPageRender vs BenchmarkT7_JSON_GoFastr (~500ns) tells you what SSR + hydration shell adds over a bare framework JSON response. The 15–16 KB witness pages take about 140–160 µs (2026-10-18) for the HTML tree build and runtime script injection.

Caveats

• httptest.ResponseRecorder removes wire latency and flushing semantics. SSE numbers are lower bounds on encoding cost, not RPS over a real network.
• Real island deployments include client-side hydration time that these benchmarks don't measure (Go can't time JS). Wire that in via Playwright if it matters.

Resource benchmarks (separate from the Tier 1-9 Go bench suite)

Resource numbers cover binary size, peak RAM during go build, and idle and under-load RAM of each running binary. They are produced by a separate runner under cmd/bench-resources/ rather than go test. They aren't expressed as Go benchmarks because the unit of measurement is "one fully-built and warmed binary", not "one operation".

make bench-resources             # default LOAD=200 requests per app
LOAD=500 make bench-resources    # override

Three bench apps under benchmarks/apps/<name>/:

Plus the two cmd binaries (gofastr, kiln) for build-only comparison. The entity-backed apps refuse invalid entity declarations at startup, before migration or serving, so a broken setup cannot yield measurements.

Output is Markdown to stdout and dist/bench/resources.md:

Measured 2026-10-18 on darwin/arm64 with Go 1.27.2 (v0.90.0):

The gofastr CLI builds to 52.2 MB and kiln to 31.2 MB.

What to look for

• Bin size delta crud − minimal is mostly the SQLite driver, the pure-Go modernc.org/sqlite (no cgo), plus the entity, CRUD and migration packages.
• Bin size delta full − crud is what every other framework feature costs at once: UI host, file storage, search backend, audit, cron, MCP, access control, multi-tenant, OpenAPI/Swagger, plugins. About +1.1 MB total. They're code paths inside the framework and its sibling packages, not separate binaries' worth of code.
• Build peak RAM is mostly compiling the SQLite driver, a large machine-translated Go package. CI machines need to budget ~1 GB headroom for the compile step.
• Idle vs Loaded RAM should be roughly equal after a warmup. A significant climb under load means GC pressure, retained slices, or goroutine leaks. Pair with Tier 8's HeapAfterLoad.

Dev-server RAM

gofastr dev runs the app plus a watcher that rebuilds on change. On examples/meridian the whole process tree idles at about 95 MB (the dev process itself at 35 MB), and a rebuild after a real edit peaks at about 256 MB with a warm build cache. A cold first build pays the full build peak from the table above.

Tier 8: operational

• ColdStart_*: time from NewApp through first request served. Reported as ns/op; with benchtime=1x that's effectively the cold- start latency for a single binary instance.
• HeapAfterLoad: drives 5000 list requests, then reads runtime.MemStats. Reports heap_alloc_bytes (live heap), heap_objects, mallocs, frees, gc_pause_total_ms, gc_cycles. A regression in live heap means we're retaining state across requests; a regression in pause time means GC pressure grew.
• GoroutinesAfterLoad: sanity check for goroutine leaks. gor_delta should always be 0 (or very small).

The Tier 8 benchmarks deliberately use -benchtime=1x because they're property assertions, not iteration timings. Reading per-run values is the point.

Adding a benchmark

Co-locate with the code being measured: bench_*_test.go in the same package. Use forEachBenchDialect if it needs a DB. Skip not fail when external infrastructure is missing.

Every benchmark should:

1. Set up state outside the timed loop (b.ResetTimer() after).
2. Report allocations with b.ReportAllocs() when allocation cost is relevant.
3. Report custom dimensions with b.ReportMetric(...) when raw ns/op isn't the whole story (drop rates, body sizes, etc.).
4. Have stable, deterministic input so runs are comparable.

Comparing runs

make bench BENCH_COUNT=10 BENCHTIME=1s
mv dist/bench/all.txt dist/bench/before.txt

# … make a change …

make bench BENCH_COUNT=10 BENCHTIME=1s
benchstat dist/bench/before.txt dist/bench/all.txt

benchstat (go install golang.org/x/perf/cmd/benchstat@latest) reports geometric mean + p-value so noise doesn't show up as a regression.

See also

• UI capability map separates composed capability from packaged convenience and measured performance.
• Runtime contract defines the UI paths being measured.
• Events and SSE documents delivery semantics behind drop-rate metrics.

Common mistakes

• Not skipping Postgres when it's unavailable. Use forEachBenchDialect, which calls b.Skip correctly. Don't fail.
• Allocating inside the timed loop. Build payloads before b.ResetTimer().
• Benchmarking through stdout-writing middleware. Some default middleware writes a request log per call; either redirect output or use WithoutDefaultMiddleware() for hot-path measurements.
• Trusting a single run. Use BENCH_COUNT=10 + benchstat for any comparison that matters.

Improvement opportunities

The prioritized list of optimizations these benchmarks surfaced, together with the specific bench that exposed each one, a verify step, and the verification results, lives in perf-results.md. Update it whenever a new hotspot appears or a fix lands.