Horizontal scaling

One replica of a GoFastr app is self-contained: sessions, rate limits, cron, queues, and live UI updates all work with no extra setup because their default state lives in the process. The moment you run a second replica behind a load balancer, every one of those defaults needs a shared backend or a deliberate single-runner strategy. This page lists what breaks, why, and the replica-safe alternative for each one.

Summary

The database is the only state the replicas share. Anything that defaults to process memory must either move into the DB (or another shared store) or run on exactly one replica.

What is process-local by default

Auth enforces the first three at boot in production mode (DevMode: false): the in-memory session store, the in-memory 2FA store and a link plugin with no TokenStore refuse to boot; a warn-only start lets a broken multi-replica deployment go unnoticed. Setting AuthConfig.AllowInMemoryStores: true acknowledges a deliberate single-node deployment: the stores then boot, and the 2FA store still leaves a WARN trace.

What is already replica-safe

• Migrations: auto-migrate takes a Postgres advisory lock, so N replicas booting simultaneously run the migration once. On SQLite the same serialization comes from a leased lock row (_gofastr_migrate_lock), the twin of the seed lease below; the versioned runner additionally re-checks the tracking table before applying each migration, so a replica whose read raced a peer converges instead of failing its boot.
• Startup seeds: RunSeeds and WithSeed hooks acquire a DISTINCT Postgres advisory lock (separate from migrations) so N booting replicas never race a seed func. Combined with the _gofastr_seeded ledger, an entity's Seed runs once globally; the other replicas see the ledger row on their locked turn and skip. A crashed lock holder's session-level lock is released by Postgres automatically; no permanent block. (WithSeed hooks have no ledger, so they serialize-per-boot but still run on every replica; keep them idempotent.) Exception: a Postgres pool capped at MaxOpenConns(1) cannot hold the advisory lock (it would deadlock the seed body), so it skips the lock with a WARN and N such replicas are NOT coordinated; keep the pool above 1 connection for multi-replica seed serialization.
• queue.DBQueue: claims jobs with FOR UPDATE SKIP LOCKED; competing workers on every replica are the intended topology.
• Webhook delivery (LeasedStore): leases deliveries so two replicas don't double-send.
• Plain CRUD/API traffic: stateless per request; scale freely.

Serve/worker roles

The first scaling step for a self-hosted app is one web process + one worker process, same binary, before replicas and before Redis. The role is picked at deploy time:

app.Start(":8080") // role from GOFASTR_ROLE: all | serve | worker | agent

GOFASTR_ROLE=serve  ./myapp   # full router; no cron/queue/outbox-relay
GOFASTR_ROLE=worker ./myapp   # cron/queue/outbox-relay; /healthz + /readyz only
GOFASTR_ROLE=agent  ./myapp   # /mcp + /a2a (if WithA2A) + health; no entity routes
./myapp                       # combined (default); today's behavior

GOFASTR_ROLE=agent (or framework.WithRole(framework.RoleAgent)) is the agent surface split: it serves the /mcp mount, the A2A exchange (/a2a, when WithA2A is configured), and health endpoints and nothing else — no entity CRUD, no OpenAPI, no browser pages. /mcp and /a2a requests forward to the app router, so session/bearer auth and owner scoping behave exactly as on a serve process (see MCP and Auth). Use it when agent traffic arrives through a dedicated tunnel or allow-listed listener and you don't want that listener to reach the rest of the app. The full contract for what the agent surface serves — including the MCP Apps widget traffic that rides it — is in agent host.

framework.WithRole(framework.RoleServe) overrides the env var; an invalid value in either fails at construction. The worker's health endpoints are the same handlers the full router serves, so LB and orchestrator probes work unchanged. Everything else runs in both roles: auto-migrate, seeds, plugins, batteries (migrations hold a lock, so either process type may boot first). Plain OnStart hooks are role-agnostic; gate custom background work on app.Role().

Recommended shapes

Two web replicas + one worker. Point sessions at EntitySessionStore, run the web replicas with GOFASTR_ROLE=serve, and one GOFASTR_ROLE=worker process that owns the cron scheduler, the DBQueue workers, and the outbox relay. This is the smallest shape with no shared-state caveats. Add WithFanout if the web replicas use SSE push.

Everything everywhere, DB-backed. All replicas run DBQueue workers (safe by design). For scheduled work, have the schedule enqueue a DBQueue job instead of doing the work inline; then it doesn't matter that every replica's scheduler fires, as long as the job is idempotent or keyed for dedup. Neither scheduler ships a distributed lock; the queue's claim semantics are the sanctioned coordination point.

Single node, on purpose. Vertical scaling is underrated. Set AuthConfig.AllowInMemoryStores: true to let the in-memory stores boot and skip this whole page until you add a replica. A restart still logs everyone out and wipes in-memory 2FA enrollment; use the entity-backed stores anyway if either matters.

SSE across replicas

Server-pushed events flow over an SSE connection to the replica the browser happened to reach. A write handled by a different replica emits on its bus and pushes to its island manager, not the one holding the connection. Two answers, in order of preference:

1. Shared fan-out: framework.WithFanout bridges the real-time lane across replicas. framework/fanout.NewPostgres(dsn, db) uses Postgres LISTEN/NOTIFY (no new infrastructure); core/fanout.NewRedis adapts a Redis client you bring. Entity _events SSE streams and island push then work from any replica. Read the "Cross-replica fan-out" section of the events doc first: with a fanout attached, On/Subscribe handlers fire on every replica, so side-effect work must move to outbox consumers and derived emits must gate on event.IsRemote(ctx).
2. Poll instead. For passive freshness such as a dashboard, a counter, or a status pill, data-cui-poll re-fetches on an interval from any replica and needs no fanout at all. Reserve SSE push for semantics that need the connection: presence, collaborative editing, sub-second updates. See Reactivity model for the full ladder.

Sessions

The uihost session is an HMAC-signed token, not a server-side record. A token issued by one replica verifies on any other replica that shares the signing secret, so any replica can serve any request.

• Set the secret in production: framework.WithSecret(secret) in code, or the GOFASTR_SECRET environment variable. Either lands the same key on every replica.
• One replica, no secret configured. The framework mints an ephemeral boot secret at startup. Sessions work, but they roll over on every restart because the next boot mints a new one. Fine for single-node; wrong the moment a second replica is on the table.
• Fanout without a secret fails at boot. A multi-replica deploy that wired WithFanout but forgot GOFASTR_SECRET (or WithSecret) refuses to start, with an error naming both. This is deliberate: silent token mismatch in production is worse than a loud boot failure.

Sticky sessions are not part of the contract. A token is portable; route the request to whichever replica the load balancer picks.

Checklist before adding the second replica

• [ ] Sessions on EntitySessionStore (or another shared SessionStore).
• [ ] 2FA store durable (if the 2FA plugin is enabled).
• [ ] Cron scheduler runs on exactly one process (GOFASTR_ROLE=worker), or jobs moved to DBQueue.
• [ ] Queue is DBQueue, not the in-memory variant.
• [ ] Rate limits on a shared store (RateLimiterConfig.Store: auth.NewSQLRateLimitStore(db, "auth_rate_limits")), or enforced at the ingress.
• [ ] SSE push crosses replicas: WithFanout attached (and side-effect handlers moved to outbox consumers), or use data-cui-poll for passive freshness that needs no fanout.
• [ ] GOFASTR_SECRET set (or framework.WithSecret in code) so the HMAC-signed uihost session token verifies on every replica. Required with WithFanout; the app refuses to boot otherwise.
• [ ] Runtime RBAC grants propagate: WithGrantStore attached when access.GrantStore is in use, so grant/revoke reaches every replica's RolePolicy without a restart. A revoke persists a revocation tombstone (access_grants_revoked), so it propagates to peers and survives replica restarts even for grants declared in code; re-granting lifts the tombstone.
• [ ] Cache backend shared (Redis) if cached data must be coherent across replicas.
• [ ] AuthConfig.AllowInMemoryStores removed; the boot warning is your regression test for the first two items.

See also

• UI capability map contrasts reconstructable and affinity-bound islands by product job.
• Events and SSE defines cross-replica fanout semantics.
• Presence documents lossy, self-healing roster aggregation.

Common mistakes

• Scaling to two replicas with default sessions. Users get randomly logged out depending on which replica the LB picks. The boot WARN about the in-memory session store is telling you this will happen; don't silence it with AllowInMemoryStores while running N > 1.
• Setting AllowInMemoryStores: true "to clean up the logs" and then scaling later. The flag is an assertion about your topology, not a log filter; remove it the moment a second replica is on the table.
• Running framework/cron on every replica because each job "checks if it already ran" against the DB. That check is a race, not a lock. Move the work to DBQueue or pin the scheduler to one process.
• Relying on SSE push without fanout. Everything appears to work in staging (one replica) and half of live updates silently vanish in production. Wire WithFanout and set GOFASTR_SECRET before adding the second replica, or pick polling for surfaces that don't need push.
• Treating the in-process login rate limit as a security boundary at N replicas. Without a shared store the budget multiplies by replica count and blocks don't propagate. Set RateLimiterConfig.Store (one SQLRateLimitStore can back every auth limiter; keys are namespaced per limiter scope) or enforce hard limits at the ingress.

See Deployment for the single-replica production checklist and Job queue for DBQueue worker sizing.