1,000 players per map

Two separate problems hide behind the 1,000 number. They have separate budgets and separate tests:

1. Server + network: simulate 1,000 players + ~4,000 monsters, and send each client only what matters.
2. Client rendering: a phone can't draw 1,000 armoured, animated characters, so it draws the right ones and fakes the rest.

◆Server tick (20 Hz = 50 ms budget, target p99 ≤ 15 ms)

Data layout: struct-of-arrays per component (positions, velocities, hp…), generational entity ids, no heap allocation in the tick (pre-sized buffers). rayon parallelises per-client interest work and per-cell AI.

The numbers the zone server reads (tick budget and targets, the spatial-hash grid, interest radii and set limits, channel caps, the scheduler's catch-up cap) live in game data, packages/game-data/data/netcode.json; this page explains them. services/zone (M1-03) runs the tick on fixed absolute deadlines: after a stall it catches up at most 4 ticks back to back and drops older backlog (proposed). Measured on one thread: logs/evidence/M1-03/tick-bench.json.

◆Interest management (what each client receives)

• Radius: candidates within 90 m (towns 70 m). Beyond that only party members (minimap) and boss/event beacons.
• Sets (proposed, M1-03): an entity enters a client's interest set inside the radius and leaves only beyond radius + 5 m (hysteresis, so walking along the edge never flickers). A set holds at most 512 entities (the 9-bit AOI table index); when more qualify, the nearest are kept. Each tick reports the enters and leaves per client.
• Priority score per entity per client, accumulated every tick: priority += base(type) × falloff(distance) × relevance. Relevance is ×8 for my target, ×6 for anyone attacking me, ×4 for party members, ×10 for bosses and ×3 for monsters casting a telegraph. The top entities that fit the packet budget are sent and their accumulators reset.
• Rates: near (≤ 30 m) effectively 20 Hz, mid (30–60 m) ~10 Hz, far (60–90 m) ~4 Hz.
• Events (cast start, hit, death, telegraph) are reliable and sent to everyone in range, independent of the movement priority.

◆Bandwidth math (downstream per client)

Hard cap per client: 32 KB/s. Upstream: 20 input packets/s × ~14 B ≈ 0.3 KB/s.

Built (M1-03B): the wire format is packages/protocol/zoen.schema.json (codec crates/zoen-protocol); bit widths, rates and the proposed values (16 mm position quantum, 10-bit HP permille, 32-frame baseline history) are in netcode.json → snapshot. Each client gets the nearest 150 of its interest set, delta-encoded against the newest frame it acked (a full snapshot when that is older than the history); leaves always go, enters and due changes share the 1,600 B packet. Measured: logs/evidence/M1-03B/snapshot-bench.json (pnpm zone:snapbench). At 1,000 clients the server sends about 10–20 MB/s, which fits on one 1 Gbit NIC with plenty of headroom.

◆Transport and prediction

• WebSocket (wss), binary frames, with Nagle disabled. WebTransport datagrams are a later experiment, kept only if they measure better on Safari/Chrome.
• Built (M1-03C): pnpm zone:run -- [--port P] [--bind ADDR] [--seconds S] [--quiet] starts the zone binary (services/zone, src/net/): plain ws:// on loopback port 7310 by default (netcode.json → server, proposed; TLS comes from the gateway later), RFC 6455 written on std (no crates), session and close codes in packages/protocol/zoen.schema.json. Per connection a reader thread (handshake, frame decode, idle timeout) and a writer thread; one bounded queue into the tick and a per-client pool of frame buffers out of it, so the tick never waits on a socket. Clients spawn around the start waypoint's zone centre. Rust test client: zoen_zone::net::client::Client; end-to-end tests in services/zone/tests/server.rs.
• Load check (M1-03D): pnpm zone:loadcheck -- [--clients N] [--seconds S] [--out PATH] [--quiet] joins 200 headless clients to an in-process server for 10 s (random walk, one input per tick, an ack per snapshot, a ping per second) and fails on a loop p99 over the target, downstream over the 32 KB/s hard cap, stale or dropped inputs, more than 1% skipped frames or dropped ticks, or an unclean close. It also reads the tick thread's CPU time next to its wall time, so waiting for a core on a busy machine shows apart from slow code. Measured: logs/evidence/M1-03D/loadcheck.json.
• Snapshot pool (M1-04B): per-client snapshots are built and handed off by server.snapshotWorkers threads (4, proposed; the tick thread runs shard 0). Clients sit in stable shards (slot mod workers); the workers read the tick's state, the tick waits for all of them, and the bytes equal the single-threaded path (services/zone/tests/snapshot_pool.rs). Std threads, no rayon: its scoped jobs allocate per tick. 1,000 bots, 80 s: loop p99 12.1 ms (logs/evidence/M1-04B/).
• Bots (M1-04): pnpm zone:bots -- [--bots N] [--ramp-s R | --steps 100,250,500,1000 [--step-s S]] [--hold-s H] [--workers W] [--snapshot-workers S] [--addr HOST:PORT] [--out PATH] runs the load-test plan below against an in-process server (or a running zone with --addr): bots on poll(2) workers, behaviour mix and pass thresholds from netcode.json → loadTest, loop phase profile in the metrics JSON. --steps is the stepped ramp (each step's new bots connect over the first half of its S s window, the second half holds; default S = loadTest.rampMinutes / steps). The metrics carry per-second series (connected bots, bytes per bot down/up, RTT p50/p99, process RSS; server loop, CPU and phases per 1 s window), the load average each minute and the memory drift over the hold (least-squares RSS slope × hold / mean RSS, limit loadTest.memoryDriftPercent). python3 tools/zone-soak-report.py <metrics.json> --png <chart.png> --summary <summary.txt> draws the dashboard PNG and the pass/fail summary.
• Own movement is predicted with the client's C# prediction subset, which must match zoen-sim on the shared golden vectors (generated from the Rust crate). Inputs carry sequence numbers; on each snapshot the client rewinds and replays unacknowledged inputs. Errors under 0.05 m are ignored, under 2 m are blended over 100 ms, and larger errors snap.
• Others are interpolated 100 ms behind (2 snapshots) and extrapolate up to 150 ms on loss.
• Skills: windup presentation is predicted; damage never is. The attacker's contact sparks may play predicted, but numbers wait for the server.
• Lag compensation (PvE): for player attacks the server checks monster positions rewound by RTT/2, capped at 150 ms, which favours the attacker. Monster telegraphs lock at cast start so dodges are fair at up to 200 ms RTT.

◆Channels

• Soft cap 900 (new players are routed elsewhere), hard cap 1,000. When every channel passes 85%, admission opens a new one.
• Parties join the leader's channel when space allows. Players switch channel at a waypoint, out of combat, with a 10 s cooldown.
• Bosses and events are per channel. Kharuun spawns in every channel at the scheduled time.

◆Crowd rendering on the client

max_visible_players (25/40/60/100/150) caps A+B+C. Nameplates are capped at 40, nearest first. Particle budgets split own 50% / party 20% / others 15% / monsters + telegraphs 15% (telegraphs are never culled).

◆Load-test plan (M1 and M6 gates)

1. Bots (services/bots, Rust): 1,000 WebSocket clients with scripted behaviours (wander, hunt, crowd the town, cast every 2–6 s, chat).
2. Ramps: 100 → 250 → 500 → 1,000 over 10 minutes, then hold 10 min (M1) / 60 min (M6).
3. Pass: tick p99 ≤ 15 ms, no disconnects, avg downstream ≤ 12 KB/s/client, crowded-town clients ≤ 32 KB/s, memory flat (± 5%).
4. Mixed: 1,000 bots + real browser clients on desktop + a phone. Record the client perf and a 10 s clip in town.
5. Evidence goes to logs/evidence/M1-04/metrics.json and the histograms as PNG (the M1 soak: logs/evidence/M1-04C/).