Controlled Discover deployment validation

The application source under test is e797d70c85d882f2437a75962215ee00525c62c8 (PR #187). This extends the earlier complete-deployment check to the new cohort middleware and per-process request admission. It is an isolated synthetic test, not a production rollout or a recommendation-quality experiment.

Protocol and current status

Passed: eight phases and 78 HTTP checks, with zero failed checks. Execution and cleanup took 420,426 ms on 2026-09-20. A separate read-only audit subsequently confirmed all three temporary service instances absent across project environments, the owned database and role absent, all four production feature/ readiness flags still off, and production Discovery Redis healthy and unconnected.

Two temporary Railway applications share a fresh test-only authentication secret, an owned Redis instance and a separate database/role on the testing PostgreSQL service. Instance A uses 100% cursor exposure. Instance B uses 0%, with only the two synthetic reader accounts allowlisted. The production application, its variables and its dedicated Discovery Redis are not modified.

The frozen protocol requires eight phases:

1. Actual production entrypoint, proxy, static HTML and built JavaScript assets.
2. Both runtimes remain healthy through the unchanged delayed startup work.
3. Actual stored authentication, private responses and CORS rejection.
4. Cross-instance cursor race/replay, ownership rejection and complete traversal of 713 eligible synthetic story families across 15 pages. Continuations on the 0% instance must drain successfully.
5. Fresh anonymous requests at 0% retain public legacy responses without a new guest cookie; an existing legacy offset visit stays legacy on the 100% instance; authenticated allowlist works at 0%; narrowed cursor scope rejects.
6. Actual client event production with device-clock skew, idempotent confirmed saves, and exclusion of saved story families including a translated sibling.
7. Cross-account cursor rejection.
8. Removal of the original app instance followed by equal retry and successful continuation on the surviving instance.

Every phase passed. Anonymous traversal returned exactly 713 distinct families over 15 pages with zero missing or duplicate families. After two saved-family exclusions, signed-in traversal returned exactly 711 families over 15 pages, again without omissions or duplicates. Concurrent cross-instance continuation and retry agreed. The 0% instance accepted existing guest cursors and allowlisted authenticated readers; its fresh anonymous response stayed legacy and public without minting a guest cookie. A legacy offset continuation stayed legacy on the 100% instance. Narrowed cursor scope and other actors' cursors were rejected.

The real client bundle produced accepted events with both +20 s and -20 s device clock skew. Repeated impression/save delivery reconciled to one canonical impression, click, confirmed save and identity link for the checked opportunity. After instance A was removed, B returned the same retry and another disjoint page. Both runtimes had remained healthy through at least 105 seconds of the unchanged startup timers before these mutable-feed checks began.

Each app has two CPUs, 2 GB memory, an eight-connection database pool and request limits of four total/two fresh requests per process. The role permits 24 database connections. Owned Redis has a 1 GB container, 128 MiB noeviction memory and authentication. These are test settings, not production capacity recommendations.

Parent traffic is capped at 160 requests and six minutes. The runtime guard caps each app at 400 application requests, 100 health requests and ten minutes alive; it denies external network calls before application startup. Measurement is on only inside this synthetic deployment to exercise canonical event receipts; experimental personalization and external analytics are off. No paid model calls are permitted. Twenty minutes is a stage-admission deadline: an admitted stage completes bounded operations before separate sequential cleanup. It is not a hard total-duration ceiling.

The runner uses one owning process and an exclusive mutation lock. It records creation/upload attempts before external effects, verifies exact deployment IDs, and never retries an uncertain upload automatically. Atomic ownership records survive failed replacement writes. Delayed or unresolved creation is reconciled by exact recorded name and run marker; repeated absence without a creation receipt cannot be certified as successful cleanup. The final result requires both all predeclared phases and confirmed removal of every owned service, database and role. Production credentials never enter the app images or test runtime.

Twenty-five local harness checks passed across the context builder, runtime guards and lifecycle helpers. Independent review found cleanup and diagnostic issues in the earlier harness; those were fixed before provisioning. The new regressions cover interrupted journal replacement, delayed creation visibility, exclusive mutation ownership, ambiguous upload rejection and credential redaction. Source CI 35520871859 passed, including actual Redis Lua regressions and the warehouse job.

Evidence identity

• Frozen protocol: ignored controlled-deployment-plan-v1.json, SHA-256 2c09b551badf0085def36a8bd1ec69c7056688652f7bb0c2273cfe882e62b90f.
• Docker context: 2,471 files / 71,768,876 bytes. Manifest SHA-256 aa24f487f50fd429eba86c87061da24c44655edb11a6f13bb16b4543ea9a1779. Only the test preload and its guard are added to the pinned production Dockerfile; application source and entrypoint are unchanged.
• Actual client beacon bundle SHA-256 3e28c75eca496d8f0d51430347c064cbf7cfd4338f27405449d750252bf22070.
• Run identity: 98c7df6209813232be10bab78724e6ec.
• HTTP result: ignored controlled-deployment-result-v1.json, SHA-256 cd955b5397b1c836761c7ed5ec2d4c4e2a6cddfef702361897201a67fb6d3cb7.
• Combined traffic/cleanup receipt: controlled-deployment-run-result-v1.json, SHA-256 5f8c72622a53b62dd4209be4d150d722d0681cd54ceea1d4e65c172f8c26ef91.
• Independent final audit, completed 2026-09-20 16:18 UTC: controlled-deployment-final-audit-result-v1.json, SHA-256 4e294b5cf19d2e3e460357b90bf74704f9ab1b130e4c01d6130bf3d2d166096e.

The first separate audit used an environment-scoped Railway inventory and could not see the production Redis it was checking. It stopped without a pass receipt. The corrected read-only query verifies the linked project's exact ID and both environment scopes before checking the complete inventory. This audit correction did not change the application, frozen deployment protocol or completed test run.

Limits

This does not test browser rendering, real-reader preference, concurrent production Neon capacity, a full production TTL cycle, or production pool interference. Percentage reduction is represented by differently configured instances; it is not a live variable redeploy or master-switch test. Actual Hono and client regression tests separately cover the master-switch restart contract.

The fresh database initially lacked the daily-recovery function, so its first scheduled invocation logged that absence. Runtime logs then confirm that the unchanged delayed installer installed the function and completed schema recovery. The test egress guard intentionally rejected a model-popularity HTTP refresh. Consequently the passing health checks do not mean every background invocation succeeded, or validate provider availability/background production workload.

Real ClickPipe CDC/archive/erasure receipts and a controlled production canary remain separate release gates. The earlier failed content-ranking evaluation is unchanged and that experimental ranking remains disabled.