Discover traffic sizing and controlled rollout

The observed traffic does not justify enabling the new feed for everyone with the default 1,000 stored visits. The dedicated Redis has a 512-MiB memory limit, while 1,000 copies of the tested 1,245,766-byte raw session would occupy about 1.25 GB in payloads alone. That fixture is not a maximum production session size. Stored visits also persist until their absolute one-hour expiry after a reader leaves or finishes a stream. A small stored-visit cap alone would turn excess new readers away; it is not a substitute for selecting a canary cohort.

Observed traffic

Read-only queries used PostHog project 318450, verified against the configured application project. The window is September 13–19, 2026 UTC, seven complete days. Origin queries select hub_serve, environment=production and surface=recommended. Browser queries additionally select $host=yumina.io. Only aggregate counts were returned; no account IDs, browsing histories, story text, raw events, session IDs or request IDs were exported.

Measurement	Result
Recorded Recommended origin responses	117,297
Average origin responses per day	16,757
Responses at offset zero	92,532
Later-page responses	24,765
Origin responses with an empty card array	424
Origin responses marked application-cache hits	27,924
Maximum responses in one second / one minute	7 / 47
Maximum first-page responses in a minute-aligned 60-minute window	880
Maximum responses in a minute-aligned 60-minute window	1,117
Browser-visible Recommended card impressions	735,932
SDK sessions with at least one such impression	18,697
Distinct SDK-session/request-ID pairs across those impressions	55,047
Maximum SDK sessions in a fixed UTC hour	196
Maximum SDK-session/request-ID pairs in a fixed UTC hour	527

Anonymous origin responses account for 33,233 of the total; authenticated origin responses account for 84,064. Daily origin response totals range from 15,759 to 18,309. The busiest first-page rolling window ends at 16:40 UTC on September 13. The maximum over 61 adjacent minute buckets is 895 first pages, a conservative bound for arbitrary 60-minute origin windows wholly inside the queried period.

Daily and hourly origin totals reconcile, all 168 hours are present, and browser hourly totals reconcile with the separate whole-window coverage query. The browser events in this window have no missing session/request identifiers. Hourly maxima use a separate unsplit query: summing anonymous and identified SDK-session counts can double-count a session that crosses login.

These are workload proxies, not cursor allocation counts. hub_serve omits aborted responses and Cloudflare cache hits; the environment label alone does not prove hostname or exclude every internal request. The Recommended surface also includes narrowed requests, so it is broader than clean cursor eligibility. Browser impressions omit unseen/prefetched cards and clients whose telemetry is unavailable. An SDK session is not a signed-in reader or a fixed-one-hour Discover visit. Empty responses alone do not establish premature exhaustion: filters, eligibility, offset and true supply also matter. The observations establish neither a future traffic ceiling nor recommendation quality.

The new cursor path bypasses the legacy feed-page cache and CDN caching and uses the primary database for current eligibility. Consequently, sizing only against the existing origin cache-miss rate would understate the changed work. server_request is also unsuitable for a naive latency percentile: the current emitter captures every response slower than one second but only 1% of the rest.

Primary database preflight

Because cursor requests use the primary, a separately frozen probe repeated the existing captured query shapes against the verified production primary in a read-only transaction. It used one connection, a two-second statement timeout, 250-ms lock timeout, a 30-second total bound, four guest recall executions and nine signed-in exclusion executions. All 13 completed; only plan counters were returned. Account identities used for the Library-size selection stayed in memory.

Guest database execution times were 21.794 ms EN, 21.001 ms ZH, 21.770 ms ES and 21.065 ms JA. Signed-in safe-English observations were:

Library entries	Newest recall, DB ms	Popular recall, DB ms	Catalog, DB ms
1	14.074	11.839	11.823
14	13.893	13.341	12.887
428	52.164	59.704	46.037

These fixed-order single observations do not establish primary capacity or a speedup over the earlier replica observations. They omit application checkout, full profile loading/ranking, concurrency and HTTP work. The exact query-builder hashes still matched the original captures. The host wall clock remains roughly 21 seconds ahead of database time; durations use the monotonic clock or DB plan timers, and no host-clock adjustment was made.

Required serving controls

The controlled-rollout change separates three limits:

1. Exposure: deterministic server-side actor cohorts, default zero percent, with an authenticated operator allowlist. Legacy offset continuations remain legacy when exposure grows. Existing owned cursors drain when exposure shrinks; the master switch returns an explicit restart when turned off.
2. Work in flight: fail-fast per-process limits applied before profile/catalog work. Fresh requests have a separate lower cap so they cannot occupy every continuation slot. Release occurs when the handler settles, never merely on a client abort or timer. Parallel read failures must also wait for launched sibling work to settle before releasing admission. These limits multiply by every simultaneously serving process, including rolling-deploy overlap.
3. Stored visits: existing atomic Redis global/per-actor admission and resumable-visit recovery. These protect retained state, not query concurrency.

Selected cursor responses and errors must remain private and non-cacheable. Master-off fresh requests and zero-percent excluded guests preserve legacy caching and do not receive a new cohort cookie. A nonzero guest cohort also requires private responses for excluded cohort-assigned guests to prevent cross-cohort CDN reuse; that changes origin demand beyond the selected group. Measure that broader cache effect before a guest percentage canary. Initial operator allowlisting at zero percent avoids this change for anonymous traffic. Legacy offset continuations keep their existing transport and cache policy. Existing cursor ownership, scope and sequence checks remain authoritative; providing a cursor never grants access to another reader's stream.

Proposed restricted canary

These are starting bounds for a measured canary, not proven production capacity. No production application flag, endpoint or cap has been changed.

Setting	Initial value
DISCOVERY_CURSOR_ROLLOUT_PERCENT	0 plus authenticated operator allowlist
DISCOVERY_MAX_ACTIVE_SESSIONS	16
DISCOVERY_MAX_ACTIVE_SESSIONS_PER_USER	2
DISCOVERY_MAX_ACTIVE_SESSIONS_PER_GUEST	2
DISCOVERY_MAX_INFLIGHT_REQUESTS_PER_PROCESS	4
DISCOVERY_MAX_INFLIGHT_FRESH_REQUESTS_PER_PROCESS	2
Session encoding	raw

Use the dedicated Redis endpoint. Keep replicas fixed and account for all of them: two serving processes at these settings allow up to eight total/four fresh requests in flight, but two old plus two replacement processes permit sixteen/ eight during a rolling deploy. Count workers as well as replicas. The guard bounds awaited feed work, not unrelated traffic or background logging/cache operations. Preserve existing production DB pool settings until measured; the eight-connection synthetic test pool is not a production recommendation. Keep experimental personalization, measurement and analytics readiness disabled.

Before widening, measure fresh/continuation latency, primary DB checkout and execution, event-loop delay, interference with existing traffic, full-visit payload distribution, Redis memory/RSS/bandwidth and overload responses. Exercise one complete TTL cycle, terminal/abandoned visits, scope changes, rejection with no resumable visit, and rollback. Scale the memory allocation and visit cap together; do not infer a safe memory bound from one synthetic fixture.

The missing ClickPipe mappings were added in a subsequent operational change. The current pipe is Running with 19 mappings, with the original 17 preserved. Live CDC/archive/erasure verification passed ten checkpoints and a separate final audit. This bounded test does not establish complete capture or approve analytics readiness. The earlier failed content-ranking evaluation remains unpromoted.

Evidence identity

• Aggregate query/result receipt: ignored discovery-traffic-aggregate-v3.json, SHA-256 b659c87d42c04b905a24e255de6e7afaeaa8ce9484e8f3d0d219cfe400851941.
• Reconciled summary: ignored discovery-traffic-summary-v1.json, SHA-256 46482b01e631b745345c296bfe227f51010505c433562f5f86a09f34001f9129.
• Primary query result: ignored production-primary-query-preflight-result-v1.json, SHA-256 2be3cc4fdc92c076e3e05bb0a6231f00b3d30ea328af1bd7479f7c559acacbee.
• Reproduction: discovery-traffic-analyze-v1.mjs validates the fixed window, completeness, coverage and cross-query totals before writing its summary.
• Independent review reconciled the receipt. Additional discovery-traffic-validate-v2.mjs checks every expected hour/actor and hour/identity key and every day/actor subtotal. Five tests pass, including four corrupted receipts with duplicates, shifted daily counts and out-of-window buckets that cannot be detected by grand totals alone. The accepted evidence and numeric conclusions are unchanged.
• One raw minute export was capped by the tool at 500 rows despite SQL requesting more; it is retained in the first receipt and excluded from all complete-window statistics. Rolling maxima were calculated inside a separate aggregate query.
• Initial queries using a documented but unsupported numeric conversion failed validation. Corrected queries use the verified numeric properties directly.

Implementation validation

The expanded isolated server run completed with 338 passes, zero failures and two skips. Both skipped cases require a local redis-server binary for actual Lua execution; Windows lacks it and the Discovery CI job installs Redis. These unchanged adapter tests are not represented as local passes. Fifteen affected client tests passed, including preserving loaded cards/manual same-cursor retry after overload and explicit offset-zero restart after the master switch is off.

Workspace typecheck completed eight tasks and hosted build completed five. Full exported offline-edition typecheck, build and test verification passed; its server suite had 1,493 passes and four existing skips, and the shared suite's opt-in live-provider check was also skipped. The export boundary test also passed with the new middleware replaced by a local-edition stub and hosted cohort/capacity implementations omitted. No hosted Redis/ranking dependencies were added to the offline application.

Independent reviews covered the rollout/route/OSS/client changes and the separate capacity/drain changes, with no remaining actionable findings. The parallel failure regressions use actual profile, recall and feed paths with controlled SQL failures and blocked sibling queries. The helper normalizes lazy Drizzle thenables once so draining cannot accidentally execute queries twice, preserves ordered successful results, and rethrows the first observed rejection only after all siblings settle. Engagement's already handled failures were verified without changing that implementation. SQL text and ranking coefficients are unchanged.

The controls are validated for review; these checks and traffic observations do not establish production concurrent capacity or approve feature activation.

Follow-up verification on source e797d70c85d882f2437a75962215ee00525c62c8: Discovery CI 35520871859 passed, including the actual Redis Lua cases unavailable on Windows and the live warehouse job. The complete controlled-deployment check also passed eight phases / 78 HTTP checks across two temporary Railway instances. It verified 713-family complete traversal, 711-family traversal after saves, 0%-cohort continuation draining, legacy cache/transport preservation, authenticated allowlisting and continuation after the origin instance was removed. Independent audit confirmed all temporary resources removed and production flags unchanged/off. These synthetic deployment checks do not replace the remaining production capacity, full-TTL or reader-outcome gates. The subsequent live ClickPipe check resolves the missing mappings and verifies one actual CDC/archive/erasure sequence; sustained collection and worker deployment remain separate requirements.