Next milestone: trustworthy discovery measurement

This specifies the next implementation boundary after the cursor/content foundation. It is a proposed engineering contract under the approved product direction, not a claim that the current events or trainer implement it. The existing audit describes why historical labels cannot establish incremental discovery value.

Identity and opportunity

The serving actor is a server-authenticated user ID or a signed first-party anonymous ID. A visit has a separate random session ID. A page has an immutable request ID; each delivered position has an opportunity ID. A viewport impression refers to that opportunity, not just a world. Guest-to-user linkage is an explicit timestamped transition, not a retroactive rewrite of all anonymous events using the identity present when a delayed batch arrives.

Every accepted event needs event_id, event_type, occurred_at, received_at, actor_id_at_occurrence, visit_id, opportunity_id, feed_request_id, world_id, language_group_id, surface, position, policy_version, feature_version and, when relevant, model_id. Event IDs deduplicate retries and mirrored warehouse copies. Server checks page membership and actor ownership; client fields never authorize access or establish identity. Client event time is bounded and accompanied by receipt time; impossible/future timestamps are quarantined from training.

Add events for visible impression, click, foreground preview close/dwell, save intent, confirmed save, play intent, confirmed consumer session start, qualified engaged interval, hide and hide undo. Technical failures remain separate from negative taste. Impression visibility requires the page to be foreground. The existing 500-ms card-view rule is not evidence of disliking a story.

play_intent is not session_started; a login prompt, failed start or closed tab cannot become a positive gameplay outcome. Attach the opportunity to the successful server-side session creation/save mutation after validating ownership and eligibility. Store a small attribution reference, never private message content.

Serving-time features

Persist the exact policy inputs used for ranking: source membership/scores, semantic/behavioral/current-interest affinities, exclusions/exposure state, quality statistics with snapshot time, model feature vector and candidate position before/after diversity. Include the actual model/policy versions and feature schema version. Log decisions about exploration and their assignment probability when randomization is introduced. Do not reconstruct historical features from today's user/library/stat rows.

Snapshots require bounded retention and storage estimates before enabling full logging. ClickHouse is the analytics/training destination; Postgres remains authoritative for state and successful mutations. Redis retains short-lived current intent and exposure state. PostHog is a product-analysis consumer of the same canonical identities and event IDs, not a second independent signal counted in training.

New-discovery labels

A story is new only if neither the world nor its language group was saved/played before the opportunity. Group identity prevents translations from being counted as fresh discoveries. Creator testing and ephemeral sessions do not qualify as consumer outcomes. Only post-exposure intervals belonging to the attributed discovery contribute; deduplicate overlapping intervals before summing.

Initial outcome windows to implement and inspect:

• Confirmed save within 24 hours of the attributed exposure.
• First qualified new play within 24 hours; initially retain the existing five-minute active-play threshold for comparability and report sensitivity at shorter/longer thresholds. This threshold is a starting hypothesis, not a validated satisfaction measure.
• Return to that newly discovered story on a later local/UTC-defined day, reported with one explicit day-boundary convention. Count distinct active days, including returns to the same ongoing chat; do not count session-row count as return-day count.
• D7 continuation and optional explicit satisfaction/hide outcomes, with full observation time before assigning negatives.

Aggregate primary outcomes per randomized actor: share of eligible Discover visits with at least one qualified new story, and distinct qualified new stories per actor. Keep CTR, depth and time as diagnostic or secondary measures; endless scrolling is not itself the optimization target. Always report source coverage and the fraction of unjoinable/censored events.

Training and experiments

Use chronological training, validation and untouched test intervals, with gaps for outcome maturation. Prevent a discovery episode and its labels from leaking across boundaries. Compare the complete candidate/rank/diversity pipeline against the incumbent, including all-zero and empty visits, rather than comparing model-only scores against served rank on positive slates alone.

Offline checks: temporal integrity, actor/opportunity joins, label maturity, feature parity, retrieval recall, calibration, successful-discovery ranking, creator/topic coverage and rejection exposure. Introduce multi-interest, negative-similarity and session features as separate ablations. Advanced model complexity follows evidence from these comparisons.

Online assignment must be stable per actor; log identity transitions and prevent one signed-in user from crossing arms due to device/session/model reload. Predeclare primary metrics, guardrails, experiment horizon and minimum detectable effect using observed actor-level variance. Use actor-clustered uncertainty, sample-ratio checks and a predefined stopping rule. Do not repeatedly peek and promote the first positive fluctuation. Model versions must be fixed or separately accounted for during a comparison.

Keep the incumbent available for immediate rollback. New model artifacts need a manifest containing training interval, feature/label versions, evaluation slices, data coverage and serving compatibility. Promotion requires offline validity and a real online outcome readout; a nightly job succeeding is not a quality gate.

Resurfacing and spending

Once durable actor-level exposure timestamps exist, distinguish merely delivered cards from actually viewed/overlooked stories. After true eligible-catalog exhaustion, allow overlooked, unsaved items after a declared cooldown; never relax saves, dismissals, blocks, publication, language or content rules to fill a page. Use measured revisit behavior to choose the cooldown, and log resurfacing explicitly for evaluation.

No additional service is required to start this milestone. Measure event throughput, feature freshness, query latency, Redis state/eviction and training duration before choosing a queue, dedicated feature store, separate vector service or GPU training. Budget flexibility allows evidence-based additions without making infrastructure expansion the success criterion.