Skip to content

Story evidence and feed storage improvement plan

Goal: address measured loss of supported story facets and Redis session transfer cost without asserting reader uplift or promoting an unmeasured model.

Architecture: keep the current serving descriptor contract intact while testing a separate, explicitly experimental semantic evidence review. Literal source provenance, bounded input/output, model identity and failure isolation remain mandatory. Optimize shared feed storage losslessly with cursor, admission, erasure and expiry semantics preserved.

Stack: existing TypeScript server, OpenRouter, Railway testing Redis, Node test runner. No new subscriptions.

Quality work

  • [x] Freeze a separate development sample of public published stories, excluding every family in the previous pilot and frozen evaluation set. Do not tune on the frozen 24-story reference.
  • [x] Add provenance-only proposal validation and diagnostic tests without allowing experimental proposals through the production descriptor validator.
  • [x] Add bounded, tool-free semantic proposal/review adapters. Review exact evidence against its full source context and a fixed facet rubric; uncertainty and production instructions must not become supported experiences. Keep proposed labels separate from accepted serving descriptors.
  • [x] Test failures, malformed/duplicate/missing decisions, literal evidence, multilingual paraphrases, denial, incidental mentions and injection contrasts.
  • [x] Run the same development inputs through lexical curation and independent semantic review; report agreement, omissions, acceptance, latency and cost separately. References are blind AI annotations, not human truth. Decide next steps from results; no automatic promotion.

Storage work (independent worker)

  • [x] Trace growing Redis bodies and choose a lossless representation with explicit compatibility behavior.
  • [x] Regression test old/new sessions, replay, CAS races, admission/ownership, absolute expiry, preference revision and erasure. Compact replay passes through the actual database tombstone guard; both encodings have explicit live-Redis CI coverage.
  • [x] Measure serialized size and CPU locally, then repeat the bounded Railway testing probe using identical inventory and traffic. Clean up only owned keys/services. Eighteen runs/3,600 attempts completed; compact saves bytes but does not establish lower latency and remains disabled.

Integration

  • [x] Independent review of changed boundaries, build/typecheck and affected suites; fix findings before committing.
  • [x] Update readiness evidence. Final publication and exact-commit CI status are recorded on PR #179 so the evidence document need not change the commit it verifies. Production rollout, full-origin capacity and reader outcome experiments remain separate gates.