Neon → ClickHouse connection — September 12, 2026

Verified at 23:37 UTC: the managed feed is Running, its initial snapshot has completed, and natural new requests have arrived through CDC. Six canonical analytics views are applied and readable by the importer and scoped reader. The application capture, admin and Free-credit release remain off.

Restart and recovery

The owner enabled logical replication in the Neon Console. The primary restarted at 23:11:58 UTC and the read replica at 23:11:57 UTC. At 23:12:17 UTC both reported wal_level=logical, authenticated as yumina_mycelium, and had zero neondb_owner sessions. Public website and game-runtime health checks returned HTTP 200; website primary/read/Redis checks were healthy.

This is separate from the credential rotation completed at 23:05 UTC and the still-unlaunched admin/pricing release.

Source preparation applied

• Dedicated role yumina_analytics_cdc: LOGIN, REPLICATION, BYPASSRLS, NOINHERIT, NOCREATEDB, NOCREATEROLE, connection limit 12. BYPASSRLS is needed because the existing source tables enable RLS; it does not grant table access. No membership in neon_superuser or the application's ownership role was granted.
• Granted CONNECT to neondb, USAGE on public, and SELECT on only the 71 selected columns across 15 tables. The 182 other current columns are also explicitly excluded in the ClickPipe request. No blanket future-column grants were added.
• Role limits: statement timeout 300s, lock timeout 2s, idle-in-transaction timeout 0 so the initial exported snapshot can remain open while its partitions are copied. Concurrency is bounded by the connector settings below.
• Snapshot setup also requires SELECT on PostgreSQL's ctid system column on each selected table. The first preflight checked the business columns but missed this: actual CTID range queries returned 42501. Added only SELECT (ctid) on the same 15 tables and verified that those queries now pass. This metadata is not in the publication or warehouse schema. Account email, all-column usage reads and wallet writes remain denied.
• Applied the reviewed source-publication.sql: publication yumina_admin_analytics, with exact column lists and the existing unique indexes as replica identity for thread_likes/post_likes. All source identity keys were validated as unique, valid, non-partial, non-null and included in the publication.
• Actual fresh connections as the source role passed selected-column queries on all 15 tables and PostgreSQL's IDENTIFY_SYSTEM replication command. The source role cannot SELECT account email/messages, UPDATE wallets or CREATE in the public schema. No test replication slot was left behind.
• Source host is the direct primary hostname, not a pooler. TLS verification is enabled.

ClickHouse preparation applied

• Organization Yumina, ID 52386b54-3b5e-4fd9-a197-a630e784a518.
• Existing service My first service, ID 8cf445bd-ff27-49aa-a8b6-8d0e8e320799, AWS us-west-2.
• Created the empty yumina_raw database and a dedicated yumina_clickpipe_destination role. Its SQL privileges cover SELECT/INSERT/ALTER/CREATE TABLE/DROP TABLE/TRUNCATE on yumina_raw.*, plus READ ON S3 for ClickHouse's import staging files. The first API validation rejected the missing S3 permission; no pipe was created by that rejected request. Adding the required staging-read grant allowed creation to pass.
• Existing analytics importer received SELECT on yumina_raw.*; its existing analytics permissions were preserved.
• Canonical views execute with invoker privileges. The analytics reader therefore received SELECT on only the selected business columns and three connector metadata columns in the 15 raw tables. It did not receive writes, internal staging-table access or broad future-column access. Its existing readonly setting remains unchanged; the verification client respects that setting instead of trying to override execution settings.
• The connector's ClickHouse query log exposed a missing SELECT(value, metric) privilege on system.metrics; granted those two metadata columns to its destination role.
• Cloud API credentials and the new source password are stored only in the ignored Windows-DPAPI store. They are not used in application browser code or committed here.

Connector

Created at 23:18:19 UTC:

• Name: yumina-admin-analytics
• ID: 767353a7-be1f-49f0-b876-f1e0af910307
• Mode: initial snapshot plus CDC, using the existing explicit publication.
• Initial-load parallelism: one reader per table, two tables at a time, 100,000 rows per partition.
• CDC: 30-second sync interval, 10,000-row pull batches.
• Destination: 15 ReplacingMergeTree tables in yumina_raw, sorted by the verified source identity keys.
• Preserve nullable columns. Do not replace unknown provider cost with zero.
• Keep deletion markers for downstream reconciliation; deletion-retention cleanup remains a separate rollout gate.
• API readback confirms the selected settings and all 15 table mappings. The service entered Running at approximately 23:34 UTC after the initial snapshot completed. No connector recreation, Neon restart or slot deletion was needed to resolve the snapshot permissions.
• At 23:25 UTC the actual raw schema passed: exactly 71 selected business columns plus the three expected connector metadata columns per table, correct identity sort keys, preserved nullable fields, JSONB tags as String and provider cost as Nullable(Decimal(24, 12)). No extra business columns were copied.
• The owner supplied the detailed 23:27:25 UTC cursor errors for user and credit_wallets. The CTID grant was applied at 23:27:31 UTC; all 15 actual DECLARE/FETCH 0 cursor permission probes passed afterward. The initial copy resumed on ClickHouse's subsequent retry. Its cumulative error counter remained at 48 through 23:37 UTC; historical counter values are not current failure indicators.
• At 23:34 UTC the slot retained approximately 55 MB while catching up. By 23:36 UTC it was active, its confirmed offset had advanced, and retained WAL had fallen to approximately 1.5 MB. Public website and runtime remained HTTP 200.

Data verification

Read comparisons used the read replica, an 8-second statement budget, and FINAL / deletion filtering on the warehouse. No balance mutation or synthetic user action was used.

Check	Result
September 11 UTC usage	Exactly 76,697 rows, 2,474,041,959 input tokens and 78,228,222 output tokens on both sides.
Historical request costs in that day	Zero rows have a recorded provider cost on either side. These costs remain NULL; this does not mean AI was free or certify a dollar-total reconciliation.
Credit ledger sample	Exactly 894 entries across 20 recently active wallets for September 11; signed and absolute sums match. The unrestricted whole-day ledger aggregate exceeded the 8-second source budget and was stopped. This is a bounded sample, not a certification of every historical ledger entry.
Account and small-table counts	31,247 accounts and wallets matched; posts, threads, likes and saved-work counts matched. Warehouse had one later world and two later check-ins when read roughly 2.5 minutes after the source; the reads are not an atomic cross-system snapshot.
Natural live requests	Three requests sampled after initial snapshot start arrived with matching token and nullable cost fields. A second sample after Running included a request at 23:35:40.906899, synced at 23:35:42.880 — about two seconds in this observation, not a latency guarantee.
Canonical views	All six applied from 002_cdc_views.sql; importer and reader SELECT checks pass. Deduplicated counts included approximately 5.02 million usage rows and 4.20 million ledger entries.
Read access	View count queries completed in approximately 38–875 ms in the verification run. This is warehouse-query timing, not end-to-end admin response performance.

Remaining release gates

1. Finish population-level historical reconciliation, source attribution and reporting checks. Keep the ledger sample's coverage explicit; don't present it as a full financial audit.
2. Deploy the separately reviewed capture changes during the application release. Verify genuine heartbeat, provider-cost records, activity and play intervals. The three new telemetry tables are empty while capture is off. Don't insert a synthetic heartbeat or invent historical play time.
3. Keep worker ANALYTICS_CDC_READY=false until capture is available and the final source/report gates pass. At 23:36 UTC Railway still had that value, with capture/admin/Bonus/Free-policy flags unset on the web and game runtime. No application release or pricing activation occurred here.
4. Verify prepared reports and reconcile Stripe/OpenRouter for identical cutoffs before enabling the new admin. Missing historical per-request provider cost remains an explicit limitation even though account-level vendor billing is available.
5. Coordinate the Free-credit activation with the owner's announcement and the previously agreed notice window.

References

• ClickHouse Neon source setup
• ClickPipes PostgreSQL FAQ: replica identity using a unique index is supported; JSON/JSONB defaults to String.
• ClickHouse Cloud API keys

Ignored operational files under packages/server/scripts/admin-analytics.local/: cdc-source-manifest.local.json, clickpipe-request.template.local.json (placeholder password only), prepare-cdc-source.local.mjs, clickpipe-live.local.mjs, fix-cdc-metadata.local.mjs, verify-cdc-live.local.mjs, cdc-verification-results.local.json, and cdc-view-verification.local.json. The setup scripts refuse to overwrite an existing named publication/role/connector automatically. Source samples remain local and are not committed.