# 32 · Global Feed Gateway: One Feed Every Product Mode Plugs Into **Rank 14 of 34** · Tier A · `app/GlobalFeedGateway` (`service/projection.ts`, `adapter/`, `workers/`, `migrations/`, `docs/`), `app/globalfeed`, `IOS/global-feed-ios` **Stack:** TypeScript on Cloudflare Workers, D1 (SQLite, FTS5), R2 multipart, KV, Queues, Media Transformations, RS256 JWT verification, generated OpenAPI 3 + AsyncAPI, Swift package client > **Where this sits.** Five product modes (Billboard, BoxOffice, Dating, Vitals, > Storefront) each wanted a feed, a profile gallery and a composer. The cheap > answer is five feeds. This is the other answer: one service, one access rule, > and a manifest per mode. It is live, it is the eighth canonical service, and > the interesting part is not the feed. It is the single function that decides > who may see an item, and the test that keeps its two copies honest. --- ## The shape ``` iOS (Swift package) web (app/globalfeed engine) │ │ └──────────┬─────────────────┘ ▼ Worker: one route table, one deploy │ ▼ projection(item, viewer, mode, surface) <- every read goes through here │ ┌─────────────┼───────────────┬──────────────┐ ▼ ▼ ▼ ▼ D1 R2 (private) Queue consumer mode adapters items, graph media bytes scan, classify resolve at read time ``` One Worker, deployed once, serving `fetch`, `queue` and `scheduled`. Measured on the working tree: **12,408 lines of TypeScript** outside tests, **4,670 lines of tests**, and **244 tests, 244 passing** under plain `node --test`. The generated OpenAPI document covers 76 operations and the AsyncAPI document 13 messages, and both have a `--check` mode that fails when the code and the contract drift. ## Why D1, not the Postgres on the origin box Every other feed in the platform reads from a single droplet that runs Cassandra, Kafka, Postgres and a dozen services. That box has a memory ceiling and has rebooted under OOM more than once. A feed is the most read-heavy thing in the product, so the decision was that **feed reads never ride the origin box**. D1 sits at the edge next to the Worker. Pull-on-read instead of fan-out-on-write, because at this scale a write-time fan-out buys latency the product does not need and costs a second copy of every access decision. That second copy is the thing this design is built to avoid. ## The one idea: three axes, one projection Every item stores three independent facts: | Axis | Set by | Values | |---|---|---| | `visibility` | the author | public, followers (reserved), subscribers, private | | `audience` | manifest, author, staff or classifier | kids, general, adult | | `moderation` | review | pending, approved, limited, rejected, removed, appealed | None is inferred from another, from a row count, or from a missing value. One function reads all three together with who is asking, in which mode, on which surface. Every list, count, search hit, story ring, notification preview, share card, bookmark list and adapter tile calls it. The reason is in the source comment. BoxOffice had shipped **seven audience leaks**, and they were the same defect in seven places: a surface that read items without the check. So the rule is written once. ### The rule exists in two forms, and a test proves they agree A list query cannot load rows and filter them in JavaScript. A single row check cannot be a WHERE clause. Two copies of an access rule drift. So each clause in `service/projection.ts` carries **both** its SQL fragment and its row check, side by side. The list query gets the SQL, the permalink gets the JavaScript. `tests/projection/matrix.test.ts` builds a fixture matrix of **4,320 items across 660 viewer, mode, audience and surface combinations** and checks it three ways, against an oracle that imports neither form. The matrix earned its keep: it found that archiving is not a visibility value, so an archived public item stayed public, and the Archived tab would have served other people's archived items to anyone who asked. That is now an `ownerOnly` flag on the surface policy, with a comment naming the leak. ### Two rules that follow from it - **Hidden means not found.** A private item, a blocked author's item and a nonexistent id return byte-identical 404 bodies. A 403 is only for an action refused on something you can already see, because confirming that a private item exists is itself the leak. - **Nothing copies an axis.** Bookmarks, notifications, adapter references and search documents store an item id and re-check at read time. An item re-rated adult falls out of a general session's saved list with no write anywhere. ## Hexagonal adapters, enforced by the schema A BoxOffice film or a Billboard track can appear in the feed. The adapter reference stores the source, the type and the native id, and nothing else. No title, no URL, no audience, no moderation state. Those are resolved from the owning service at read time, because a copied access fact goes stale and a stale one is a leak. `gf_adapter_refs` has **no column** for any of them, so the rule is enforced by the table definition rather than by code review. The gateway side (`adapter/port.ts`, `resolve.ts`, `boxoffice.ts`) is built. BoxOffice's own resolve endpoint is not, which is stated in the build record. ## Writing the spec first, and what it found The specification is 14,897 lines with 1,415 identifiers and more than 1,600 acceptance criteria, plus a deployment checklist of gates. Writing it against the real code surfaced blockers before any feed code existed, each verified at a file and line: - **BoxOffice's hourly reconcile adopted every ready video asset in the shared Stream account that had no catalog row.** Any other product putting video there would have leaked into the BoxOffice catalog within an hour. The feed later moved video to R2 entirely, so a feed upload can never reach that job. - The privacy policy promised EU storage and no third-party ads, which constrained where media could live and how ads could be served. - The staff roles CHECK constraint had no Vitals mode and no ads roles. - Storefront had no creator subscription model, so a subscribers-only visibility value had nothing to check against. ## Media: fail closed, by design Uploads go through R2 multipart. Video is parsed by range reads, refused unless it has exactly one `moov`, one `mdat`, one H.264 track and at most two AAC tracks, then streamed back out with `udta`, `meta` and stray tracks dropped (that is where GPS and device tags live) and the index moved to the front. Verified with ffmpeg: zero decode errors and identical frame checksums. Then a scan gate. Frames and images go through a classifier, and a known-content hash port answers for exact files and for perceptual image matches. **No vendor is wired to that port, so in production nothing releases.** That is deliberate. A media path that publishes without a known-content check is not a feature. ## Bugs worth naming - `pageLimit()` capped every unbounded list to **one item** when no `?limit=` was passed. `URLSearchParams.get` returns `null`, and `Number(null)` is `0`, not `NaN`, so the "invalid, use the default" branch never ran. Present since the first build, found while extending it, fixed with a comment explaining why. - Comment @mentions now pass the same visibility gate as everything else. A security review asked for it, and it is deployed. - Profile edits use `If-Match` with an ETag and creation uses `If-None-Match: *`, which is optimistic concurrency on a full-replace PUT. A client that omitted both was refused rather than silently overwriting. ## The scaling finding, from someone else's benchmark [FeedBench](29-feedbench-runtime-study.md), the runtime study, ran this contract on Workers, Go and Spring. Every feed section's predicate is `mode AND audience IN (...) AND archived_at IS NULL AND (ownerBranch OR visitorBranch)`, and the existing partial indexes require `state = 'ready'` or `visibility = 'public'`. SQLite uses a partial index only when the query implies its WHERE. This one does not, so D1 read the whole table and sorted. At 200,000 items the suggested section took about 1.35 s median per query. Two indexes, no code change, and saturation rose by an order of magnitude on all three runtimes. The production feed has the same predicate shape. It was reported, not silently patched. ## Honest gaps - **No hash vendor, so no media releases in production.** Text and link posts work. The vendor contract test is ready to run against an adapter. - **No real email provider** exists in the account; the email port is built. - BoxOffice's resolve endpoint for adapters is not built. - The web UI for Billboard's feed is on a branch, not merged. - The tuned indexes from the benchmark have not been applied to production. - Video has no transcoder: HEVC, VP9 and fragmented MP4 are refused by name. - Owner overrode the spec's own "no chat" boundary, so the gateway carries a DM system that duplicates GlobalChat. That decision is recorded, with the recommended alternative, in the repo.