Additional project 2 of 4 · app/GlobalCloudflareAdapter, app/workers/global_upload_service, app/BillboardGateway/workerStack: Go (adapter + Kafka/Redis adapters + KV proxy), TypeScript Workers (Hono), R2 multipart, Cloudflare Stream, Durable Objects, TUS
Where this sits. Large-file ingest is a genuinely hard distributed problem
— resumability, partial failure, orphaned parts, and a coordinator that must
be a single writer — and this is the system that solves it across five
product modes.
Resumable ingest, and the single writer that makes it possible
Presigned URLs are not available for R2 multipart on Workers, so the upload is proxied and coordinated by a Durable Object, the only construct at the edge with single-writer semantics. Equal 10 MiB parts keep the part table uniform; the coordinator is what stops two clients racing the same part index.
Two halves
The Go adapter — app/GlobalCloudflareAdapter (7,855 lines)
It is the bridge between the edge and the host: a Worker that cannot open a
TCP socket still needs Kafka, Redis, and Cassandra, so the adapter terminates
those protocols host-side and exposes them over HTTP. The kv_proxy is the
same idea for key-value access.
This is the general form of the constraint that shaped
the Activity Graph: the edge cannot speak
your protocols, so something host-resident has to.
The edge worker — app/workers/global_upload_service
One ingest surface serving five modes:
BillboardUploadService.ts music
BoxOfficeUploadService.ts video
StorefrontUploadService.ts commerce
BeaconUploadService.ts beacon
BeaconAgroService.ts
DatingDistroService.ts DatingStreamBridge.ts
auth.ts jwt.ts routes.ts service.ts RedisAdapter.ts
Upload v2 — the design that shipped
Decision
Why
R2 multipart, no presigned URLs
Presigning hands the client a credential you cannot revoke mid-upload; multipart through the Worker keeps authorization on every part
10 MiB equal parts
Uniform parts make part numbering, retry, and progress arithmetic trivial
Audio probe on ingest
Reject an unplayable file at upload time, not at first play
IngestDO Durable Object
The upload session needs exactly one writer; a DO is that primitive at the edge
TUS made genuinely resumable ([W-3])
It advertised resumability it did not have
publishDraft made atomic ([A-9])
A half-published draft is worse than a failed one
Auth on the media path
[F-2] — global-media-worker gained real RS256 verification with identity
bound to the token, rather than trusting an identity supplied in the request
body. On an upload path, body-trusted identity means anyone can write to
anyone's library.
Delivery
billboard-gateway-edge serves audio from R2 with HTTP range requests,
which is what makes seeking work without downloading the whole object. Video
goes through Cloudflare Stream with HLS/DASH manifests and a playback token
that refreshes ([H-4]/[H-6]).
One shipped defect worth naming: [seed] fix truncated Stream uids — the seeded catalog would not have played. A truncated id fails at playback, far from
where it was introduced.
Interview surface this opens
Multipart upload: part sizing, retry, orphaned-part cleanup, completion races
Presigned URLs vs proxied uploads — the revocation trade-off
Durable Objects as a single-writer session coordinator
HTTP range requests and byte-range streaming
Terminating TCP protocols host-side for a socket-less edge runtime