How TEEI Runs Its Platforms on Cloudflare Workers

A learner picks a volunteer who is free right now, and the volunteer’s screen rings. A company manager invites employees into TEEI’s programmes, and each invitation is saved before an email goes out. Both run on Cloudflare Workers, like the rest of TEEI’s platforms: the website, TEEI Language, Skills Academy, TEEI Mentorship, Evenhall and the admin our own staff work in, all on one Cloudflare account. In Cloudflare’s sampled analytics for the 30 days to 14 October 2026, our production Workers answered about 5.44 million requests, and about 120 of them ended in a runtime error.

TEEI is a nonprofit. Our free programmes connect people from Ukraine, and women from Latin America and the Caribbean, with volunteers from many countries who practise a language with them, mentor them and teach live classes. Companies take part through Evenhall, where their managers see what their employees did through TEEI’s programmes and turn it into a report they can file.

This is an engineering note about what all of that runs on: how the account is laid out, the two days in June when we moved everything onto it, the Durable Object that rings a volunteer’s screen, why our Workers run next to the database instead of next to the user, the queues behind Evenhall, the limits we hit, and how we deploy. Every number comes from our Cloudflare account, our logs or our code. We measured them on 14 October 2026 unless we say otherwise, and the last section lists what you can check yourself.

Built by volunteers, supported by companies. TEEI’s software platforms are built through volunteer work, with NOK 0 spent on paid software development. Companies support this work through cloud credits, sponsored services, technology and technical support. Infrastructure and third-party costs are separate from the software development figure.

What people do on these Workers

Most of this note is about Workers, bindings and limits. This is what they carry. A learner can find TEEI on the website, practise a language on TEEI Language, talk to a volunteer in a practice room, and meet a mentor on TEEI Mentorship. Each of those screens is answered by a different Worker, or by the same Worker doing a different job.

Oksana T. learned this way, in one-to-one sessions with volunteers on TEEI Language. Her film is one of those on the stories wall, and it plays from the same account as the rest of this note: the file sits in an R2 bucket and reaches the browser through the bucket’s public domain.

On Skills Academy, volunteers teach live workshops, clinics, expert Q&As, webinars and classes. Participants can combine language practice, skills learning and mentoring according to their needs, without following a fixed sequence. People earn certificates and badges as they go, drawn by a renderer on AWS Lambda and stored in R2. And when a volunteer gives their time through an employer, the company’s managers see it in Evenhall.

The same infrastructure also serves TEEI Network Partners. NGOs, universities, churches and other community organisations bring their learners, members, staff and volunteers into TEEI’s free programmes. They use their own Network Workspace to share TEEI Language, TEEI Mentorship and Skills Academy, contribute teachers and mentors, and follow participation and verified activity. We do not build a separate platform for each organisation. Their workspace reads participation from the same records as the programmes, using the same infrastructure and reporting systems. The applications remain within our Cloudflare environment, with the same protection through Project Galileo that covers the rest of TEEI’s platforms.

The screens come from the public tours of TEEI Language and TEEI Mentorship, from Evenhall’s demo workspace and from a practice room with demo personas. Every name in them belongs to a sample profile, and none of the faces or pictures shows a participant.

At a glance

The requests are not evenly spread. Most of them reach the website, while the programme platforms do more work for each request they answer: TEEI Language made about 7.7 million calls to other services to answer its 1.39 million requests.

The last line holds the redirects for our other domains, the schedulers, the monitors, a tail Worker and the smaller portals. Its 50 or so runtime errors all came from Evenhall’s scheduler Worker between 14 and 17 September. Because Cloudflare’s analytics sample requests, the errors arrive in steps of ten. Clients that disconnect before a response is finished are recorded separately and are not counted as errors here. A runtime error is a Worker invocation that did not complete, such as an uncaught exception or an exceeded limit. A page our code answers with an HTTP 4xx or 5xx status, a request the firewall blocks before it reaches a Worker, and whether a service was available are different measurements, so this table is neither a count of HTTP errors nor an uptime figure.

Day by day, each Worker has its own rhythm, and the daily lines show things the totals hide.

The website’s line follows the week. TEEI Language is almost flat at 25,000 to 56,000 requests a day, apart from 19 and 20 September, when about 306,000 requests arrived in two days. The daily median wall time on those two days was about 23 milliseconds, against 430 on 14 September, so at least half of those requests finished in a fraction of the time a page takes. TEEI Mentorship has stayed above 9,500 requests a day since 5 October, against 3,300 to 5,900 a day between 14 and 25 September.

One account, five core application Workers

The parts are easier to see together.

The website is an Astro 7 site that runs as a Worker with Static Assets. The Worker runs first for every request. We use Worker-first routing so our entry can apply request handling before the asset layer. The website binds KV for rate-limit counters and R2 for videos, course certificates and the share cards for published sessions. The share cards have a bucket of their own. A separate renderer outside Cloudflare writes them with an R2 API token, which is scoped to buckets, not to prefixes within them. R2’s temporary credentials can be narrowed to a prefix, but the renderer does not use them. We use separate buckets to keep the share-card renderer’s write access separate from certificate storage, so a fault in the renderer cannot overwrite the certificates people put on LinkedIn. Two cron triggers run the website’s scheduled jobs.

TEEI Language and TEEI Mentorship are two products on one platform, built from one React Router 8 codebase and deployed as two Workers. Language is where people practise a language through real conversations; Mentorship connects mentees with professional mentors. Skills Academy is served from a separate hostname on the Mentorship Worker; sharing a Worker does not decide which programme a participant takes first. Each Worker has its own Durable Object namespace for live presence and runs next to the database through a Placement Hint. Both send their logs to R2 through Logpush, so the logs outlast the few days Workers Logs keeps.

Evenhall is where a partner company’s managers invite employees into TEEI’s programmes, see what they did there, and turn it into a report they can file. It is a React Router 8 Worker with three Cloudflare Queues and a small scheduler Worker of its own.

Admin is where TEEI’s staff run the programmes, and it binds the same R2 bucket the website reads credentials from. Programme credentials are rendered with Playwright and Chromium on AWS Lambda and stored in Cloudflare R2. Admin sends its trace events to a Tail Worker and uses Smart Placement.

By bytes, those certificates and badges were most of what R2 held on 14 October 2026, and the films came next. Ihor R.’s, below, is one of them: 6.7 MB in the same square cut the stories wall plays, and Oksana T.’s above is 9.0 MB. Both reach the browser through the bucket’s public domain, in the byte ranges the player requests when it plays. The player is set to preload="none", which asks the browser not to fetch the video ahead of playback; its poster image loads with the page.

By object count the order flips: the platform logs hold more than eleven objects for every certificate or badge.

Around these five sit smaller Workers: one that sends our Norwegian domains and our .com domain to the main site, the Women Buddy Programme portal, a health monitor, and Evenhall’s scheduler.

Some things are deliberately not on Cloudflare. The database is Turso, a hosted libSQL database with its primary in AWS Europe (Ireland): our programmes need one relational database that decides what is true, and running the Workers next to it was part of the performance work described below. Video calls run on Amazon Chime SDK, and their audio and video never pass through our Workers; the Workers decide who may join and when. Features built on language models run on Amazon Bedrock. Credential artwork is rendered on AWS Lambda before it is stored in R2. Email goes out through Resend and Amazon SES, errors go to Sentry, and search for help answers and job titles runs on Algolia.

Two days in June: moving everything to a new Cloudflare account

On 28 and 29 June 2026 we moved TEEI’s Workers, domains and buckets to a new Cloudflare account, using the Cloudflare API, Wrangler and rclone.

The plan had one rule: deploy everything on the new account first, then move the nameservers, and let a domain go only when everything behind it already answered from the new side. We started with teei.no, which only redirects, because a domain whose only job is to send people elsewhere is the cheapest place to learn what breaks. Both accounts ran the same redirect Worker and the same routes while the nameserver change spread, so it moved with no downtime, and the same steps then ran for each domain in turn. The main domain needed one more step. It carried a zone hold on the old account, which we released before the domain could be added to the new one, and its DNS went across with 38 records after we dropped nine staging and retired hostnames. It became active on the new account about seven and a half minutes after its nameservers changed, and every subdomain answered from the new account. We had allowed for up to 30 minutes of downtime per site.

The data was the simple part. Twenty-two R2 buckets were copied with rclone and verified byte for byte, eight empty log and staging buckets were created again, and two buckets in Cloudflare’s EU jurisdiction were copied through the EU endpoint. Of all the bindings in our configuration, only two kinds of identifier change from one account to another: the ID of a KV namespace and the ID of a D1 database. R2 buckets, Durable Objects and Vectorize indexes are bound by name. Vectorize indexes cannot be exported cleanly, so ours were created empty on the new account.

The traps were in the deploys. These are the ones we wrote down in the runbook as we met them:

1. A new workers.dev subdomain. The new account needed its own workers.dev subdomain, registered once before the first deploy, and every Worker on it got a new workers.dev address.
2. New IDs, edited on a copy. The old account kept serving production until each domain moved, so the KV and D1 IDs were swapped in a copy of each configuration, never in the repository.
3. Deploy what the build produced. Our React Router apps are deployed from the configuration their build generates, not from the source configuration. Bundling the source directly fails on a virtual module that only the framework’s build provides.
4. Logpush and Tail Workers. Both were in our configuration and neither is available on the Free plan, which is where the new account started. Logpush answered with error 10023, so both were switched off for the first deploys.
5. Cron triggers off until cutover. Both accounts talked to the same database. If a Worker’s schedule ran on both, every job would run twice.
6. Empty buckets must exist. A Worker that binds an R2 bucket cannot be deployed until the bucket exists, even if it is empty.
7. Secrets that only lived in Cloudflare. A Worker secret cannot be read back, so the few secrets that existed only on the old account had to be found in their original source or created again.
8. Workers without source. Six old Workers had no source in any repository. The API endpoint for a Worker’s content refused us with error 10405, but the endpoint for the script itself returned the code, so we deployed five of them again and left the sixth behind.

We also planned to stay on the Free plan, and that held for less than a day. The website’s Worker was 4.94 MiB compressed, more than the Free plan allowed at the time, so we turned on Workers Paid on 28 June. DNS went in through the API rather than through a zone file import, because a BIND import drops the proxied flag on each record and every record would have needed fixing by hand.

The move took two days; the loose ends took a week. On 3 July we found that a video bucket had never been created on the new account, and on 7 July a deploy tool’s cache still pointed at the old account. The Tail Worker for Language had no source left to deploy from, so we replaced it with Workers Logs, Sentry and a health monitor Worker.

From Pages to Workers

Two of our applications had left Cloudflare Pages for Workers before the move. Evenhall started in May: a small proxy Worker took over the hostname and sent each path either to the old Pages project or to the new React Router Worker, and paths moved across one prefix at a time. Admin followed the same pattern and switched completely on 14 May 2026, when its CNAME moved from the Pages project to the Worker.

One detail made Evenhall’s switch invisible to the people using it. Evenhall counts requests in KV to rate-limit them, and the new Worker was bound to the same KV namespace the Pages project had used, “so counters keep their state across the cutover”. A limit that someone had reached a minute before the switch was still reached a minute after it.

The website now runs as a Worker with Static Assets as well, and its old Pages project was deleted on 4 September 2026.

What Cloudflare’s cache does for the zone is easiest to see by day.

About 7.7 million of the 10.7 million requests in those 30 days, seven in ten, were answered from Cloudflare’s cache. About 2.7 million carry the status none, bypass or updating, and the dynamic and miss layers together are under 300,000. The hit share moves with the traffic mix: from about half on 19 September, the day of the TEEI Language burst, to about 86 percent on 27 September.

One codebase, two Workers, and calls between platforms

TEEI Language and TEEI Mentorship share one repository, one database and most of their code, but they are separate products with their own words, rules and hostnames. We deploy the same code twice, as two Workers. The environment is chosen when the code is built, not when it is deployed: our framework’s build flattens the configuration and drops the named environments, so a build for staging and a build for production are different artefacts from the start.

Hostnames are not in the Worker configuration at all. The comment above where the routes would be explains why:

// Production custom domains are intentionally NOT declared here. Wrangler
// 4.107.0 reconciles every declared custom-domain list as replacement state.
// Keeping route/routes absent makes ordinary Worker deploys domain-neutral; the
// separately approval-gated Workers Domains API contract owns the exact
// hostname-to-service mapping instead.
"workers_dev": false,

An ordinary deploy can therefore never add or remove a hostname. The mapping lives in one place, is changed through the Workers Domains API, and needs its own approval. The account has seven Workers custom domains attached this way, including the two hostnames on the Mentorship Worker.

The two Workers talk to each other, and to Evenhall, through service bindings. Language owns every cron schedule on the platform; Mentorship has none. When a scheduled job needs to act inside Mentorship, Language calls the Mentorship Worker through a binding instead of its public hostname, and since 13 October Skills Academy’s scheduled jobs reach Mentorship the same way. Evenhall exports two named entrypoints, one for activating a company’s application and one for the admin’s review of applications, and Language and admin each hold a binding to the one they need. A call between them never leaves our account and has no public URL for anyone else to find.

Every call a Worker makes while it answers, to the database, to another Worker through a binding, to KV or R2, or to a service outside Cloudflare, is counted as a subrequest, so the ratio says how many calls each Worker makes to answer a request.

Admin makes about 20 calls for each request it answers and Evenhall about 12. Language makes between five and six, which is lower than the database trips per page quoted below because the ratio counts every request, including the ones for assets and the API, not only pages.

We learned the hard way that a plain fetch() between our own Workers is not the same thing. On 21 September a small public Worker that reads data from our website answered every call with “temporarily unavailable” on its first staging deploy, while the same URL answered 200 from anywhere else.

A Worker’s global fetch to a hostname that another Worker on the same account serves through a zone route is refused with error 1042, unless the Worker has the global_fetch_strictly_public compatibility flag. Without the flag, the request goes to the zone’s origin, and our website has no origin behind its Worker.

We set the flag rather than adding a binding. With it, the request goes through the public front door and the same security settings as any visitor, so that Worker sees exactly what the public sees, which was the point of it. Evenhall’s scheduler Worker met the same error from the other side, and there a service binding was the right answer: it only needs to reach Evenhall.

The rooms needed one more setting. Background blur and noise reduction in our Amazon Chime SDK rooms need SharedArrayBuffer, and the browser only provides that to a page that is cross-origin isolated. The Language Worker therefore adds two headers to the room pages, and only to those:

if (isVideoSurfacePath(pathname)) {
  cloned.headers.set('Cross-Origin-Opener-Policy', 'same-origin');
  cloned.headers.set('Cross-Origin-Embedder-Policy', 'credentialless');
}

credentialless rather than require-corp lets the room load images from other origins without each of them sending a header of its own. Keeping the headers to the room pages means no other page pays for the isolation.

Live presence on a Durable Object

Some conversations do not wait for a booking. A volunteer can mark themselves available now, and a learner who sees them can start a session straight away. The volunteer’s screen then rings, and the volunteer answers or declines. If nobody answers, the session must be cancelled and the learner told, even if the volunteer’s browser has been closed.

That needs something that can hold a connection open and keep time after every browser has closed, which a request and a response cannot do. That is a Durable Object. The head of the file says what it does, and also what it does not:

// LivePresence Durable Object: the real-time plane for snap ("Available now").
//
// ONE instance per environment (idFromName('live')). It is a THIN notification
// hub: Turso remains the single source of truth for live windows and bookings;
// this DO only
// (a) holds the WebSockets of connected clients (hibernation API, so idle
//     sockets cost nothing),
// (b) fans out "live set changed" ticks to learners and "ring" takeovers to
//     volunteers,
// (c) tracks transient ring state (claimed -> answered | declined | timeout)
//     and enforces the ring timeout with a DO alarm: an unanswered ring cancels
//     the snap booking in Turso and tells the waiting learner, even if every
//     browser tab died.
//
// Failure posture: everything here is best-effort enhancement. If the DO is
// cold, broken, or unbound, the product degrades to what shipped before it
// existed (push + email + loader revalidation), never worse.

The booking is committed first. The Worker then notifies the Durable Object and waits at most about two seconds; if that fails, the booking stands.

Three choices carry the design. There is one instance per environment, so every connected person reaches the same object and there is nothing to route. The sockets use the WebSocket Hibernation API, so an idle connection does not keep the object awake, and the keep-alive ping is answered by the runtime through setWebSocketAutoResponse without waking the object at all. And the database stays the source of truth: the object sends change signals, and the page loads persisted outcomes from the database. Three kinds of state are kept apart. The booking in Turso is the durable record. Pending rings and their deadlines are kept in the object’s own durable storage. The open sockets, and who is online at this moment, are transient.

Running one object for months showed us three mistakes of our own.

A value that could only grow. The same object keeps watch over sessions that may need a rescue, and for each watch it kept a list of the five-minute boundaries it had already checked. Retirement measured a watch’s age from the newest boundary, and the newest boundary moved forward five minutes on every check, so on-time checks never let the watch grow old enough to retire. Production writes failed with SQLITE_TOOBIG on 6 August, including the alarm’s writes. Removing old entries needed a write too. The fix measures age from a fixed anchor stamped once and caps the history when it is read, so a later write can store a smaller value. The errors had stopped before the fix shipped; the fix stops the value from growing.

Lost rings. Durable Objects run with an input gate that holds back new events while the object waits for its own storage; external I/O allows other events to interleave. Our alarm read the map of open rings, waited for the database and for the notice that tells the volunteer they missed the request, and wrote the whole map back. That notice is in the app and mirrored to Web Push, rather than email. A ring that arrived during that wait could be overwritten. We found this in code review and tests, not in a measured production booking. The fix has two parts: it reads the map fresh, and it applies each decision only to the generation of the ring it evaluated. We did not reach for blockConcurrencyWhile, because on the single object that serves everyone it would have blocked every socket message while the alarm waited. The test harness hid it: its fake storage handed back the live object, so the old test passed with the bug in place. The revised fake returns detached copies so the test can expose stale writes.

Ghost sockets. In a test run on AWS, with the object logging every socket it accepted and closed, each page load by a mentor opened two sockets. React’s StrictMode starts every effect twice in development, and the first socket was dropped while it was still connecting. Calling close() on a connecting socket abandons it in the browser only, so the object never heard about it. Over the run it accepted eight sockets and saw four close, and it kept answering that the mentor was online for 19 seconds and more after the last real socket had closed. The page now lets such a socket finish its handshake and closes it then.

Two smaller lessons came from the same object. We changed the integration to keep personal identifiers out of logs. And when the database times out while a socket is being opened, the Worker now answers 503 instead of 500, so the browser knows to try again later instead of treating it as a bug.

Durable Objects also shaped how we test. When we set this up, Preview URLs were not generated for a Worker that implements a Durable Object, so every agent and developer who needed a live copy of the platform got a Worker of their own. They accumulated, so on 22 September we deleted the preview Workers that nobody was using any more.

The same day, Cloudflare released Worker Previews, which give every Preview a Durable Object namespace of its own. They need Wrangler 4.135.0 or later, and we already run 4.144.0. We are looking at them now. One thing we will have to plan around: a service binding from a Preview calls the production deployment of the Worker it binds, so a Preview of Language would reach production Mentorship and Evenhall.

Running the Worker next to the database

By September, TEEI Language was quick in Western Europe and slow almost everywhere else. Over the week before 21 September, we measured the Worker’s own time on page requests and grouped it by the data centre where the request came in. Request-weighted regional averages of the data centres’ median Worker times were 247 milliseconds in Western Europe, 1,193 in the western United States, 1,656 in Japan and 2,723 in Oceania. The code was the same everywhere, but the mix of pages was not controlled. In the week before the release, a page averaged about 17 database trips, some concurrently; the critical path was the number of dependent stages. Summed database-fetch time divided by trips averaged about 123 milliseconds, including database work and queueing as well as distance. The dashboard learners open first made about 72 trips.

Distance was the part we could act on. A Worker runs in the data centre closest to the person by default, and our database has one location, in AWS Europe (Ireland). Every trip crossed that distance again, so the further away someone was, the more each trip could cost them.

We had tried Smart Placement on staging at the end of August. Smart Placement needs consistent traffic from several locations before it decides where to run a Worker, and our staging Worker carried about 175 requests in a week, nearly all through Oslo. The trial never reached a decision. We knew the database’s region, so on 21 September we committed a Placement Hint. The commit did not move production traffic; the release that did went out on 22 September at 05:30 UTC for Language and 05:34 UTC for Mentorship. This is the configuration, with its comment from 21 September, written from figures that still counted API calls with pages:

// Targeted placement (2026-09-21). The only database is the Turso primary in
// aws-eu-west-1, and a page averaged 13 to 15 sequential-ish round trips to it
// (72 on /home) at ~123 ms each from the data centre that served the user. This
// runs the fetch handler next to that primary instead. Static assets are
// unaffected (served nearest the user, this Worker is assets-first), the
// Durable Object places itself, and scheduled() is not a fetch handler. Proof
// of placement is the `cf-placement` response header (remote-<location>) and the
// per-request dbTotalDurationMs/dbRoundTrips in Workers Logs; rollback is
// removing this key.
"placement": {
  "mode": "targeted",
  "region": "aws:eu-west-1"
},

Only the fetch handler moves. Static assets are still served from the data centre nearest the person, the Durable Object places itself, and scheduled jobs are not affected. We checked the move on 14 October 2026: requests to Language from Oslo answered with cf-placement: remote-LHR, which means the Worker ran in London, and a cf-ray that ended in OSL, where the request came in. Rolling it back means deleting four lines. The mode line is optional; region on its own sets the same hint.

Before the change, a preview Worker let us measure what the hop itself costs: at most about 130 milliseconds for a request that has to travel from Oslo to London and back. That price is paid once per request, while the database trips it shortens are paid many times.

The change went out on 22 September as part of a larger release, and the first 41 minutes looked remarkable: a median page of 95 milliseconds, against 433 the week before. That did not hold over whole days, and we do not quote it as the result. We measure the result over a whole week on each side, on the same kind of request: a successful page request that reads the database, timed inside the Worker. In the seven days to 14 October, across about 132,000 such requests on Language, the median was 177 milliseconds, against 433 in the seven days before the release. The 90th percentile was 462 milliseconds, against 1,747. For Mentorship, the median over the same week was 147 milliseconds.

Placement, loader changes and indexes shipped together. The before-and-after figures describe that combined release and the traffic in each measurement window; they do not isolate the effect of placement. The later window averaged 55 milliseconds per database trip, against 123; a shorter distance is a plausible reason for part of that, alongside changed SQL work and indexes. Code removed trips. A request-local read coalescer batches eligible SQL reads into fewer transport calls; it does not deduplicate identical statements. Loader rewrites and indexes reduce other database work, and the later window averaged 5.8 trips per page against 17.1. The page-time change is the combined result, with later code and a different traffic mix. On a Samsung phone, the response headers of the dashboard learners open first took 3.4 to 4.9 seconds to arrive before the release and 1.0 to 1.5 seconds after it, although the before was measured on Wi-Fi and the after over a mobile network, so that comparison is only indicative.

Cloudflare’s own analytics show the same change from the outside, on a different measure: the wall time of every invocation, including API calls, assets and scheduled jobs, not only pages.

The medians here are lower than the page times above because they take in everything the Worker answers, and on 19 and 20 September, the two days of the burst in the daily chart near the top of this article, the TEEI Language median drops to about 23 milliseconds. The CPU line hardly moves all month and sits far under the wall line, which is consistent with time spent waiting on calls rather than computing. After the 22 September release the distance between wall time and CPU time narrows on both Workers, and that distance is what the placement change was made to close.

We considered enabling Workers Cache and decided against it for personalised pages. Without explicit private or no-store directives, enabling that cache could allow a personalised response to be reused. A contract test keeps the cache setting absent from the Language Worker configuration.

Queues behind Evenhall

Evenhall sends email, takes in applications from companies and syncs employee lists from HR systems. None of that should keep the person who pressed the button waiting, so Evenhall uses three Cloudflare Queues: applications, email and HR system sync. Each has a dead-letter queue.

This is where one of those messages starts. A manager pastes the employees’ email addresses and presses Send invitations. For each address the Worker saves an invitation and an outbox row in separate database writes, then publishes a queue message that references the row. If publication fails after both records have been written, the invitation and its outbox row remain saved. This is not the same atomicity guarantee as a transactional outbox written with the domain change, and recovery differs between message families.

The queue settings show the retry and concurrency budgets. Evenhall had an earlier name when this code was written, and the queue names still carry it:

Adapted for publication.

# The message references the outbox row and carries envelope metadata.
# Recipient details and email content stay in the database.
# …

[[queues.consumers]]
queue = "teei-csr-email-production"
max_batch_size = 10
max_batch_timeout = 5
max_retries = 8
dead_letter_queue = "teei-csr-email-dlq-production"
max_concurrency = 2

[[queues.consumers]]
queue = "teei-csr-email-dlq-production"
max_batch_size = 10
max_batch_timeout = 5
max_retries = 3
max_concurrency = 1

Queue messages identify work held in the database rather than carrying the recipient list or message content. The email envelope holds the outbox row’s ID, its kind, a version and optional trace identifiers, never an address, a name or a list of employees. The HR sync queue follows the same rule, and its comment says “roster data never enters Queues”. Before sending, the consumer reads the row and acquires a conditional claim. That keeps contact data out of the queue and keeps the database as the record of attempts, provider acceptance and delivery receipts. A row with a provider message ID or a final status is acknowledged and skipped. Consumers use guarded claims and provider idempotency where supported to handle retries. The claim admits one attempt while its lease lasts; a repeated attempt also needs the provider’s idempotency key and an unchanged request within the provider’s retention window. That reduces duplicate sends within those conditions; it is not exactly-once delivery. On exponential retry paths the delay doubles from one minute and stops growing at an hour. A message retried after every failed delivery reaches its dead-letter queue after eight retries, and its family handler runs before acknowledgement. The dead-letter consumer has a separate retry budget for its own failures. Recovery and terminal handling differ between message families. A final status on an outbox row, a message in a dead-letter queue, the provider accepting an email and the email reaching its recipient are four different things.

Two problems taught us more than the happy path did.

The first was a database client created once and shared between requests. On 9 September, in our browser test lane with several clients in parallel, requests hung and the runtime warned of a “promise resolved from a different request context”; our evidence documents no production incident. The fix that day moved the client cache into each request.

The second was silence. Evenhall’s daily sync went stale on 12 July 2026, and nobody was told for two months. Now a sweep runs every five minutes and sends an alert when a scheduled job goes stale. It has been on in production since 28 September.

The limits we hit

Workers have limits, and we have hit a fair number of them. Each one taught us something about our own code.

128 variables. On Workers Paid, a Worker can carry 128 environment variables and secrets together. In September, Language carried 34 variables and 94 secrets, 128 in all, and adding one more secret was refused at the moment of the write, with error 10055: “found a total of 129”. We added a check that counts them before a deploy, moved feature flags into the database, and combined related secrets into one JSON secret. On 12 October the same limit refused two more writes to Language, and since then each change has made room before it added anything. On 15 October 2026 we counted 25 variables in the committed production configuration and 97 secret names live on Cloudflare, 122 against the limit of 128.

Size. For months, our size check compared the uncompressed bundle with a number someone had picked, and the number was raised 17 times. On 7 August we found that the limit Cloudflare applied then was the compressed size: 3 MB on the Free plan and 10 MB on Paid. Our August investigation concerned the compressed-size limits then in force. Cloudflare removed those limits on 4 September 2026; the platform now checks a 64 MiB uncompressed bundle limit. Our smaller gzip ceiling remains an internal release budget. The comment on our check records that start-up time matters too, and that we had not measured ours. Language’s measured build on 15 October 2026 was 4.19 MB compressed, below the 4.25 MB ceiling we set on 13 October when Spanish and Portuguese were added.

Twenty thousand files. Astro’s fallback for other languages wrote one redirect page per language for every city page on the website, 12,115 small files in all, which took the build to 22,618 files. Cloudflare documents 20,000 static asset files on the Free plan and 100,000 on Paid; we enforced our own 20,000-file ceiling for the website. Pruning 12,115 generated fallback pages brought that build under our release budget; the records do not establish that Cloudflare rejected it. Astro still generates those pages, and a step after the build removes them before it ships. In the July investigation, the rule returned a 301 in development but the production build returned a 404 for the affected paths. Our inference was that Astro’s language routing answered before our middleware; we did not trace that ordering through the framework. A dynamic redirect rule on the zone runs before the Worker, so that is where those redirects live now.

Six connections. A Worker invocation can have six connections waiting for a response at once, and further calls wait for one of them. In August our scheduled ticks still started every job at the same moment, 22 of them within three seconds. On 12 August one quarter-hour tick read 8.2 million rows, and while it ran, database reads for real people timed out. Since then the scheduler starts jobs four at a time. That was not quite enough. On 3 October four quarter-hour ticks had database writes that aborted after 10 seconds without ever leaving the Worker, because the heaviest job and the smaller checks on the same trigger could still start in the same second. Since 11 October the trigger that carries that job runs two at a time.

A shared promise. Language used to hang now and then, for 80 seconds to almost 9 minutes, 31 to 44 times a day. Our diagnosis, reached by ruling out every other explanation rather than by catching it in the act, was a promise that was created once and shared between requests: when the request that created it was torn down, the promise could never settle for the others. The fix on 15 September gives every caller its own 1.5-second budget. From 8 to 12 October, Workers Logs show no hangs of that kind on Language or Mentorship.

Deploy and rollback

We deploy often. Language has had 235 production deploys since 27 June 2026, and Mentorship 107. Every production deploy is written to a deploy log and probed. If the probe fails, the release script restores the previous version on its own, as long as that version still works with the database schema; when a migration has made the old version unsafe, we go forward with a fix instead.

That has happened twice, and both times the probe was the problem. On 22 July a Language deploy failed its probe at 07:10:12 UTC, because the probe expected a heartbeat the new version had only just added and that had not yet been seen; the previous version was back six seconds later, and a release with the corrected probe went out at 08:05 and passed. On 28 July a Mentorship deploy rolled back because the probe still expected the closed service Mentorship had been before it opened on 24 July.

Seen week by week, the deploys also carry the story of the limits above.

The busiest week, with 45 deploys, about a fifth of the 235, began on 21 September, the day we committed the Placement Hint; it reached production the next morning. The week of 3 August had none.

Rolling back is one command because Workers keep versions. In a rehearsal on a staging Worker on 2 July 2026, the command took 4.3 seconds to put the older version on all traffic, and 4.1 seconds to restore the newer one. That rehearsal does not time a production rollback today, which first checks the live schema, then reads the deployments back and probes. For an urgent fix there is a separate lane that never deploys whatever happens to be newest, and that stops on a failed probe and leaves the rollback decision to that same schema check: it uploads one version with wrangler versions upload, and deploys exactly that version by its ID with wrangler versions deploy. The approval for it is bound to the checksum of that artefact. A checksum identifies what was uploaded, not what production serves, so as with our other releases we then read the deployments of that Worker to check which version is serving traffic.

We keep logs long enough to answer a question asked next quarter, not only one asked this week, and Workers Logs keep a few days. So Language and Mentorship send their logs to R2 through Logpush, and the bucket held 318,916 log objects in the 14 October 2026 snapshot. Logpush needs a job on the account and logpush: true on each Worker. Turning on Workers Logs does neither, and that is the trap: with observability on and logpush off, the Logpush job exists, reports no error and delivers nothing from that Worker. The two settings now sit next to each other in the configuration, with a comment that says so.

Project Galileo in practice

The zone for TEEI’s main domain, which also carries the platforms’ hostnames, runs on Cloudflare’s Business plan through Project Galileo. Most of what that plan does is invisible, so here is what it did in three days.

From 11 to 14 October 2026, Cloudflare’s analytics count about 1.03 million requests across that zone from people and bots. The Cloudflare Managed Ruleset blocked about 14,000 of them, or 1.4 percent. More than 10,500 matched the rule for version control files such as .git. Most of the rest were requests for common configuration files, fake search engine crawlers, a known code injection in a JavaScript template library, and probes for WordPress and DotNetNuke. The site has not run WordPress since it moved to Astro. We turned on the Managed Ruleset on 22 September. Probes that tried script injection and SQL injection were answered 200 before and 403 after, while 38 legitimate requests we tested answered exactly as before.

We read what those requests asked for from the rules they matched.

The same zone carries the redirects that keep old links alive. The website used to run on WordPress, and other sites still link to its old addresses, among them news sites and job boards in Ukraine. On 12 October we found 39 of those addresses that answered 404 and added them to a Bulk Redirect list, 78 entries with both slash forms. All 117 checks we ran answered 301 in one hop, kept the query string, and ended on a page that answered 200.

What this means for TEEI’s work

TEEI’s programmes are for people who are often on a phone, on a mobile network, in a country far from our database. In the week before 21 September, the regional summary of Worker time was 2.7 seconds for Oceania, against a quarter of a second in Western Europe. Placement and fewer dependent reads address that repeated distance, but we have not measured that regional gap with a comparable after-window. Keeping the platforms on one account means a small team sees all of them in one place: one set of analytics, one set of logs and one view of every Worker.

The numbers in this article come from the same analytics, logs and configuration we use to run the platforms, and the reports companies prepare in Evenhall are built from data that passes through the same Workers.

A counter readers can check

The public statistics that the Language Worker publishes said, on 14 October 2026 at 13:38 UTC, that 12,517 sessions and 9,435.5 hours of conversation have taken place across TEEI Language and TEEI Mentorship, with members in 148 countries.

The website shows the same counter. Its impact page pins each published figure, so that a citation stays true, and under the record it prints the counter as it stands now. The website’s Worker asks the Language Worker for that number, so the reader’s address never reaches the counter, and reuses each reading for 30 seconds. The capture below was taken two hours after the reading above, so its live line is a little higher.

Thank you, Cloudflare

Through Project Galileo, Cloudflare gives TEEI the Business plan on the zone for our main domain and the firewall rules that come with it. That support protects the website that people from Ukraine, Latin America and the Caribbean, and volunteers from many countries, open first. Thank you, Cloudflare.

Verify this

• See where our Worker runs. From a terminal, send curl -sI https://language.theeducationalequalityinstitute.org/. The response carries cf-placement, which shows the Worker ran next to our database, and cf-ray, whose last three letters name the data centre that received your request.
• Read the live numbers. The public statistics for TEEI Language and TEEI Mentorship are answered by the Language Worker as you open them.
• Compare them with the website. The impact page pins each published figure and prints the live counter under it.
• If you are building with Cloudflare or with TEEI, talk to us and we will walk you through the code.

Appendix: engineering notes

For developers who want the detail behind the sections above. The excerpts reflect the code reviewed for the 14 October 2026 account snapshot, with ticket numbers, identifiers and unrelated lines removed, and long comments rewrapped to fit the page. Technical corrections were checked against source again on 17 October.

The Durable Object binding

The live presence object is declared once and migrated as a SQLite-backed class. Named environments do not inherit it, so the staging block repeats the binding.

"durable_objects": {
  "bindings": [{ "name": "LIVE_PRESENCE", "class_name": "LivePresence" }]
},
"migrations": [{ "tag": "v1", "new_sqlite_classes": ["LivePresence"] }],

Keep-alive without waking the object

The constructor tells the runtime to answer the browser’s ping itself. A hibernating object is never woken just to say that it is still there.

class LivePresenceBase extends DurableObject<AppEnv> {
  constructor(ctx: DurableObjectState, env: AppEnv) {
    super(ctx, env);
    // Keepalive without waking from hibernation: the runtime answers the ping
    // pair itself. Browsers use this to detect a dead socket and reconnect.
    this.ctx.setWebSocketAutoResponse(new WebSocketRequestResponsePair(LIVE_PING, LIVE_PONG));
  }
}

Service bindings with named entrypoints

Language calls Mentorship and Evenhall without a public URL. Evenhall’s binding names the class it exports, so Language can reach only that one entrypoint.

// Language is the only cron owner. Its narrowly authenticated scheduled-mentor
// release enters the Mentorship Worker through this direct binding, not the
// public hostname.
"services": [
  {
    "binding": "MENTORSHIP_SCHEDULED_WAVE_SERVICE",
    "service": "teei-mentorship-rr7"
  },
  {
    "binding": "CSR_APPLICATION_ACTIVATION",
    "service": "teei-csr-cockpit-rr7",
    "entrypoint": "CsrApplicationActivation"
  },
  // …
],

Logs that outlast the window

The comment is the lesson.

"observability": {
  "enabled": true
},

// Two DIFFERENT switches, and having only the first is the trap (2026-07-27).
// `observability` turns on Workers Logs, which is a rolling window of a few
// days that cannot be queried afterwards; `logpush` is what actually lets the
// Logpush job ship those trace events out to R2. With observability on and
// logpush off, the job exists, reports no error, and delivers nothing forever.
// …
"logpush": true,

Scheduled work from a queue

The scheduler starts jobs four at a time instead of all at once. The comment records the measurement that led to it.

/**
 * How many cron jobs may hold a Turso connection at the same instant.
 *
 * Every job used to be dispatched into its own waitUntil the moment the tick
 * fired, so all 22 of them started within three seconds and competed for one
 * primary in eu-west-1. Measured 2026-08-12: a single fifteen-minute tick read
 * 8,2 million rows, the baseline between ticks was ~200 000 per five minutes,
 * …
 *
 * The work is unchanged; only its shape is. Four at a time spreads the same
 * jobs across the window instead of stacking them into one burst, and the
 * tick's wall time is dominated by its slowest single job either way (digests,
 * 139 s), so nothing is delayed materially by the queue.
 */
const CRON_DISPATCH_CONCURRENCY = 4;

A switch that does not travel with the code

During maintenance on the database, an operator’s switch in KV stops the Language and Mentorship Workers from starting new writes. It is a KV key and not an environment variable for a reason that belongs to Workers. In the words of the code, “a var or a secret travels with the Worker version”, so a version prepared in advance would carry the switch as it stood when the version was made. A key in KV is read at the moment of the write, whichever version is running. The switch stops new work from starting, and it does not promise more than that: a change in KV can take a minute or more to reach every location.