One Cron Owner, Still Duplicate Ticks

We schedule two products from one Cloudflare Worker. In October 2026 we recorded four periods in which our ticks arrived twice, and we have not established the cause. We give the schedule one owner, guard effects inside individual jobs and check their outcomes separately from tick delivery. On 28 June we recorded a fresh heartbeat for the scheduler while one of its jobs had never run. If you deploy the same code to more than one Worker, you need to decide who owns the schedule, how you prevent duplicate effects and how you find jobs that never start.

TEEI Language and TEEI Mentorship are two products on one platform, built from one React Router 8 codebase and deployed as two Workers; How TEEI runs its platforms on Cloudflare Workers describes the account as a whole. Behind them runs scheduled work: meeting rooms created ahead of a session, reminders, expiry of unanswered requests, digests, health probes, and since 13 October the scheduled jobs of Skills Academy.

Built through volunteer contributions. TEEI’s software platforms are built through volunteer work, with NOK 0 spent on paid software development. Infrastructure and third-party services are separate from that figure.

One owner for the schedule

A Cron Trigger calls the Worker’s scheduled() handler with controller.cron, the expression that fired. Several triggers on one Worker all call the same handler, and the value of controller.cron “must match character-for-character, including spacing” the string in your configuration. Cloudflare’s own example switches on it, and so do we.

The Language Worker declares three expressions and selects a job set by the one that fired. The five-minute expression runs the jobs that cannot wait a quarter of an hour. The quarter-hour expression three minutes past the quarter runs the heaviest job in an invocation of its own, and since 11 October also four read-only probes. Everything else is the maintenance tick.

Adapted for publication.

"triggers": {
  // */15    = the maintenance set (attendance, expiry, recurrence, win-back, health).
  // 1-59/5  = meetings created ahead of time, reminders, push retries and Skills room
  //           clean-up. Reminders ride this tick ONLY: adding them back to */15
  //           duplicates provider work and creates races on the same rows.
  // 3-59/15 = lifecycle deliveries, isolated in its own invocation so the
  //           heaviest job gets a full budget.
  "crons": ["*/15 * * * *", "1-59/5 * * * *", "3-59/15 * * * *"]
},

We deploy the same code to Mentorship with a separate configuration and an explicitly empty cron list.

Verbatim excerpt.

  // Mentorship is web-only. Language remains the sole scheduled owner.
  "triggers": {
    "crons": []
  },

Some scheduled work still belongs to Mentorship’s data. Skills Academy runs on the Mentorship Worker, and since 13 October its scheduled jobs run there too. We schedule them from Language’s trigger and call Mentorship through a service binding, which has no public URL (What We Bypass with Cloudflare Service Bindings covers the binding). There is no local fallback: running the job inside Language would claim Mentorship’s rows under Language’s feature flags, so a failed call goes to the logs and to Sentry. We record a completed run only when Mentorship reports a job outcome; a response without one is a dispatch failure, and a job that ran and degraded is a completed run with a degraded result.

Ownership has to survive configuration changes, so we guard it in three places.

• One owner, tested. A test reads the expressions from both Workers’ configuration files rather than from a hand-typed copy, because a hand-typed copy would let a fourth expression added to the Language configuration run the whole maintenance set a second time with every test green. As of 15 October 2026, it requires the three expressions in Language production and staging, pins the five declared Mentorship cron arrays to empty lists, and requires every other declared Language cron array to be empty.
• An opt-in per Worker. We require a per-Worker opt-in for reminders, no-show handling and the main maintenance set. We added the switch after a production deployment on 26 June left the scheduler armed against demonstration data.
• An explicit empty list in every other environment. On 3 August a preview environment that declared no triggers of its own came out with production’s three expressions; our upload guard refused that first build, which is the only reason we noticed. We observed this inheritance in our build; we cannot quote it from the environment documentation. Cloudflare does document that an empty crons list removes every trigger, and that “commenting out the crons key will not disable a Cron Trigger”. So every environment that should not run the scheduler declares "crons": [].

Inside one tick

Until August, every job started in its own waitUntil the moment the tick fired, 22 of them within three seconds, all against one database primary. On 12 August we observed database reads for people using the platform timing out during these bursts, and we changed the dispatcher to a bounded queue; the platform note gives the measurement.

Before the queue, a deployment without database access refuses to run the scheduler. We then start a Sentry check-in for the expression that fired, a heartbeat for the handler itself and a ping to an external dead-man’s switch outside Cloudflare, and none of them blocks dispatch: they can remain in flight while the jobs run.

Adapted for publication.

const drainCronQueue = async (concurrency = CRON_DISPATCH_CONCURRENCY): Promise<void> => {
  const queued = cronQueue.splice(0, cronQueue.length);
  let next = 0;
  const workers = Array.from(
    { length: Math.min(concurrency, queued.length) },
    async () => {
      while (next < queued.length) {
        const task = queued[next++]!;
        await task();
      }
    },
  );
  await Promise.all(workers);
};

Each invocation drains a snapshot of its own queue through a fixed number of lanes: as of 15 October 2026, up to four, and up to two on the lifecycle tick. The lanes share one cursor and advance it before awaiting a task, and the job wrapper catches failures, so one failing job does not stop its lane. Two concurrent invocations have separate queues, so we limit concurrency within an invocation and rely on per-job guards when invocations overlap. A test goes red if the fan-out comes back, and every branch that queues work registers its drain under one waitUntil; a test for the video-deletion job once found a queue that was filled on one path and never drained.

Most queued tasks are local route jobs. We call the React Router request handler in process for them, using the same feature-flag overlay as page requests. For each call we create a fresh RouterContextProvider and populate it with the flagged environment, execution context and logger, the same typed context contract the web loaders use. One job is queued as a direct function call instead, and the Skills Academy jobs go to the other Worker.

Around each local route job, three choices are deliberate. Completion and outcome are separate: a job that answered 500 did finish, so its “last completed” marker stays fresh while we record its outcome as degraded. We await a bounded outcome-write attempt inside the drain’s existing waitUntil, because on 2 July a heartbeat write that we registered under a second, nested waitUntil from the last of several concurrent jobs was lost on every tick, and that job looked as if it had never run. If the write still fails after its retries, we log the completion as unrecorded rather than as a normal completion. And the wrapper never throws into scheduled(): we log a failing job, send it to Sentry, and the rest of the tick carries on.

A repeated tick needs a guard in the job

A single owner does not prevent a repeated delivery. Our ticks arrived twice in four periods in October 2026: on 2, 3 and 4 October and from 7 to 8 October. We have not established the cause. We guard each job’s logical effect, such as a booking, recipient or reporting period; the raw scheduledTime alone does not identify one. Here is the smallest of them, in the job that expires requests a volunteer never answered:

Adapted for publication.

it('two runs in flight expire one stale request exactly once', async () => {
  const id = await seedRequested({ expiresAtMsFromNow: -60_000, slotId: 'slotTwice' });

  const [first, second] = await Promise.all([expireStaleRequests(ENV), expireStaleRequests(ENV)]);

  expect(first.expired + second.expired).toBe(1);
  const stored = await db.execute({ sql: 'SELECT status FROM bookings WHERE id = ?', args: [id] });
  expect(stored.rows).toEqual([expect.objectContaining({ status: 'declined' })]);
  // … one slot released, one set of notices
});

The test runs against a database built from the real migration chain, not a hand-written copy of the schema. The guard starts with one condition on one UPDATE:

Adapted for publication.

// The booking write is guarded on status = 'requested' so a concurrent accept/decline
// that already moved the row wins (rowsAffected 0 -> skip).
{
  sql: `UPDATE bookings
           SET status = 'declined', terminal_reason = ?, updated_at = ?
         WHERE id = ? AND status = 'requested' AND request_initiator = ?`,
  args: [terminalReason, now, bookingId, requestInitiator],
},
// … after the batch commits:
if (Number(results[transitionIndex]?.rowsAffected ?? 0) !== 1) {
  result.skipped += 1;
  continue;
}

The status condition gives one run the transition; in the same batch, the slot is released only while still held for this booking, and notice rows carry a unique key. After the commit, the losing run sees zero rows touched and skips its reporting.

As of 15 October 2026, sixteen test files each run a job twice at the same moment with Promise.all and require the effect once, and we showed each one failing with its guard removed. A passing concurrent run alone would not show that the guard made the difference; removing the guard must break the same persisted-effect assertions.

We test three boundaries separately. The configuration test reads declared arrays. A dispatcher proof runs the real scheduled() with the expressions read from configuration, but replaces the router and heartbeat writers, so it proves which jobs were selected, not that a row was written; in a negative control on 12 September 2026, adding a fourth expression, */7, made the job sets overlap and turned it red with 26 collisions. The concurrent job tests inspect persisted effects. None of them proves which triggers Cloudflare actually fires. For that boundary we read the fired expression in Workers Logs, and that is where our next lesson came from.

Changing an expression is not one deploy

On 5 October we moved the five-minute expression from */5 to 1-59/5, one minute off the five-minute grid.

What the move taught us is about deploys. In our release lane, we upload a version and then deploy it (wrangler versions upload, then wrangler versions deploy), and neither command changes the triggers. A plain wrangler deploy would replace them with the configuration’s list; our lane changes them separately with wrangler triggers deploy. So the new code went live while production still fired the old expression. Because the handler compares controller.cron character for character, every old */5 tick would have matched neither named expression and run the whole maintenance set, 30 jobs instead of 4, while the meeting rooms and the reminders never ran. A test with a negative control now proves that. The handler accepts both expressions for one release:

Verbatim excerpt.

export const PRECREATE_CRON_PREVIOUS = '*/5 * * * *';

/** Whether a fired cron expression is the 5-minute tick, under its new or its previous name. */
export function isPrecreateCron(cron: string): boolean {
  return cron === PRECREATE_CRON || cron === PRECREATE_CRON_PREVIOUS;
}

Cloudflare documents that changes to triggers “may take several minutes (up to 15 minutes) to propagate”. We measured longer. On 6 October, Cloudflare’s schedules API returned the new expression for the production Worker from 07:56 UTC, and the old expression kept firing for another 44 minutes. On staging on 5 October, the first 1-59/5 invocation came more than four hours after the deploy. The schedule the API reports is not the schedule that fires. So our rule for removing the alias is the fired expression: Workers Logs must show $workers.event.cron as 1-59/5 and no */5 tick for a full day, on production and on staging.

When a tick runs out of time

A scheduled invocation has at most 15 minutes of wall time, by Cloudflare’s limits. On 24 September two quarter-hour ticks in a row reached it and were cut off at 900 seconds, and every job still waiting in the queue behind them was lost. Three fan-out jobs used serial recipient loops. In measurements on 28 and 29 September 2026, the “available now” fan-out took over nine minutes on its own.

On 29 September we replaced the serial loops with a bounded sweep: four recipients in flight. The dispatcher bounds the jobs in an invocation; the sweep bounds the recipients in a job. Two of the notice jobs also stop starting work after four minutes and leave unstarted items for the next tick. Tests pin both stops: after a failed lane or a spent budget nothing new starts, and work already in flight settles before a failure is rethrown. In an adaptively sampled read of Workers Logs for the seven days to 15 October 2026, the longest maintenance tick in the sample used about a fifth of the wall-time ceiling.

None of this saves the dispatcher’s own queue, which lives in memory and is lost if the invocation is cut off. Resuming safely depends on each job keeping its unstarted work durably.

Knowing when a job did not run

A job that throws is easy to see; a job that never starts leaves nothing behind. On 28 June the handler’s heartbeat was fresh while one job had never run. Since then we keep a heartbeat for each job in a monitored register, with markers for the last run, the last success and a degraded outcome, and a self-check on the maintenance tick warns in Sentry when a job that should run has gone stale. Around it sit three wider checks: an external dead-man’s switch outside Cloudflare, which still works if the whole account is what stopped; a monitor Worker that reads the health of both programmes every five minutes and escalates; and a Sentry cron monitor per expression, which says that the tick started and nothing more.

That last check once raised a false alarm. On 27 September Sentry declared the 11:00 UTC maintenance tick missed, while Workers Logs show that it ran for 583,504 milliseconds, under the 900-second ceiling, and every job had run. We called captureCheckIn at the start of the tick, but in @sentry/cloudflare that only queues the check-in; the transport sends what is queued when the client is flushed, after every waitUntil of the invocation has settled. So the check-in left when the tick ended. We now start sending it straight away:

Adapted for publication.

try {
  captureCheckIn(
    { monitorSlug: monitor.slug, status: 'ok' },
    {
      // The expression that actually fired, so the monitor expects the trigger Cloudflare runs.
      schedule: { type: 'crontab', value: cron },
      // … margin, thresholds and time zone
    },
  );
  const transport = client.getTransport();
  if (!transport) return undefined;
  const sent = Promise.resolve(transport.flush(CHECK_IN_SEND_TIMEOUT_MS)).then(
    (drained) => ({ drained: drained === true }),
    () => ({ drained: false }),
  );
  ctx.waitUntil(sent);
  return sent;
} catch {
  // Observability cannot stop a scheduler tick.
  return undefined;
}

We flush only what is already queued; the tick’s later events still leave at the end. A real-SDK regression test holds a scheduled job open and requires one check-in before that job settles, and still exactly one after the final flush. A second test makes fetch reject and requires every promise registered under waitUntil to settle anyway, because a failure to reach Sentry must not reject the scheduler tick. Until 27 September, check-in lateness mixed scheduler start delay with execution time, so we do not read our older check-in data as start delay. How We Join React Router Errors Across Browser and Worker covers the rest of our Sentry setup.

In Evenhall, partner companies report what their employees did through TEEI’s programmes. Its daily jobs run from a small scheduler Worker that, unlike Language’s, rethrows a failed dispatch so that it shows in Cloudflare’s invocation status. Evenhall’s daily sync went stale on 12 July, and for two months the scheduled jobs did not reach the Evenhall Worker. The schedule was intact and the scheduler did fire; in the words of our runbook, “it fails on the outbound call”. In a six-day window we measured on 6 September, 27 of 29 invocations ended in scriptThrewException, and the Evenhall Worker received none of their requests. That window is a sample and does not show every tick of the two months. The scheduler we had deployed had no service binding, and we repaired delivery with production deploys on 17 September.

The silence had a cause of its own. When we wrote the alert row for a job that had never run, we also gave it a “last run” of that moment, so the job reported healthy for ever. We fixed that on 6 September, the same afternoon we split reading the health of the jobs from writing alerts about it, and alerting for stale jobs has been on in production since 28 September. The comment on that switch says it plainly: “nobody was told for two months”. Two things were wrong at once, the binding and the silence, and we have not measured them apart.

A test now starts with no recorded runs, runs the alerting step and requires the run history to stay empty and the job to remain unknown. We evaluate health from recorded rows and a clock, without a database client, so reading health cannot manufacture evidence that a job ran.

When this pattern is the wrong choice

• One owner for two products. The Language Worker’s ticks carry both products’ jobs under one 15-minute ceiling, and Mentorship cannot be scheduled on its own. If your two Workers have different owners or different release cadences, give each its own schedule and its own data.
• Guards per job. Overlap protection lives in each job, not in the dispatcher, so every new job needs its own guard written and proven. If a job’s effect has no key, a lock can limit overlap and a queue can control delivery, but neither replaces an effect identity and a documented retry and recovery rule. Why We Freeze Email Payloads on Cloudflare Queues and Durable Object alarms: stale decisions, moving deadlines describe the two approaches we use elsewhere.
• Throwing or not. Language’s dispatcher catches a job’s failure, records it and alerts, so a green Cloudflare status does not show that the tick’s jobs succeeded. Evenhall’s scheduler throws, so the status is a signal, but one failing call ends the invocation. Choose by which signal you will actually watch.
• We have not measured what it would cost or save to spread this work over more Workers. There is no comparison to report.

Verify this

This is a minimal example of our own, not code from our platform. It reproduces two invocations of the same job at the same moment, not the platform’s own double delivery, and it shows where duplicate suppression ends.

1. Create a D1 database, bind it to a Worker, and create two tables. The unique key in claims identifies one effect for one job; effects records what happened:

   Adapted for publication.

   CREATE TABLE claims (job TEXT NOT NULL, effect_key TEXT NOT NULL, UNIQUE (job, effect_key));
   CREATE TABLE effects (job TEXT NOT NULL, effect_key TEXT NOT NULL, at INTEGER NOT NULL);
   

2. In scheduled(), key the claim on the effect, here one digest per UTC day, not on the delivery. Claim first, and perform the effect only if the claim inserted a row.

   This example demonstrates duplicate suppression, not crash-safe execution. A process that stops after claiming and before performing the work can leave the work undone; the failure case in step 5 demonstrates that boundary.

   Adapted for publication.

   export default {
     async scheduled(controller, env) {
       const key = new Date(controller.scheduledTime).toISOString().slice(0, 10);
       const claim = await env.DB
         .prepare('INSERT OR IGNORE INTO claims (job, effect_key) VALUES (?, ?)')
         .bind('digest', key)
         .run();
       if (claim.meta.changes !== 1) return;
       await env.DB
         .prepare('INSERT INTO effects (job, effect_key, at) VALUES (?, ?, ?)')
         .bind('digest', key, Date.now())
         .run();
     },
   };
   

3. Apply the schema to the local database and start with both tables empty. Start the Worker with wrangler dev --test-scheduled and send two requests, within the same UTC date, to its local scheduled route at the same moment:

   Adapted for publication.

   PORT=8787
   URL="http://localhost:$PORT/cdn-cgi/local/scheduled?cron=*/5+*+*+*+*"
   curl -s "$URL" & curl -s "$URL" & wait
   

4. Count the effects with wrangler d1 execute <your-database> --local --command 'SELECT COUNT(*) FROM effects'. Expected: 1. Then clear both tables, remove the claim and its check, and send the two requests again. Expected: 2.

5. Restore the claim and its check, clear both tables and add a deliberate throw immediately after a successful claim. Invoke once, remove the throw, then invoke again within the same UTC day without clearing either table. Expected: one claim and zero effects. The claim and the effect are separate writes, so the claim that suppresses a duplicate also suppresses the retry; resuming after an interruption needs a rule of its own.

Appendix: versions

As of 15 October 2026, TEEI Language and TEEI Mentorship run on React Router 8 and Wrangler 4, and the website on Astro 7.

If you are building with Cloudflare or with TEEI, talk to us and we will walk you through the code.