How we tick thousands of monitors every minute

A worker on a one-minute cron sounds simple until you do the math: thousands of monitors, each on its own interval, all sharing a single D1 database with a finite write budget. Advancing a per-monitor next_run_at cursor on every tick would spend that budget on bookkeeping alone.

So we don't keep a cursor. This post walks through the scheduler that decides what is due right now in one query, writes nothing to schedule, and still fans out cleanly across the whole fleet.

#The write-budget problem

Every persisted "when did this last run, when does it run next" is a row you have to write back on each check. At fleet scale that is thousands of writes a minute doing nothing but moving a timestamp forward. D1 is generous but not infinite, and those writes compete with the ones that actually matter: check results, incidents, hourly rollups.

The way out is to notice that a monitor's cadence is already a pure function of two numbers we never have to store: its interval, and the current wall-clock minute.

#The clock is the cursor

A monitor is due this minute when its interval, in minutes, evenly divides the current minute-of-epoch. That is a modulo, and SQLite can evaluate it in the WHERE clause, so "what is due" becomes a read instead of a read-then-write.

// Due = monitors whose interval (in minutes) divides the current minute.
// No per-check write to advance a cursor — the clock IS the cursor.
const due = await env.DB.prepare(
  `SELECT id, url, interval_sec
     FROM monitors
    WHERE enabled = 1
      AND auto_paused = 0
      AND (? % (interval_sec / 60)) = 0`
).bind(minute).all();

There is no next_run_at column, no update statement, and no lock contention on a hot cursor. Create a monitor and it simply starts matching the modulo on its next aligned minute. Pause one and it drops out of the read the same tick. A 5-minute monitor fires when minute % 5 == 0; a 1-minute monitor matches every minute because minute % 1 is always zero.

The interval floor is enforced in the same expression. Instead of the raw interval_sec / 60, the live query clamps up to the owner plan's minimum, so a downgraded account keeps ticking but at its plan's slowest cadence, without ever rewriting the stored interval.

#The seconds-versus-milliseconds trap

The one place this bites hard: Cloudflare's cron hands you epoch seconds, but almost everything we persist — rollups, plan expiry, Apple's transaction dates — is epoch milliseconds. Compare the two directly and every window is off by a factor of a thousand, which looks like "nothing is ever due" or "everything expired in 1970."

The slot query above sidesteps this entirely because it only needs the minute number, not a timestamp diff. The trap shows up the moment a feature does compare against stored time: maintenance windows, dormancy sweeps, subscription expiry. Keeping the unit boundary explicit in one place saved us from rediscovering it in each of them.

#Fanning out without falling over

Once the due set is in hand, the checks themselves fan out in bounded batches so a single invocation stays inside the platform's subrequest and wall-time limits:

• Chunk the due monitors into groups of ~50 and Promise.allSettled each batch, so one slow or hanging host can't stall the rest of the fleet.
• Results, not schedules, are the only writes: a rollup upsert, and on a state transition, an incident row.
• Maintenance windows and paused monitors are filtered inside the query, so excluded checks cost nothing — they never enter the batch at all.

#What we'd do again

Derive state you can recompute; don't store it. The interval and the clock were always enough to know what is due — persisting a cursor only added writes, contention, and a whole class of drift bugs. The fleet now ticks on one cron, and the database spends its write budget on things a user can actually see.

The next post in this series takes the same "recompute, don't store" idea into billing, where enforcing a plan downgrade turned out to need almost no stored state at all.