Why not a simple request counter per minute?

Fixed windows burst at the boundary. Attackers and eager clients learn the reset. Sliding windows cost more to store and are honest about “the last N seconds.”

Where do you store the window?

In a shared store the API instances already trust — for us, Redis-class infrastructure next to the Node services — so multiple machines agree on the same user and route.

August 17, 2026 Infrastructure

Sliding-window rate limiting for a 3-million-user Node API

Spicychat did not need a blog-post rate limiter. It needed limits that still meant something at 3 million users: chat, generation, push, moderation, TTS. I shipped sliding-window limits on the endpoints that would otherwise take the platform down.

Why fixed windows fail in public

A “60 requests per minute” counter that resets on the clock looks fine in a unit test. In production, everyone retries on the reset. Premium TTS and character generation are expensive. A burst at :00 is a self-inflicted outage. Sliding windows ask a different question: how many hits in the last N seconds, continuously.

What we actually limited

Not every route deserves the same budget. Login, search, and static assets are not the same as model calls or voice. We put the strict windows on the costly paths — generation, moderation hooks, TTS fan-out — and kept human-facing reads cheaper. Personalized AI push had its own budget so marketing could not stampede the same keys as chat.

Shared state or it did not happen

Multiple Node instances behind a load balancer will each believe the user is under limit if the counter lives in process memory. The window has to live in Redis (or equivalent) with a key that includes user, route class, and window. That is boring. It is also the difference between a limit and a decoration.

What broke before it held

The first version is always too coarse (one bucket for the whole API) or too fine (a key per tiny path, cardinality explosion). We tightened classes until ops could explain a 429 in one sentence: which product surface, which plan, which window. Logging the class next to the 429 mattered more than a clever algorithm name.

This is not a copy-paste recipe

I am not publishing a bypass guide or a drop-in exploit. The useful part for hiring and for other engineers is the product framing: expensive AI routes, shared counters, honest windows, budgets per surface. The same discipline shows up in search and in moderation — you budget the scarce thing.

If you want the rest of the Spicychat work — push, moderation, TTS over WebSockets — it is on the case study. I am open to product and platform roles that have this shape of problem.

FAQ

Why not a simple request counter per minute?: Fixed windows burst at the boundary. Attackers and eager clients learn the reset. Sliding windows cost more to store and are honest about “the last N seconds.”
Where do you store the window?: In a shared store the API instances already trust — for us, Redis-class infrastructure next to the Node services — so multiple machines agree on the same user and route.