Timeouts are not a budget

Name the clock

An idle timeout measures a period without progress. An absolute deadline measures total elapsed time from a chosen starting event. A client that sends a little data often enough can remain below an idle timeout while consuming resources for much longer than that timeout's value.

Consider a transfer with a five-second idle timeout. If another byte arrives every four seconds, the transfer may continue indefinitely unless another limit stops it. A twenty-second absolute deadline would instead end the operation twenty seconds after its start, regardless of intermittent progress.

Neither policy is always better. A long download might need a generous total lifetime with an idle check. An interactive lookup may need a strict total deadline. Choose based on the operation, not on a desire to set every timeout to the same number.

Time is not capacity

A timeout bounds some kind of duration. It does not, by itself, bound the number of simultaneous operations. If new work arrives faster than old work completes, active work accumulates.

For a stable system, Little's law relates average work in the system to average arrival rate and average time spent there. As a rough planning example, ten completed requests per second spending two seconds in the system corresponds to about twenty requests in the system on average. This is an average under stated assumptions, not a worst-case sizing rule.

A concurrency cap limits simultaneous work. A rate limiter controls admission over time. A request-size limit bounds bytes accepted for one request. Memory and process limits bound different resources again. A usable policy is a set of compatible limits, not one magic timeout.

Keep the layer in view

A request passes through several stages before an application can act on it: a transport connection, possibly a TLS handshake, HTTP parsing and finally request handling. A limiter that only runs after HTTP headers are read cannot account for every connection still waiting to reach that point.

This is why documentation details matter. Nginx's HTTP connection limiter does not count every raw open socket: it counts connections with a request being processed after the whole request header has been read. With HTTP/2, concurrent requests are counted separately. A name such as “connection limit” is not enough to infer the actual boundary.

Likewise, a limit on one server does not automatically protect an earlier proxy or a downstream service. Record the scope in plain language: “limits completed HTTP request admission here,” rather than “prevents overload everywhere.” Source addresses may also be shared behind a proxy; a per-address rule can accidentally become a limit for many unrelated readers.

Test the boundary, not just the number

Use a controlled environment and a small, bounded test. Check ordinary traffic first. Then test the specific case you intend to constrain: no progress, slow intermittent progress, a short burst or more simultaneous requests than the configured cap.

• State the starting event and the expected terminal event.
• Record observed elapsed time as well as the configured value.
• Verify whether the connection, request or downstream task actually ended.
• Check that an ordinary request still works after the test.
• Keep a hard duration and concurrency cap on the test itself.

Do not use a production system, someone else's service or a shared environment as an unbounded load generator. A configuration parser can establish that syntax is accepted. It cannot establish capacity, fairness or a complete resource-lifetime guarantee.

Good limits make a promise you can describe and observe. If that promise is narrower than you hoped, changing the label will not widen it; another control at the correct layer might.