API 101: Timeouts, Retries, Backoff, Idempotency
API 101: Timeouts, Retries, Backoff, Idempotency
- Imagine booking a train ticket during the morning rush, and the app just keeps spinning.
- Networks fail, servers get overloaded, and responses get lost. These four techniques help an API client and server fail safely.
Timeouts
- A timeout limits how long you're willing to wait.
- Clients limit how long they wait for a response, so the app can show an error or retry instead of spinning forever.
- Servers limit how long they spend processing a request, so slow work doesn't tie up threads and connections.
- Common types:
- Connection timeout: how long to wait to establish a connection.
- Read (response) timeout: how long to wait for data once connected.
- Overall deadline: the total time budget for the whole operation, including retries.
- In a chain of services (App → Booking → Payment), pass the remaining deadline downstream. There's no point in the Payment Service working on a request the app has already abandoned.
const res = await fetch("/api/bookings", {
method: "POST",
signal: AbortSignal.timeout(5000), // give up after 5 seconds
body: JSON.stringify(booking),
});
- A timeout doesn't cancel or undo anything. The client stops waiting, but the server may still complete the booking. After a timeout, the outcome is unknown: the request may have failed, or succeeded with the response lost.
Retries
- You still need the ticket. Maybe the server crashed before responding, or the network dropped the response.
- A retry sends the request again, giving it another chance to succeed. Many failures are transient and go away within seconds.
- Retry only when it makes sense:
- Retry: network errors, timeouts,
429 Too Many Requests, 502, 503, and 504.
- Don't retry:
400 Bad Request, 401 Unauthorized, 403, 404, or validation errors. Sending the same bad request again will fail the same way.
- If the server sends a
Retry-After header, wait at least that long.
Exponential backoff and jitter
- You're not the only one retrying. Millions of users hitting Retry, while their original requests are still being processed, can overload a struggling server and keep it down. This is a retry storm.
- Exponential backoff increases the delay after each failed attempt: 1 s, then 2 s, then 4 s, then 8 s, up to a maximum delay.
- Always add a retry limit (for example, 3 to 5 attempts) or an overall deadline, then show the error to the user.
- Add jitter: randomize each delay. Without it, clients that failed at the same moment retry at the same moment, causing synchronized spikes.
delay = random(0, min(maxDelay, baseDelay × 2^attempt))
- At the system level, a circuit breaker stops calling a dependency that keeps failing and fails fast for a while, giving it time to recover.
Idempotency
- Your money was already debited, but the response never arrived. Will retrying charge you twice?
- An operation is idempotent if doing it multiple times has the same effect as doing it once.
GET, PUT, and DELETE are idempotent by definition in HTTP: setting a seat to "booked" twice still leaves it booked.
POST is not: "create a booking" twice can create two bookings and two charges.
- Idempotency keys make unsafe operations safe to retry:
- The client generates a unique key for the booking attempt, such as a UUID.
- It sends the key with the first request, and reuses the same key on every retry.
- The server stores each key with the result of the request.
- If a request arrives with a key that already succeeded, the server returns the stored result without charging again.
- Server details that matter:
- Store the key and result atomically with the operation, for example in the same database transaction.
- If a retry arrives while the first request is still in progress, return a conflict (
409) or wait, rather than processing it twice.
- If the same key arrives with a different request body, reject it as an error.
- Expire keys after a reasonable period, such as 24 hours.
Putting it together
async function bookTicket(booking, maxAttempts = 4) {
const idempotencyKey = crypto.randomUUID(); // same key for every retry
for (let attempt = 0; attempt < maxAttempts; attempt++) {
try {
const res = await fetch("/api/bookings", {
method: "POST",
headers: {
"Content-Type": "application/json",
"Idempotency-Key": idempotencyKey,
},
body: JSON.stringify(booking),
signal: AbortSignal.timeout(5000),
});
if (res.ok) return res.json();
const retryable = [429, 502, 503, 504].includes(res.status);
if (!retryable) throw new Error(`Booking failed: ${res.status}`);
} catch (error) {
const retryable = error.name === "TimeoutError" || error instanceof TypeError;
if (!retryable || attempt === maxAttempts - 1) throw error;
}
const delay = Math.random() * Math.min(8000, 1000 * 2 ** attempt);
await new Promise((resolve) => setTimeout(resolve, delay));
}
throw new Error("Booking failed after retries");
}
Summary
- Use all four together: timeouts detect problems, retries recover from them, backoff keeps retries from making things worse, and idempotency makes retries safe.
References