Skip to content
Shubham JangleResources
Scalability & reliability

Explore topics

API 101: Timeouts, Retries, Backoff, Idempotency

API 101: Timeouts, Retries, Backoff, Idempotency

  • Imagine booking a train ticket during the morning rush, and the app just keeps spinning.
  • Networks fail, servers get overloaded, and responses get lost. These four techniques help an API client and server fail safely.

Timeouts

  • A timeout limits how long you're willing to wait.
    • Clients limit how long they wait for a response, so the app can show an error or retry instead of spinning forever.
    • Servers limit how long they spend processing a request, so slow work doesn't tie up threads and connections.
  • Common types:
    • Connection timeout: how long to wait to establish a connection.
    • Read (response) timeout: how long to wait for data once connected.
    • Overall deadline: the total time budget for the whole operation, including retries.
  • In a chain of services (App → Booking → Payment), pass the remaining deadline downstream. There's no point in the Payment Service working on a request the app has already abandoned.
const res = await fetch("/api/bookings", {
  method: "POST",
  signal: AbortSignal.timeout(5000), // give up after 5 seconds
  body: JSON.stringify(booking),
});
  • A timeout doesn't cancel or undo anything. The client stops waiting, but the server may still complete the booking. After a timeout, the outcome is unknown: the request may have failed, or succeeded with the response lost.

Retries

  • You still need the ticket. Maybe the server crashed before responding, or the network dropped the response.
  • A retry sends the request again, giving it another chance to succeed. Many failures are transient and go away within seconds.
  • Retry only when it makes sense:
    • Retry: network errors, timeouts, 429 Too Many Requests, 502, 503, and 504.
    • Don't retry: 400 Bad Request, 401 Unauthorized, 403, 404, or validation errors. Sending the same bad request again will fail the same way.
  • If the server sends a Retry-After header, wait at least that long.

Exponential backoff and jitter

  • You're not the only one retrying. Millions of users hitting Retry, while their original requests are still being processed, can overload a struggling server and keep it down. This is a retry storm.
  • Exponential backoff increases the delay after each failed attempt: 1 s, then 2 s, then 4 s, then 8 s, up to a maximum delay.
  • Always add a retry limit (for example, 3 to 5 attempts) or an overall deadline, then show the error to the user.
  • Add jitter: randomize each delay. Without it, clients that failed at the same moment retry at the same moment, causing synchronized spikes.
delay = random(0, min(maxDelay, baseDelay × 2^attempt))
  • At the system level, a circuit breaker stops calling a dependency that keeps failing and fails fast for a while, giving it time to recover.

Idempotency

  • Your money was already debited, but the response never arrived. Will retrying charge you twice?
  • An operation is idempotent if doing it multiple times has the same effect as doing it once.
    • GET, PUT, and DELETE are idempotent by definition in HTTP: setting a seat to "booked" twice still leaves it booked.
    • POST is not: "create a booking" twice can create two bookings and two charges.
  • Idempotency keys make unsafe operations safe to retry:
    1. The client generates a unique key for the booking attempt, such as a UUID.
    2. It sends the key with the first request, and reuses the same key on every retry.
    3. The server stores each key with the result of the request.
    4. If a request arrives with a key that already succeeded, the server returns the stored result without charging again.
  • Server details that matter:
    • Store the key and result atomically with the operation, for example in the same database transaction.
    • If a retry arrives while the first request is still in progress, return a conflict (409) or wait, rather than processing it twice.
    • If the same key arrives with a different request body, reject it as an error.
    • Expire keys after a reasonable period, such as 24 hours.

Putting it together

async function bookTicket(booking, maxAttempts = 4) {
  const idempotencyKey = crypto.randomUUID(); // same key for every retry

  for (let attempt = 0; attempt < maxAttempts; attempt++) {
    try {
      const res = await fetch("/api/bookings", {
        method: "POST",
        headers: {
          "Content-Type": "application/json",
          "Idempotency-Key": idempotencyKey,
        },
        body: JSON.stringify(booking),
        signal: AbortSignal.timeout(5000),
      });

      if (res.ok) return res.json();
      const retryable = [429, 502, 503, 504].includes(res.status);
      if (!retryable) throw new Error(`Booking failed: ${res.status}`);
    } catch (error) {
      const retryable = error.name === "TimeoutError" || error instanceof TypeError;
      if (!retryable || attempt === maxAttempts - 1) throw error;
    }

    const delay = Math.random() * Math.min(8000, 1000 * 2 ** attempt);
    await new Promise((resolve) => setTimeout(resolve, delay));
  }
  throw new Error("Booking failed after retries");
}

Summary

TechniqueProblem it solvesKey rule
TimeoutWaiting foreverA timeout means "unknown", not "failed"
RetryTransient failuresRetry only retryable errors, with a limit
Exponential backoff + jitterRetry stormsIncrease and randomize the delay
Idempotency keyDuplicate side effectsReuse the same key for every retry
  • Use all four together: timeouts detect problems, retries recover from them, backoff keeps retries from making things worse, and idempotency makes retries safe.

References