fix(engine): widen 429 backoff budget, correct the rate-limit explanation
Measured directly against the live forge: nginx never sends a Retry-After header on its 429s (the branch handling it is dead code in practice), the limit is a leaky bucket rather than a fixed ban (a 20-25 request burst took up to ~15s to drain), it is IP-based rather than token-quota-based (a valid FORGEJO_TOKEN made no difference to a reproduced burst), and it triggers well below "13 calls in a loop" — 20 concurrent anonymous requests reproduced it directly. The old default (retries: 3, ~7s worst case) was tuned for a hard ban that doesn't exist. fetchWithRetry now defaults to retries: 5 with a maxDelayMs: 8000 cap (23s worst case), covering the measured drain time without one attempt blocking for a full uncapped exponential step. CLAUDE.md's explanation is corrected to match; test count in README/CLAUDE.md updated for the two new tests (111 -> 113). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W1ZJFViVYpr8cvf4fs91j1
This commit is contained in:
parent
568b8e374a
commit
e6b0f04021
4 changed files with 61 additions and 9 deletions
|
|
@ -827,12 +827,27 @@ export function loadRegister(path = REGISTER_PATH) {
|
|||
// a false SKIP, which this repo's own rule says is never a pass.
|
||||
const defaultSleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
|
||||
|
||||
export async function fetchWithRetry(url, options, { fetchImpl = fetch, retries = 3, baseDelayMs = 1000, sleep = defaultSleep } = {}) {
|
||||
// Measured 2026-08-04 against the live forge: nginx never sends a
|
||||
// `Retry-After` header on its 429s, so the exponential fallback below is the
|
||||
// ONLY path that ever actually runs — the branch above it is dead in
|
||||
// practice, kept only because a future proxy config could add the header.
|
||||
// The 429 itself is a leaky-bucket burst limit, not a fixed-duration ban: a
|
||||
// 20-25 request burst took up to ~15s to fully drain, and a 20s pause always
|
||||
// cleared it. `retries: 3` (7s worst case) was tuned for a hard ban that
|
||||
// turned out not to exist; `retries: 5` with `maxDelayMs: 8000` (23s worst
|
||||
// case) covers the measured drain time without one attempt blocking minutes.
|
||||
export async function fetchWithRetry(
|
||||
url,
|
||||
options,
|
||||
{ fetchImpl = fetch, retries = 5, baseDelayMs = 1000, maxDelayMs = 8000, sleep = defaultSleep } = {},
|
||||
) {
|
||||
for (let attempt = 0; ; attempt += 1) {
|
||||
const res = await fetchImpl(url, options);
|
||||
if (res.status !== 429 || attempt >= retries) return res;
|
||||
const retryAfter = Number(res.headers?.get?.('retry-after'));
|
||||
const delayMs = Number.isFinite(retryAfter) && retryAfter > 0 ? retryAfter * 1000 : baseDelayMs * 2 ** attempt;
|
||||
const delayMs = Number.isFinite(retryAfter) && retryAfter > 0
|
||||
? retryAfter * 1000
|
||||
: Math.min(baseDelayMs * 2 ** attempt, maxDelayMs);
|
||||
await sleep(delayMs);
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1102,6 +1102,31 @@ test('no Retry-After header falls back to exponential backoff from baseDelayMs',
|
|||
assert.deepEqual(calls, [1000, 2000]);
|
||||
});
|
||||
|
||||
test('exponential backoff is capped, so a long retry budget does not wait minutes between attempts', async () => {
|
||||
// Measured 2026-08-04 against the live forge: 20 parallel requests from one
|
||||
// IP produced 429 with NO Retry-After header at all (nginx never sends one
|
||||
// here) — the exponential fallback is the only path that ever runs in
|
||||
// practice. Recovery was gradual, not a fixed-duration ban: a burst that
|
||||
// size took up to ~15s to fully drain, and a 20s manual pause cleared it.
|
||||
// Uncapped doubling would reach 32s on a single attempt; capping at 8s and
|
||||
// extending the retry budget covers the measured recovery window without
|
||||
// one attempt blocking for excessive time.
|
||||
const calls = [];
|
||||
const responses = [
|
||||
fakeResponse(429),
|
||||
fakeResponse(429),
|
||||
fakeResponse(429),
|
||||
fakeResponse(429),
|
||||
fakeResponse(429),
|
||||
fakeResponse(200),
|
||||
];
|
||||
const fetchImpl = async () => responses.shift();
|
||||
const sleep = async (ms) => calls.push(ms);
|
||||
await fetchWithRetry('https://x', {}, { fetchImpl, sleep, baseDelayMs: 1000, maxDelayMs: 8000, retries: 5 });
|
||||
// Uncapped, the 5th delay would be 1000 * 2**4 = 16000.
|
||||
assert.deepEqual(calls, [1000, 2000, 4000, 8000, 8000]);
|
||||
});
|
||||
|
||||
test('retries are bounded — a persistent 429 returns the 429, not an infinite loop', async () => {
|
||||
let calls = 0;
|
||||
const fetchImpl = async () => fakeResponse(429);
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue