`--selftest` slept six seconds and then photographed whatever was on screen. On the first cold run of the freshly packaged 2.4.0 — Gatekeeper checking the bundle, a first DNS lookup, the catalog still in flight — six seconds was not enough, and it reported an empty window as SELFTEST FAILED. Four runs immediately afterwards passed with all 13 cards. The window was fine; the guess about how long somebody else's machine takes was not. It now polls for a settled window — a card, or the gate — and gives up only at a 30-second ceiling. The common case got *faster* than the old fixed wait (the run finishes in about three seconds rather than always at least six), and the cold case passes. A timeout is deliberately not a failure by itself: the report is taken anyway and the existing checks decide, so a genuinely empty window still fails for the right reason instead of as a bare timeout. This is a diagnostic, not the product: nothing here runs without `--selftest`, which is why 2.4.0 shipped as it is rather than being retagged over it. I made the flake marginally likelier by adding one more startup request — the service descriptor — so fixing it is mine to do. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -4,7 +4,20 @@ import {
|
||||
asRecord, readBoolean, readNumber, readOptionalString, readString, readStringArray
|
||||
} from '../../infrastructure/json/JsonRecord'
|
||||
|
||||
const SETTLE_DELAY_MS = 6_000
|
||||
/**
|
||||
* How long to keep waiting for the window to have something on it.
|
||||
*
|
||||
* This used to be a flat six-second sleep, which is a guess about somebody else's
|
||||
* machine: on a cold start — a freshly built app, Gatekeeper checking it, a first DNS
|
||||
* lookup and the catalog still in flight — six seconds is sometimes not enough, and the
|
||||
* run reported an empty window as a failure. It was not a failure; it was a stopwatch.
|
||||
*
|
||||
* Now it polls for a settled window and only gives up at the ceiling, so the common
|
||||
* case is *faster* than the old fixed wait and the cold case still passes.
|
||||
*/
|
||||
const SETTLE_POLL_MS = 400
|
||||
const SETTLE_CEILING_MS = 30_000
|
||||
const SETTLE_DELAY_MS = 1_000
|
||||
const SWITCH_SETTLE_DELAY_MS = 8_000
|
||||
const SHOT_FRAME_DELAY_MS = 400
|
||||
|
||||
@@ -55,12 +68,14 @@ export class SelfTestRunner {
|
||||
private readonly shotPath: string | null = process.env['SELFTEST_SHOT'] ?? null
|
||||
) {}
|
||||
|
||||
/** A short first wait; `run` does the rest of the waiting itself. */
|
||||
public get settleDelayMs (): number {
|
||||
return SETTLE_DELAY_MS
|
||||
}
|
||||
|
||||
/** True when the window is in a state a user could work with. */
|
||||
public async run (): Promise<boolean> {
|
||||
await this.awaitSettled()
|
||||
const report = await this.readReport()
|
||||
console.log(JSON.stringify(report, null, 2))
|
||||
|
||||
@@ -93,6 +108,27 @@ export class SelfTestRunner {
|
||||
return passed
|
||||
}
|
||||
|
||||
/**
|
||||
* Wait until the window is showing something, or until the ceiling.
|
||||
*
|
||||
* "Something" is a card or the gate: those are the two states a person could act on,
|
||||
* and between them they cover every way this application legitimately ends up. Timing
|
||||
* out is not treated as a failure here — the report is taken anyway, and the checks
|
||||
* below decide, so a genuinely empty window still fails for the right reason rather
|
||||
* than as a timeout with no detail.
|
||||
*/
|
||||
private async awaitSettled (): Promise<void> {
|
||||
const deadline = Date.now() + SETTLE_CEILING_MS
|
||||
while (Date.now() < deadline) {
|
||||
const ready = await this.evaluate(
|
||||
"String(document.querySelectorAll('.card').length > 0 || " +
|
||||
"!document.getElementById('gate').hidden)"
|
||||
)
|
||||
if (ready === 'true') return
|
||||
await delay(SETTLE_POLL_MS)
|
||||
}
|
||||
}
|
||||
|
||||
private async readReport (): Promise<SelfTestReport> {
|
||||
const record = asRecord(JSON.parse(await this.evaluate(`JSON.stringify({
|
||||
cards: document.querySelectorAll('.card').length,
|
||||
@@ -206,3 +242,9 @@ export class SelfTestRunner {
|
||||
return typeof result === 'string' ? result : JSON.stringify(result ?? null)
|
||||
}
|
||||
}
|
||||
|
||||
async function delay (milliseconds: number): Promise<void> {
|
||||
await new Promise<void>((resolve: () => void): void => {
|
||||
setTimeout((): void => { resolve() }, milliseconds)
|
||||
})
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user