Keegan's blog

Handling timeouts in Vitest on Bazel

Recently I have been trying to get a handle on flakiness in our test infrastructure at work. The most common thing left was timeouts while running Vitest. However, Vitest when receiving a SIGTERM just cleans up and calls process.exit without any information on what was running.

We run Vitest in Bazel, so my first attempt was looking for a way to pass down the timeout Bazel will use onto Vitest. However, it doesn't support a global timeout (only per test, hook, etc). Then I looked into any other mechanism it might have, but came up short.

Finally I realised I could hook into it's rather nice Reporter interface. It's slightly hacky because we need to register a SIGTERM handler before Vitest does. In practice though it works.

Out of interest Vitest's logger does provide a onTerminalCleanup that could be useful here. However, it will be called for both SIGTERM and exit without anyway to distinguish the two.

Also I don't really write typescript code, so no idea if this looks like decent TS code. Also stored in a gist

import type { TestCase, TestSuite, Vitest } from 'vitest/node'
import type { Reporter } from 'vitest/reporters'

/**
 * Catches SIGTERM from Bazel so we can report what tests are currently
 * running. Vitest currently does not support reporting this information nor
 * configuring a global timeout.
 */
class TimeoutReporter implements Reporter {
    private runningTests = new Set<{ id: string; name: string; file: string; startTime: number }>()
    private runningSuites = new Set<{ name: string; startTime: number }>()

    public onInit(_: Vitest): void {
        this.runningTests.clear()
        this.runningSuites.clear()

        // Vitest registers a SIGTERM listener in logger.ts which calls
        // process.exit(). So we need to make sure we run before it.
        process.prependOnceListener('SIGTERM', () => {
            const now = Date.now()
            process.stderr.write('\n\nSIGTERM received!\n')

            for (const test of this.runningTests) {
                const duration = now - test.startTime
                process.stderr.write(`Currently running test ${test.name} (${test.file}) - running for ${duration}ms\n`)
            }

            for (const suite of this.runningSuites) {
                const duration = now - suite.startTime
                process.stderr.write(`Currently running suite ${suite.name} - running for ${duration}ms\n`)
            }

            process.stderr.write('Vitest was terminated by Bazel due to timeout\n')
        })
    }

    public onTestCaseReady(testCase: TestCase): void {
        this.runningTests.add({
            id: testCase.id,
            name: testCase.fullName,
            file: testCase.module.moduleId,
            startTime: Date.now(),
        })
    }

    public onTestCaseResult(testCase: TestCase): void {
        for (const test of this.runningTests) {
            if (test.id === testCase.id) {
                this.runningTests.delete(test)
                break
            }
        }
    }

    public onTestSuiteReady(testSuite: TestSuite): void {
        this.runningSuites.add({ name: testSuite.fullName, startTime: Date.now() })
    }

    public onTestSuiteResult(testSuite: TestSuite): void {
        for (const suite of this.runningSuites) {
            if (suite.name === testSuite.fullName) {
                this.runningSuites.delete(suite)
                break
            }
        }
    }
}