Flaky Test Triage

Run failing tests repeatedly, classify each failure as flaky or real, and fix only confirmed regressions.

0 views•0 uses•by solo lemon.

// parameters

Triggermanual

Max iterations10

Exit conditionAll failures classified; real regressions fixed; flaky tests documented or stabilized

// run_this_loop

Copy the kickoff into your agent first. The blueprint below explains how the loop evaluates progress and when it should stop.

Open in Cursor Open in Claude

Use loop copies the kickoff prompt. Share copies the loop link. Open in Cursor / Claude only pre-fill that prompt — they do not install hook files. Download loop saves the loop as flaky-test-triage.md.

Start the "Flaky Test Triage" loop.

Goal: classify failing tests as flaky vs real and fix only real regressions
Max iterations: 5
Between iterations run: npm test -- --testPathPattern=<failing-suite>
Exit when: every failure is classified and real regressions are fixed or explicitly deferred

Step 1: Run the failing suite multiple times. Classify each failure, fix real ones, and document flaky behavior.

Self-pace this loop. After each iteration, run the check command, read the output, and only continue if the exit condition is not met. Stop when the exit condition passes or max iterations is reached. Give a short status update each pass.

// blueprint

The loop contract: target outcome, execution sequence, checks, stopping condition, and safety constraints.

Goal

classify failing tests as flaky vs real and fix only real regressions

// loop_sequence

// loop_sequence_diagram.json

closed loop

Trigger

Manual start

Actions block

4 steps

1Run failing tests

2Classify failures

3Fix real failures

4Confirm stability

Feedback gate

Evaluate: state progress

Run the check, read the output, then decide whether the loop is complete.

Check failed

Repeat actions with the latest feedback.

Success

All failures classified; real regressions fixed; flaky tests documented or stabilized

Safety capMax 10 iterations

// steps

1
Run failing tests
Run the failing test file or suite 3–5 times. Record pass/fail pattern per test.
npm test -- --testPathPattern=<failing-suite>
2
Classify failures
Label each failure as flaky (intermittent) or real (consistent). Note timing, ordering, or env dependencies.
3
Fix real failures
Fix confirmed real failures with minimal changes. For flaky tests, propose stabilization (retries, isolation, mocks).
4
Confirm stability
Re-run the suite multiple times to confirm real failures are gone and flakiness is reduced or documented.
npm test -- --testPathPattern=<failing-suite>

// related_recipes

Testing

Pre-Commit Guard

Hook that runs tests before git commit commands to block commits when the suite is red.

#hooks#testing#git#pre-commit

compatible with:

Testing

Post-Edit Test Guard

Hook-based loop that runs related tests after file edits to catch regressions early.

#hooks#testing#regression

compatible with:

Testing

Post-Merge Regression Guard

Hook that runs smoke tests after git merge or rebase to catch integration regressions immediately.

#hooks#testing#git#merge

compatible with: