Green/Yellow/Red auto-evaluation

Green/Yellow/Red auto-evaluation is an automated system that assesses every pull request and assigns a quality tier that determines what happens next. Unlike traditional CI (which gives a binary pass/fail on tests and lint), the traffic-light system is nuanced:

• Green - The PR meets all quality criteria: tests pass, coverage is maintained, lint is clean, the AI review agent found no issues, the diff is within safe size limits, and the changes touch no high-risk areas. Green PRs are candidates for auto-merge.
• Yellow - The PR passes basic checks but requires human review for specific reasons: it touches security-sensitive code, modifies a core shared interface, introduces a new dependency, or has an AI review comment flagged as requiring human judgment. Yellow PRs route to the appropriate human reviewer.
• Red - The PR has blocking issues: failing tests, lint violations, AI review agent flagged a high-confidence security or correctness issue, or the changes touch architectural boundaries that require explicit approval. Red PRs are returned to the author for remediation.

The evaluation is fully automated - a CI pipeline step (or the AI review agent itself) evaluates the PR against defined criteria and sets the status. No human decides the color; the algorithm does. The criteria are explicit, documented, and consistent.

This system is the bridge between L3's automated first-pass review and L4's auto-merge. Before you can auto-merge Green PRs, you need a trustworthy definition of what "Green" means. The traffic-light evaluation makes that definition explicit and machine-executable.

The binary pass/fail model of traditional CI has a fundamental limitation: it treats all passing PRs as equivalent. A PR with 100% test coverage, zero lint errors, and an AI review that found nothing is equivalent, in CI's eyes, to a PR that barely passes the minimum checks. Both get a green checkmark.

The traffic-light system introduces nuance that the binary model can't express:

• Enables proportional response - Green gets auto-merged; Yellow routes to the right reviewer; Red goes back to the author. Each outcome is calibrated to the actual risk of the change.
• Makes policy explicit - What does "requires human review" actually mean? The Yellow criteria answer that question precisely and consistently. This is better than the implicit judgment call every reviewer currently makes.
• Provides a quality signal - Tracking the Green rate over time tells you whether development process quality is improving or degrading. A falling Green rate is an early warning signal.
• Eliminates ambiguity - Developers know exactly what determines their PR's path. If they want to go Green, the criteria are explicit. This drives quality improvement at the source (developers write better code to avoid Yellow/Red) rather than at review time.
• Scales review capacity - Human reviewers only see Yellow and Red PRs. If 60% of PRs go Green, human review load is reduced by 60%. The same reviewer bandwidth handles significantly more PRs.

The specific criteria for each color tier will vary by team and codebase. What matters is that the criteria are explicit, documented, and enforced algorithmically - not dependent on reviewer judgment to apply consistently.

1. Define your initial Green criteria explicitly - Start with what's unambiguously safe to auto-merge: all tests pass (and there are adequate tests for the change), lint is clean, AI review found no issues above a severity threshold, diff size is under 300 lines, and the changes don't touch any files in the high-risk list. Write these criteria down before implementing them.
2. Define your Yellow criteria - What conditions require human review but aren't blocking? Common Yellow conditions: changes to authentication/authorization code, new external dependencies, modifications to public API contracts, AI review comments that were flagged as "requires human judgment," PRs from developers below a seniority threshold.
3. Define your Red criteria - What automatically blocks a PR? Failing tests, failing lint, AI review flags a high-confidence security or critical correctness issue, coverage drops below threshold, changes exceed size limits without explicit exemption.
4. Implement the evaluation as a CI step - This can be a GitHub Action, a GitLab CI step, or a custom script. The evaluation reads the CI results, the AI review agent's output, the diff metadata, and the list of affected files, then sets a status check with the appropriate color and an explanation.
5. Post the evaluation as a PR comment - When a PR is evaluated Yellow or Red, post a comment explaining exactly what triggered the status and what the author needs to do. Transparency about the criteria makes the system legible and actionable.
6. Calibrate the criteria with data - After 60 days, analyze outcomes: What proportion of Green PRs introduced post-merge bugs? What proportion of Yellow PRs were trivially approved by human reviewers? Use this data to tune the criteria toward the optimal balance.

Starting with a permissive Green definition. If too many PRs qualify as Green before confidence is established, auto-merge will introduce regressions. Start strict (few Green PRs) and loosen as the evaluation proves reliable. It's better to have human reviewers briefly annoyed that too many PRs are Yellow than to have an auto-merge cause a production incident.

Making the criteria opaque. If developers don't understand what determines their PR's color, the system feels arbitrary and generates frustration. Publish the criteria explicitly in the team wiki. The evaluation step should post a comment explaining the decision in terms of the criteria.

Not tracking the Green rate. The Green rate (percentage of PRs that score Green over a period) is a leading indicator of development process quality. If the rate drops, something in the process has changed - higher risk areas are being touched, test quality is degrading, AI review is catching more issues. Instrument and monitor it.

Conflating Yellow and Red. Yellow means "review needed" not "broken." Red means "blocked." If Yellow PRs routinely sit in a reviewer queue for days without action, the distinction isn't working - Yellow is functionally Red. Ensure Yellow PRs have a defined SLA for human review.