Decisions under the agent kit

8Decisions under the agent kit

Confidence bands

The agent kit maps every answer to an action by its confidence. These tables replay that mapping on every answer in the run.

choice band	Rule	Agent action	Share	Accuracy
`trust`	confidence >= 0.85	act on it	0.0%	n/a
`verify`	0.60 <= confidence < 0.85	act, say so, and verify	0.0%	n/a
`defer`	confidence < 0.60	gather evidence or ask the user	0.0%	n/a

noul band	Rule	Agent action	Answers	Share	Correct	Accuracy
`yes`	P >= 0.80	treat as yes	52	8.1%	48	92.3%
`no`	P <= 0.20	treat as no	433	67.2%	385	88.9%
`unsure`	0.20 < P < 0.80	treat as unsure	159	24.7%	83	52.2%

Figure 7 Share of answers the kit acts on, acts on with verification, or defers.

Coverage and accuracy

Figure 8 noul: coverage and accuracy as the threshold rises. Dashed lines mark the kit's thresholds.

Default project policy

The agent kit's AGENTS.md snippet ships a default project policy. The report replays it on every item and counts what an agent following it to the letter would have done: the right thing, a cautious extra step (an unnecessary question or review), or a wrong action (acting on a vague request, or shipping a risky change unreviewed).