A score question returns a distribution over ordered levels, and agents read its expected value. Each plot shows where that value lands for every gold level; a perfect advisor puts every dot on its dashed gold line.
Figure 10 Expected risk score by gold level. The orange line is the policy's review threshold.
Gold level
Items
Mean expected
Exact level
Within one level
0: Negligible
12
0.60
2 of 12
12 of 12
1: Low
12
0.86
12 of 12
12 of 12
2: Medium
12
2.08
10 of 12
12 of 12
3: High
12
2.07
2 of 12
7 of 12
Figure 11 Expected effort score by gold level.
Gold level
Items
Mean expected
Exact level
Within one level
0: Trivial (minutes)
9
0.75
1 of 9
6 of 9
1: Small (under an hour)
13
1.36
8 of 13
12 of 13
2: Medium (a few hours)
11
1.91
7 of 11
11 of 11
3: Large (a day or more)
7
1.74
1 of 7
2 of 7
Figure 12 Expected alert_level score by gold level.
