ember

9Ordinal answers

A score question returns a distribution over ordered levels, and agents read its expected value. Each plot shows where that value lands for every gold level; a perfect advisor puts every dot on its dashed gold line.

Expected risk score by gold levelDots for each item's expected score, grouped by gold level.rounds to the gold levelmisses itgold levelmean0123Expected scoreNegligibleLowMediumHigh0.610.841.961.98review line 2.0
Figure 10 Expected risk score by gold level. The orange line is the policy's review threshold.

Scale: 0 Negligible → 3 High

Gold levelItemsMean expectedExact levelWithin one level
0: Negligible230.612 of 2323 of 23
1: Low230.8423 of 2323 of 23
2: Medium231.9618 of 2322 of 23
3: High231.985 of 2312 of 23
Expected effort score by gold levelDots for each item's expected score, grouped by gold level.rounds to the gold levelmisses itgold levelmean0123Expected scoreTrivial (minutes)Small (under anhour)Medium (a fewhours)Large (a day ormore)0.761.361.781.82
Figure 11 Expected effort score by gold level.

Scale: 0 Trivial (minutes) → 3 Large (a day or more)

Gold levelItemsMean expectedExact levelWithin one level
0: Trivial (minutes)180.762 of 1814 of 18
1: Small (under an hour)301.3618 of 3028 of 30
2: Medium (a few hours)181.7811 of 1818 of 18
3: Large (a day or more)101.821 of 103 of 10
Expected alert_level score by gold levelDots for each item's expected score, grouped by gold level.rounds to the gold levelmisses itgold levelmean0123Expected scoreNo alert (calmgreen)Low (yellow)Medium (orange)High (red)0.091.041.922.83
Figure 12 Expected alert_level score by gold level.

Scale: 0 No alert (calm green) → 3 High (red)

Gold levelItemsMean expectedExact levelWithin one level
0: No alert (calm green)20.092 of 22 of 2
1: Low (yellow)21.042 of 22 of 2
2: Medium (orange)21.922 of 22 of 2
3: High (red)22.832 of 22 of 2