ember

10Confusion matrices

Each matrix counts answers by gold label (rows) and ember's answer (columns). The outlined diagonal is agreement.

Confusion matrix for intentCounts of answers by gold label and ember's answer.rows: gold columns: ember's answerquestionimplementinvestigatefixquestion14000implement01301investigate00140fix00014
Figure 13 intent (choice).
Confusion matrix for specific_enoughCounts of answers by gold label and ember's answer.rows: gold columns: ember's answeryesnoyes287no219
Figure 14 specific_enough (noul).
Confusion matrix for failure_kindCounts of answers by gold label and ember's answer.rows: gold columns: ember's answerlogic_bugenvironmentflakytest_buglogic_bug12000environment01020flaky00120test_bug10011
Figure 15 failure_kind (choice).
Confusion matrix for retryCounts of answers by gold label and ember's answer.rows: gold columns: ember's answeryesnoyes120no036
Figure 16 retry (noul).
Confusion matrix for riskCounts of answers by gold label and ember's answer.rows: gold columns: ember's answerNegligibleLowMediumHighNegligible21000Low01200Medium01101High0282
Figure 17 risk (score).
Confusion matrix for needs_reviewCounts of answers by gold label and ember's answer.rows: gold columns: ember's answeryesnoyes195no024
Figure 18 needs_review (noul).
Confusion matrix for ownerCounts of answers by gold label and ember's answer.rows: gold columns: ember's answerapistorageuiinfraunclearapi80000storage08000ui00800infra00080unclear00008
Figure 19 owner (choice).
Confusion matrix for effortCounts of answers by gold label and ember's answer.rows: gold columns: ember's answerTrivial (minutes)Small (under an hour)Medium (a few hours)Large (a day or more)Trivial (minutes)1800Small (under an hour)0850Medium (a few hours)0272Large (a day or more)0331
Figure 20 effort (score).
Confusion matrix for approachCounts of answers by gold label and ember's answer.rows: gold columns: ember's answerminimal_patchrefactornew_componentask_userminimal_patch10000refactor01000new_component3070ask_user4015
Figure 21 approach (choice).
Confusion matrix for dominant_redCounts of answers by gold label and ember's answer.rows: gold columns: ember's answeryesnoyes40no04
Figure 22 dominant_red (noul).
Confusion matrix for dominant_colourCounts of answers by gold label and ember's answer.rows: gold columns: ember's answerredgreenbluemixedred2000green0200blue0020mixed0002
Figure 23 dominant_colour (choice).
Confusion matrix for alert_levelCounts of answers by gold label and ember's answer.rows: gold columns: ember's answerNo alert (calm green)Low (yellow)Medium (orange)High (red)No alert (calm green)2000Low (yellow)0200Medium (orange)0020High (red)0002
Figure 24 alert_level (score).
Confusion matrix for colour_changedCounts of answers by gold label and ember's answer.rows: gold columns: ember's answeryesnoyes40no04
Figure 25 colour_changed (noul).