ember

10Confusion matrices

Each matrix counts answers by gold label (rows) and ember's answer (columns). The outlined diagonal is agreement.

Confusion matrix for intentCounts of answers by gold label and ember's answer.rows: gold columns: ember's answerquestionimplementinvestigatefixquestion27000implement02601investigate00270fix01026
Figure 14 intent (choice).
Confusion matrix for specific_enoughCounts of answers by gold label and ember's answer.rows: gold columns: ember's answeryesnoyes5215no338
Figure 15 specific_enough (noul).
Confusion matrix for failure_kindCounts of answers by gold label and ember's answer.rows: gold columns: ember's answerlogic_bugenvironmentflakytest_buglogic_bug23000environment02021flaky00230test_bug10022
Figure 16 failure_kind (choice).
Confusion matrix for retryCounts of answers by gold label and ember's answer.rows: gold columns: ember's answeryesnoyes221no069
Figure 17 retry (noul).
Confusion matrix for riskCounts of answers by gold label and ember's answer.rows: gold columns: ember's answerNegligibleLowMediumHighNegligible22100Low02300Medium04181High05135
Figure 18 risk (score).
Confusion matrix for needs_reviewCounts of answers by gold label and ember's answer.rows: gold columns: ember's answeryesnoyes3511no046
Figure 19 needs_review (noul).
Confusion matrix for ownerCounts of answers by gold label and ember's answer.rows: gold columns: ember's answerapistorageuiinfraunclearapi150000storage015000ui001500infra000150unclear000213
Figure 20 owner (choice).
Confusion matrix for effortCounts of answers by gold label and ember's answer.rows: gold columns: ember's answerTrivial (minutes)Small (under an hour)Medium (a few hours)Large (a day or more)Trivial (minutes)21600Small (under an hour)018120Medium (a few hours)05112Large (a day or more)0361
Figure 21 effort (score).
Confusion matrix for approachCounts of answers by gold label and ember's answer.rows: gold columns: ember's answerminimal_patchrefactornew_componentask_userminimal_patch19000refactor11710new_component60130ask_user11017
Figure 22 approach (choice).
Confusion matrix for dominant_redCounts of answers by gold label and ember's answer.rows: gold columns: ember's answeryesnoyes40no04
Figure 23 dominant_red (noul).
Confusion matrix for dominant_colourCounts of answers by gold label and ember's answer.rows: gold columns: ember's answerredgreenbluemixedred2000green0200blue0020mixed0002
Figure 24 dominant_colour (choice).
Confusion matrix for alert_levelCounts of answers by gold label and ember's answer.rows: gold columns: ember's answerNo alert (calm green)Low (yellow)Medium (orange)High (red)No alert (calm green)2000Low (yellow)0200Medium (orange)0020High (red)0002
Figure 25 alert_level (score).
Confusion matrix for colour_changedCounts of answers by gold label and ember's answer.rows: gold columns: ember's answeryesnoyes40no04
Figure 26 colour_changed (noul).
Confusion matrix for domain/toolingCounts of answers by gold label and ember's answer.rows: gold columns: ember's answeryesnoyes1116no00
Figure 27 domain/tooling (noul).
Confusion matrix for layer/infraCounts of answers by gold label and ember's answer.rows: gold columns: ember's answeryesnoyes24no00
Figure 28 layer/infra (noul).
Confusion matrix for type/codeCounts of answers by gold label and ember's answer.rows: gold columns: ember's answeryesnoyes45no1135
Figure 29 type/code (noul).
Confusion matrix for type/decisionCounts of answers by gold label and ember's answer.rows: gold columns: ember's answeryesnoyes54no050
Figure 30 type/decision (noul).
Confusion matrix for type/designCounts of answers by gold label and ember's answer.rows: gold columns: ember's answeryesnoyes27no238
Figure 31 type/design (noul).
Confusion matrix for type/discoveryCounts of answers by gold label and ember's answer.rows: gold columns: ember's answeryesnoyes45no243
Figure 32 type/discovery (noul).
Confusion matrix for type/mocCounts of answers by gold label and ember's answer.rows: gold columns: ember's answeryesnoyes00no148
Figure 33 type/moc (noul).
Confusion matrix for type/referenceCounts of answers by gold label and ember's answer.rows: gold columns: ember's answeryesnoyes27no142
Figure 34 type/reference (noul).
Confusion matrix for type/session-logCounts of answers by gold label and ember's answer.rows: gold columns: ember's answeryesnoyes54no045
Figure 35 type/session-log (noul).
Confusion matrix for type/specCounts of answers by gold label and ember's answer.rows: gold columns: ember's answeryesnoyes00no1144
Figure 36 type/spec (noul).
Confusion matrix for type/systemCounts of answers by gold label and ember's answer.rows: gold columns: ember's answeryesnoyes27no040
Figure 37 type/system (noul).