Project
Reject length mismatches and unknown labels, keeping zero counts for classes never observed.
AI coding workspace
Classification confusion matrices
A count is only useful when its labels line up. MatrixTern is a Windows AI coding workspace for examining classification code, fixed label orders and the cases a summary must retain.
● ● ●MatrixTern●
A concrete starting point · Classification confusion matrices ×
def confusion(expected, predicted, labels):
if len(expected) != len(predicted) or len(set(labels)) != len(labels):
raise ValueError("labels or lengths")
index = {label: i for i, label in enumerate(labels)}
matrix = [[0] * len(labels) for _ in labels]
for actual, guess in zip(expected, predicted):
if actual not in index or guess not in index:
raise ValueError("unknown label")
matrix[index[actual]][index[guess]] += 1
return matrix✳Build a confusion matrix from expected and predicted labels.
✓ ChangesReview before applying ↗
Illustrative workspace
Review before applying
Compare actual behavior with the expected result. If a check fails, investigate it before applying more changes.
Connection
Base URL, credentials and a reachable server.
A concrete starting point
def confusion(expected, predicted, labels):
if len(expected) != len(predicted) or len(set(labels)) != len(labels):
raise ValueError("labels or lengths")
index = {label: i for i, label in enumerate(labels)}
matrix = [[0] * len(labels) for _ in labels]
for actual, guess in zip(expected, predicted):
if actual not in index or guess not in index:
raise ValueError("unknown label")
matrix[index[actual]][index[guess]] += 1
return matrixMake the evaluation code as reviewable as the result
MatrixTern gives developers a Windows AI workspace for examining the code behind an evaluation result. Open the project folder and state what the report is supposed to measure before asking for changes. The conversation can then connect the calculation, its inputs and the code that presents the result rather than treating a printed number as sufficient evidence.
For a classification report, ask the model to inspect how expected and predicted labels become a confusion matrix. Supply the declared label order and a small fixture whose counts you can check yourself. That request provides a concrete way to discuss the implementation without assuming the report has already been independently verified.
Use the proposed edits to clarify assumptions that affect interpretation: whether unknown labels are rejected, how empty classes are represented and whether input lengths must match. Review the affected files and run your evaluation tests. Compare the displayed labels with the underlying matrix as well as checking that the totals add up.
An abliterated AI model may be available through the OpenAI-compatible or Anthropic endpoint you configure in MatrixTern. Add credentials when required and check the connection. The selected model and server govern restrictions, context and tools; their output remains something to evaluate alongside the project evidence.
Checking the software behind a report
What is a useful first fixture for a classification report?
Use a short sequence with a known correct prediction and a known misclassification, keeping label order explicit. Calculate the expected counts independently. This makes swapped axes or inconsistent label indexing easier to identify during review.
Should labels absent from a sample disappear from the output?
Decide whether the report uses a fixed label set or derives it from each sample. Show that requirement to the model and inspect the presentation code too. A disappearing class can make reports difficult to compare.
Does MatrixTern rank the connected models for me?
The workflow described here concerns reviewing your evaluation software. A model response is not a measured benchmark result. You define the experiment, run its checks and inspect the resulting evidence before drawing conclusions about any system.
How should I handle sensitive evaluation samples?
Use representative sanitized fixtures and review the model endpoint and connected tools before sharing data. Even a separately running local model server does not make every tool local, so inspect the complete data path.
MatrixTern · Updated 2026-10-25
