Back to Report Blocks

Score/Rubric Consistency Check

Preflight
Related check

Checks whether a ScoreVersion's code and prompt appear consistent with that same version's rubric.

What It Answers

  • Does this score version implement its rubric?
  • Should I investigate a code/rubric mismatch before evaluation?
  • Is an item useful as spot-check context?

Use When

  • Before promotion.
  • Before a feedback evaluation.
  • When evaluation results suggest the prompt and rubric may disagree.

Avoid When

  • You need feedback item curation; use FeedbackContradictions.
  • You need a full evaluation; this is only a preflight check.

Run From The CLI

Direct report commands are the simplest path for one-off usage. Saved report configurations use `plexus report config create` and `plexus report run`.

plexus score contradictions \
  --scorecard "Customer Service QA" \
  --score "Medication Review: Dosage" \
  --version abc123-version-uuid \
  --format json

Minimal Configuration

# This is not a ReportBlock. It is a score command.
plexus score contradictions \
  --scorecard "Customer Service QA" \
  --score "Medication Review: Dosage" \
  --version abc123-version-uuid

Example Result

This check returns a compact verdict instead of a report block.

{
  "status": "potential_conflict",
  "paragraph": "The rubric says dosage does not require a separate customer repeat-back, but the score prompt still asks for explicit dosage acknowledgment.",
  "score_version_id": "abc123-version-uuid",
  "model": "gpt-5-mini"
}

How To Interpret It

  • `consistent` means no obvious mismatch was found; it is not a replacement for evaluation.
  • `potential_conflict` should block promotion until a human reviews the finding.
  • `inconclusive` usually means the rubric or code was missing or too ambiguous.