Report Blocks

Report blocks are reusable analysis components that generate metrics, charts, review queues, and supporting context for Plexus reports. These pages show how to run each report, configure it, and interpret the rendered dashboard block.

Run Reports From The CLI

Use direct feedback report commands for quick analysis, or create a saved report configuration when you want a reusable report template.

# Direct one-off report
plexus feedback report alignment --scorecard "Customer Service QA" --score "Medication Review: Dosage" --days 30

# Reusable report configuration
plexus report config create --name "Dosage Feedback Overview" --file dosage-report.md
plexus report run --config "Dosage Feedback Overview"

`FeedbackAnalysis` is still accepted as an alias for `FeedbackAlignment`; new documentation uses `FeedbackAlignment`.

Feedback quality

FeedbackAlignment

Analytics
Report block

Measures agreement between AI score results and human feedback using AC1, accuracy, class distribution, and confusion matrices.

Answers

  • How aligned is this score with human feedback?
  • Which labels are being confused?
  • Which scores need attention first?

FeedbackContradictions

Curation
Report block

Finds feedback edits that appear to contradict the current rubric, grouping likely invalid feedback into reviewable topics.

Answers

  • Which feedback items may be invalid under the current rubric?
  • What policy themes explain the contradictions?
  • Which examples should be reviewed before feedback curation?

RecentFeedback

Review
Report block

Shows the most recent feedback items for a score or scorecard, including reviewer edits and item identifiers.

Answers

  • What are reviewers correcting right now?
  • Which recent examples should I inspect?
  • Is new feedback arriving for this score?

Trends

FeedbackAlignmentTimeline

Trend
Report block

Plots stored feedback alignment across time buckets with rolling minimum samples so teams can see whether agreement is improving or drifting.

Answers

  • Is alignment improving over time?
  • Did a release or rubric change move AC1?
  • Which completed periods had no feedback?

FeedbackVolumeTimeline

Trend
Report block

Shows how much feedback arrived in each completed time bucket.

Answers

  • Is feedback volume stable?
  • Were there enough recent examples to trust an alignment change?
  • Which periods need more review coverage?

ScoreChampionVersionTimeline

Optimizer
Report block

Shows score champion-version changes during a requested time window, with per-score timelines, optimizer cost, SME context, diffs, and feedback/regression evaluation metrics.

Answers

  • Which scores changed champion versions during this window?
  • Did champion feedback and regression metrics improve?
  • What changed between the prior champion and the latest champion?

AcceptanceRateTimeline

Trend
Report block

Tracks score-result acceptance rate over time for stakeholder-friendly monitoring.

Answers

  • Are reviewers accepting more AI decisions over time?
  • Did acceptance improve after a score release?
  • Which periods had weak acceptance?

ScorecardHistory

Stakeholder
Report block

Summarizes starred score-version changes over a time window, with champion coverage, high-level change bullets, and optional evaluation gauge context.

Answers

  • What changed across this scorecard in the requested window?
  • Which changes were champion-related (all, some, or none)?
  • What stakeholder/SME questions should we ask next?

Operational

AcceptanceRate

Marketing
Report block

Measures the share of score results, and optionally items, accepted by reviewers without correction.

Answers

  • How often do reviewers accept AI decisions?
  • Can we show a simple positive trust metric?
  • Which recent items were fully accepted or corrected?

CorrectionRate

Operations
Report block

Shows how often feedback changes AI answers, which is the inverse operational view of acceptance.

Answers

  • How many reviewed results were corrected?
  • Which recent items required changes?
  • How large is the feedback correction workload?

CostAnalysis

Cost
Report block

Summarizes score-result cost and call-count distribution across scorecards, scores, or windows.

Answers

  • Which scores are expensive?
  • What is average cost per item?
  • Where should we optimize model or prompt cost?

Analysis

TopicAnalysis

NLP
Report block

Clusters text into topic groups using the report topic-analysis pipeline.

Answers

  • What topics appear in this text corpus?
  • Which examples represent each topic?
  • What keywords describe each cluster?

ExplanationAnalysis

NLP
Report block

Clusters score-result explanations to reveal recurring model reasoning patterns.

Answers

  • What reasons does the model give most often?
  • Which reasoning patterns correlate with misses?
  • Where are explanations repetitive or vague?

VectorTopicMemory

Memory
Report block

Rebuilds a topic-memory view using vector clustering so recurring topics can be tracked across runs.

Answers

  • Which topics are recurring?
  • Which topics are new or trending?
  • What has persisted in memory?

ActionItems

Workflow
Report block

Turns alignment and topic-memory findings into prioritized score-improvement action items.

Answers

  • What should the team work on next?
  • Which score/topic combination needs attention?
  • Which examples support the action item?

ScoreInfo

Context
Report block

Displays compact score metadata inside a report.

Answers

  • Which score is this report about?
  • What is the score description?
  • When was this score last updated?

Related checks

Score/Rubric Consistency Check

Preflight
Related check

Checks whether a ScoreVersion's code and prompt appear consistent with that same version's rubric.

Answers

  • Does this score version implement its rubric?
  • Should I investigate a code/rubric mismatch before evaluation?
  • Is an item useful as spot-check context?

Related Documentation

Reports concept overviewCLI command referenceRubric memory