Reports

Core Components

The reporting system is built around four key concepts:

• Report Configuration (`ReportConfiguration`)

  This is the template or blueprint for a specific type of report. It's typically defined using Markdown mixed with simple configuration blocks. It specifies the static content (text, headers), the dynamic analysis blocks to include, and any parameters needed for those blocks. Configurations are stored and managed within Plexus, allowing reuse.

  Think of it like a recipe for generating a specific kind of analysis.

  # Score Performance Report
  
  This report analyzes the performance of the 'Agent Professionalism' score.
  
  ```block name="Professionalism Score Analysis"
  class: ScorePerformanceBlock
  scorecard: "Customer Service v2"
  score: "Agent Professionalism"
  time_range: last_30_days
  ```
  

• Report (`Report`)

  This represents a specific instance of a report generated from a `ReportConfiguration` at a particular time, potentially with specific runtime parameters. `Report.output` stores the report template markdown, while execution results are stored in the related `ReportBlock` records.

  If the Configuration is the recipe, the Report is the finished cake, baked on a specific day.

• Report Block (`ReportBlock`)

  These are the reusable Python components that perform the actual data fetching and analysis for specific sections within a report. Examples might include `ScorePerformanceBlock`, `FeedbackTopicAnalysisBlock`, or `SentimentTrendBlock`. When a report is generated, the system executes the Python code for each block defined in the configuration.

  Each block generates structured data (usually JSON) containing its findings (e.g., metrics, lists of feedback, chart data) and optionally logs. Large outputs and logs may be stored as S3-backed attachments and referenced from the block record.

• Task Integration (`Task`)

  Report generation is tracked with the existing `Task` system. Dashboard-dispatched runs typically execute in workers, while CLI runs execute synchronously in-process, but both create and update Task records for progress/status visibility. The `Report` record itself is directly linked to its corresponding `Task` record.

How It Works

The typical workflow for using reports involves these steps:

1. Define a `ReportConfiguration`

   Create a template (using Markdown and block definitions) for the type of report you need. This is often done once and then reused. This can be done using the Plexus CLI, the Dashboard UI, or programmatically via the API/SDK (used by AI agents).

   # Example: Creating a config from a file
   plexus report config create --name "Agent Prof Report" --file agent_prof_report.md

2. Run the Report

   Trigger the generation of a new `Report` based on a chosen `ReportConfiguration`. This creates/updates a `Task` for observability; worker-dispatched and synchronous CLI execution paths both use the same core generation flow.

   # Example: Running the report
   plexus report run --config "Agent Prof Report"

3. Monitor the Task

   Track the progress of the report generation via the associated `Task`.

4. View the `Report`

   Once the task is complete, view the generated `Report`. This includes the rendered output (e.g., Markdown display) and access to the structured data generated by each `ReportBlock` for deeper dives or visualization.

   # Example: Viewing the latest generated report
   plexus report last

Benefits

• Flexibility: Define custom analyses without needing new API endpoints or dedicated UI pages for each report type.
• Reusability: Report Configurations and Report Blocks can be reused across different reports.
• Consistency: Leverages the standard `Task` system for progress tracking and status updates.
• Extensibility: New analysis types can be added by creating new Python `ReportBlock` classes.
• Structured Data: Provides both human-readable output (Markdown) and machine-readable structured data (JSON) from analysis blocks.

Next Steps

Explore how Reports integrate with other core concepts like Scores and Evaluations. Check the CLI documentation for detailed commands on managing and running reports, or browse the available report blocks to see what types of analysis you can include in your reports.

For stakeholder update workflows, the ScorecardHistory report block summarizes featured score-version changes across a scorecard window, including champion coverage, high-level change summaries, and evaluation gauge context.