Evaluate Quality

Score calls against scorecards and view detailed evaluation results from the CLI

Create scorecards with weighted criteria, evaluate calls against them, and get per-criteria breakdowns with evidence -- all from your terminal.

The Workflow

You have a call. You want to know if the agent followed your quality standards. Here is how to score it from the CLI.

Step 1: Find your scorecard

chanl scorecards list

Scorecards
──────────────────────────────────────────────────────────────
ID                  Name                    Status   Categories
scorecard_abc123    Customer Service QA     active   3
scorecard_def456    Compliance Check        active   2
scorecard_ghi789    Sales Performance       draft    4

Step 2: Evaluate a call

chanl scorecards evaluate 695d6957e21c0ceb325c394d \
  --scorecard scorecard_abc123

Evaluation Complete
───────────────────────────────
Call:      695d6957e21c0ceb325c394d
Scorecard: Customer Service QA
Score:     85%
Status:    PASSED ✓

Category Scores:
  Communication:  90%
  Resolution:     80%
  Compliance:     85%

Step 3: See the detailed breakdown

chanl scorecards results 695d6957e21c0ceb325c394d

Evaluation Results: 695d6957e21c0ceb325c394d
─────────────────────────────────────────────

Customer Service QA (85% - PASSED)
  Communication (90%)
    ✓ Proper Greeting: 100%
      "Thank you for calling Acme Corp, this is Sarah speaking."
    ✓ Clear Explanation: 80%
      "Agent explained the refund process clearly."

  Resolution (80%)
    ✓ Issue Identified: 90%
    ✗ Solution Provided: 70%
      Suggestion: Could have offered expedited processing.

  Compliance (85%)
    ✓ Recording Disclosure: 100%
      Matched: "this call may be recorded"

Create a Scorecard

From CLI flags

chanl scorecards create \
  --name "Customer Service QA" \
  --description "Evaluates customer service interactions" \
  --threshold 75 \
  --status draft

✓ Created scorecard: scorecard_abc123

From a JSON file

chanl scorecards create --file scorecard.json

scorecard.json:

{
  "name": "Customer Service QA",
  "description": "Evaluates customer service interactions",
  "status": "draft",
  "passingThreshold": 75
}

Add Categories

Categories group related criteria with weights that sum to 100%.

chanl scorecards categories scorecard_abc123 add \
  --name "Communication" \
  --description "How well the agent communicates" \
  --weight 40

✓ Added category: cat_comm01

chanl scorecards categories scorecard_abc123 add \
  --name "Resolution" --weight 35

chanl scorecards categories scorecard_abc123 add \
  --name "Compliance" --weight 25

Add Criteria

Prompt criteria (AI-evaluated)

The AI reads the transcript and scores against your description:

chanl scorecards criteria scorecard_abc123 add \
  --category cat_comm01 \
  --key proper_greeting \
  --name "Proper Greeting" \
  --type prompt \
  --description "Agent greeted the customer professionally, stating their name and company"

✓ Added criterion: proper_greeting

Keyword criteria (pattern matching)

Checks for specific phrases in the transcript:

chanl scorecards criteria scorecard_abc123 add \
  --category cat_comp01 \
  --key recording_disclosure \
  --name "Recording Disclosure" \
  --type keyword \
  --keywords "this call may be recorded,call is being recorded" \
  --match-type must_contain

✓ Added criterion: recording_disclosure

Criteria types

Type	How It Works	Options
`prompt`	AI evaluates against your description	`--description "..."`
`keyword`	Pattern matches specific phrases	`--keywords "...,..."` `--match-type must_contain\|must_not_contain`
`response_time`	Measures response latency	`--max-ms 3000`
`talk_time`	Measures speaking duration ratio	`--min-percent 30` `--max-percent 70`
`silence`	Detects excessive silence	`--max-silence-ms 5000` `--max-count 3`
`interruptions`	Detects speaking overlap	`--max-count 2`

Evaluate Commands

Evaluate a single call

chanl scorecards evaluate 695d6957e21c0ceb325c394d \
  --scorecard scorecard_abc123

Force re-evaluation

chanl scorecards evaluate 695d6957e21c0ceb325c394d \
  --scorecard scorecard_abc123 \
  --force

JSON output for scripting

chanl scorecards evaluate 695d6957e21c0ceb325c394d \
  --scorecard scorecard_abc123 \
  --json

{
  "id": "result_xyz789",
  "callId": "695d6957e21c0ceb325c394d",
  "scorecardId": "scorecard_abc123",
  "score": 85,
  "passed": true,
  "categoryScores": {
    "Communication": 90,
    "Resolution": 80,
    "Compliance": 85
  }
}

View Results

All results for a call

chanl scorecards results 695d6957e21c0ceb325c394d

Filter by scorecard

chanl scorecards results 695d6957e21c0ceb325c394d \
  --scorecard scorecard_abc123

JSON output

chanl scorecards results 695d6957e21c0ceb325c394d --json

{
  "results": [
    {
      "scorecardId": "scorecard_abc123",
      "scorecardName": "Customer Service QA",
      "score": 85,
      "passed": true,
      "criteriaResults": [
        {
          "key": "proper_greeting",
          "name": "Proper Greeting",
          "score": 100,
          "passed": true,
          "evidence": "Thank you for calling Acme Corp, this is Sarah speaking."
        }
      ]
    }
  ]
}

Batch Evaluation

Evaluate all recent calls against a scorecard:

chanl calls list --status ended --json | \
  jq -r '.calls[].id' | \
  xargs -I {} chanl scorecards evaluate {} --scorecard scorecard_abc123

With rate limiting:

chanl calls list --status ended --json | \
  jq -r '.calls[].id' | \
  while read id; do
    chanl scorecards evaluate "$id" --scorecard scorecard_abc123
    sleep 1
  done

Export Results to CSV

echo "call_id,score,passed" > results.csv
chanl calls list --status ended --json | \
  jq -r '.calls[].id' | \
  while read id; do
    result=$(chanl scorecards results "$id" --json 2>/dev/null)
    if [ -n "$result" ]; then
      echo "$result" | jq -r '[.callId, .score, .passed] | @csv' >> results.csv
    fi
  done

Complete Example: Build and Use a Scorecard

# 1. Create the scorecard
chanl scorecards create \
  --name "Sales Quality" \
  --threshold 70 \
  --status draft

# 2. Add categories
chanl scorecards categories scorecard_new add \
  --name "Opening" --weight 30

chanl scorecards categories scorecard_new add \
  --name "Presentation" --weight 40

chanl scorecards categories scorecard_new add \
  --name "Closing" --weight 30

# 3. Add criteria
chanl scorecards criteria scorecard_new add \
  --category cat_opening \
  --key greeting \
  --name "Professional Greeting" \
  --type prompt \
  --description "Agent introduced themselves professionally"

chanl scorecards criteria scorecard_new add \
  --category cat_presentation \
  --key product_knowledge \
  --name "Product Knowledge" \
  --type prompt \
  --description "Agent demonstrated strong product knowledge"

chanl scorecards criteria scorecard_new add \
  --category cat_closing \
  --key next_steps \
  --name "Clear Next Steps" \
  --type prompt \
  --description "Agent provided clear next steps or call to action"

# 4. Activate it
chanl scorecards update scorecard_new --status active

# 5. Evaluate a call
chanl scorecards evaluate 695d6957e21c0ceb325c394d \
  --scorecard scorecard_new

Troubleshooting

Scorecards Commands

Full command reference for scorecards

Analyze Calls

Import calls to evaluate

SDK: Evaluate Quality

Build quality evaluation into your app

Platform: Scorecards

Scorecard concepts and dashboard usage

Evaluation returns 0% for prompt criteria

Keyword criteria not matching

Scorecard not found

Evaluation taking too long

Scorecards Commands

Analyze Calls

SDK: Evaluate Quality

Platform: Scorecards

On this page