Skip to main content

Testing Prompts

Scope includes a built-in test panel that lets you run prompts against real LLM providers, compare outputs across models, and review performance metrics — all before promoting to production.

Using the Test Panel​

  1. Open a prompt and select the version you want to test
  2. Click Test to open the test panel
  3. Configure the test:
    • Provider — select a configured provider (e.g., OpenAI)
    • Model — select a model from the provider (e.g., gpt-4o)
    • Variables — fill in values for each detected variable
    • Model config (optional) — set parameters like temperature, max_tokens
  4. Click Run

Understanding Results​

After execution, the test panel displays:

MetricDescription
ResponseThe LLM's generated output
Resolved contentYour prompt after variable substitution
Prompt tokensNumber of tokens in the input
Completion tokensNumber of tokens in the output
Total tokensCombined input + output tokens
LatencyResponse time in milliseconds
CostEstimated cost based on the model's pricing

Multi-Model Comparison​

Compare the same prompt across multiple models simultaneously:

  1. In the test panel, select Multi-model mode
  2. Add up to 10 provider/model combinations
  3. Fill in variable values (shared across all models)
  4. Click Run All

Results appear side-by-side, making it easy to compare response quality, latency, and cost across models.

tip

Multi-model comparison is useful for choosing the best model for a prompt or validating that a cheaper model produces acceptable results.

Testing with History​

Re-run a prompt using parameters from a previous execution:

  1. Select a previous execution from the test history
  2. Click Re-test or use the "Test with History" feature
  3. Scope runs the prompt with the same variables and model configuration
  4. Compare the new output with the original output side by side

This is useful for regression testing when you modify prompt content — you can verify that the new version produces similar or better results with the same inputs.

Next Steps​

Was this page helpful?