🧪 Agent Regression Pack Workbench

Run deterministic fixture validation, then create named sanitized regression packs and compare versions by stable fixture ID.

Offline validation only: this workbench does not replay HTTP requests, call tools, execute fixtures, or contact providers. Bindings may reference past steps only.

Pack nameVersion

Fixture packet JSON

{
  "fixtures": [
    {
      "id": "rpc-request",
      "type": "jsonrpc-request",
      "payload": {
        "jsonrpc": "2.0",
        "id": 1,
        "method": "users.get",
        "params": {
          "userId": "u1"
        }
      },
      "expect": {
        "valid": true
      }
    },
    {
      "id": "rpc-response",
      "type": "jsonrpc-response",
      "payload": {
        "jsonrpc": "2.0",
        "id": 1,
        "result": {
          "userId": "u1",
          "name": "Ada"
        }
      },
      "expect": {
        "valid": true
      }
    },
    {
      "id": "tool-call",
      "type": "tool-call",
      "payload": {
        "name": "notify_user",
        "arguments": {
          "userId": "{{steps.rpc-response.payload.result.userId}}",
          "authorization": "Bearer sk-example-secret-123456"
        }
      },
      "expect": {
        "valid": true
      }
    },
    {
      "id": "typed-error",
      "type": "typed-error",
      "payload": {
        "code": "NOT_FOUND",
        "message": "No user"
      },
      "expect": {
        "valid": true
      }
    }
  ]
}

Run summary

Deterministic report · utilito.agent-fixtures.v1Sanitized fixture packetVersioned regression pack · utilito.agent-regression-pack.v1

Compare with a baseline pack

Paste an older exported pack. Diff evidence lists only stable fixture IDs as added, removed, changed, or unchanged.

Baseline pack JSON

Limits: 20 fixtures, 256,000 bytes, depth 30, pack names 80 characters, and versions 32 characters. Secrets are redacted before pack or diff export.

Runs in your browser

Fixtures and packs stay in this browser. Exports are sanitized, but review them before sharing.

Include current input in shared/debug payload

What this tool does

Validates agent fixtures and packages sanitized named versions for deterministic offline regression comparison.

Why it is useful

Stable fixture packs make provider and tool-call contract changes reviewable without executing requests or preserving secrets.

Deterministic pack diff

Packs compare fixtures by stable ID and report added, removed, changed, and unchanged IDs only.

No execution

Validation and diffing never replay HTTP requests, call tools, or contact model providers.

Privacy note

All parsing, redaction, versioning, and diffing run locally.

Important limitation

A passing fixture validates supported structure only; it does not prove the real provider or tool will succeed.

Quick answers

Are secrets exported?

Recognized secret keys and token patterns are redacted before pack export.

Can this replay requests?

No. Execution and outbound replay are explicitly disabled.

Common Use Cases

Version a tool-call contract

Save a sanitized pack before changing schemas, then compare the next version.

Review agent error changes

Diff typed-error and JSON-RPC fixtures by stable ID without exposing payload secrets.

View all in Developer →