Testing Introduction

@llmist/testing provides utilities to mock LLM responses in your test suite. Write deterministic tests for AI-powered applications without making real API calls.

Why Mock LLM Responses?

Deterministic Tests

Get the same response every time - no flaky tests due to LLM variability

Fast Execution

No network calls means tests run in milliseconds, not seconds

No API Costs

Run thousands of tests without spending on API calls

Edge Cases

Test error handling, timeouts, and unusual responses easily

Key Features

MockBuilder - Fluent API to define mock responses based on model and message content
testGadget() - Test gadgets in isolation without an agent
createMockGadget() - Create spy gadgets to verify calls
Stream Helpers - Collect and inspect streaming responses
CLI Helpers - Test CLI applications with mocked stdin/stdout

Quick Example

import { describe, it, expect } from 'vitest';
import { mockLLM, createMockClient } from '@llmist/testing';

describe('My AI Feature', () => {
  it('should process user requests', async () => {
    // Set up mock response
    mockLLM()
      .forAnyModel()
      .whenMessageContains('hello')
      .returns('Hello! How can I help you today?')
      .register();

    // Create client with mock provider
    const client = createMockClient();

    // Run your agent
    const response = await client.createAgent()
      .withModel('sonnet')
      .askAndCollect('hello');

    expect(response).toContain('Hello');
  });
});

What You Can Test

Scenario	Tool
Agent responses	`mockLLM()` + `createMockClient()`
Gadget execution	`testGadget()`
Gadget call verification	`createMockGadget()`
Multi-turn conversations	`mockLLM().times(n)`
Error handling	`mockLLM().throwsError()`
Streaming behavior	`collectStream()`

Next Steps

Installation Add @llmist/testing to your project

Quick Start Write your first mock test

MockBuilder API Full mocking capabilities

Testing Gadgets Test gadgets in isolation