Add support for streaming

[thelovekesh]

Summary

Fixes #100

This PR introduces the core changes needed for stable streaming support in text generation.

Here is a script to test streaming with text generation. It supports streaming of reasoning, content and tool calls.

stream-text.php

<?php
/**
 * Usage:
 *   AI_API_KEY=sk-... php stream-text.php "Tell me a short joke"
 *
 * Optional env:
 *   AI_BASE_URL  Base URL incl. version path. Default: https://api.openai.com/v1/
 *                DeepSeek: https://api.deepseek.com/   Groq: https://api.groq.com/openai/v1/
 *   AI_MODEL     Model id. Default: gpt-4o-mini
 */

declare(strict_types=1);

use GuzzleHttp\Client;
use WordPress\AiClient\Messages\DTO\MessagePart;
use WordPress\AiClient\Messages\DTO\UserMessage;
use WordPress\AiClient\Providers\DTO\ProviderMetadata;
use WordPress\AiClient\Providers\Enums\ProviderTypeEnum;
use WordPress\AiClient\Providers\Http\DTO\ApiKeyRequestAuthentication;
use WordPress\AiClient\Providers\Http\DTO\Request;
use WordPress\AiClient\Providers\Http\Enums\HttpMethodEnum;
use WordPress\AiClient\Providers\Http\HttpTransporter;
use WordPress\AiClient\Providers\Models\DTO\ModelConfig;
use WordPress\AiClient\Providers\Models\DTO\ModelMetadata;
use WordPress\AiClient\Providers\Models\Enums\CapabilityEnum;
use WordPress\AiClient\Providers\OpenAiCompatibleImplementation\AbstractOpenAiCompatibleTextGenerationModel;
use WordPress\AiClient\Tools\DTO\FunctionDeclaration;

require_once __DIR__ . '/vendor/autoload.php';

if (!class_exists(Client::class)) {
    fwrite(STDERR, "Guzzle is required. Install it:\n  composer require --dev guzzlehttp/guzzle\n");
    exit(1);
}

$apiKey = getenv('AI_API_KEY') ?: '';
if ($apiKey === '') {
    fwrite(STDERR, "Set AI_API_KEY, e.g.:\n  AI_API_KEY=sk-... php stream-text.php \"Hello\"\n");
    exit(1);
}

$modelId = getenv('AI_MODEL') ?: 'gpt-4o-mini';
$baseUrl = getenv('AI_BASE_URL') ?: 'https://api.openai.com/v1/';

$prompt = $argv[1] ?? 'Plan a 12-day trip across Japan and South Korea in May 2026 for 2 adults '
    . 'and 1 child, budget about $9000, prefer trains between cities, with a couple of activities per stop.';

$model = new class ($modelId, $baseUrl) extends AbstractOpenAiCompatibleTextGenerationModel {
    private string $baseUrl;

    public function __construct(string $modelId, string $baseUrl)
    {
        parent::__construct(
            new ModelMetadata($modelId, $modelId, [CapabilityEnum::textGeneration()], []),
            new ProviderMetadata('openai-compatible', 'OpenAI-compatible', ProviderTypeEnum::cloud())
        );
        $this->baseUrl = rtrim($baseUrl, '/') . '/';
    }

    protected function createRequest(
        HttpMethodEnum $method,
        string $path,
        array $headers = [],
        $data = null
    ): Request {
        return new Request($method, $this->baseUrl . ltrim($path, '/'), $headers, $data);
    }
};

$planTrip = new FunctionDeclaration(
    'plan_trip',
    'Build a detailed multi-city travel itinerary from the user request.',
    [
        'type' => 'object',
        'properties' => [
            'travelers' => [
                'type' => 'object',
                'properties' => [
                    'adults' => ['type' => 'integer', 'minimum' => 1],
                    'children' => ['type' => 'integer', 'minimum' => 0],
                ],
                'required' => ['adults'],
            ],
            'date_range' => [
                'type' => 'object',
                'properties' => [
                    'start' => ['type' => 'string', 'description' => 'ISO 8601 date, e.g. 2026-05-01'],
                    'end' => ['type' => 'string', 'description' => 'ISO 8601 date'],
                ],
                'required' => ['start', 'end'],
            ],
            'budget_usd' => ['type' => 'number', 'description' => 'Total budget in USD.'],
            'primary_transport' => [
                'type' => 'string',
                'enum' => ['flight', 'train', 'car', 'mixed'],
            ],
            'stops' => [
                'type' => 'array',
                'description' => 'Ordered list of cities to visit.',
                'items' => [
                    'type' => 'object',
                    'properties' => [
                        'city' => ['type' => 'string'],
                        'country' => ['type' => 'string'],
                        'nights' => ['type' => 'integer', 'minimum' => 1],
                        'activities' => [
                            'type' => 'array',
                            'items' => ['type' => 'string'],
                        ],
                        'accommodation' => [
                            'type' => 'string',
                            'enum' => ['hostel', 'hotel', 'ryokan', 'apartment'],
                        ],
                    ],
                    'required' => ['city', 'country', 'nights'],
                ],
            ],
        ],
        'required' => ['travelers', 'date_range', 'stops'],
    ]
);

$config = new ModelConfig();
$config->setFunctionDeclarations([$planTrip]);
$config->setSystemInstruction(
    'You are a meticulous travel planner. First write one short sentence telling the user what '
    . 'you are about to do, then call the plan_trip tool with detailed, structured arguments.'
);
$config->setCustomOption('tool_choice', 'required');
$model->setConfig($config);

$model->setHttpTransporter(new HttpTransporter(new Client()));
$model->setRequestAuthentication(new ApiKeyRequestAuthentication($apiKey));

fwrite(STDERR, "Streaming from {$baseUrl} ({$modelId})\n\n");

try {
    $stream = $model->streamGenerateTextResult([new UserMessage([new MessagePart($prompt)])]);

    $section = '';
    $toolFragments = 0;
    foreach ($stream as $chunk) {
        $reasoning = $chunk->getReasoningDeltaText();
        if ($reasoning !== '') {
            if ($section !== 'reasoning') {
                echo "\n[reasoning]\n";
                $section = 'reasoning';
            }
            echo $reasoning;
            flush();
        }

        $answer = $chunk->getDeltaText();
        if ($answer !== '') {
            if ($section !== 'answer') {
                echo PHP_EOL . '[answer] ';
                $section = 'answer';
            }
            echo $answer;
            flush();
        }

        foreach ($chunk->getToolCallDeltas() as $toolCall) {
            $name = $toolCall->getFunctionName();
            if ($name !== null) {
                echo PHP_EOL . "[tool-call] {$name} args (each ⟨…⟩ is one streamed chunk):" . PHP_EOL;
                $section = 'tool';
            }
            $fragment = $toolCall->getArgumentsFragment();
            if ($fragment !== '') {
                echo '⟨' . $fragment . '⟩';
                $toolFragments++;
                flush();
            }
        }
    }
    echo "\n";

    $result = $stream->getFinalResult();

    foreach ($result->getCandidates() as $candidate) {
        foreach ($candidate->getMessage()->getParts() as $part) {
            $functionCall = $part->getFunctionCall();
            if ($functionCall !== null) {
                fwrite(STDERR, sprintf(
                    "\n[assembled call] %s(%s)",
                    (string) $functionCall->getName(),
                    (string) json_encode($functionCall->getArgs(), JSON_PRETTY_PRINT)
                ));
            }
        }
    }

    $usage = $result->getTokenUsage();
    fwrite(STDERR, sprintf(
        "\n\n[done] tool-call fragments received: %d | tokens: %d prompt + %d completion = %d total\n",
        $toolFragments,
        $usage->getPromptTokens(),
        $usage->getCompletionTokens(),
        $usage->getTotalTokens()
    ));
} catch (\Throwable $e) {
    fwrite(STDERR, "\n[error] " . $e->getMessage() . "\n");
    exit(1);
}

Tasks

• Add stream support in Request and Response.
• Add EventStreamParserInterface and SSE Parser.
• Add streaming support for text generation covers both reasoning and content.
• Update AbstractOpenAiCompatibleTextGenerationModel to use streaming in text generation.
• Add initial implementation for chunk aggregation.
• Add event dispatch on text result generation with streaming.
• Update prompt builder and ai client to consume streaming for text generation.
• Add streaming support for function calls.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.20%. Comparing base (052a468) to head (1bd7ca9).

Additional details and impacted files

@@             Coverage Diff              @@
##              trunk     #255      +/-   ##
============================================
+ Coverage     88.16%   90.20%   +2.04%     
- Complexity     1217     1448     +231     
============================================
  Files            61       69       +8     
  Lines          3945     4493     +548     
============================================
+ Hits           3478     4053     +575     
+ Misses          467      440      -27     

Flag	Coverage Δ
unit	90.20% <100.00%> (+2.04%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[github-actions]

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: thelovekesh <thelovekesh@git.wordpress.org>
Co-authored-by: felixarntz <flixos90@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

[thelovekesh]

Will add docs once the API is reviewed and approved.

[thelovekesh]

If a model doesn't implement this interface, it will fail even if the model supports text generation. It fails because streaming is gated behind this opt-in interface that no provider actually implements. We can't discover our way around it either, since streaming isn't a CapabilityEnum, so discovery can't tell which models stream and may still pick one that doesn't.

The clean solution is to remove StreamingTextGenerationModelInterface and rather add streamGenerateTextResult(array $prompt): StreamedGenerativeAiResult to TextGenerationModelInterface. Streaming is a networking primitive, not a real model capability, so every text model can stream: natively where the provider's API supports it, or by emulating it (one generateTextResult() call yielded as a single chunk) where it doesn't. Adding a method to the interface sounds like a breaking change, but if we put the emulation default in the shared base (AbstractApiBasedModel) that every provider already extends, they all inherit it and nothing breaks.

Another way is to add a new capability and wire discovery to find a model with it, but that isn't standard given streaming isn't a model-level capability, it's a networking primitive, not something a model generates. We'd also still need a method on an interface plus every provider opting in, so it's more machinery for something that isn't really model-level.

[thelovekesh]

Created a tiny server to see streaming in action - https://gist.github.com/thelovekesh/a05a99570989672ad4b493b81a4b6dc3

[justlevine]

Didn't do a real review yet, just a quick note: can you make sure the PHPDoc types use FQCN? E.g.

Suggested change

	* @return StreamedGenerativeAiResult The streamed result.
	* @throws InvalidArgumentException If the prompt or model validation fails.
	* @throws RuntimeException If the model does not support streaming text generation.
	* @return \WordPress\AiClient\Results\StreamedGenerativeAiResult The streamed result.
	* @throws \InvalidArgumentException If the prompt or model validation fails.
	* @throws \RuntimeException If the model does not support streaming text generation.