Running.

createJev#

createJev(options) makes a client with run and stream attached. Make one per process and share it: the concurrency limit and the batching queue live on the client.

import { createJev } from "jevchain";

const jev = createJev({
  model: "jev-1.13.0",          // pin a version instead of jev-latest
  timeoutMs: _000,             // per attempt
  retry: { maxRetries: 3 },     // merged over the defaults
  maxConcurrency: 16,
  batch: { windowMs: 5 },       // wait 5ms for siblings before sending
});

Retries cover timeouts, connection failures and HTTP 408, 409, 429, 500, 502, 503, 504 and 529. On a 429 the client honours retry-after (or retry-after-ms) up to maxRetryAfterMs; otherwise it backs off exponentially with jitter. Every retry lands in the trace.

import { createJev, type JevClient } from "jevchain";

const fake: JevClient = {
  model: "jev-latest",
  usdPerMillionTokens: 0.042,
  async ask(state, questions) {
    return { answers: cannedAnswersFor(questions), model: "jev-1.13.0",
             usage: { inputTokens: 0, outputTokens: 0 }, costUsd: 0, latencyMs: 1, attempts: 1 };
  },
};

const jev = createJev(fake); // run() and stream() work as usual, no network

run#

jev.run(chain, input, options?) runs to completion and resolves with the output and the trace. The input is type-checked against the chain's input type.

Statuses#

run doesn't throw for things that go wrong at runtime. Network down, key revoked, your step exploded: you get a result with a status and the trace up to that point. RunResult is a discriminated union, so TypeScript knows output only exists when status is "ok".

const result = await jev.run(desk, "My toaster whispers my name at 3am.");

switch (result.status) {
  case "ok":
    reply(result.output);          // typed as the chain's output
    break;
  case "halted":
    log(result.trace.halted?.summary);
    break;
  case "error":
  case "aborted":
    alert(result.error.code);      // a JevChainError, never undefined here
    break;
}

save(result.trace);                 // always there, whatever happened

Streaming#

jev.stream(chain, input, options?) starts the same run and hands you its events as an async iterable. s.result resolves to the same RunResult run would have returned. Break out of the loop and the run is aborted; a stream nobody iterates just runs to the end.

const s = jev.stream(desk, ticket);

for await (const event of s) {
  if (event.type === "decision") console.log(event.path, event.decision.summary);
}

const { status, trace } = await s.result;

To keep a live view, fold events with reduceTrace. The runtime builds its own trace with this same function, so what you render mid-run and the trace you store at the end can't disagree. traceFromEvents(events) does the whole fold at once, handy for replaying a recorded event log.

import { reduceTrace, type Trace } from "jevchain";

let trace: Trace | undefined;
for await (const event of jev.stream(desk, ticket)) {
  trace = reduceTrace(trace, event);   // pure: returns a new object, safe for React state
  render(trace);                       // partial traces are valid traces (status "running")
}

Cancellation and deadlines#

Two ways to stop a run early, and they report differently:

const controller = new AbortController();
stopButton.onclick = () => controller.abort();

const result = await jev.run(desk, ticket, {
  signal: controller.signal,  // abort → status "aborted"
  timeoutMs: _000,           // deadline for the whole run → status "error", code "timeout"
});

• Your signal aborts in-flight requests, sleeps between retries and queued batches, and the run ends with status "aborted".
• The run deadline (timeoutMs on run) cancels the same way, but the reason is a JevTimeoutError, so the status is "error" with error.code === "timeout". You asked for an answer by a time; not getting one is a failure.
• Either way, spans still running are closed with status error and that same reason as their error (code aborted or timeout), and the partial trace comes back as usual.
• Stopping means stopping: calls still queued for a concurrency slot are never sent, a retry backoff wakes up and quits, and nothing new starts.
• Leaving a stream's for await loop early (break, return, a throw) aborts the run the same way. s.result resolves with status "aborted".

A failure inside a parallel cancels its sibling branches too; they close with a cancelled error whose cause is the real failure (see parallel). Your own code gets the signal as ctx.signal; pass it along to anything that can be cancelled. Calls a step makes through ctx.jev are cancelled with it already:

step("lookup", async (ticket: Ticket, ctx) => {
  const res = await fetch(`/api/customers/${ticket.customerId}`, { signal: ctx.signal });
  return res.json();
});