Prompts

Manage prompt text as a first-class artifact, run it from Python or TypeScript, and keep production prompt versions tied to code.

A Prompt is a managed LLM prompt that can be drafted and tested in AGNT5, then committed with your application code for production. Use Prompts when the prompt itself is an artifact you want to version, evaluate, and deploy, instead of burying instructions inside function bodies.

In development, a Prompt can resolve through AGNT5’s prompt store so you can test drafts quickly. In production, AGNT5 resolves Prompts from files bundled with the deployed code artifact. That keeps production behavior reproducible: the code version and prompt version move together.

Run a Prompt

Python:

from agnt5 import lm
from agnt5.lm import Prompt

response = await lm.generate(
    model="openai/gpt-4o-mini",
    prompt=Prompt(
        id="support_reply",
        variables={
            "customer": {"name": "Ada"},
            "topic": "shipping",
        },
    ),
)

print(response.text)

TypeScript:

import { LM } from "@agnt5/sdk";

const lm = LM.openai();

const response = await lm.generate({
  model: "openai/gpt-4o-mini",
  prompt: {
    id: "support_reply",
    variables: {
      customer: { name: "Ada" },
      topic: "shipping",
    },
  },
});

console.log(response.text);

prompt replaces inline messages for the call. If you provide a Prompt, do not also pass raw prompt text or message arrays for the same generation request.

Commit a production Prompt

Create a prompt file under prompts/<id>.mdx in your application repo:

---
id: support_reply
version: 3
version_id: 018f0000-0000-7000-8000-000000000003
model: openai/gpt-4o-mini
temperature: 0.2
max_tokens: 600
variables:
  - customer.name
  - topic
response_format: text
---

<System>
You are a concise support agent.
</System>

<User>
Reply to {{customer.name}} about {{topic}}.
</User>

AGNT5 parses the front matter for routing and generation settings, then renders the body into ordered chat messages. The body supports <System>, <User>, and <Assistant> blocks. If no block is present, the body is treated as a single user message.

Production Prompt resolution looks in this order:

AGNT5_PROMPT_OVERRIDE
AGNT5_PROMPTS_MANIFEST
prompts/<id>.mdx
prompts/<id>.md
AGNT5 prompt-run API fallback for non-production draft and test workflows

Prompt files must be Markdown or MDX. AGNT5 does not load prompt text from prompts.lock or JSON files; prompts are meant to be readable authored artifacts.

In production environments, the Prompt must be bundled with the deployed artifact. Missing Prompts fail closed and do not fall back to control-plane state.

Select a version

Use version when constructing the Prompt object, or pass the SDK’s version option for the call.

Python:

response = await lm.generate(
    model="openai/gpt-4o-mini",
    prompt=Prompt(id="support_reply", version="version-3"),
)

TypeScript:

const response = await lm.generate({
  model: "openai/gpt-4o-mini",
  prompt: { id: "support_reply", version: "version-3" },
});

The version can match either the prompt file’s version or version_id.

Override runtime settings

Use runtime LLM overrides when you want to test the same prompt with a different model or generation settings. This is useful in the Playground, experiments, and one-off comparisons because workflow code can stay unchanged.

Python:

ctx.runtime.llm.model = "openai/gpt-4o"
ctx.runtime.llm.temperature = 0.6
ctx.runtime.llm.max_tokens = 800
ctx.runtime.llm.top_p = 0.9

response = await lm.generate(
    model="openai/gpt-4o-mini",
    prompt=Prompt(
        id="support_reply",
        variables={"topic": "shipping"},
    ),
)

TypeScript:

ctx.runtime.llm.model = "openai/gpt-4o";
ctx.runtime.llm.temperature = 0.6;
ctx.runtime.llm.maxOutputTokens = 800;
ctx.runtime.llm.topP = 0.9;

const response = await lm.generate({
  model: "openai/gpt-4o-mini",
  prompt: {
    id: "support_reply",
    variables: { topic: "shipping" },
  },
});

For workflows with multiple prompts, set prompt-specific overrides by prompt id:

ctx.runtime.prompts["draft"] = LLMRuntimeOptions(
    model="anthropic/claude-3-5-haiku-20241022",
    temperature=0.7,
)
ctx.runtime.prompts["review"] = LLMRuntimeOptions(
    model="openai/gpt-4o",
    temperature=0.3,
)

await lm.generate(model="openai/gpt-4o-mini", prompt=Prompt(id="classify"))
await lm.generate(model="openai/gpt-4o-mini", prompt=Prompt(id="draft"))
await lm.generate(model="openai/gpt-4o-mini", prompt=Prompt(id="review"))

The Prompt file remains the source of truth for prompt text. Runtime overrides only change model execution settings for the current run. Prompt-specific overrides win over the global runtime default.

Use Prompts inside workflows

Prompts work anywhere you can call the language model SDK. Inside workflows, keep the LLM call inside a checkpointed step so replay can reuse the completed result.

from agnt5 import function, FunctionContext, lm
from agnt5.lm import Prompt

@function
async def draft_reply(ctx: FunctionContext, customer_name: str, topic: str) -> str:
    response = await lm.generate(
        model="openai/gpt-4o-mini",
        prompt=Prompt(
            id="support_reply",
            variables={
                "customer": {"name": customer_name},
                "topic": topic,
            },
        ),
    )
    return response.text

When this function is called through ctx.step() from a workflow, the generated reply is checkpointed with the rest of the step result.

Compatibility

Older SDK code may still use PromptRef, prompt_ref, or promptRef. Those names remain supported for compatibility, but new docs and new code should use Prompt as the concept:

# Supported, but prefer prompt=Prompt(...)
await lm.generate(
    model="openai/gpt-4o-mini",
    prompt_ref="support_reply",
    variables={"topic": "shipping"},
)

// Supported, but prefer prompt: { id: "support_reply" }
await lm.generate({
  model: "openai/gpt-4o-mini",
  promptRef: "support_reply",
  variables: { topic: "shipping" },
});

Use PromptRef only when maintaining older code that already uses that terminology.