Formalising Prompts as First-Class Research Objects

Most LLM-powered systems treat prompts as strings. They live inside application code, concatenated at runtime, shaped by whatever felt right during a debugging session at 2am. They are rarely versioned independently of the code that hosts them. They are almost never tested in isolation. And when something breaks — when the model starts producing garbage after what should have been a routine deployment — nobody can point to the diff that caused it.

This is not an acceptable state of affairs for a core component of system behaviour.

We argue that prompts deserve the same engineering rigour that we apply to source code: explicit types, version control, portability guarantees, and formal specifications. This is the thesis behind promptel, our declarative prompt specification language, and the extraction tooling in blogus.

The Problem with Prompts-as-Strings

Consider a typical LLM integration. Somewhere in your codebase, buried inside a function, there is a string literal or template that constitutes the actual instruction to the model. It might look like this:

def summarise(text: str, model: str = "gpt-4") -> str:
    response = client.chat.completions.create(
        model=model,
        messages=[
            {"role": "system", "content": "You are a concise summariser. "
             "Output exactly 3 bullet points. No preamble."},
            {"role": "user", "content": f"Summarise:\n\n{text}"}
        ],
        temperature=0.3
    )
    return response.choices[0].message.content

This code conflates at least four concerns: the prompt text, the model selection, the inference parameters, and the I/O plumbing. The prompt itself — the actual instruction that determines system behaviour — is tangled into the application logic. You cannot version it independently. You cannot test it without invoking the full function. You cannot port it to a different provider without rewriting the integration code.

Worse, there is no schema. Nothing declares what text should look like, what the expected output shape is, or what constraints the prompt assumes. The contract between the prompt author and the model is entirely implicit.

Prompts as Typed Artefacts

promptel takes a different approach. A prompt is a standalone, declarative specification:

# summarise.yml
name: Summariser

meta:
  version: "1.0"
  description: Produce a three-bullet summary of input text

params:
  text:
    type: string
    required: true
  audience:
    type: string
    required: false
    default: "general"

body:
  text: |
    You are a concise summariser. Output exactly 3 bullet points
    for a ${params.audience} audience. No preamble.

    Summarise:

    ${params.text}

constraints:
  maxTokens: 256
  temperature: 0.3

The same prompt can be written in promptel’s native .prompt DSL, and the CLI converts between the two (promptel --convert prompt -f summarise.yml -o summarise.prompt); both compile to the same AST.

Several things are different here. The prompt has a name and a version. Its parameters are declared and typed — string, number or boolean — with required flags and defaults. The template variables are explicit. The inference parameters are declared alongside the prompt, not buried in application code. And nothing in the file names a provider: that is chosen when the prompt is executed.

This is what we mean by treating prompts as first-class objects. They have identity, structure, and a defined interface.

Why Types Matter

Typed inputs and outputs enable several things that string prompts cannot support.

Validation before inference. If a parameter is declared as a required string, a call that omits it or passes the wrong type can be rejected before it ever reaches the model, and an optional parameter falls back to a declared default rather than to whatever the calling code happened to pass. A prompt rendered with a missing variable is one of the commonest silent failures in LLM code; catching it at the interface is cheaper and more reliable than hoping the model copes.

Output verification. The input side is where promptel’s types are today. The output side is the natural next step and the harder one: if a prompt declared that it returns exactly three strings, a test harness could check every response structurally — did it actually produce three bullets? — and regression testing would no longer need a human to read every case. promptel lets a prompt declare an output format, such as markdown, but structural output schemas are a direction rather than a shipped feature, so for now that check belongs in your tests.

Contract-driven development. When the prompt’s interface is explicit, downstream code can depend on it. If someone renames a parameter or makes an optional one required, the change is visible in the prompt file’s diff and checked against the declared parameters when the prompt runs, rather than discovered later as a strange answer.

Portability and the Provider Problem

One of the less discussed problems with prompts-as-strings is provider lock-in. A prompt tuned for GPT-4’s instruction-following style may produce poor results on Claude or Llama. But because the prompt is embedded in provider-specific code, switching providers means rewriting the integration layer.

promptel addresses this by separating the prompt specification from the provider binding. The prompt file says what to ask; the provider is an execution option — executePrompt(prompt, params, { provider: 'claude' }), or openai, or groq — and the runtime handles the translation into each provider’s API. Changing provider is a change to one argument, not to the prompt.

This does not solve the fundamental problem that different models respond differently to the same text. But it does make the problem tractable. You can now test the same prompt specification against multiple providers and compare results systematically, rather than maintaining parallel implementations that diverge silently.

Extracting Prompts from Existing Codebases

For most teams, the prompts-as-typed-artefacts vision runs into an immediate practical obstacle: existing codebases already contain hundreds of prompts embedded in application code. Rewriting everything from scratch is not realistic.

This is where blogus fits in. blogus is a Python tool that describes itself as package.lock for prompts. blogus scan finds the LLM API calls in a codebase — OpenAI and Anthropic calls, in Python and JavaScript — and reports which are unversioned. blogus init creates .prompt files: a YAML header naming the prompt, its model settings and its variables, followed by the template text. blogus lock writes a prompts.lock with a content hash per prompt, blogus fix rewrites the call sites to load the versioned prompt, and blogus verify fails a CI run when code and lock file disagree.

Note that blogus’s .prompt files are its own format — YAML frontmatter plus a template body — and are not promptel’s .prompt DSL, despite the shared extension. The two tools address different stages and do not depend on each other.

Discovery is necessarily heuristic — prompts constructed dynamically at runtime cannot always be resolved statically. But many prompts follow predictable patterns (string literals, f-strings, template rendering), and the output is a catalogue of the prompts a codebase actually uses. That catalogue is a starting point: review it, correct it, lock it, and you have a versioned prompt library without a ground-up rewrite.

Implications for Reproducibility

Reproducibility is a persistent challenge in LLM research. Results depend on the exact prompt text, the model version, the inference parameters, and often the system prompt or few-shot examples. Papers frequently omit one or more of these, making replication difficult.

promptel specifications are designed to be self-contained. A .prompt or YAML file captures most of what is needed to reproduce a particular LLM interaction: the full prompt text, the typed parameters, and the inference settings, including the model if you pin one in constraints. Record the provider and the parameter values alongside it and you have a complete record. You can commit it to a repository, attach it to a paper, or share it as a standalone artefact.

This is not a complete solution to reproducibility — model versions change, API behaviour drifts, and stochastic sampling means exact output matching is often impossible. But capturing the prompt specification precisely eliminates one of the most common sources of irreproducibility: ambiguity about what instruction was actually sent to the model.

What This Does Not Solve

We want to be clear about the boundaries of this approach. Formalising prompts does not make prompt engineering easier. It does not automatically produce better prompts. It does not eliminate the need for empirical testing against actual models.

What it does is make the engineering process around prompts tractable. Version control becomes meaningful when the artefact has structure. Testing becomes possible when inputs and outputs have types. Portability becomes a concrete property rather than a vague aspiration.

The analogy to programming languages is deliberate. We did not stop writing buggy code when we adopted type systems and version control. But we made bugs easier to find, changes easier to track, and collaboration easier to manage. That is the proposition for prompts.

Next Steps

In the next post in this series, we will discuss the prompt lifecycle in detail: how prompts move from initial drafting through testing, deployment, monitoring, and eventual retirement. We will cover the specific problem of prompt drift, and where version locking with blogus and declarative specification with promptel each help.