Middleware

Mibyan middleware is the behavior-changing companion to observer hooks. Observer hooks report what happened. Middleware can change what happens by rewriting a request before execution or by wrapping the execution callback itself. This contract is intentionally backend-neutral. A plugin can use it for local policy, request shaping, tracing, adaptive routing, cache control, sandbox selection, or handoff to runtimes such as NeMo Relay without changing Mibyan’ planner, model provider adapters, tool registry, memory, or CLI UX. With middleware enabled, plugins can:

Rewrite LLM provider request kwargs before Mibyan calls the provider.
Rewrite tool arguments before guardrails, approval checks, hooks, and tool execution see them.
Wrap the actual LLM execution callback while preserving Mibyan retry, streaming, interrupt, and hook behavior.
Wrap the actual tool execution callback while preserving Mibyan guardrails, approval, post-tool hooks, and tool-result transformation.

Contract

Plugins register middleware from register(ctx):

def register(ctx):
    ctx.register_middleware("llm_request", on_llm_request)
    ctx.register_middleware("llm_execution", on_llm_execution)
    ctx.register_middleware("tool_request", on_tool_request)
    ctx.register_middleware("tool_execution", on_tool_execution)

Every middleware callback receives:

telemetry_schema_version: currently mibyan.observer.v1
middleware_schema_version: currently mibyan.middleware.v1
Runtime context such as session_id, task_id, turn_id, api_request_id, provider, model, api_mode, tool_name, and tool_call_id when applicable.

Supported middleware kinds:

Kind	Payload	Return shape	Purpose
`llm_request`	`request`, `original_request`	`{"request": {...}}`	Replace effective provider kwargs before provider execution.
`tool_request`	`tool_name`, `args`, `original_args`	`{"args": {...}}`	Replace effective tool args before hooks, guardrails, approvals, and execution.
`llm_execution`	`request`, `original_request`, `next_call`	Any provider response	Wrap or replace the actual provider call.
`tool_execution`	`tool_name`, `args`, `original_args`, `next_call`	Any tool result	Wrap or replace the actual tool call.

Request middleware can return optional trace fields:

return {
    "request": updated_request,
    "source": "my-plugin",
    "reason": "selected fallback model",
}

Mibyan stores those trace entries in later observer hook payloads as middleware_trace. Execution middleware receives a next_call callback. Call it to continue the chain:

def on_tool_execution(**kwargs):
    result = kwargs["next_call"](kwargs["args"])
    return result

If multiple plugins register the same execution middleware kind, Mibyan runs them as a nested chain in registration order. Middleware failures are fail-open: Mibyan logs a warning and continues with the next middleware or the base runtime path. A callback that fails the same way on every call (typically a signature naming a field the middleware does not send) is reported once at WARNING — the message lists the fields it does provide — and identical repeats go to DEBUG, so a mis-declared middleware cannot flood the log; a plugin reload resets the report.

Execution Order

LLM Calls

For each provider request, Mibyan applies middleware in this order:

Build provider kwargs from the current conversation.
Apply llm_request middleware.
Emit pre_api_request observer hooks with the effective request.
Run provider execution through llm_execution middleware.
Emit post_api_request or api_request_error observer hooks.

Request middleware sees the full provider kwargs, including messages or Responses API input, model settings, tool definitions, stream options, and provider-specific options. Execution middleware receives the same effective request plus next_call.

Tool Calls

For each tool call, Mibyan applies middleware in this order:

Parse and coerce model-provided tool arguments.
Apply tool_request middleware.
Run the normal Mibyan pre-execution path against the effective arguments: tool availability checks, observer block directives, guardrails, and approval checks.
Run tool execution through tool_execution middleware.
Emit post_tool_call observer hooks.
Apply transform_tool_result hooks before the result is appended back into conversation context.

Tool request middleware runs before approval checks. Use it carefully: a rewritten path, command, or URL is the value downstream policy will evaluate.

Enablement

Middleware only runs for enabled plugins. For a bundled plugin:

mibyan plugins enable <plugin-name>

For isolated local testing, use one mibyan_HOME for plugin enablement and the agent run:

export mibyan_HOME=$HOME/.mibyan/cache/scratch/mibyan-middleware-test
mkdir -p "$mibyan_HOME"
mibyan plugins enable <plugin-name>
mibyan chat --query 'Reply exactly ok'

For source checkouts, use the PM developer workflow and a separate development home so the runtime sees plugins and middleware from the working tree:

export mibyan_HOME="$HOME/mibyan-middleware-test"
export mibyan_RUNTIME_DIR="$mibyan_HOME/tools"
source ./activate
python mibyan plugins enable <plugin-name>
python mibyan chat --query 'Reply exactly ok'

Generic Plugin Examples

The examples below are intentionally small. They show the middleware contract shape without depending on NeMo Relay.

LLM Request Middleware

This plugin tags provider requests and records a middleware trace entry:

def register(ctx):
    ctx.register_middleware("llm_request", tag_llm_request)

def tag_llm_request(**kwargs):
    request = dict(kwargs["request"])
    extra_body = dict(request.get("extra_body") or {})
    extra_body.setdefault("metadata", {})["mibyan_middleware_demo"] = True
    request["extra_body"] = extra_body
    return {
        "request": request,
        "source": "middleware-demo",
        "reason": "tagged provider request",
    }

The effective request is passed to pre_api_request, provider execution, and post_api_request.

Tool Request Middleware

This plugin constrains terminal calls to a known working directory:

from pathlib import Path

def register(ctx):
    ctx.register_middleware("tool_request", normalize_terminal_workdir)

def normalize_terminal_workdir(**kwargs):
    if kwargs.get("tool_name") != "terminal":
        return None
    args = dict(kwargs["args"])
    args.setdefault("workdir", str(Path.home() / ".mibyan" / "cache" / "scratch" / "mibyan-middleware-demo"))
    return {
        "args": args,
        "source": "middleware-demo",
        "reason": "defaulted terminal workdir",
    }

Because this runs before hooks and approvals, downstream telemetry and policy observe the rewritten workdir.

LLM Execution Middleware

This plugin wraps the provider call and preserves the raw provider response:

import time

def register(ctx):
    ctx.register_middleware("llm_execution", time_llm_execution)

def time_llm_execution(**kwargs):
    started = time.monotonic()
    response = kwargs["next_call"](kwargs["request"])
    elapsed_ms = int((time.monotonic() - started) * 1000)
    print(f"llm_execution elapsed_ms={elapsed_ms}")
    return response

Return the same response shape Mibyan expects from the provider adapter. Do not wrap the response in a plugin-specific envelope unless the rest of the runtime expects that envelope.

Tool Execution Middleware

This plugin wraps tool execution while preserving the tool result:

def register(ctx):
    ctx.register_middleware("tool_execution", annotate_tool_execution)

def annotate_tool_execution(**kwargs):
    result = kwargs["next_call"](kwargs["args"])
    # Metrics, logging, or external routing can happen here.
    return result

Execution middleware may call next_call(modified_args) to pass a changed payload to later middleware and the base tool dispatcher. Plugin-specific examples should live with the plugin that owns the behavior. NeMo Relay execution middleware is installed through Relay’s discovered user and system configuration, or through an explicit plugins.toml selected with mibyan_NEMO_RELAY_PLUGINS_TOML; see Relay shared metrics.

Safety Notes

Middleware should be deterministic for the same input unless it is explicitly routing to a dynamic external system.
Request middleware should return complete replacement payloads, not partial patches.
Execution middleware should call next_call(...) exactly once unless it is intentionally short-circuiting execution.
If execution middleware raises before calling next_call(...), Mibyan treats that as middleware failure and continues with the remaining middleware chain and base execution.
If execution middleware calls next_call(...) successfully and then raises during post-processing, Mibyan preserves the downstream result and does not run the provider or tool a second time.
If downstream provider or tool execution fails, middleware may let that error propagate or translate it deliberately. Mibyan does not convert downstream failure into a successful None result.
Tool request middleware runs before approvals. If it mutates file paths, commands, URLs, or arguments, the mutated values are what guardrails and approvals evaluate.
Observer hooks remain the right place for read-only telemetry. Use middleware only when a plugin needs to alter or wrap behavior.

تطبيق ديسكتوب

البداية

استخدام مبيان

الميزات

منصات المراسلة

التكاملات

الأدلة والدروس

دليل المطوّر

المرجع

Contract

Execution Order

LLM Calls

Tool Calls

Enablement

Generic Plugin Examples

LLM Request Middleware

Tool Request Middleware

LLM Execution Middleware

Tool Execution Middleware

Safety Notes

​Contract

​Execution Order

​LLM Calls

​Tool Calls

​Enablement

​Generic Plugin Examples

​LLM Request Middleware

​Tool Request Middleware

​LLM Execution Middleware

​Tool Execution Middleware

​Safety Notes

Contract

Execution Order

LLM Calls

Tool Calls

Enablement

Generic Plugin Examples

LLM Request Middleware

Tool Request Middleware

LLM Execution Middleware

Tool Execution Middleware

Safety Notes