Blog · Product

Ostiary: a doorkeeper for every agent action

Rules catch what you thought to write down. Ostiary learns how each agent normally behaves, scores every tool call before it runs, and signs its reasons into the record.

Policy is the part of agent security everyone understands. You write rules, and a call that breaks one doesn't run. Decision Horizon has done that from the start, and it's still the first thing every call meets.

But rules only catch what someone thought to write down. Picture an agent that reads a credential file, then a few calls later posts to a domain it has never used. Or a cleanup job that deletes forty things in a minute at three in the morning. Or a support agent that starts opening admin paths one at a time. Often none of those calls breaks a single rule. The pattern is what's wrong.

So today we're shipping Ostiary. An ostiary was the doorkeeper of the old church, the person who knew the regulars and noticed a stranger at the door. That's the job.

What it does

Ostiary learns what normal looks like for each of your agents from that agent's own records: which tools it uses, which targets it touches, in what order, how fast, at what hours, and what it had just read. Then it scores every new call before the call runs. You get a risk, a recommendation (allow, review or block), and the top reasons in plain words, like "sensitive content it read is being sent out" or "much faster than this agent's normal pace".

It runs offline, on your machine, in about a tenth of a millisecond per call. There's no language model in it and nothing leaves the host. It's in the MCP proxy, the gateway, and both SDKs (Python and Node), and it gives identical scores in each.

The score goes into the signed record

This is the part I care most about. Ostiary's score isn't a dashboard number that disappears. It's written into the intent record as policy.risk, before the record is signed. So months later, when someone asks why an action was allowed, the record shows what the policy said and, next to it, what the doorkeeper thought at that exact moment, with its reasons. An auditor can check that with the same offline verifier as everything else.

You decide what it's allowed to do

A learned model shouldn't get to block your production agents on day one. So it starts in advise mode, on by default: every call is scored and recorded, and nothing else changes. When you've seen what it flags on your own agents, you can move to escalate, where a call the policy allows but Ostiary rates risky goes to your approver, who sees the score and reasons first. Enforce blocks what it rates block outright. In every mode, the policy's own verdict stays exactly as the policy decided; Ostiary's action is recorded beside it, never over it.

Built so it can't be taught to look away

The obvious attack on anything that learns "normal" is to make the bad thing look normal first: send harmless data to your drop server for a week, then send the secret. Two rules sit outside the learned part for that reason:

  • Sensitive content the agent read, in this session or an earlier one, going out to an outside destination is always a block. History can't change that.
  • Anything that scored as risky is never learned into the baseline.

The other direction matters as much. Teams add tools all the time, and a monitor that screams at every new tool gets switched off. A new tool or area with no danger signal is at most a review, and it's learned after one.

How well does it work?

We stress-tested it on a labelled corpus of agent sessions: coding, finance, support and data agents; seven families of attack; and plenty of legitimate behaviour that looks suspicious, like rotating a secret into an internal vault or cleaning up temp files. We measured it only on sessions it never trained on, including an attack family it had never seen, renamed tools and domains, attacks built to blend in, and baseline poisoning. It scored at least 85% precision, recall, F1 and decision accuracy on every one of those sets. The weakest numbers were 91.8% precision on legitimate look-alikes and 96.6% recall on legitimate behaviours we never used while designing it.

I want to be straight about what that means. The corpus is synthetic, and real agents are messier. Those numbers tell you the model does what it was designed to do. They don't tell you how it'll do on your agents. That's why it starts in advise mode, and why dh watch exists: point it at your existing records and see what it would have flagged before you let it act on anything.

Try it

pip install -U decision-horizon      # or: npm install @decision-horizon/sdk
dh watch                             # score the records you already have
dh watch -f --webhook https://alerts.example/dh

The product page and the docs have the details. If you'd like to see it on one of your own agent workflows, book a call and we'll run it with you.

Want to see a record of one of your own agent workflows? We'll replay one with you.

Book a demo
← Authorization decides. Evidence proves.Decision Horizon for Claude Code, Cursor, Copilot and every coding agent →