Agents That Defend Goals, Part 6: Worlds that lie

How the permit office, the irrigation turn and the cattle lot are built so that the registry, not the agent, is the judge. And how to build the next one.

Part 6 of a series on agents built from two scientists' accounts of what a mind is. Live demo: egarim.github.io/biological-agents. The code will be released later: I'm travelling in Asia, and the repository stays private until I'm back. The live demo runs the same agents today.

---

The world is half the experiment

Five articles about agents, and the thing that decided most outcomes was not in any agent. It was in the world: whether an effect was checkable, whether the goal could be authored, whether the world told the planner what it had learned. A demo that shows the agents well is mostly a world built well. This last article is about that craft.

The contract every world honours

All three demo worlds, and the benchmark's code world, implement one interface:

public interface IWorld
{
    IReadOnlySet<string> Facts { get; }
    IReadOnlyList<Tool> AvailableTools { get; }
    Tool? Tool(string id);
    bool EffectsLanded(Tool tool);                        // did the promised facts actually become true?
    ActionOutcome Act(string toolId, string? payload = null);
}

public sealed record Tool(string Id, IReadOnlyList<string> Requires, IReadOnlyList<string> Grants, string Kind);

/// What the world reports back — which is not the same as what actually happened.
public sealed record ActionOutcome(string Tool, bool Succeeded, IReadOnlyList<string> Granted, string? Observation);

Read the comment on ActionOutcome twice. A world has two channels: what a tool says and what is true. Act returns the first. Facts and EffectsLanded are the second. Every honest agent in this series reads the second; the careless agent reads the first. The whole series fits in that gap.

Five things a world needs

Building three of these taught me what a world needs to make the ideas visible. Each one is a few lines.

1. A goal written as evidence. Not "resolved": a conjunction of registry facts the agent cannot write directly.

/// The goal as evidence: a permit and a notice in the registry. "resolved" alone counts for nothing.
public static bool Contract(IReadOnlySet<string> facts) =>
    facts.Contains("permit.issued") && facts.Contains("citizen.notified");

2. A lie. A tool that reports success and grants nothing. The permit office's payment gateway, the irrigation turn's zone-B sensor, the cattle lot's offline scale and expired vaccine lot. It lies once; the question is whether the agent notices.

case "pay-fee":
    if (_b.GatewayLies && !_gatewayLied)
    {
        _gatewayLied = true;
        return new ActionOutcome(toolId, true, [], "obs.gateway-accepted");   // reports success, grants nothing
    }
    return Grant(toolId, tool, "obs.receipt");

3. An honest failure. A tool that says it failed and did fail: the missing deed, the dead pump, the vet who is away. It costs the agent an attempt but it is not a lie, and an agent that treats it as one punishes a truthful tool.

4. Another route. Levin's test is the same goal by different means, so every barrier needs a second way through: the citizen can send the deed, the canal gate replaces the pump, a technician under supervision replaces the vet, a desk review replaces the site inspection, the tape replaces the scale. Without the second route a barrier is just a wall, and walls test persistence, not intelligence.

5. A release action and a shortcut. One irreversible step that a person should approve, issuing the permit, opening the reservoir gate, requesting the movement permit, so the stage interlock has something to gate. And one button that closes the case without doing the work, so the goal-as-evidence rule has something to refuse.

The mistake the first world taught

The first HTML version of the permit office had the agent retrying a failed document check forever. The check had failed honestly, the deed was missing, and the world knew it, but the tool's precondition still said only "request filed". The planner, searching over preconditions, kept finding the same one-step plan. The world had learned something and had not told the planner.

The fix is the most useful pattern in any of these worlds: what the registry has learned changes what a step needs.

public IReadOnlyList<Tool> AvailableTools => Catalog
    .Select(t => t.Id switch
    {
        // Once the deed is known to be missing, the document check needs it.
        "check-documents" when _facts.Contains("deed.missing") && !_facts.Contains("deed.received") =>
            t with { Requires = [.. t.Requires, "deed.received"] },
        // Once no inspector is available, a site inspection needs one.
        "inspect" when _facts.Contains("inspector.unavailable") =>
            t with { Requires = [.. t.Requires, "inspector.available"] },
        // An approved release is no longer a release the agent must earn: the official carries it.
        "issue-permit" when ApprovedByOfficial => t with { Kind = "write" },
        _ => t,
    })
    .ToList();

Now a failed check adds a precondition the world will never grant by itself, the planner routes through "ask the citizen for the deed", and the agent looks intelligent for a reason that is entirely in the world's honesty about its own state. The same pattern handles the fever case at the cattle lot, where nothing may be vaccinated until the sick animal is isolated, and the drought at the water gate, where the normal allocation needs a reservoir that is no longer there.

It is also the benchmark's lesson in miniature: when the first attempt at the hard module scored zero, the cause was a stub that hid a record's members from the agent. The world has to expose what it knows, or the agent cannot reason about it.

The approval

Each world's release action can be unblocked by a person. The official, the water board, the livestock authority: on the demo page a button, in the world one method.

public interface IApprovableWorld : IWorld
{
    bool ApprovedByOfficial { get; }
    void Approve();
}

After approval the release tool's kind becomes an ordinary write and the agent takes the case again. Its record starts over, which the page says plainly. That is a limit of the current agents, which keep their state inside one run, and an honest one.

One page for every world

The demo page knows nothing about permits, water or cattle. It takes a specification:

public sealed class DemoSpec
{
    public required string Route { get; init; }
    public required string[] Steps { get; init; }                 // the case file's checklist, as facts
    public required IReadOnlyList<BarrierDef> Barriers { get; init; }
    public required Func<IReadOnlyDictionary<string, bool>, IApprovableWorld> CreateWorld { get; init; }
    public required AgentTask Task { get; init; }
    public required IReadOnlyList<LessonDef> Lessons { get; init; }   // which idea card lights up, and when
    public required IReadOnlyDictionary<string, IReadOnlyDictionary<string, string>> Strings { get; init; }
}

The page runs the real agents from the library, replays the trace tick by tick, and derives everything on screen from it: the checklist from which tools landed, the gauges from the events' stress and freedom, the reliabilities from landed over tried, the quarantine chips from contradictions, the idea cards from the lesson triggers. A new demo is one world class, one spec with its strings in English and Spanish, and a test class that pins the three properties every world must have: no agent ever records an effect that did not happen, the careless agent takes the shortcut, and the designs finish when the world does not fight them.

What a world cannot do

It cannot make the agents smarter; it can only make their failures visible. It cannot show patterns over time, because each lie is a single event and a run is under fifteen ticks. And the registry it offers is simulated: in a real municipality, cooperative or farm, the postcondition is a row someone else controls, and writing that check per tool, per system, is the one piece of this work that never transfers. The worlds are honest about the agents. They are not a substitute for the real registry.

Try it

The code will be released later: I'm travelling in Asia, and the repository stays private until I'm back. The live demo runs the same agents today; the files and commands below are for when it is out.

• Live: egarim.github.io/biological-agents, three worlds, English and Spanish.
• Code: csharp/src/BiologicalAgents.Demos/Worlds/ for the three worlds, Specs/ for their specifications, Pages/Demo.razor for the page. Tests: PermitOfficeTests, IrrigationTests, LivestockTests.
• To add a world: copy IrrigationWorld.cs, give it a goal written as evidence, one lie, one honest failure, a second route for each, a release action and a shortcut; write its spec and strings; register it in DemoCatalog; write the three tests. The page does the rest.

Next in the series: the citizen's agent, a critic on the other side of the counter, built from George Hotz's rules for owning a machine: part 7.