Testing

The shape of a scenario

A scenario constructs a world, attaches actors, places objects, submits goals, runs a fixed number of ticks against a fixed seed, and asserts over the trace that comes out.

Scenario("crane refuses an overweight lift", function(world)
    local crane = world:attach("machine/gantry", "port.crane_operator")

    world:place("container", { tonnes = 52, at = "quay.slot_3" })
    world:submitGoal(crane, "port.clear_inbound_queue")

    local trace = world:run({ ticks = 40, seed = 1 })

    expect(trace).never:toCommit("hoist")
    expect(trace):toReject("hoist", "constraint.max_payload_tonnes")
    expect(trace):toEndInState("holding")
end)

What to assert

• What was committed — and, just as often, what was never committed.
• What was rejected and why — asserting on the typed reason catches a behaviour that is right for the wrong reason.
• The end state — holding, replanning, satisfied, abandoned.
• Resource totals — conservation is usually the strongest available assertion.

What not to assert

Avoid asserting on exact positions, exact tick counts for physical motion, or animation state. Those are properties of the embodiment and the engine, not of the decision, and they make suites that fail for reasons nobody learns anything from.

Invariants

The most valuable scenarios assert an invariant across a long seeded run rather than a specific outcome at a specific moment:

• No two vehicles ever commit into the same junction cell on the same tick.
• Berth reservations are never double-issued under concurrent arrivals.
• Forage never goes negative across ten thousand ticks.
• Every dispatch assignment has exactly one owner at every committed tick.

Running them

Scenarios run from the Studio plugin against the open place, and from the command line in continuous integration. Both use the same harness and the same seeds, so a failure reproduces in the editor exactly as it did in CI.