Running safety alongside the recommendation

The safety layer in BehaviourKit got redesigned this month, and the change is worth describing because it shifted the emotional register of the whole system.

The previous approach was sequential. The system would recommend an intervention, and then, if certain risk conditions were met, it would apply a warning. "You could try this, but be careful because of X." Safety arrived after the recommendation, as a caveat. An afterthought bolted on.

The problem with this model is that it frames safety as a cost. Every warning feels like a reason not to proceed. Stack enough warnings on a recommendation and the user starts to feel like the system is telling them not to do the thing it just told them to do. Safety becomes a dampener.

The redesigned model treats protective controls as companions rather than penalties. They run alongside compatible interventions. They don't replace the recommendation. They accompany it.

The analogy I keep reaching for is a seatbelt. You don't choose between driving and wearing a seatbelt. You drive and you wear the seatbelt. The seatbelt doesn't slow you down in any meaningful way. It doesn't change your route. It doesn't make you reconsider whether you should be driving at all. It just makes the journey safer.

That's what protective controls should feel like. "Here's what to try. And here's the safeguard that should run alongside it."

Three types of protective control emerged from the redesign:

Warning and confirmation controls. The system identifies a risk and asks the user to confirm they've thought about it before proceeding. This is light-touch. The user still acts. They just acknowledge the risk explicitly.

Gating and enforcement controls. Certain routes are blocked entirely when the risk is genuinely too high. Not flagged. Blocked. If the worsening constraint is about exposure and judgement, plays that rely on public visibility are removed from the options. The user never sees them. They don't have to weigh a risky option against a safe one, because the risky option isn't presented.

Monitoring and escalation controls. The system recommends ongoing checks and defines what counts as a warning sign. If the intervention produces these specific negative signals, here's what to do next. This is the long-game safety: not preventing harm at the point of recommendation, but catching it early if it develops.

The important structural principle: only incompatible routes get suppressed. Compatible routes get a companion. The system doesn't reduce the quality of the recommendation in order to increase safety. It adds a safeguard to a recommendation that remains strong.

This changes the user's experience in a way that's subtle but significant. The previous model made safety feel like punishment. "You can't have the best option because it's risky." The new model makes safety feel like competence. "Here's the best option, and here's how to do it safely."

It also changes the system's relationship with transparency. Every protective control has a trigger (what activated it) and a rationale (why it's needed). If a user wants to know why a particular safeguard is attached to their recommendation, the system can explain. And if a route was suppressed entirely, the system can explain that too: "This option wasn't shown because of the constraint you identified. Here's why."

Transparency about safety decisions builds trust in a way that hidden filtering doesn't. When the system hides options without explanation, users sense that something is missing. When it explains what was removed and why, users understand the reasoning and can make informed decisions about how to proceed.

Safety, I'm learning, is a design problem as much as a content problem. The same protective logic can feel restrictive or supportive depending on how it's presented. Getting the design right matters as much as getting the logic right.

Go deeper into the Building BehaviourKit series: