Before You Let AI Into the Control Room: Five Questions Every OT Leader Should Ask

O T  S E C U R I T Y  ·  A I  R E A D I N E S S

By Sanjeev Sharma | Industrial Control Systems Security

 

Every few weeks, a fresh headline reminds us that the safety guardrails around advanced AI systems are not as solid as their builders would like. A model is coaxed into producing something it was explicitly designed to refuse. The vendor issues a measured statement. The industry debates. And then we move on — until the next time.

For those of us who run operational technology in refineries, pipelines, and process plants, these stories deserve more than a passing glance. Not because our DCS or PLC networks are about to be overrun by rogue chatbots, but because the AI conversation is arriving in our world whether we invite it or not. Predictive maintenance platforms, anomaly-detection engines, copilots embedded in engineering tools, vendor “AI-assisted” diagnostics — the pitch decks are already on our desks.

The instinct of a good OT engineer when a new technology shows up at the plant boundary is not enthusiasm and it is not refusal. It is interrogation. We ask hard questions before we let anything touch a live system. AI deserves exactly the same discipline. Here are the five questions I keep coming back to.

 

1.  What happens when the model is wrong — and who notices?

In OT we have spent decades engineering for failure. We assume instruments drift, valves stick, and signals lie. We build interlocks, voting logic, and independent layers of protection precisely because we do not trust any single element to be right all the time. An AI model is no different in principle, but it fails in less obvious ways. It does not throw a fault code; it produces a confident, plausible answer that happens to be wrong. The question is not whether the model is accurate — every vendor will say yes. The question is what your detection and response looks like when it is wrong, and whether a human in the loop is positioned to catch it before the error propagates into a decision.

 

2.  Is this advisory, or is it in the loop?

There is a world of difference between an AI that recommends a maintenance window and one that adjusts a setpoint. The first is a decision-support tool; the second is part of the control system, with all the safety, reliability, and regulatory weight that implies. A surprising number of “AI” deployments blur this line by design. Before adoption, draw the boundary explicitly: what can this system change, what can it only suggest, and is that boundary enforced by architecture or merely by policy? In OT, we trust architecture far more than we trust policy.

 

3.  Where does the data go, and what comes back?

Many AI offerings are cloud-anchored. Your process data — historian trends, alarm logs, equipment health, sometimes configuration detail — leaves the plant to be scored, and an output returns. Each leg of that round trip is an attack surface and a data-governance question. This is familiar territory. It is the same defense-in-depth reasoning we apply to any external connectivity: segmentation, brokered data flows, strict egress control, and a clear-eyed view of what proprietary information we are willing to expose. The novelty of AI does not exempt it from the zone-and-conduit thinking we already practice under IEC 62443.

 

4.  Can the vendor explain its own safety model — in our terms?

When a jailbreak story breaks, what it really exposes is the gap between how robust a safety control looks and how robust it is. A keyword filter that can be defeated by clever rephrasing was never a strong control; it only appeared to be one. We know this pattern intimately. A firewall rule that can be trivially bypassed offers the illusion of protection, which is more dangerous than

acknowledged exposure. So ask the vendor to explain, in engineering terms, what their model will refuse to do and why that refusal holds up under pressure. If the answer is hand-waving about “advanced safety layers,” treat it the way you would treat a contractor who cannot explain his own trip logic.

 

5.  What is our exposure if we do nothing — and if we wait?

The honest counterweight to all this caution is that standing still has a cost too. Reliability gains, earlier fault detection, and better use of scarce engineering time are real benefits, and competitors capturing them is a real risk. The goal of interrogation is not to say no. It is to say yes deliberately, with the boundaries drawn and the failure modes understood.

 

The discipline travels

None of this requires us to become AI specialists. It requires us to apply the discipline we already have. The principles that keep a refinery safe — defense in depth, separation of advisory from control, distrust of single points of failure, architecture over policy, and a relentless focus on how things fail rather than how they work — map almost perfectly onto the questions AI raises.

The technology is new. The thinking is not.

 

That, for those of us responsible for operational integrity, is reassuring news. We have been preparing for this conversation our entire careers; we just called it process safety.

Sanjeev Sharma writes on industrial control systems security, instrumentation, and reliability. Views are his own.