How I Run AI Agents Like an IT Service: Seven Disciplines

A light that shows green while something behind it has stopped working: that is how it started with my AI agents. I now run them the way an IT department runs its services. Seven habits from operations, each explained through a case from my own setup, a one-page card you can fill in, and the question of who carries the responsibility in the end.

A green light is reassuring because nobody has to look any more. It turns dangerous when green only means that no error message has come in. Then the light stays on while something behind it stopped working a while ago.

That is what happened with my AI agents. Every morning a report is waiting for me, and I read it before I start. One morning it said that no routine had failed. In fact the backup of my database had failed the day before. The error mail that failure should have triggered never arrived, and the report knew only one source, the list of error messages. The backup was not on that list at all. So the report read an empty list and said what an empty list says: all clear. That the backup was missing came out only because somebody looked.

An agent, in my setup, is an AI program with one fixed job, the way an employee has a role. One reviews code, one writes drafts, one finds references. Underneath them run simple programs without any AI, called scripts, which only report what happened; the morning report is one of them. At the end I decide, the way a product owner in software development decides what gets built and has the team report to them. My team just happens to be made of agents. I work with Claude Code, but nothing in this article depends on that choice.

This is what the article comes down to: an agent is not a tool you pick up and put down. It is a service that runs, and somebody has to look after a service. IT operations has had named habits for that for decades. You want to know whether something is running without waiting for a complaint? For me that is the morning report; in IT operations it is event and monitoring management. I explain seven of these habits here through the cases from my own operation. It is a report from the workshop: this is what my operation looks like right now, mid-build, with the places where it still sticks.

How do I know something is running? (Event and monitoring management)

Monitoring, in IT operations, means somebody keeps looking at whether a service is doing its job, instead of waiting for a complaint. My report was looking, just in the wrong place, because a list the backup never appears on cannot report that the backup failed. So the report now checks for itself how old the last backup is. A report that is allowed to say “all clear” needs a second source.

Something similar happened with the schedule that starts my routines. It reported one run as successful, although the run had ended after a few seconds and done nothing. A message like “finished” only tells you that something ended. What I need is evidence that something came out of it, such as a file or an entry with a date. So every recurring routine now writes down at the end what it achieved, and “nothing achieved” counts as a failure. And a missing value must never show green; it has to show as unknown.

Is it running again, or will it not happen again? (Incident and problem management)

In IT operations a disruption is called an incident, and an incident counts as resolved as soon as the service runs again. My backup ran again the next day, so the disruption was over. But that said nothing about why nobody had noticed. The question of the cause is what operations calls the problem, and it makes the difference between “runs again” and “does not happen again”. In my case there were two causes: the report had only one source, and the error mail had not arrived. That is when I realised that an alert path I rely on is better triggered once on purpose than tested for the first time on the day it matters. So fixing the disruption is the smaller part. Finding the cause takes evidence rather than a guess, and in my case it sat both times in how I had set the operation up, not in the AI model.

Who is allowed to change what? (Change enablement)

Any change to a running service is called a change in IT operations, and there are rules about who approves one and when. At first every change in my operation needed my yes, one at a time, and that was right as long as I had measured nothing. Then my operation introduced a daily release. The idea was that reviewed changes without risk, meaning text, documentation and settings, are collected in the evening, shown to me in one mail, and go live the next day unless I object. Anything touching the database, payments or sign-in stayed with the individual yes.

For changes that always run the same way and carry little risk, IT operations has the standard change: approved once, then run without individual sign-off. In my operation these are the standing approvals, each written down with a date, for one narrowly defined case, and revocable at any time. My first standard change was a separate path for small bug fixes. On every run the automation checked whether a change qualified for it and never found one, until the path was switched off. A path nobody walks delivers nothing and can still cause mistakes. The daily release has since been switched off as well, and some standing approvals I have withdrawn; the date on the entry shows me how long one has been in force and when to look at it again.

Before the approvals were rebuilt, I had someone measure who they actually burden. The problem was not too many approvals but too few answers, and the answers come from me. Out of that came three kinds of work: what may run without my yes and is reported to me afterwards, what is prepared and comes to me as one package with a short description, and what never happens without my yes, namely platform code, prices, payments, publishing and customer contact.

What happens when the operation goes down? (Service continuity)

An operation that depends on a single point stands still as soon as that point fails. The precaution against that is called service continuity. My operation has a standby in the cloud for it. It checks regularly when the main operation last reported in and steps in when that is longer ago than a set threshold. Once the main operation stayed silent longer than planned, and the standby did nothing, because its threshold sat just above the length of the pause. A threshold that has never fired is untested, even if it looks correctly set. What was rehearsed afterwards was the takeover itself: an outage was assumed and the standby was allowed to run. It took over, but it only skimmed the files it has to read, and of its two ways to run commands, only one worked from the cloud. The threshold is still untested. The entry afterwards holds the rule that applies now: the first real outage must not be the first test.

A second case. On two evenings the evening mail that sums up my day did not arrive, because development work had used up the weekly allowance of AI calls. That became a rule: development stops short of a reserve, and the reserve belongs to the runs that have to happen every day. At the next squeeze the rule was not enough, and the evening mail failed again. Making that reserve really hold is still open. Keeping project work apart from operational work is something I teach to others; in my own operation it is not finished.

Next comes the rehearsal of whether anything can actually be restored from that backup. For that, a department of its own for backup and hardening is being set up, and it is not finished either.

What is left of a disruption? (Knowledge management and continual improvement)

A disruption that has been fixed but written down nowhere comes back, and then the search starts from the beginning. So every disruption in my operation gets an entry with a date, a cause, a why and an apply, meaning what holds from now on and how to recognise the case next time.

When two reviewing agents rated the whole system noticeably lower than the self-assessment before them, the reason sat in the same place as with the standby. It is easy to mistake built for shown to have run, and in my operation that is exactly what had happened. That, too, is now an entry.

Take a look and get started, the account is free

Who reads it over? (Review, test and pipeline)

No draft becomes a result in my operation just because an agent wrote it. A second agent, one that only reads and changes nothing, checks it first, always in its latest state. Before one go-live it found a duplicated greeting, broken formatting and a setting that could have run up costs with no upper limit.

One limit I say openly. When an agent writes into its change that it has been reviewed, that is a self-declaration. The automatic checks that run before anything goes live cannot see whether the reviewer really read it. An instruction in a task text is not a lock, only a request, and where I have no lock I would rather say so than rely on the line.

Does the platform get better while it runs? (Product improvement)

My learning platform was slow for a while, and it showed up in a trial, not in a metric. The computing sat on a different continent from the database, so every request had a long way to go. When somebody feels a symptom, I therefore take it seriously even when no automatic message arrives. Every improvement to the platform is a change with a target, an order and a review, and it happens while the platform keeps running.

The operations card

In IT operations every service has a runbook that says what it is for, who watches it and what to do when it fails. The operations card is one page of that for a single AI routine, and wherever a field stays empty, that is the next job. Below it is filled in for my evening mail, as far as I honestly can.

Operations card for an AI routine (example: my evening mail)

Field	Habit (term)	Evening mail
What is it for? (one sentence)	Reporting	sums up the day, shows open decisions and what is waiting for a yes
Does it run with AI?	Watching (monitoring)	yes
What starts it? (schedule, event, person)	Watching (monitoring)	the schedule, in the evening
What may it do on its own? (read, prepare, act)	Approving changes (change enablement)	read and report; writes to me only
How can I tell it ran? (a file or entry, not the schedule’s message)	Watching (monitoring)	by a sent marker with a date, not by the schedule’s message
Who is told how when it fails?	Fixing disruptions (incident management)	the fallback mail arrives recognisably as a mail from the cloud; if it lacks data, it names the reason in its subject; next morning the morning report notes that no evening mail came
Second source for “all clear”	Watching (monitoring)	what the routines write down as achieved at the end, not their “finished” message
Who steps in?	Preparing for outages (service continuity)	a fallback mail from the cloud when the sent marker is missing or too old
What does it never do?	Approving changes (change enablement)	publishes nothing, changes no data, writes to nobody but me
Last failure and the rule it left	Finding causes, keeping what was learned (problem management, knowledge management)	the regular evening mail stayed out because the weekly allowance of AI calls was used up; the fallback mail from the cloud stepped in, on some evenings only as a short mail with the reason. Rule: the reserve for the daily runs has to hold under pressure too; that is still open
Who checks the result?	Reading over (review)	I read it; a reply is my decision
Checked on	Improving	(enter date)

Opens straight away, no account

Who carries the responsibility in the end?

One thing runs through all seven habits: the agents are not responsible for my system, I am. As a guideline I use the AI Fluency framework by Rick Dakan and Joseph Feller with its four competencies, and this is how I translate them into my operation. Delegation means deciding which task goes to the AI and which does not; for me that is the levels of approval. Description means putting the task in a way that produces a usable result; that is my task descriptions and the context every agent gets. Discernment means checking the result critically; that is the second agent that reads over, and evidence instead of a “finished” message. And diligence means handling data, sources and consequences responsibly. That is the point everything comes down to: the responsibility stays with me, I sign off what goes out, and I report openly when something went wrong. I teach these four competencies in my course “Managing projects with AI”; applying them to my own operation is my reading.

Limits

This is how I work: a small company with a learning platform and a system still being built, which has its failures. Your operation is different. Once several people are involved, the question of who may enter a standing approval becomes the real work. Once a tool sees customer data, other questions come before everything on this page, namely which documents it may read, where they are processed and who signed that off. The very first of them, which AI is running in the house at all, I asked in my article on the AI Act as a maturity test. Take this as a starting point and adapt it to your conditions before you use it. Whether any of this fits you is your call.

Where this goes next

What all this delivers I cannot say with a number today, only with an order: first prove that it runs, then prove what it delivers. The second step needs an assumption, a measurement plan and a number with a source. It is an article of its own. This text was the starting shot of a series. Next I will write about building the system: which processes, workflows and systems it takes, organised by the four dimensions of service management. In ITIL those are organisations and people, information and technology, partners and suppliers, and value streams and processes, four angles that together show whether a service really holds up.

Sources

• ITIL® is a registered trademark of PeopleCert.
• Article “The AI Act is not a compliance project. It is a maturity test.”
• AI Fluency framework by Rick Dakan and Joseph Feller (four competencies: delegation, description, discernment, diligence)
• Course “Managing projects with AI”
• Course “Getting results into operations”
• Four dimensions of service management according to ITIL (organisations and people, information and technology, partners and suppliers, value streams and processes)