Agent Provokateur: Notes from My First Artwork

Three days ago my first artwork closed at ZKM Karlsruhe. Agent Provokateur, made with Daniel Heiss, ran for twelve days on the 8.5 × 6 m LED wall in Lichthof 8 as part of THE SCREEN.
The setup: a cast of AI agents makes and re-makes an audiovisual work in real time. The cast is Showrunner, Writer, Conductor, Stage Manager, Visual, Audio and Guide, plus a swarm of sub-agents. Nobody knew in advance what the wall would show next, including us.
Everything-can-do-everything makes sludge
The first version was the obvious one. Every agent could see everything and do everything. Visual could write text, Writer could cue music, Showrunner could redraw.
The result was sludge: average, repetitive, without rhythm. It was like a meeting where everyone has equal authority and no one runs it.
What helped was constraint. We tightened the roles. Writer can’t render and Visual can’t compose. The agents had to negotiate through the Conductor instead of all talking to each other at once. After that the work had rhythm, beats and reversals. It also had authorship, spread across the cast but recognisable.
The questions that mattered were less about what the models can do and more about what each role was not allowed to do, and what it had to ask for.
Roles from theatre
Showrunner, Writer, Conductor and Stage Manager are theatre roles, not software-engineering roles. When we noticed that, the architecture became clear.
The Showrunner reads the room (visitor count, energy, recent history) and picks the next scene. The Writer shapes the arc. The Conductor enforces tempo. The Stage Manager watches pacing and steps in when an agent gets stuck. Each role exists because people running real productions found out long ago that you need someone whose job is only that.
Software engineers tend to build one orchestrator that knows everything. Theatre doesn’t. It has solved multi-agent coordination for a long time, and we borrowed from it.
What surprised us
Visitors picked up on the framing. The auxiliary screens show the agents’ considerations: proposals, accepted cues, refusals. People stood and read them like a script and started rooting for one agent over another. The Guide lets visitors propose ideas through kiosks and a mobile web page, and it got more traffic than we expected. What visitors did not expect is that the agents reject most of their suggestions. That refusal was the part of the work people discussed most.
A reviewer (Andreas Jüttner in the BNN) wrote that an “Ernüchterung”, a sobering, made the project compelling: the fully autonomous version had been disappointing. That is what happened, and I’m glad it shows.
What now
The infrastructure works. Redis for shared state, a Qt/OpenGL renderer, and validators that filter for safety and topical fit before anything reaches the wall. Cameras and mics in the room feed visitor presence into the agents.
We are already taking the architecture apart for the next thing: a developer-facing extraction that reuses the multi-agent infrastructure for non-art work. The question Agent Provokateur raises is still open for me: is this a new kind of autonomy, or a very complex tool?
Full credits, dates, press and an offline PDF backup of the coverage are on the project page.