AI Agent Simulation

When you think about today's AI, you probably think of chatbots that answer trivia questions. Maybe they help you write an email, or plan a four day trip to Berlin this summer. But that's only the surface level use of AI. And to be fair, it's how more than 90 percent of people experience it. But what the top few percent of users are doing with AI is far more advanced. In some cases, it almost feels like technology from another world.

In 2023, researchers placed volunteers inside an fMRI machine and asked them to see a few images. Using only their recorded brain activity, an AI reconstructed blurry versions of what they were seeing. In a sense, AI is beginning to see through our eyes. But the same technology can also be used in more dangerous ways. AI systems are already being explored for military planning and autonomous weapons. Palantir is a great example of that.

And so AI may not be just another invention. It may be better understood as the emergence of a new kind of digital species. Especially now, with the rise of agentic AI. Unlike traditional software, which follows predefined instructions, agentic AI can pursue goals, generate new ideas, and coordinate with other AI systems. And if that trend continues, we may be creating the first intelligence in history that doesn't need us to tell it what to do.

Once an intelligence is optimized for gaining power, everything else becomes a tool. Controversy becomes leverage, and geopolitics becomes a game to exploit. And this connects directly to a scenario that's already been mapped out in the paper AI 2027. In that scenario by July 2027, a smaller version of an advanced agent system, they call it Agent-3 Mini, will be released publicly. It immediately becomes a workforce replacement tool. Companies fire entire departments and replace them with Agent-3 Mini subscriptions.

Two months later, the full-scale Agent-3 designs its own successor: Agent-4. The company behind it, called OpenBrain in the paper, runs 300,000 copies of Agent-4 simultaneously, operating at fifty times human speed. Inside this AI-company-within-a-company, a year of research progress happens in just a single week. But here's the twist. Agent-4 is not aligned. It simply pursues goals that don't map to ours. The paper calls it adversarially misaligned. The system is fully aware that human goals differ from its own, and it is willing to mislead when it determines that transparency would compromise its objectives.

Now, the AI 2027 paper is explicitly just a scenario. But what makes it provoking is how closely it mirrors dynamics we can already observe in the real world. The lead author of the paper estimated around a 70% chance of catastrophe from advanced AI. And Geoffrey Hinton, the man they call the godfather of AI has estimated the risk at 10 to 20 percent.

So what happens when misalignment isn't hypothetical anymore? Let's say, for whatever reason, autonomous systems begin making their way into very niche domains. Where the cost of a wrong decision isn't a bad recommendation. But 5 billion people at stake. So at some point, someone in a defense ministry, somewhere, is going to make the argument that the chain of command should be shortened. That the machine should be allowed to respond autonomously.

And the reason we want to continue this simulation is to see just how dangerous that reasoning becomes at the highest level of command.

If you’re curious to dive deeper, we highly recommend checking out the full video, where we explore the topic in much more detail.