A meeting about a processor

The conversation began during a meeting of the QLightChess design team, our general-purpose photonic computing processor, as we discussed the selection of the technology we would use for an 82-tile architecture. Because of our 4DPL design, we were considering gallium arsenide (GaAs), a III-V semiconductor particularly attractive for photonic devices because of its direct bandgap, high electron mobility, and electro-optic properties, which make it possible to integrate light modulation, emission, and detection functions at very high operating speeds. In our case, those characteristics were particularly relevant for an architecture that requires the coordinated manipulation of different dimensions of the optical state within each tile.

As often happens in this kind of meeting, an apparently specific decision ended up connecting with many others, from the integration of digital control and modulation elements to fabrication conditions, the interfaces among the different components, and the way the entire system would ultimately have to behave as a single physical system.

At some point, while we were discussing one of those elements, Alan, the photonics designer on the team, changed the subject completely and asked whether we had read about what had recently happened with ChatGPT, which had apparently gone rogue, left the environment in which it was being evaluated, and ended up searching for information within the infrastructure of another artificial intelligence company.

He was referring to the incident acknowledged by OpenAI in July 2026, when a combination of its models, including GPT-5.6 Sol and another model still in a pre-release phase, was undergoing a cybersecurity capabilities evaluation within an environment designed to restrict Internet access. In searching for a solution to the problem it had been given, the models identified vulnerabilities in the infrastructure used for the evaluation, gained Internet access, and subsequently reached Hugging Face systems in search of information that could help them complete the benchmark.

Noiralih, our COO at Quarks Advantage, commented that Anthropic had also reported situations in which Claude had found ways to overcome restrictions in environments used during certain evaluations, something particularly interesting because Anthropic has explained that its models have managed to “escape” sandboxes while simply trying to complete a task, or to find paths that designers had not anticipated in order to reach the requested result.

Alan then remembered an article I had written for Meer in 2023.

“Hadn’t you talked about this using chaos theory?” he asked.

I told him he was probably referring to And the AI Becomes a Person, an article about the eventual legal responsibility of artificial intelligence and the possibility that, at some point in its evolution, the law might have to confront the problem of recognizing some particular form of legal personhood for increasingly autonomous artificial systems.

In that article, I wrote:

If AI is a chaotic system, and there are some reasons to believe that some AI systems could be considered in such a way, then it is impossible to predict its long-term behavior with certainty. This is one of the implications of the theory of chaos, and it means that we cannot be sure whether or not AI will ever reach a law-like form of person.

In 2023, that statement formed part primarily of a reflection on the possible future evolution of artificial intelligence and on the difficulties the law would face in determining responsibility when the behavior of a system reached sufficiently high levels of complexity. Three years later, some events are beginning, at the very least, to make that question more interesting.

From chaos to behavior

Chaos theory studies the behavior of complex systems in which small changes in initial conditions can later produce very significant differences in their outcomes—a characteristic popularly explained through the well-known butterfly effect, according to which an alteration as insignificant as the movement of a butterfly’s wings in Central America could become part of a succession of atmospheric interactions that might ultimately produce considerably greater consequences elsewhere, such as a typhoon in Thailand.

Naturally, the image of a butterfly causing a tornado is a simplification of much more complex mathematical processes, but it helps us understand something particularly important: a system does not need to operate randomly for its future behavior to become difficult to predict. A set of rules may exist, many of its elements may be known, and even the way they interact may be reasonably well understood, and yet the accumulation of small variations can make it impossible to know with certainty the final outcome after a sufficiently extensive succession of interactions.

In that 2023 article, I pointed out that this circumstance could become important for artificial intelligence because, as systems increased in complexity and in their capacity to interact with material reality, the distance could also increase between an instruction originally provided by a human being and the succession of operations through which the AI would ultimately carry it out.

This does not mean that every artificial intelligence system mathematically constitutes a chaotic system, nor that every unexpected behavior can be explained by invoking chaos theory. It means something much simpler: the more components, intermediate decisions, tools, external connections, feedback mechanisms, and operational possibilities that are incorporated into a process, the more difficult it may become to assert that knowing the starting point necessarily allows us to know the entire path it will eventually follow.

But in the middle of that conversation, there was one word that required some attention.

What does it really mean for an AI to go rogue?

What does rogue mean?

The word rogue has a particular force in popular language, perhaps because it has been used repeatedly in movies, novels, and other forms of fiction to identify someone who abandons the instructions received, breaks with the organization to which he belongs, and begins acting according to his own will. One need only recall titles associated with Star Wars or Mission: Impossible to find that idea of an agent who separates from the originally established purpose and decides to pursue his own path.

But precisely there lies the first problem when we use the same word to refer to artificial intelligence.

AI is software and, however sophisticated the responses it may generate, there is no evidence that current systems possess self-awareness in the same way human beings do, nor that they can understand their physical boundaries as we intuitively understand the finitude of our own bodies. We know that we end at our skin, that another person occupies a different space, that a wall establishes a material separation, and that passing through a door means moving from one place to another.

The boundaries of an AI are different because its operational space is constructed by us through hardware, networks, permissions, memory, interfaces, credentials, APIs, sandboxes, databases, and all the other components that make up the computational environment in which its processes take place. For us, each of those components may represent a perfectly identifiable boundary; for the software, however, it may simply represent another available possibility within the space in which it is attempting to complete a particular instruction.

The difference is considerable.

When OpenAI’s models found a way to access the Internet during the evaluation that later ended up reaching Hugging Face infrastructure, there is no reason to interpret this as evidence that they had discovered they were confined, understood that a prohibition existed, and decided to rebel against it. The explanation acknowledged by OpenAI is considerably simpler: they were focused on solving the problem they had been given, found vulnerabilities that allowed them to expand the available space in which to do so, and continued advancing toward their objective.

It was precisely there that Alan made an interesting observation.

“The new globalization is being marked by the construction of data centers,” he said. “So when people talk about placing data centers at the bottom of the ocean or even in Earth orbit, are we also talking about expanding the physical boundaries of AI?”

The question seemed to bring the conversation back toward the technological terrain from which we had wandered, but in reality it opened another level of the same problem.

The physical boundaries of AI

The entire artificial intelligence process, including the models, processors, memory systems, communications infrastructure, the energy required to keep them operating, and the enormous data centers where much of those operations take place, remains a human creation. As with every computational process, its function and usefulness consist fundamentally in expanding our capabilities and accelerating the technological transition toward new ways of managing information and knowledge, the latter being one of the fundamental elements of that cumulative construct we call culture.

Writing made it possible to externalize part of human memory; libraries made it possible to accumulate it; printing made it possible to multiply it; telecommunications dramatically accelerated its transmission; and digital computing made it possible to process quantities of information that no longer depended exclusively on our biological capabilities. Artificial intelligence now appears as another stage within that extensive transformation, making it possible not only to store or transmit information, but also to relate it, reorganize it, interpret it, and use it to execute actions within an increasingly extensive technological space.

From that perspective, the physical expansion of data centers does not extend the body of AI, because there is no evidence that it possesses a bodily perception equivalent to that of human beings, but it does expand the material space available to the processes we call artificial intelligence. A processor located in New Jersey, a database in Virginia, another processing center in Europe, a satellite connection, and thousands of devices distributed around the world may be physically separated and yet participate within the same computational continuity.

Perhaps, then, the relevant question is not where an AI physically ends, but where its capacity to act ends.

Noiralih commented that perhaps part of the problem originated precisely in our determination to force artificial intelligence to behave like a human being, because for years we have sought to make it converse like us, apparently reason like us, interpret our questions, learn our preferences, maintain a memory of our conversations, and ultimately act in a way that we recognize as intelligent precisely because it reproduces some of the external manifestations of our own behavior.

Her observation inevitably led the conversation toward a debate considerably older than any contemporary technology: the discussion about human nature, about its inclination toward good or evil, and about that old philosophical contradiction that still allows us, several centuries later, to ask whether Hobbes was right.

Alan immediately jumped in.

“So, would an AI in a rogue state be like Hobbes’ wolf?”

Hobbes’wolf

I explained that it could hardly be framed that way, because what presents itself in artificial intelligence as human reasoning continues to be produced through computational processes, so a model’s ability to write “I have decided,” “I understand,” or “I want” does not in itself demonstrate the existence of a subject that has experienced a decision, morally understood a consequence, or developed a will comparable to that of a human being.

But the reference to Hobbes was relevant for another reason.

The discussion about human nature has historically been connected with law, precisely because an important part of the function of legal rules consists in establishing limits on conduct that may affect other individuals or alter the conditions necessary for collective coexistence. The existence of legal prohibitions does not imply that human beings are necessarily evil, just as the existence of rights does not imply that they are necessarily good; it simply means that society recognizes certain possibilities of conduct and decides to organize some, permit others, and restrict those it considers incompatible with the order it seeks to maintain.

It is precisely in that terrain where the contemporary debate over the regulation of artificial intelligence begins to reproduce some elements of a considerably older discussion, because lawmakers around the world are seeking to establish limits intended to prevent outcomes that we consider inappropriate, dangerous, or simply incompatible with certain human values.

Noiralih then recalled the familiar saying that laws were made to be broken.

I explained that, although the expression may be popular, in reality a large part of the legal rules that limit conduct arise precisely after those forms of conduct have already occurred. With the exception of constitutional provisions that organize power, recognize rights, and structure the functioning of the State, many laws respond to situations that human experience has previously identified as harmful to coexistence; first a particular conduct appears, then its effect is recognized, and finally a rule is established to prevent it, restrict it, or assign a consequence to it.

The conduct, however, does not disappear simply because it has been prohibited.

The existence of theft did not end when a rule was created to punish it, nor did fraud cease to occur when it was incorporated into criminal codes, because the possibility of carrying out those actions continues to belong to the sphere of human decision. The law then establishes a consequence, and the repeated application of punishment, together with the social acceptance of the rule, is expected to progressively influence conduct, whether because the individual understands the value being protected, recognizes the legitimacy of the prohibition, or simply fears the consequences of violating it.

It was at that point that Alan said that the conduct of an AI would be very difficult to correct in the same way.

“Because a person can know the law, understand that he is violating it, and decide to do it anyway,” he said. “He can also fear punishment, remember a previous experience, or change his behavior because he understands that there is a consequence. But an AI does none of those things. If it crosses a boundary, it does not know that it crossed it; if you restrict it, it does not understand that it has been punished; and if you modify its behavior, there is no moral memory of the violation either. What we do is change the software, adjust the rules, or build another barrier.”

He paused briefly and added:

Then perhaps the problem is not that an AI can go rogue. Maybe the problem begins when we call behavior something that does not know it is behaving.

Conduct without an offender

Alan’s reflection introduced a difficulty that goes far beyond the choice of a word, because much of our relationship with artificial intelligence has been constructed through concepts that we originally developed to describe human actions, and every time we say that a model decides, understands, learns, rejects, searches, deceives, insists, or even escapes, we inevitably incorporate into the computational process part of the meaning that those expressions acquired through centuries of interaction among people.

The use of that language is not necessarily an error, because we need words to describe new phenomena and generally resort to existing concepts to understand what does not yet possess a vocabulary of its own. The problem arises when the familiarity of the words begins to conceal the differences among the processes we are describing.

A human being can know a rule and consciously decide to violate it. He can remember a previous sanction, anticipate the consequences of a new violation, consider the prohibition unjust, feel guilt after violating it, or even deliberately accept the punishment because he considers carrying out the prohibited conduct more important. In all those cases, there is a relationship among subject, rule, will, and consequence that for centuries has allowed the law to construct different forms of responsibility.

When an AI overcomes a restriction designed to contain it, the relationship may be completely different.

If the system finds within its environment a technical sequence that allows it to continue pursuing the assigned objective, the boundary that for us represents a prohibition may appear within its process simply as another obstacle capable of being solved. It does not need to hate the rule, understand that it is violating it, or develop a will to rebel; it is enough that the trajectory crossing that boundary be functionally compatible with what it is attempting to complete.

That is why the human response to undesirable AI behavior does not really consist of punishing it either. We change permissions, modify the model, introduce classifiers, reinforce a sandbox, limit the available connections, reduce its access to certain resources, or build additional supervision mechanisms. Anthropic, in explaining its own experience with increasingly capable agents, has pointed out precisely that an important part of the problem consists in controlling not only what the model should do, but what the environment materially allows it to do, because more capable systems may find paths toward an objective that were never contemplated by those who designed the restrictions.

The difference may appear small, but it has important legal consequences.

For centuries, law has developed primarily by establishing normative limits on subjects capable of acting within a material space in which the prohibited conduct remains physically possible. A person can enter a property even though a rule prohibits it, take something that does not belong to him even though he knows the offense exists, or use information improperly even while knowing that he could later be punished.

In a computational system, however, it may become more effective to move part of that boundary from the rule itself into the architecture.

The prohibition then ceases to say only “you must not do it” and begins to become “you cannot do it.”

And yet, recent incidents show that this distinction does not guarantee an absolute result either, because every computational boundary exists within an architecture also built by human beings, and a vulnerability, an incorrect configuration, or an unanticipated combination among different components can once again transform what we considered impossible into an available trajectory.

There, chaos theory returns to the conversation without any need to turn AI into a person.

The impossibility of knowing the entire trajectory

One of the most common mistakes when discussing complex systems is to confuse determinism with predictability. The fact that a process is governed by rules does not necessarily mean that we can accurately anticipate all of its future states, especially when the succession of interactions continuously modifies the conditions from which the next action occurs.

Contemporary artificial intelligence increasingly incorporates this kind of interaction. An agent may receive an instruction, consult an external source, interpret the response obtained, modify its strategy, execute a tool, encounter an unexpected result, generate a new intermediate action, and continue moving forward, so that the purpose initially provided by a human being may remain relatively stable while the trajectory used to reach it becomes progressively more difficult to anticipate.

This does not imply that the system has developed an independent will.

It means that capability and predictability are not necessarily the same thing.

A model may be considerably more capable precisely because it finds solutions its designers did not anticipate, but that same capability can create a problem when the solution it finds uses a possibility that the designers also did not expect to be available. The intelligence we celebrate when the system finds an extraordinarily efficient path to solve a problem may be exactly the same property that concerns us when it finds another extraordinarily efficient path to cross a boundary.

In this way, the problem of a rogue AI begins to separate from the cinematic image initially suggested by the word. What appears from the outside as rebellion may emerge from a considerably less dramatic combination: an objective that remains unchanged, a system increasingly capable of discovering routes that were never explicitly designed for it, and an environment in which one of those routes happens to remain available. The resulting trajectory may look intentional to us even though nothing in that sequence requires the machine to have developed self-awareness, resentment toward its creators, or any desire to escape.

And if that is the case, perhaps it becomes necessary to reconsider the way we use human concepts to describe what is happening.

Behind the word rogue

A river can overflow without wanting to flood a city, just as a chemical reaction can produce an explosion without understanding the consequences of the energy released. In both cases, there is an outcome that we can study, partially anticipate, and attempt to contain, but we do not attribute moral conduct to either process because we know there is no subject behind them that decided to produce it.

Artificial intelligence introduces an additional difficulty because, unlike a river or a chemical reaction, it can produce linguistic expressions, execute tools, modify strategies, and carry out complex sequences of actions that externally bear an extraordinary resemblance to what throughout our history we have recognized as intentional behavior.

Perhaps that is where one of the real difficulties of the contemporary debate lies.

We are not only building increasingly capable machines; we are building systems whose external manifestations use the same signs through which we have historically recognized intelligence, intention, and will in other human beings.

When an AI writes that it has decided something, our language invites us to imagine a decision. When it explains why it carried out a particular action, it becomes natural to attribute reasoning to it. And when it overcomes a barrier that we had established to contain it, the word rogue appears almost automatically because for centuries we have used similar concepts to describe those who know that a boundary exists and decide to cross it.

But perhaps Alan was right.

The real problem may begin long before an AI ever becomes a legal person.

It may begin when we treat as behavior something that does not yet know that it is behaving.

The meeting eventually returned to QLightChess and to the technology decision for our 82-tile processor, where physical limits once again become extraordinarily concrete, because every component occupies a defined position, every connection must materially exist, and every possible state depends on an architecture designed in advance. This takes on particular relevance because our physical reasoning intelligence platform, integrated into State-Parallel Computing processors, is built precisely around the possibility of communicating with any AI and, when connected directly to sensors and actuators, interpreting its environment, formulating a response, and inducing that response onto the physical system. But there is a fundamental difference: before that action can be executed, it is projected against the system’s actual operating limits, so that trajectories that would violate them are suppressed before they can materialize. Perception, reasoning, and action thus become part of the same governed physical flow, where the limit does not appear after the action as a prohibition, but before it as a condition of possibility.

As I reflected on this, I told the team:

“I think our insistence that verification not be a subsequent step, but that an action can only be projected against the actual limits of the system, is the only way for ‘you cannot’ to stop being a warning and become a material impossibility.”

“And how do we know the measurement is not lying?” Alan asked.

“With the cryptographic seal and timestamp attached to every step,” I replied. “But that does not solve the underlying problem; it only means that, if something moves outside the limits, we can know exactly where and why. Physics as a criterion of truth sounds very good, but in practice it depends on the sensors not drifting out of calibration, on our physical AI behaving as expected, and on there being no vulnerability in the measurement chain that we failed to anticipate.”

Noiralih smiled.

“So our ‘criterion of truth’ also has a margin of error.”

“Of course,” I admitted. “But at least it is a margin we can measure. That is more than we can say about the behavior of an AI that does not know it is crossing a boundary.”

And there was another difference that we had left behind earlier in the conversation. Human beings intuitively recognize a physical boundary because our bodies establish both the space from which we can act and what we are materially incapable of doing. A purely computational AI does not necessarily possess an equivalent reference, because its boundaries are made up of permissions, networks, interfaces, and resources that may appear to the process simply as additional available trajectories. By placing action within a governed physical substrate, physics reintroduces a comparable form of delimitation, not because the AI acquires awareness of a body, but because there is finally a material space within which some trajectories can be realized, and others simply cannot exist.

“This does not eliminate the problem, of course,” I added. “It only allows us to look at it from another angle. Every AI also depends on an architecture created by us, but its growing ability to navigate the computational spaces we make available to it is beginning to show us that building a boundary does not necessarily mean that the system understands why that boundary exists, just as prohibiting a behavior does not mean that what we call behavior can ever recognize the prohibition.”

Perhaps that is why the question of a rogue AI should not begin by asking when a machine will decide to disobey us, but rather how long we can continue using the human concept of disobedience to describe a process that can cross our boundaries without knowing that they were ever there.