![](http://expert.com.ua/wp-content/uploads/2025/01/giganet_banner-900x90-2.png")
Recent speculation about the significant increase in the complexity of AI models, in particular Astra from OpenAI, raised concerns about their transparency and monitoring capabilities. However, OpenAI's chief scientist, Jakub Pachoki, allayed these concerns, stating that the architecture of the latest models has not undergone a dramatic increase in complexity.
Concerns about “deep recurrence”
Reports, including from The Information, have suggested that OpenAI is testing a technology called “recurrent depth.” This technology, unlike traditional linear architectures like Transformer (which ChatGPT, Claude, and Gemini are based on), could make it harder to understand the decision-making process of models. While standard Transformers process information in stages and can visualize their “chain-of-thought,” “deep recurrence” involves using the same layers of the network repeatedly to update the internal state. This means that much of the computation and logic can remain hidden in the internal states of the model, making it less transparent to researchers.
This prospect has sparked heated debate in the AI community, as it could complicate external monitoring and, as a result, reduce the overall transparency of AI systems.
OpenAI's answer: complexity under control
Jakub Pachoki posted a statement on the X platform (formerly Twitter) to clarify the situation. He emphasized that his goal is to prevent panic that could arise from inaccurate media reports that AI is moving towards unmonitorable. Pachoki assured that the computational graph depth of OpenAI’s most advanced models, including Astra, is no more than twice that of GPT-4. This means that the architectural leap in complexity that was feared did not occur.
“We really value this technique because it allows us to understand how models match the distribution of their training data. I think it’s quite fragile right now, and unfortunately, for some reasons unrelated to the architecture change, it’s moving in an undesirable direction. But we can take steps to strengthen it, and that’s one of the key goals of our current research projects,” Pachoki noted.
Monitoring the “chain of thought” remains a priority
OpenAI has been committed to monitoring the “chain of thought” since the early days of its models. The technique is critical to understanding how models learn and adapt to training data. While Pachoki acknowledged that current monitoring methods are “fragile” and need improvement, he stressed that this task is a top priority for the OpenAI research team.
According to the chief scientist, the monitoring issues that are being observed are not so much related to changes in the architecture, but to other factors that will be discussed in more detail later. This indicates that OpenAI is actively working to improve mechanisms for monitoring and understanding the internal processes of its AI systems.
What is the depth of a computational graph?
Computation-Graph Depth is a key measure of the complexity of a neural network architecture. It measures the number of successive computational layers that a model goes through from input to output. The higher this value, the more complex the model's internal structure and the number of computations it performs.
Jakub Pachoki's statement reassures the community, emphasizing that OpenAI continues to develop advanced AI models while maintaining a focus on their controllability and transparency. Additional monitoring and strengthening of control mechanisms are an integral part of the company's ongoing research work.
