Code Runs the World
· outdoors
Codependent on Chaos: Why Math Matters in a World Run by Machines
The International Congress of Mathematicians’ meeting in Philadelphia last month highlighted the growing reliance on machines that are rewriting the rules of human endeavor. Jacob Tsimerman’s decision to leave academia for AI safety is symptomatic of a deeper issue: our world is rapidly becoming reliant on code we don’t fully understand.
Mathematicians have recently used AI to prove an 80-year-old problem, known as the Erdős unit distance conjecture. This breakthrough has far-reaching implications beyond mathematics. It highlights the paradox of our time: while machines are accelerating discovery, they’re also creating a bottleneck that’s stifling innovation. Human confirmation is now the scarce resource.
The rise of generative AI has made it possible for developers to write large quantities of code with natural language prompts. However, this convenience comes at a cost: the resulting code is often opaque and difficult to vet. Our infrastructure for certifying discoveries is woefully inadequate, making it challenging to verify the accuracy of machine-generated code.
Formalization – translating AI outputs into precise forms that can be checked automatically – is key to unlocking this bottleneck. However, building the necessary infrastructure requires a national-scale effort. This involves more than just mathematics; it demands collaboration across industries and disciplines.
Researchers have been developing systems for proving that software will do exactly what it was designed to do for decades. These formal methods have made it possible to automatically validate AI output, but they’re not a panacea. Formal verification can’t eliminate all errors in complex human systems – but it can eliminate whole classes of errors in critical software.
The Leiden Declaration warns that AI threatens the verifiability of proof, but mathematics offers the infrastructure to make generative AI more trustworthy. The United States should treat mathematical rigor as a national mission, investing in public infrastructure for verified software and sustaining investment in mathematics research and education.
Companies operating critical infrastructure shouldn’t rely solely on reactive patching and bug bounties; they should aim to build in auditing and formal guarantees from the start. This isn’t about verifying every line of code but about specifying what that logic must and must not do.
Mathematics has weathered crises of trust before – geometry, infinity, logic, and proof each threw the field into foundational doubt. Each time, the response was more precise languages, stricter standards, and better verification. Software now needs the same transformation as AI writes more of the code that runs our world. Blind trust is no longer good enough.
The United States stands at a crossroads: it can continue down the path of reactive patching or invest in a future where machines and humans work together to create code that’s both efficient and trustworthy. The stakes are clear: our world runs on code, and it’s time to take responsibility for its integrity.
Reader Views
- TTThe Trail Desk · editorial
The "Code Runs the World" narrative glosses over a critical aspect: the uneven distribution of expertise required for formal verification. While mathematicians and computer scientists are acutely aware of this issue, the reality is that few industries outside academia possess the necessary talent to develop and implement robust formal methods. This expertise gap threatens to exacerbate the bottleneck created by machine-generated code, highlighting the need for a more inclusive approach to developing the infrastructure required to verify AI outputs.
- JHJess H. · thru-hiker
While it's true that math is increasingly essential in a world dominated by machines, we shouldn't overlook the elephant in the room: who's going to pay for this infrastructure? The article highlights the need for national-scale collaboration and funding, but what about the human cost? What happens when researchers like Jacob Tsimerman leave academia because they can no longer secure grants or industry partnerships? Can we really afford to let math literacy fall by the wayside in the name of progress?
- MTMarko T. · expedition guide
The article raises important questions about our reliance on code and the limitations of machine-driven innovation. However, I'm struck by the lack of attention given to the human element: what happens when these systems fail? We're so focused on developing formal methods for verifying AI output that we're neglecting the need for skilled professionals who can actually debug these systems when they go wrong. In other words, as we create more sophisticated machines, do we also need to revamp our education and training programs to ensure a workforce capable of dealing with the unintended consequences?