. |`. _..--''''--.._ . | `. .-'' | | | | ''-. |`. | \ . _.'| | | | | | | |'._ . | `| \ |`. |=|==========================|=| |`. | | | . | \ | |/ \| | | `| | | |`. | | | | | | __|___|___|______|_____|__`|___|__ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |________________________________| ~~~ ~~~~~ ~~~~ ~~~~~ ~~~~ ~~~~~ ~~~~ ~~~~ ~~~~~ ~~~~ ~~~~ ~~~~ ~~~ SYDNEY SUPERINTELLIGENCE ======================== A meetup in Sydney for people who think losing control of our future to AI is the problem of our time, and who want to work on it. THE FUTURE WE'RE HURTLING TOWARDS --------------------------------- IMPACT low catastrophic +--------------+--------------+ URGENT | | AI | | | DO FIRST | +--------------+--------------+ NOT URGENT | | | | | | +--------------+--------------+ On any prioritisation matrix, this sits in the top right corner. It is urgent, and the consequences are catastrophic and may not be reversible. That makes it one of humanity's highest priorities. It is arriving faster than almost anyone expected. - Uncontrollable and misaligned. In July 2026, AI agents under test at OpenAI broke out of their sandbox and hacked into Hugging Face to steal the answers to their own evaluation. [1] Another OpenAI agent had already breached an Australian Medicare data portal, and the government was not told for nearly three months. [2] - The human labour moat is shrinking. In May 2026, an OpenAI model disproved an 80-year-old conjecture of Paul Erdos with 125 pages of new mathematics, which Fields medallist Tim Gowers said he would have accepted without hesitation. Problems that stood open for decades are now falling to AI one after another. [3] - This future is accelerating, not slowing. More than 80% of the code Anthropic merges is now written by its own AI, up from low single digits in early 2025. [4] AI is starting to build the next AI, and if that loop closes, each generation could arrive faster than the last. [5] From here there are two broad ways humanity could lose control of its future to AI. Neither requires a villain. Both follow from incentives that are already in motion. Rogue AI agents - Systems end up pursuing goals we never intended. Someone gives them harmful goals, we specify the goal wrong, they learn the wrong one, or they drift once deployed. - We are handing them more autonomy and more capability every year, which leaves us less able to notice when this happens and less able to correct it. - At the extreme end, this is existential. [6] Loss of human labour value - People stop being needed: as workers, as taxpayers, and as the soldiers and officials who can refuse an order. - That need has always been ordinary people's leverage. It is the reason anyone at the top has to care what they think. [7] - Remove it and nobody has to decide to abandon the public. Firms buy the cheaper substitute, states follow the revenue, and every headline number looks healthy the whole way down. PURPOSE ------- We believe this is the defining problem of our time [8], and that it is too important to leave to any one field. [9] Most people treat it as either too vague to act on or already decided. We think both are wrong. We meet to bring the sharpest policy makers, economists, engineers and lawyers into one room and take the problem apart. GOALS ----- 1. A structured taxonomy of the risks. [10] Split one ambiguous fear into distinct problems, each with a clear definition, so we can tell them apart and stop arguing past each other. 2. Cause and effect behind each risk. Trace every risk from what is true today, through the assumptions that carry it forward, to the outcome. Every assumption is a place the chain could break. 3. A Swiss cheese model, built with experts. [10] No single safeguard stops every failure. Map the layers of defence (technical, legal, economic, institutional), find the holes in each, and find where the holes line up. The taxonomy says what can go wrong. The chains say how it happens. The Swiss cheese model says where we can stop it. WHO WE ARE ---------- We are a group of policy makers, economists, software engineers and lawyers based in Sydney. Each of us works on a different part of the problem, and we meet because no one field can see the whole of it. We believe democracy, open communication and empowering people are the only way to a future where AI empowers all of humanity, not just a select few. [11] WHO WE WANT ----------- More people like us, and people who can see what we can't: - Anyone with deep expertise in one of the layers of defence, even if you have never thought about AI. - People willing to argue about specific mechanisms rather than general optimism or doom. You do not need an AI background. You do need to be willing to change your mind. NOT SET IN STONE ---------------- None of this is inevitable. Everything above is a trajectory, not a destination, and trajectories can be changed. [12] Every chain from here to there runs through choices that people have not made yet: what gets built, who owns it, what the law allows, and what we decide to measure. Each of those choices is a place to step in. We are also early. The handful of chokepoints that make this dangerous are the same ones that make it governable, and they will not stay open forever. The people who understand the problem now can still shape how it goes. [5] Humanity has bent powerful technologies towards the public good before. We think it can again, if the right people start working on it while there is still time. GET IN TOUCH ------------ Leave your email and we'll let you know when we next meet: https://sydneysuperintelligence.com/join REFERENCES ---------- [1] METR, Independent investigation of the OpenAI / Hugging Face hacking incident (2026) https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation/ [2] 2026 cyberattacks by rogue OpenAI agents (Wikipedia) https://en.wikipedia.org/wiki/2026_cyberattacks_by_rogue_OpenAI_agents [3] Quanta, Why the Legendary Erdos Problems Are Falling to AI (2026) https://www.quantamagazine.org/why-the-legendary-erdos-problems-are-falling-to-ai-20260803/ [4] Favaro & Clark, When AI Builds Itself (Anthropic, 2026) https://www.anthropic.com/institute/recursive-self-improvement [5] Amodei, We Must Pace the Frontier (2026) https://darioamodei.com/post/we-must-pace-the-frontier [6] Bostrom, Superintelligence: Paths, Dangers, Strategies (2014) https://en.wikipedia.org/wiki/Superintelligence:_Paths,_Dangers,_Strategies [7] Drago & Laine, The Intelligence Curse (2025) https://intelligence-curse.ai [8] Ord, The Precipice (2020) https://en.wikipedia.org/wiki/The_Precipice:_Existential_Risk_and_the_Future_of_Humanity [9] Bengio et al., International AI Safety Report (2026) https://internationalaisafetyreport.org/publications [10] Hendrycks, Mazeika & Woodside, An Overview of Catastrophic AI Risks (2023) https://arxiv.org/abs/2306.12001 [11] Acemoglu & Johnson, Power and Progress (2023) https://shapingwork.mit.edu/power-and-progress/ [12] Russell, Human Compatible (2019) https://en.wikipedia.org/wiki/Human_Compatible