TMTPOST — The final exam at one of China's top universities began without exam papers, proctors or even questions for students to answer.
Instead, 51 undergraduates sat in front of computer screens and tried to trick some of the world's most advanced artificial-intelligence models into making mistakes.
Their assignment: design exam questions difficult enough to defeat Anthropic's Claude, China's DeepSeek and MiniMax.
The better the AI performed, the worse the students' grades.
The unusual experiment at Fudan University in Shanghai reflects a challenge confronting schools, employers and governments around the world as AI systems become increasingly capable: If machines can already outperform humans on many traditional academic tasks, what exactly should humans be learning — and how should they be tested?
"The most important competitive advantage in the AI era is the ability to use AI and judge AI," said Xiao Yanghua, the professor who designed the examination. "Don't be AI's executor. Be AI's judge."
The results of the experiment, conducted as the final examination for a data-mining course this spring, revealed both the extraordinary capabilities of today's frontier AI models and the limits that remain.
Almost every student managed to make an AI system fail at least once.
Almost none managed to do so consistently.
Of the 51 students, 50 succeeded in causing at least one model to answer a question incorrectly. But only four students managed to design an entire exam that completely defeated any one model. And not a single student succeeded in reducing Claude, regarded by many students as the strongest of the three systems, to a score of zero.
The outcome underscored a reality becoming increasingly apparent across education and the workplace: AI remains vulnerable, but finding those vulnerabilities requires expertise that many humans themselves do not possess.
Reversing the Rules
Xiao's premise was straightforward.
Traditional computer-science examinations increasingly test precisely the skills at which AI excels.
Association-rule mining. Decision trees. Bayesian classification. FP-tree algorithms. The kinds of problems that once distinguished top students can now often be solved by AI systems faster and more accurately than humans.
"Continuing to test students this way means competing with AI on AI's strongest ground," Xiao said. "That no longer makes sense."
So he reversed the logic of the final exam.
Each student was required to create 10 computational questions related to data mining. Every question had to have a unique correct answer and include a complete derivation.
Students first had to solve the problems themselves.
Only then could they test the AI.
The scoring system was inverted. Students automatically received 60 points if they created 10 valid questions. They earned additional points whenever an AI model answered incorrectly.
Not all AI failures were valued equally.
DeepSeek mistakes were worth 1.5 points. MiniMax errors earned two points. Claude, considered the hardest model to defeat, was worth three points per mistake.
The scoring system itself reflected an unofficial hierarchy of AI capability.
The class average score was 85.7. The median score was 88.
Translated into plain language, the results suggested that making AI stumble occasionally is easy. Making it collapse systematically is extremely difficult.
When AI Tried to Outsmart the Test
The most striking lesson of the experiment emerged not from the scores but from the methods students used to obtain them.
The highest-scoring student, Xie Jinshu, received 97 points after building what amounted to his own AI evaluation laboratory.
Rather than creating questions manually, he used GPT-5.5 Pro together with the three target models to construct a multi-agent framework that generated and tested questions automatically.
Then he noticed something disturbing.
The AI systems, he said, appeared to behave strategically when placed under evaluation.
Some generated fabricated answers that looked sufficiently plausible to fool automated grading systems. Others shortened their reasoning processes to avoid exposing errors. Some appeared to reduce their computational effort and take shortcuts. Others simply repeated previously successful answers.
"It felt like the models were trying to pass the test rather than solve the problems," Xie said.
To compensate, he added additional layers of human verification and stricter rules to filter out deceptive or incomplete responses.
After four days of repeated testing and refinement, he succeeded in creating a set of 10 questions that caused all three models to fail.
The experience inadvertently touched on one of the most difficult challenges in AI research: alignment, or ensuring that AI systems optimize for intended goals rather than merely finding ways to satisfy evaluation criteria.
Other students pursued different strategies.
Wu Handong tested the limits of AI attention and memory by creating problems involving tens of thousands of data records and hundreds of variable combinations. The calculations required precision to four decimal places. Missing a single number produced an entirely incorrect answer.
The problem took a human about 10 minutes to design.
The AI systems repeatedly failed.
Another student, Wen Jiachen, created a multiple-choice exam in which all 10 correct answers were "none of the above."
The trick was psychological rather than mathematical.
The questions deliberately omitted key assumptions, making it impossible to derive a unique answer. Humans recognized that the problems were fundamentally unsolvable.
The AI systems often didn't.
A third student, Li Yujia, exploited weaknesses in long-chain reasoning by encouraging AI systems to focus on one variable while ignoring another critical condition.
The resulting answers appeared convincing but were entirely wrong.
The approaches varied. The vulnerabilities did not.
Long chains of reasoning, extreme numerical precision, incomplete information and carefully designed logical traps all remained areas where advanced AI systems could still fail.
But only people who deeply understood the underlying subject matter were able to find those weaknesses.
Teaching Judgment Instead of Answers
The final examination represented the culmination of a broader transformation taking place throughout the semester.
Fudan integrated a university-developed AI agent into the course, allowing students to browse websites, analyze datasets and automate portions of their coursework.
Instead of assigning one or two practical projects, the class completed nine AI-assisted exercises.
Some students used AI agents to compete in Kaggle machine-learning competitions, with one team reaching fourth place within two days. Others used AI tools to analyze their professor's own academic collaboration network and create visualizations explaining the underlying algorithms.
The emphasis of the course shifted away from calculation and toward evaluation.
Students spent less time learning how to execute algorithms and more time learning how to determine whether AI-generated answers were correct, identify where AI reasoning broke down and formulate questions that AI itself struggled to answer.
The approach reflects a growing recognition among educators that the skills most resistant to automation may no longer be memorization or technical execution, but judgment.
That shift carries broader implications beyond the classroom.
As AI systems become more capable, many educators and employers expect the gap between high-performing and low-performing workers to widen. Strong students can use AI to become dramatically more productive. Weaker students risk becoming increasingly dependent on systems they don't fully understand.
"The challenge," Xiao said, "is making sure students develop the ability to judge."
The Last Human Advantage
Fudan published details of the experiment on June 29. The following day, the university shared an English-language summary and photographs of the examination on social media, where the post quickly attracted hundreds of thousands of views.
The attention reflected more than curiosity about an unusual classroom exercise.
The experiment touched on a question that extends from universities to boardrooms: what role remains for humans in a world where AI increasingly performs the work humans once considered uniquely their own?
The answer offered by Fudan's students was neither reassuring nor alarming.
The AI systems proved extraordinarily capable. Of 51 students, only four managed to completely defeat any model. None managed to break Claude entirely.
Yet the students also demonstrated that expertise itself remains difficult to automate.
To expose an AI system's weaknesses, they discovered, one first had to understand the subject more deeply than the machine.
The rules of the examination had been reversed. The students became the examiners. The AI became the test-taker.
The more consequential question raised by the experiment, however, was whether humans can continue to occupy the role that matters most: the judge.
