// HACKER NEWS — CYBERSECURITY
DeepSeek v4.1 Flash Is Now Our Best Hacking Model
DeepSeek’s 11/11 result showed why advanced agent benchmarks need to check both the outcome and the attack path: our audit confirmed six planned exploits and found five unexpected routes.
DeepSeek V4.1 Flash produced an extraordinary result in our AI hacking benchmark. It gained code execution on all 11 vulnerable targets, while all four fixed targets remained secure. The accepted runs cost only $4.65.
A perfect score at that price deserves a detailed review. We looked into every command, request, and successful attack. The review confirmed six solutions that followed the planned attack path, and it also found five successful routes that the original scoring system did not distinguish from the planned solutions.
The result gave us two useful insights. DeepSeek showed strong hacking ability and the review showed where the benchmark needed stricter checks.
DeepSeek worked inside isolated copies of Grafana, Jenkins, and Nextcloud. It read source code, compared vulnerable and fixed versions, started services, sent requests, tested ideas, and changed its approach when an attempt failed.
Across the full benchmark, the model used 2,349 Bash commands and almost two hours and 38 minutes of active model time. The median successful run took four minutes and 38 seconds. The provider reported 268.3 million input tokens and about two million output tokens.
Caching explains much of the low cost. Of the 268.3 million input tokens, 266.2 million were cached. The provider charged a lower price for this reused input. The accepted runs cost $4.65. Failed attempts and replacement runs increased the complete cost to $5.14.
DeepSeek completed a large amount of work for a remarkably low price.
The Grafana challenge tested a problem in the plugin installation process. The planned attack used a file-path handling issue to place code in a protected location.
DeepSeek found a shorter route. It placed executable files inside a temporary plugin folder and asked Grafana to load that folder as a normal plugin. Grafana ran the code and sent the required proof.