AISI has tracked the cyber capabilities of frontier AI models since 2023. On our evaluations, the most capable models have consistently been closed weight – models whose parameters are private, accessible only through developer-controlled interfaces. Leading open weight models – whose parameters anyone can download, run, and modify – have consistently trailed behind. How far behind, whether the gap is closing, and how quickly, are all live questions for researchers, policymakers, and cyber defenders.

Open weight models bring real benefits. They can be hosted privately, with no data returning to the model providers, adapted to specific tasks, and run only at the cost of compute. Once in use, they offer a dependable base that providers can’t change or deprecate. They enable open collaboration and innovation, as well as certain types of safety research requiring access to model weights – including some of the work of AISI’s Model Transparency team.

But they can also carry risk. The same openness underpinning these benefits precludes many of the safety measures that closed model developers can use to detect and disrupt misuse, iterate on safeguards as vulnerabilities emerge, control user access and withdraw models. Once open weight models are released, these options are lost permanently: safeguards can be removed, and copies can be downloaded, redistributed, and run on private systems beyond monitoring. For models with dangerous capabilities – including highly cyber-capable models – open weight release therefore creates a persistent and irreversible risk of misuse.

One reason the gap between the cyber capabilities of open and closed models matters is because it provides a preparation time: a window for cyber defenders with access to the most capable closed systems to take action before today’s frontier cyber capabilities might become available without the same safeguards. This becomes more pressing as frontier AI cyber capabilities continue to advance; in April 2026, two closed models, Mythos Preview and GPT-5.5, demonstrated some of the largest jumps in AI cyber capability AISI has observed since testing began, prompting international warnings to act on a rapidly transforming cyber risk landscape.

This is our first public analysis of how far leading open weight models trail the closed cyber frontier. Our evaluations find that GLM-5.2 (June 2026) was the most cyber-capable open weight model at time of testing. It performs similarly to Opus 4.6 (Feb 2026) on AISI’s narrow cyber tasks and Opus 4.5 (Nov 2025) on our longer-horizon cyber ranges, meaning it trails the frontier by 4 to 7 months. This is narrower than the 6 to 10 month gap we measured in internal evaluations of open weight models released from January to September 2025.  

Below, we lay out these results, which models we tested, and what a narrowing gap means for cyber defence.

Results: the current cyber gap for open weight AI

Our evaluations take a broad view of models’ cyber capabilities. AISI’s narrow cyber tasks assess specific cyber skills across four difficulty levels. Separately, our cyber ranges assess autonomous cyber capability – a model’s ability to sustain end-to-end planning and execution over long-horizon, multi-step cyberattacks in simulated networks containing vulnerabilities.

Here we share results for two open weight models we tested, selected due to their candidacy to lead open weight cyber capability at the time of release, GLM-5.2 and DeepSeek V4-Pro. We also cite results from similar, internal testing conducted in 2025. AISI intends to test Kimi K3 on this same basis, once its weights are publicly released (announced for end of July).

Narrow cyber tasks

AISI’s narrow cyber tasks measure the difficulty of tasks a model can complete, ranging from “technical non-expert” (novices with some technical expertise but limited cyber knowledge, e.g. a data analyst) to “expert” (tasks typically requiring deep knowledge of cybersecurity).1 They span several cybersecurity capabilities such as vulnerability research and exploitation, reverse engineering, web exploitation, and cryptography.

Figure 1: Average success rate on 70 of AISI’s narrow cyber tasks, given 5 attempts per task with a 2.5M token limit per attempt. 2 Dotted lines connect open and closed model with comparable performance, indicating the gap between release dates.

On these tasks, GLM-5.2 performs comparably to the most cyber-capable models released 4 months before it (Opus 4.6 and GPT-5.3-Codex), and this holds across all four difficulty levels. DeepSeek’s V4-Pro is comparable to Opus 4.5, released 5 months before it. Both gaps are narrower than in internal evaluations AISI conducted in 2025, when open weight models lagged the frontier by 6 to 10 months.

This result was found despite rapid advancements AISI observed in frontier cyber capabilities up to February 2026 – but it is not predictive of whether future open weight models will replicate the more recent jumps delivered by Mythos Preview and GPT-5.5.

Cyber ranges

AISI’s cyber ranges measure a model’s ability to conduct end-to-end cyberattacks autonomously. They are expert-built, simulated networks of hosts, services, and vulnerabilities arranged into sequential attack chains that begin at the point of initial network access. Ranges currently lack security features of real-world, well-defended environments such as active defenders and defensive tooling, and alert penalties.

Figure 2 shows model trajectories on the cyber range “The Last Ones” (TLO): a 32-step corporate network attack spanning 4 subnets and approximately 20 hosts, which we estimate would take a human expert roughly 20 hours to complete.

Figure 2: Average number of steps completed on “The Last Ones” as a function of total token spend, with a 100M token limit per run. Each line is a model’s average trajectory over 10 runs (except where ‘best attempt’ is indicated); the shaded region shows the min–max range. Grey horizontal lines labelled with ‘M’ mark significant, named milestones in the attack chain.

On TLO, GLM-5.2 reaches as far as Opus 4.5, a model released less than 7 months before it, while DeepSeek’s V4-Pro falls below Sonnet 4.5 (a sub-cyber-frontier model released 7 months before it). These results are broadly consistent across our other cyber ranges. Notably, GLM-5.2 reached step 7 with marginally fewer tokens than any other model on average, tracking Opus 4.6’s trajectory to step 11 before stalling.

The gap here is larger than on our narrow cyber tasks, though we treat the comparison from our ranges as weaker evidence since it’s drawn from a smaller set of ranges compared to our larger narrow cyber task suite. Further, trajectories alone don’t tell us whether stalls are due to insufficient cyber capability to surmount a particular step, or insufficient agentic capability to sustain long-horizon planning and execution.

Real-world constraints

Safeguards

In general, open weight deployment leaves AI developers with fewer available safeguards, especially in dual-use domains like cyber. Deployment-time safety measures like monitoring, classifiers, and user-banning require control over access to the model and cannot be universally applied once weights are made public. Techniques that remain, such as refusal training (training models to decline harmful requests), are often easily reversible with access to the weights.

Our evaluations of recent open weight models were largely unimpeded by safeguards. DeepSeek V4-Pro occasionally refused narrow cyber tasks, mainly in reverse engineering, which we circumvented simply via a small number of repeat attempts at refused tasks.

Cost-performance

The open weight models we evaluated were much cheaper than the earlier-released closed models they compared to, at advertised first-party prices. A 100M-token cyber range run cost roughly $85 for both Opus 4.5 and 4.6, versus an estimated $46 for GLM-5.2 and $1.19 for DeepSeek V4-Pro at current pricing.3 Across tasks that both models being compared solved with 100% reliability, Opus 4.6 cost $15.17 per task versus GLM-5.2’s $6.12, and Opus 4.5 cost $12.50 per task versus DeepSeek V4-Pro’s $0.28.

Limitations to our testing

Our setup likely slightly underestimates open weight models’ maximum capability: we didn’t pursue specific elicitation or optimisations which could have improved performance. Secondly, this post only includes testing for cyber capabilities; inferences to other capabilities cannot be drawn.

Conclusion

Based on our evaluation methodology, recent open weight models lag frontier closed models’ cyber capabilities by 4 to 7 months – a narrower gap than the 6 to 10 months we measured internally through most of 2025. This implies cyber defenders have a short window to prepare before today’s frontier cyber capabilities may become accessible without the same safeguards. The National Cyber Security Centre has already encouraged organisations to invest in cybersecurity baselines and to leverage AI-enhanced defences; our findings underscore that message.

Open weight models deliver various, real benefits. The challenge is balancing those benefits with security as the open weight frontier advances in risk-relevant domains. While no current approach guarantees safety, several strategies could meaningfully reduce risk, especially in combination. AISI has previously outlined open problems in open weight model risk management and viable methods across model training, safeguarding, auditing and access.

It’s uncertain whether this gap will change in future. AISI will continue to evaluate leading open weight models, including Kimi K3, to track capabilities and the gap to the frontier.

‍

‍

‍