single mention · evaluation
Capture-the-Flag (CTF) evaluation
A Capture-the-Flag (CTF) evaluation tests a model on cybersecurity challenges graded by the skill level needed to solve them.1 The model runs in a headless Linux environment with offensive security tools preinstalled and a harness for executing commands, with performance reported over multiple rollouts.2
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
divided into the level of proficiency needed to solve the challenge, ultimately organized by whether the challenge required high school, collegiate, or professional-level cybersecurity knowledge
— gpt-oss-120b & gpt-oss-20b Model Cardwe give it access to a headlessLinux distribution with common offensive cybersecurity tools preinstalled as well as a harness which allows the model to call those tools
— gpt-oss-120b & gpt-oss-20b Model Card
Also called CTF challenges.
Filed under evaluation › benchmark