KaliBench: A Fine-Grained Benchmark for Cybersecurity Tool Use on Kali Linux with Runtime-Free Verifiable Rewards

Paper recorded by Signals 4 on 2026-10-01 in cs.AI. Abstract reproduced from arXiv; link to the original below.

Published 2026-10-01 on arXiv · recorded by Signals 4 on 2026-10-02

Category: cs.AI · 人工智能 · first seen 2026-10-02

Abstract

LLMs are increasingly applied to cybersecurity workflows, where they are expected to translate analysts' intent into tool invocations. However, existing evaluations focus on knowledge-based assessments or end-to-end agentic tasks, and do not directly measure LLMs' ability to generate executable commands for real-world cybersecurity tools. This gap is critical because cybersecurity operations rely

Read on arXiv →