Machine Learning Approaches to Identify Security Hotspots in Code

2024-2026

The linked repository GitHub Repository hosts a paper describing an ML project I built to detect security "hotspots" in code. These hotspots are valuable to security engineers, developers, and security compliance programs at scale.

Abstract

This paper demonstrates an application of specialized machine learning models that reliably identify security-sensitive operations in code and deliver meaningful cost and operational utility advantages compared to LLM alternatives while maintaining classification performance. The models achieve 74.6% precision, 77.5% recall, and 80.1% accuracy in human-labeled experiments, as well as a 95.42% precision rate in modified usage during real-world testing, enabling reliable detection of privileged operations in previously unseen codebases. This subjective task has applications for developers implementing security improvements, security engineers reviewing unseen codebases, and organizational security efforts at scale.

I start by constructing a labeled dataset using AST-derived representations and a multi-pass LLM pipeline, then train and evaluate multiple machine learning model architectures individually and in ensembles. Finally, I validate real-world applicability by conducting a human-labeled alignment experiment and demonstrating generalization and operational value. The results show that building custom models is an efficient and practical method to identify security-sensitive code that significantly outperforms LLMs in terms of cost and operational utility, and provides a mechanism that materially improves security implementation and review workflows at scale.

View the full paper here.