Microsoft’s CyberBattleSim environment effectively leverages Reinforcement Learning to simulate network intrusions and lateral movement, but its current implementation has limitations. In this paper, we extend the CyberBattleSim framework to support VLAN-based (Virtual Local Area Network) network segmentation. This modification enables researchers to design more realistic corporate network topologies, simulating both local and remote traffic management between isolated network segments. We present a novel methodology for integrating Access-Control Lists (ACLs) to enforce segmentation rules and demonstrate its application in a reinforcement learning (RL) setup. After implementing these enhancements, we benchmark the performance of several RL agents in the modified environment. The results show that network segmentation is effective at slowing an attacker attempting to move laterally through a simulated environment. Our work not only enhances the CyberBattleSim framework but creates opportunities for more robust research in attack-path prediction, lateral movement, and intrusion detection.
References
[1]
Raza, M. (2023) What Are TTPs? Tactics, Techniques & Procedures Explained. https://www.splunk.com/en_us/blog/learn/ttp-tactics-techniques-procedures.html
[2]
Chandramouli, R. (2021) Guide to a Secure Enterprise Network Landscape (NIST Special Publication No. 800-215). National Institute of Standards and Technology. https://doi.org/10.6028/NIST.SP.800-215
[3]
Internet Engineering Task Force (2012) Firewall Considerations for Operations and Management. https://www.ietf.org/archive/id/draft-ietf-opsawg-firewalls-01.txt
[4]
Microsoft Research (2020) CyberBattleSim: A Platform for Simulating and Evaluating Cyber Defense Strategies. Microsoft Research. https://www.microsoft.com/en-us/research/project/cyberbattlesim/
[5]
Kunz, T., Fisher, C., La Novara-Gsell, J. and Nguyen, C. (2024) A Multiagent CyberBattleSim for RL Cyber Operation Agents. Systems and Computer Engineering, Carleton University.
Microsoft (2024) Cached and Stored Credentials Technical Overview. https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/hh994565(v=ws.11)
[9]
Siddiqui, L. (2024) Security Event Logs: A Beginner’s Guide. Splunk. https://www.splunk.com/en_us/blog/learn/security-event-logs.html#:~:text=Network%20infrastructure%20or%20firewall%20logs&text=They%20track%20connection%20attempts%2C%20blocked,security%20teams%20can%20detect%20cyberattacks
[10]
Franco Terranova, A., Lahmadi, A. and Chrisment, I. (2024) Leveraging Deep Reinforcement Learning for Cyber-Attack Paths Prediction: Formulation, Generalization, and Evaluation. The 27th International Symposium on Research in Attacks, Intrusions and Defenses (RAID 2024), Padua, 30 September 2024-2 October 2024, 1-16. https://doi.org/10.1145/3678890.3678902
[11]
Moschovakis, J.R. Markov’s Principle, Markov’s Rule, and the Notion of Constructive Proof. UCLA Mathematics.
[12]
Hong, Q.A., Li, J.Q., Guo, X.Z., Xie, P. and Zhai, L.D. (2023) Assessing the Effectiveness of Deception-Based Cyber Defense with CyberBattleSim. In: Goel, S. and Nunes de Souza, P.R., Eds., Digital Forensics and Cyber Crime. ICDF2C 2023. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, Vol. 571, Springer, 224-243. https://doi.org/10.1007/978-3-031-56583-0_15
[13]
Fortinet (2025) How to Prevent Lateral Movement in Cybersecurity? Fortinet. https://www.fortinet.com/resources/cyberglossary/lateral-movement
[14]
MITRE Corporation (2024) T1003.003: OS Credential Dumping: NTDS. https://attack.mitre.org/techniques/T1003/003/
[15]
Baeldung (2021) Q-Learning vs Deep Q-Learning vs Deep Q-Network. Baeldung. https://www.baeldung.com/cs/q-learning-vs-deep-q-learning-vs-deep-q-network#:~:text=Essentially%2C%20deep%20Q%2DLearning%20replaces,%2C%20Q%2Dvalue
