Cyber Security Exercises and Reality

Cybersecurity Exercises and The Reality

Alexandre Dulaunoy a@foo.be

version 0.1 - 2017-11-30

When you are facing a potential threat, the most difficult aspect is to understand what you are fighting against. Evaluating a threat in information security is a complex aspect especially when you have no simple ways to scale the threat and know if you have the organisational and technical capabilities to respond to such threat.

In the past years, many cyber security exercises appear at local, national or international levels with the aim to improve the capabilities at organisational or/and technical levels. There are many different organisations involved in such exercise and there are many models depending of their respective focus. After being involved in many of those (including designing or/and playing), I compiled my thoughts and especially the shortcomings in such approach. The idea behind this series of notes is to improve such exercise or experiment other approaches.

Synthetic information/evidences

While participating to some exercises, a lot of the evidences used are synthetic and rarely reflect realities from operational security. This gives a perception to the players that the evidences are like this in real cases. But it’s usually not the case, the collection of the evidences (and its complexity) is often discarded from such game. Any digital forensic investigator knows how complex is to gather, collect and acquire evidences. So it’s not by playing or participating to such exercise that would help you or your organisation to grasp the complexity and improve your team capabilities.

Reducing operational security aspects to simple games

A critical issue in my eyes with cyber security exercises is the over simplification of cyber security threats at a level which make these understandable for the political or non-operational managerial level. There are some significant risks to reduce complexity of the reality. When operational security teams face real and concrete incidents, their work can be seen as like solving a challenge. In incident response, it’s quite common to face complex topics, with different contexts and ultimately being incapable to reach a complete solution of the analysis from partial evidences, multi-compromised infrastructures.

Ideas and improvements to make “exercises” useful

Proposal Description
Take real cases, evidences and investigations Avoid at all cost synthetic or fake data when creating exercises. If you take real data, don’t mix-up with synthetic data.

References