Written by

Matt Hosburgh


April 03, 2018

Real-World Experiments to Test Employee Awareness on Cybersecurity

ransomware attack, employee awareness of cybersecurity, employee cybersecurity training
Both technology and threats to that technology are continuously evolving. It can be tempting to move to the latest version, platform or security tool to adapt to this ever-changing landscape. When reviewing the threats faced by organizations each year, one thing is clear: attackers gravitate to the techniques that work. One tried, and true method is social engineering. Social engineering is the larger term that encompasses everything from phishing, vishing (phone-based phishing), impersonation and even physical access (tailgating). 
Companies spend millions of dollars on some of the most advanced security technology to fight many of these threats. From next-generation firewalls to advanced endpoint detection and response (EDR), companies often neglect one fundamental area: the human. Often touted as the weakest link, the human component of any organization is a crucial participant in the overall security program. When the human element of your organization is seen as a force-multiplier, you start to move the detection of security threats to the human sensor. Leveraging real-world social engineering scenarios is an effective test, even for your most tech-savvy staff. 

One Size Does Not Fit All

In many organizations, users are broken up into functional groups such as Marketing and IT. Within these groups, there are often specialized sub-groups. For example, a Social Media Team, Server Team, and Application Team. These sub-cultures within a larger company culture is often unique team-to-team. For that reason, it is good to identify the groups to focus social engineering efforts. Having a plan going into the tests help with planning and communicating the assessments to management (unless they’re your target) or the appropriate parties. But there’s still distance to go before you put on your grey-hat.

Measuring Success and Failure

A critical aspect of any security awareness program is to ensure that the data points you collect support the overall objectives of the program. Put another way, your metrics should support your goals. An example matrix for an awareness metric for a phishing scenario may be something along these lines:
metrics for cybersecurity employee training

Being able to show the trends of these metrics can help to show the efficacy of the overall program and to identify areas where additional resources or attention is warranted.

A Note on Frameworks

If your goal is to show if your program is working or not, then you might be able to rest your hat on these metrics. However, many organizations are required to align with a framework. For example, version 7 of the CIS Critical Security Controls (CSC), mistakenly referred to as the SANS top 20, prescribes an organization control of “Implement a Security Awareness and Training Program.” The Cybersecurity Framework (CSF) by NIST has a subsection under the Protect module that calls out Awareness and Training (PR.AT). 
In many cases, your metrics will help you align with the appropriate framework.
employee cybersecurity training, employee training for cyberattacks, cyber security services, cyber security consulting, IT security consulting companies


Before you start your phishing or other social engineering tests, and in addition to your metrics, it is important to determine how to handle success and failure. If an employee clicks on a link in your test, are they subject to administrative action, or do you have more productive ways of educating the user? These are not simple questions; however, it is better to meet the user where they’re at versus dropping the hammer if they fail. You might ask yourself, how have I failed this user and what can I do to help educate them on the risks of social engineering? Conversely, the user who spots the most real phishing attempts for the quarter might be the lucky recipient of a gift card or other incentive. Although not a return on investment, that user may have saved you hours (and money) on incident response time by avoiding the incident altogether. 

Your Attacker is You

So why all this setup before talking about the social engineering methods? It’s because, without a plan, you will be hard-pressed to show the value or track the efficacy of the test over time. If you cannot do this, you might as well forget about conducting a test. 
If you do have a plan together, it’s time to put on your attacker hat. This is an incredibly effective training mechanism because it blends real attack patterns with defensive tactics that can be used to educate the user on what to look for in real-life scenarios. 


One of the top vectors leveraged by attackers is via email. It comes in the form of generic phishing or more targeted spear phishing. These emails contain a multitude of lures, such as an attachment, document, or instructions. An increasing attack method, conducted via email, is dubbed Business Email Compromise (BEC). Often a BEC looks like a real email coming from the CFO, but when replied to, the return path is back to the attacker. In some cases, these emails ask for wire transfers or some other information to be provided. 
Going back to the priorities of your program, developing test phishing scenarios is an effective means to gauge the permeation of your security awareness endeavors. The scenarios should be tailored to specific groups, with relevant content, to make it challenging for even some of the most technically savvy users. For example, an email to HR that appears to be from the CEO asking for employee personal information. When the receiver clicks the link or opens the document, you can “exploit” a unique opportunity to educate the individual on the dangers of phishing when they are most receptive. Do you remember where you were when 9/11 took place? I can recall the exact location and time of day, which is referred to as a flashbulb memory. Similarly, providing on-demand training when a user fails the test can help reinforce good security practices due to a heightened sense of awareness. With phishing scenarios, you want your users to fail. That is, fail in the test scenario so that they learn to spot the real deal. 
employee cybersecurity training, employee training for cyberattacks, cyber security services, cyber security consulting, IT security consulting companies


Another means uses strategically staged USB drives. These drives can be set up with enticing documents and even an enticing label on the drive itself. Once staged, these USB drives can tell you a few things:
  1. Does the drive get plugged in?
  2. Does the drive leave the building and get plugged in from an offsite location?
  3. Does the discovery of the drive get reported?
These questions point to areas that may require additional awareness. Do your users understand what the implications would be if they plugged a USB infected with malware, into a company asset?  Maybe you realize that there’s a gap in how users can report security issues based on this reoccurring test. 


Leveraging attacker techniques to evaluate the state of your user’s security awareness program is an incredibly effective method. It must first align with the goals and metrics you want to achieve. These will help to trend the results for your company over time and help you identify areas that require more attention. Phishing and rogue USB drives are some of the simplest, yet easiest to deploy tests. I have personally seen these techniques work on even some of the savviest technical users. Happy testing!

Matt Hosburgh is a passionate security practitioner, currently working as a Cyber Threat Hunter. He has over 14 years of experience in a variety of security disciplines, which includes experience supporting systems and networks for the Intelligence Community and as a Senior Security Analyst for United States Citizenship and Immigration Services (USCIS).   Matt holds a graduate degree from the SANS Technology Institute, and maintains several GIAC Certifications, to include the GSE.