Scans vs. Assessments vs. Pentests

At White Oak we work with a lot of companies across a wide range of industries, sizes, geographic locations, etc.  Many of these organizations have very mature information security programs.  Many do not.  We do not discriminate - if you're trying to improve security and lower risk at your organization we are happy to work with you and help you and your team move things forward.

This is the first in a series of posts that will hit on some key points for companies and teams that are new to pentesting.  This series isn't going to be technical - it's going to be informational, with a goal towards better preparing you for making decisions about what type of testing you need, how to work with your testing partner effectively, what to expect during testing, and how to most effectively address post-testing activities.

In this post we're going to talk about different types of security testing, what the terms mean, and where the types fit in your security program.  (I'll also try to avoid ranting too much about marketing departments at some product vendors and some low-end testing providers misleading clients on what a penetration test really is...)

Testing Hierarchy

The first thing that we're going to talk about is testing 'types' - these types just designate different breakpoints in what's really a continuous testing model.  In other words, all of the testing types we're going to be discussing here are built on top of each other - each including and then improving on the one that came before...

Here's my quick, but very important rant - pay attention not just to the labels that any testing partner uses with their services.  Pay attention to descriptions of what's involved in those services.  The term 'Penetration Test' or 'Pentest' is now often used to describe something that's not a penetration test.  You end up getting skewered when it comes time to provide your client or auditor or board with the results of a penetration test only to find out that what you got was a scan.  This happens A LOT.  It's extremely frustrating.  It should be illegal…but it's not.  So please ask questions!  Thanks.  Rant over.


A 'Scan' is testing that is performed using an automated scanning solution.  There are a number of scanning tools that you and testing firms can utilize to automatically scan your network, your applications, or your cloud environments.  There are many tools out in the market and there are more being added constantly.  Keep in mind that there are also a bunch of open source scanning tools as well that are really quite good. 

Scans, due to the fully automated nature of the exercise, have their uses but are also very limited.  There are a number of factors that make scans useful, but that also limit their effectiveness.  This is the first stage and lowest rung of the testing hierarchy, but it certainly has its place in a security program.


Where scanning fits really well into a mature information security program is for compliance reasons (i.e. it's required), for continuous monitoring, and for gap testing between more rigorous testing exercises.  It's just generally impractical for organizations to perform deeper testing exercises on their network or on a deployed and stable application in a constant manner.  Scanning can help you watch your assets in between penetration testing.  It can also help you discover what others at your organization are deploying - maybe without going through a proper security review...


A 'Vulnerability Assessment' is the next step up the hierarchy.  A vulnerability assessment builds on (and typically extends) a scanning exercise by incorporating multiple scanners (with differing strengths) and manual testing that will find issues that the scanners simply can't.  This includes both technical vulnerabilities that the scanners aren't good at finding as well as business-logic issues that the scanners simply can't find. 

If the team doing the vulnerability assessment has any pride (and/or understands the hell they will be creating if they don't) the vulnerabilities will also be verified manually to ensure that the end result doesn't include false positives. 

Vulnerabilities should be reported using a format and structure that is significantly more useful and effective than auto-generated scanner reports.

An additional step should also be completed here, but often isn't.  Vulnerabilities are risks.  Each risk needs to be properly understood and put into the right context so that someone at your organization can make a decision about remediation priorities.  The team doing your vulnerability assessment (whether an internal team or an external partner) should try to understand this context as best as possible and should actively work to include that context where they are able.


Vulnerability assessments are useful in taking your testing to a level of insight that should go well beyond basic scanning.  It's particularly useful in situations where you want to avoid testing risk to an asset or environment.  As vulnerability assessments do not include safe exploit of vulnerabilities it can be a safer option (than a full penetration test) if you are concerned about the target asset's stability.  That doesn't mean that a scan or a vulnerability assessment can't take down systems - they can. 


Penetration Testing or 'Pentesting' is the next step up.  It builds on the previous two steps to provide additional, very useful, information regarding the target asset or environment.

In a penetration test, a wide variety of automated scanners are typically utilized first to find the easy-to-identify vulnerabilities and to gather information for the manual work.  Then the steps of a vulnerability assessment are performed to test for manual vulnerabilities, business logic issues, and other problems that the scanners won't find.  Vulnerabilities are also manually verified to ensure that false positives won't be an issue.

Once those steps are completed vulnerabilities are analyzed to better understand their potential as 'entry points'.  These are vulnerabilities that might conceivably provide an attacker a way to penetrate your environment or application.  Once these potential entry points are identified the penetration tester(s) will attempt to safely exploit these issues. 

This 'safe exploit' exercise is extremely important as it allows an organization to understand their potential vulnerability risks to a much higher degree and creates a situation that allows you to prioritize remediation far more effectively. 

However, safely exploiting vulnerabilities does come with risks and requires that your penetration testers have both significant expertise and experience.  A good penetration tester understands not only how to exploit vulnerabilities, but how to do so safely and how to translate their results into business-relevant information.


Penetration testing will provide you with more useful, more actionable, and more relevant data than either scanning or vulnerability assessments - if it's done correctly.  The risks involved with penetration testing can make this sort of testing problematic if done poorly or if done without giving the penetration testing team information that they need to do their work effectively and safely. However, if working with a strong and experienced team, a penetration testing exercise is an extremely effective way to better understand the technical vulnerabilities and the business logic issues (and their proper priority) in critical assets and environments.

So - that's your primer on Scanning vs. Assessment vs. Pentesting. 

Now I have a favor to ask… If you are just now starting to look for a penetration testing partner please, please, please ask questions.  Challenge your potential partners on how they do pentesting.  What tools do they use?  What methodologies do they follow?  What is their experience in the industry?  How do they rate vulnerabilities discovered?  How do they address false positives?  (btw - I'm going to do another post on what to look for in a security testing partner.)

'Pentesting' is a term that many people use incorrectly and, far too often, it's used incorrectly on purpose…to mislead…  Information security is a growth industry and it's pretty easy to download a free scanner, play with it a bit, and start charging people for 'penetration tests'.  This is something that we run into quite often.  It's really, really frustrating…  Real penetration takes time, expertise, a great deal of experience, a strong process, technical ability, and the right tools.  It costs actual money but can be extremely valuable if done correctly.


Thank you and if you have any comments or questions please let us know via the contact form and we'll get back to you right away.  

Alex Crittenden