When it comes to security testing, there are two very common tests that you may want to consider. The first is vulnerability testing and the second is penetration testing. This blog post aims to explain what each kind of test is, and then define the difference between penetration testing and vulnerability scanning.
Vulnerability Testing
Vulnerability testing can be grouped into vulnerability scanning and vulnerability assessment. Vulnerability scanning is simply the act of using automated tools to identify where vulnerabilities are in any given system. Vulnerability assessment builds on a vulnerability scan which will quantify and prioritise the vulnerabilities identified in a vulnerability scan. This information will be delivered as a report that highlights exactly where the vulnerabilities are and what rating they have been assigned.
Penetration Testing
A penetration test is an authorised attack against a given system. Whilst it will often begin with multiple vulnerability scans, a penetration test aims to exploit the system in order to demonstrate the impact of any weaknesses. By exploiting a weakness, a penetration test may allow a tester to gain further insight into a system and therefore identify a vulnerability that would not have been discovered by vulnerability testing.
The report for a penetration test delivered by LRQA consultants will show exactly where the problem is and make recommendations as to the next steps for remediation. Additionally, resources that your technical teams can refer to for additional advice will be provided. Anything that is unclear can be discussed during a technical debrief between your technical team and the LRQA consultant who performed the penetration testing.
The difference between vulnerability scanning and penetration testing
The biggest difference between vulnerability scanning and penetration testing comes down to the depth of testing. This is easier to describe using an example.
Imagine a website with a login page. On the login page, there is a password field and the developer has not included an “autocomplete=off” statement inside the password field tag. This is a reasonably common occurrence and will be picked up by a vulnerability assessment (low severity) as PCI compliance requires this to be present.
During a penetration test, the tester will identify the missing autocomplete statement and then look at the login system as a whole. The missing statement will still be identified but what comes next is what distinguishes the difference between vulnerability assessment and penetration testing. The consultant will check the login page for account lockouts, check the logic behind any forgot password function, and test to see if it is possible to confirm the presence of a registered user by the errors provided by the site.
To demonstrate the difference between vulnerability assessment vs penetration testing let’s say for this example there is no account lockout or timeout feature on this login page (medium severity); the password reset uses the same codes in a reset email to reset any password, which would allow any user to reset any other users password (critical severity), and the site gives different login errors for a known username vs an unknown username (medium severity).
In this case, vulnerability assessment and penetration testing have revealed different volumes of problems. In the vulnerability assessment, you get to discover that there is a low-severity problem with the login page, which is only a problem for you if you need to be PCI compliant and is normally easy to fix. In the penetration test, the same low severity is discovered, but the additional depth of testing also reveals an additional two medium severity problems and a critical flaw within the login system.
If we change the scenario slightly so that the login page does contain an “autocomplete=off” inside the password field, then the vulnerability testing will completely miss any problem with this login page and you will receive a clean bill of health, as it were, for this part of the test. However, the reality may be somewhat different. When it comes to vulnerability testing vs penetration testing, the clear difference is that vulnerability testing would not identify the additional problems that would be revealed during a penetration test. Vulnerability testing exists as a way to efficiently gain an understanding of vulnerabilities easily identified in an organisation’s infrastructure, focusing very much on breadth rather than depth.
Deciding Between Vulnerability Assessment and Penetration Testing
When deciding whether to perform either penetration testing or vulnerability scanning it comes down to the needs of your organisation and the level of assurance you may need from the test.
If you require a high level of assurance that a given system is secure then you need a penetration test. With a penetration test you will get the full depth that you are expecting, along with recommendations on what you need to do to make changes and fixes to improve security; as well as time with the consultant after the test to discuss any problems that were identified.
If you require a low level of assurance that a system is secure or wish to get a broad overview of the security posture of your infrastructure, you may wish to opt for vulnerability testing instead. The depth of testing will be less and there will be no recommendations for further steps, any security remediation required would be decided by your in-house security experts.