Recently I came across an article which nicely put forward a point I’ve been arguing for a while, often without much success: . Unlike Babar, I’m not amazed that there’s a confusion between vulnerability scanning and pen testing. The aim of both is to find vulnerabilities, and if the report is being read by someone who is not a security specialist and knows what they’re looking for then a vulnerability scan looks awfully similar.
The difference is vital. A good penetration test will often start with a vulnerability scan, a bad one stop there (aside from replacing the logo on the output). I’ve seen plenty of reports which I could have spun out of Nessus or Qualys myself from budget pen tests. This isn’t always the fault of a pen test company (although a good one should always advise customers of the requirements for effective engagement) – if you’re given only a couple of days engagement for a black box test (one where you have minimal information about the target) then you aren’t going to get much more than a vulnerability scan out of it.
The problem comes when those short engagements, with pen testers not provided proper information, are counted as pen tests for security. They aren’t – they will not discover anything an automated scan doesn’t, and so a company is simply wasting money on security theatre. Either don’t bother pretending that a pen test has occurred, or engage a company fully and effectively to assess a system over a longer period of time, with all the information and documentation available internally. This is one of those cases where a budget solution is worse than none at all.
This isn’t intended to diminish vulnerability scanning – it’s a vital activity that everyone should carry out on a regular basis (in my perfect world near-realtime, in reality, often monthly) to assess and deal with vulnerabilities. In a perfect world, these results can simply be handed over to a penetration testing company to save you a little money – pen testers are not the enemy, they are there to help you find holes, giving them additional information means that time isn’t wasted looking for non-existent holes and everyone gets better intelligence out at the end.