A Penetration Test Is Not a Vulnerability Scan

A vulnerability scanner (Nessus, Qualys, OpenVAS, the scanning built into Microsoft Defender for Cloud) fingerprints hosts and services, then matches what it finds against a database of known issues: unpatched software versions, missing security headers, default credentials, weak TLS configurations, exposed management interfaces. It is fast, cheap, repeatable, and it should run continuously, not once a year. For the patch-applications and patch-operating-systems strategies in the ACSC Essential Eight, an authenticated scan is the honest way to measure your maturity, because it reports what is actually missing across the fleet rather than what you believe is deployed.

What a scanner cannot do is reason. It reports that a host is missing a patch. It does not know whether that host is reachable from the internet, whether the vulnerable service is even running, or whether exploiting it leads anywhere. It produces false positives (a backported fix the vendor applied without bumping the version string) and, more dangerously, false comfort: a medium-rated finding in isolation may be the first link in a chain that ends in full compromise. The scanner rates the link. It never sees the chain.

A penetration tester starts from an objective and works the way an attacker does: enumerate, find a foothold, escalate, pivot, reach the goal. The value is not in finding one vulnerability. It is in understanding how ordinary conditions combine into an extraordinary outcome. A tester reads your application's behaviour, not just its version banners: how it authenticates, how it authorises each individual request, what it trusts, and where it assumes a step happened that an attacker can simply skip.

Three classes of finding live almost entirely outside a scanner's reach. Access control flaws: can user A read user B's invoice by changing an ID in the URL (an insecure direct object reference), and does the server actually check, or does it trust the frontend to hide the button? Authentication logic: can you reset a password without proving you own the account, or reach an admin route because the only check runs in JavaScript? And chained exploitation, where individually minor issues compose into a serious breach. A scanner sees none of these, because none of them is a signature. They are consequences of how your specific system was built.

Consider a hypothetical that mirrors what these engagements routinely surface. A scanner looks at a web application and reports three findings, all rated low or medium: a verbose error page that leaks a stack trace, an account-enumeration quirk where the login form distinguishes wrong password from no such user, and a file-upload feature that does not restrict file types tightly. Read in isolation, they get deferred by a busy team. Read as a chain, they are a path to compromise.

The stack trace names an internal library and a directory path. The enumeration flaw lets a tester confirm valid usernames without noise. The upload accepts a file that the server will later execute, because of how the path exposed by the first finding is configured. None of the three is critical on its own. Together they are remote code execution. A scanner cannot assemble that story, because assembling it means holding three unrelated facts in mind and reasoning about how this particular system connects them. That reasoning is the product you are buying when you buy a penetration test.

The findings that most often cause real financial loss are not memory-safety bugs or missing patches. They are business-logic flaws: places where every request is technically valid, but the sequence or the values break an assumption the business depends on. A checkout that recalculates the total on the client and trusts it. A refund endpoint that never checks whether the order was already refunded. A promo-code field that stacks. A multi-step workflow (apply, approve, disburse) where step three can be called directly without steps one and two ever running.

No scanner has a signature for 'this discount can be applied twice', because that is not a defect in any library. It is a defect in your rules, and only your system has those rules. Finding it takes a human who understands what the application is for and then deliberately tries to make it do something it was never meant to do. This is also why secure code review pairs so well with testing on custom software: the reviewer reads the intent in the source, the tester attacks the running behaviour, and between them the logic flaws surface that a black-box scan would sail straight past.

Match the instrument to the question. If you need to know what is unpatched and misconfigured across many hosts, and you need to know it regularly, buy scanning and run it on a schedule. It is the right tool for measuring hygiene and for feeding your patch-maturity evidence. If you need to know whether a specific application, network, or cloud tenancy can actually be broken into by someone who is trying, buy a penetration test, and scope it by objective and target rather than by raw host count.

Settle a few questions before either engagement. What is the crown-jewel outcome you are trying to prevent (data theft, funds movement, tenant escape)? Is the target internet-facing or internal? Do you need point-in-time assurance for a customer or auditor, or ongoing hygiene? For a web application, insist the test is authenticated and covers access control and business logic, not just an unauthenticated surface sweep. And be wary of any penetration test priced and delivered like a scan: a report that is nothing but scanner output under a new cover page is a scan, whatever the invoice says.

You need both, and they are not substitutes for one another. Run scanning constantly as a hygiene baseline and as evidence for Essential Eight and similar frameworks. Commission human testing when something material is at stake: a new product going live, a customer or regulator asking for assurance, a cloud migration, or a system that moves money or holds personal information you are obliged to secure under APP 11 of the Privacy Act.

Ask the provider to state plainly which one you are getting, and what the tester will actually attempt beyond running a tool. The answer to that single question tells you whether you are buying an inventory or an answer.