A vulnerability scan is not a penetration test.
A vulnerability scan is automated breadth, cheap and worth running continuously; a penetration test is a human adversary chaining findings toward an objective, and most businesses that hold anything worth taking eventually need both.
Plenty of what is sold in Australia as a penetration test is a vulnerability scan with a new cover page. The two are different instruments that answer different questions, and the difference decides what gets found, what you pay for, and whether the report survives an auditor or an enterprise customer's security questionnaire.
A vulnerability scan is software: it fingerprints your systems and matches them against a catalogue of known weaknesses, missing patches, outdated versions, exposed services, and common misconfigurations. A penetration test is a person: someone who attacks your systems the way a real adversary would, exploits what they find, and chains individually minor weaknesses into an attack path with a real objective at the end. One is breadth. The other is depth.
The two are complements, not competitors. Scans are cheap enough to run against everything you own, continuously, and they catch the drift that accumulates between tests. A penetration test answers the question a scan cannot: what could a determined human actually do to us? At Black Shard we use scanners inside our tests as instruments, but every finding in our reports is validated by hand. If we report it, we reproduced it.
Side by side.
| Criteria | Penetration test | Vulnerability scan |
|---|---|---|
| What it finds | Exploitable attack paths, including the classes no signature describes: business-logic flaws, broken access control, authentication weaknesses, tenancy-isolation failures, and chains of minor issues that combine into a major one. | Known, catalogued weaknesses: missing patches, outdated software versions, exposed services, weak TLS configurations, and common misconfigurations matched against a signature database. |
| False positives | Every finding is validated by hand before it reaches the report. Nothing lands on your desk that was not reproduced against your systems. | False positives are built in. Scanners infer from version banners and response patterns, so the output always needs human triage, and that triage is your problem unless someone is paid to do it. |
| Exploitation depth | Findings are exploited and chained toward an objective. A verbose error message, a predictable identifier, and a weak password-reset flow are three low-severity entries to a scanner and one account takeover to an adversary. | Detection stops at identification. A scanner flags a weakness; it does not exploit it, cannot combine it with anything else, and cannot tell you what it is worth to an attacker. |
| Reporting | Ranked findings with severity, business impact, reproduction steps a developer can follow, and a concrete fix for each, written in plain English. | A machine-generated list, typically ordered by generic CVSS score rather than by what the weakness would cost your business. Useful as a work queue, not as an assessment. |
| Compliance acceptance | What auditors and enterprise security questionnaires mean when they ask for penetration testing: a human-led test with a named methodology, a dated report, and evidence. We test against the OWASP testing guides and ASD guidance. | Satisfies continuous-monitoring and patch-verification expectations, and supports vulnerability-management controls. Rarely accepted as a substitute where a penetration test has been asked for by name. |
| Cost basis | Priced on scope. We run offensive work as a fixed-scope engagement with a defined target and timeframe, so the scope is settled before the work starts. Human time is the cost, which is also the value. | Tooling or subscription pricing, low enough per asset to run across your whole estate. The low cost is the point: it is what makes continuous coverage affordable. |
| Cadence | Point in time. Annually for most businesses, and after anything material changes: a new application, a major release, a re-platform, an acquisition. | Continuous or scheduled. Weekly or monthly at minimum, plus after every patch cycle, so that regressions and drift are caught in days rather than at next year's test. |
When you need a penetration test
- You run bespoke code: a customer-facing application, an API, a client portal. Scanners match signatures; your business logic has no signature.
- You hold data with real consequences: health records, legal matters, client funds, or personal information at scale.
- An enterprise customer or an auditor has asked for your latest penetration test report. A scan export will not close that request.
- You are about to launch, re-platform, or make a major architecture change, and you want it attacked before the public gets the chance.
- You operate multi-tenant software and no human has ever tried to cross the tenancy boundary.
- Your systems have only ever been scanned. If no person has attacked them, you do not yet know where you stand.
When a vulnerability scan is enough, for now
- Your surface is small and static: standard office IT, off-the-shelf SaaS, a brochure site you did not build and cannot change.
- You run no bespoke code and expose no application of your own to the internet, so the weaknesses that matter to you are the known, patchable kind a scanner is built to find.
- You need hygiene between annual tests: missed patches, newly exposed services, and configuration drift caught as they appear rather than at the next engagement.
- You want to verify that a patch cycle actually landed across the whole fleet, not just on the machines someone remembered.
- Budget forces a choice and nothing has ever been scanned. Scan first, fix the obvious, then bring in a human to test what remains. Testing before basic hygiene wastes tester hours on findings a scanner would have handed you cheaply.
What does a vulnerability scan actually do?
A scanner probes your hosts and applications, fingerprints what is running, and compares the results against a database of known weaknesses: unpatched software, deprecated protocols, exposed administrative interfaces, default credentials, weak cipher suites. It can sweep an entire estate overnight and repeat the sweep every week without fatigue. That breadth and repeatability are genuinely valuable. For known, patchable weaknesses, a scanner finds them faster and cheaper than any human ever will.
The limits are structural, not a matter of buying a better scanner. A scanner detects what has a signature. It cannot understand what your application is for, so it cannot see that an invoice endpoint returns other customers' invoices when the identifier is changed. It cannot exploit what it finds, so it cannot tell you which of three hundred flagged items combine into a path to your data. Broken access control is the clearest example: the flaw is in what your application permits, not in what it runs, and no signature can describe that.
What does a penetration tester do that a scanner cannot?
A tester starts with an objective, not a checklist: reach the client data, take over an account, cross the tenancy boundary. They read your application the way its developers wrote it and an attacker will read it. They exploit what they find, then use each foothold to look for the next one, chaining findings a scanner would score as trivial into paths a scanner cannot represent at all. We work to the OWASP testing guides, take only the access the work requires, and document exactly how far we got and how.
The output is different in kind, not just in depth. You get a ranked report where every finding was reproduced by hand, with the reproduction steps, the business impact, and a concrete fix for each. No raw scanner output, no severity theatre, no three hundred undifferentiated line items. And when your team has shipped the fixes, we re-test to confirm the holes are actually closed. The mechanics are laid out on our approach page; the posture behind them is on our trust page.
Why do most businesses end up needing both?
The two answer different questions on different clocks. A scan answers: is anything known-broken exposed right now? That question needs asking continuously, because estates drift, patches get missed, and new services appear without ceremony. A penetration test answers: what could a determined adversary do to us? That question needs asking at least annually, and whenever something material changes, because the answer depends on how your systems fit together, not on any single weakness.
The mature pattern is a human-led test on an annual or per-change cadence, with continuous scanning in between. The scan keeps the known weaknesses swept up so the next test spends its hours on the flaws only a human can find. What does not work is buying one and believing you bought the other. A scan relabelled as a penetration test leaves the expensive question unanswered while charging as though it were asked.
Questions, answered
- Is a vulnerability scan the same as a penetration test?
- No. A scan is software matching your systems against a catalogue of known weaknesses. A penetration test is a person exploiting and chaining weaknesses toward an objective, including whole classes a signature cannot describe: business logic, access control, authentication flows. The confusion persists because both produce a findings report. The resemblance ends there.
- How can I tell whether a quoted penetration test is really just a scan?
- Ask three questions. Who is doing the testing, and for how many days? Does every finding come with reproduction steps a developer can follow? Is a re-test of the fixes included? A rebadged scan has no human hours to account for, exports generic vulnerability descriptions instead of reproduction steps, and the provider disappears once the PDF is delivered. Judge the category by those markers, not by the label on the proposal.
- How much does a penetration test cost compared with a scan?
- A scan costs a fraction of a test, which is exactly why it should run continuously. A penetration test is priced on what the engagement has to cover: how much attack surface, how many environments, how many tenancies. Both the scope and the price are fixed before testing begins. Anyone quoting a price before understanding your scope is pricing a product, not an engagement.
- How often should we run each?
- Scan continuously, or monthly at minimum, plus after every patch cycle. Test annually, or after any material change: a new application, a major release, a re-platform. Between tests, the scan holds the line; the test is what tells you whether the line was drawn in the right place.
- Does Black Shard use scanners during a penetration test?
- Yes, as instruments for coverage, the same way a surveyor uses a level. But no scanner output goes into the report. Every finding is reproduced against your systems, written up with reproduction steps and a concrete fix, and ranked by what it would cost you. On fixed-scope offensive work, a re-test of your fixes is included.
A scoping conversation settles which one you need.
Australia-wide, from our Brisbane head office. Someone will contact you as soon as possible.