Why the wrong assessment wastes the budget
Security assessments are sold under overlapping names, and the overlap costs buyers real money. A scan gets quoted as a penetration test. A penetration test gets scoped like a red team. A board asks for assurance, procurement asks three providers for a pentest, and the three quotes describe three different pieces of work. The purchase then turns on price, which is the one dimension that says least about what you will actually receive.
The fix is unglamorous: know what each assessment type finds, what it structurally cannot find, and what evidence it leaves behind. A vulnerability scan answers whether you are missing patches. A web application penetration test answers whether your application can be talked into breaking its own rules. A red team answers whether a patient attacker reaches something that matters before anyone notices. Buying one when you need another produces a confident report about the wrong question.
This guide compares the seven assessment types we are most often asked to disentangle, then covers how to match them to your organisation's maturity, how to sequence them over a year, why a red team is usually the wrong opening move, and how to read a proposal before you sign it.
Seven assessment types, side by side
The table compares the seven types on the dimensions that matter at purchase time. Durations are qualitative and describe the market generally, because the honest number for any specific scope only comes out of a scoping conversation. Read the two middle columns together: what an assessment cannot find is as much a part of the purchase as what it can.
Australian compliance regimes lean on different rows of this table, and it pays to know which before you buy. The Essential Eight expects routine vulnerability scanning as part of its patching strategies, with the required cadence tightening at higher maturity levels. PCI DSS requires both regular vulnerability scans and periodic penetration testing for environments that handle card data. APRA CPS 234 requires regulated entities to test the effectiveness of their information security controls systematically, using appropriately skilled and functionally independent testers. ISO 27001 does not name a specific test type; the auditor checks that your risk assessment led to technical testing proportionate to the risk and that findings were treated. SMB1001 sets tiered requirements for smaller organisations, with more assurance expected as the tiers rise. If a specific clause is driving the purchase, name it in the scoping call, because different regimes accept different evidence and the deliverable has to satisfy the assessor who reads it.
| Assessment | What it finds | What it cannot find | Typical duration | What you receive | Prerequisites | When it is the right choice |
|---|---|---|---|---|---|---|
| Vulnerability scan | Known, catalogued flaws: missing patches, expired certificates, weak configurations | Business logic flaws, chained attacks, anything without a known signature | Hours to days, repeatable on a schedule | Automated finding list with severity ratings, often noisy | Asset list, authorisation to scan, network access | Baseline hygiene, recurring patch evidence, before any manual testing |
| Web application pentest | Authentication bypass, access control failures, injection, business logic abuse | Infrastructure weaknesses around the application, staff susceptibility | Days to weeks per application | Manual report: reproduction steps, impact, recommended fixes | Test accounts for each role, a test environment or agreed window | Anything handling customer data or money, before launch, after major change |
| Network or infrastructure pentest | Exploitable services, weak segmentation, credential and privilege escalation paths | Application logic flaws, cloud control plane misconfiguration | Days to weeks depending on scope | Manual report with attack paths and prioritised fixes | Defined scope and authorisation; internal access for internal testing | Office networks, servers, VPNs; after network change or acquisition |
| Cloud configuration review | Public storage, over-broad identities, missing logging, weak tenant settings | Code flaws, proof of exploitability, on-premises exposure | Days for most tenants | Configuration findings mapped to fixes, often benchmark aligned | Read-only access to the tenant or subscriptions | Anything running on Microsoft 365, Azure, AWS or Google Cloud |
| Secure code review | Injection, crypto misuse, hardcoded secrets, authorisation gaps at source | Runtime and deployment issues, misconfiguration outside the repository | Days to weeks, scales with codebase size | Line-level findings with fixes a developer can action | Source access and a contact who knows the code | Before launch, high-value code paths, inherited or vendor-delivered code |
| Red team | Whether a realistic attacker reaches an agreed objective unnoticed | Broad coverage; it proves one path was open | Weeks, run quietly over a longer window | Attack narrative, detection and response gaps, executive debrief | Mature controls, working detection, executive sign-off | After clean pentests, to test detection and response |
| Purple team | Which attacker techniques your monitoring detects and which it misses | Unknown vulnerabilities; it measures detection coverage | Days per exercise, best repeated | Technique-by-technique detection results and tuning actions | A monitoring stack and defenders in the room | Building or validating detection after a SIEM or MDR rollout |
Seven security assessment types compared at purchase time
Match the assessment to where you actually are
Assessment value tracks maturity. A scan against an environment that has never been scanned produces hundreds of findings, and that is useful exactly once: it establishes the baseline and feeds the patching program. A manual penetration test against the same environment spends senior tester hours rediscovering what the scanner would have listed anyway. The reverse mismatch is just as wasteful. An organisation that patches promptly, scans on a schedule and has already fixed two rounds of pentest findings learns very little from another scan, because everything a scan can see has already been seen.
Three questions locate you on the ladder. First, when did you last see a scan report for your own environment, and did the finding count fall between two consecutive scans? If the answer is never, or the count is not falling, stay at the bottom of the table. Second, has a human ever attacked your most important application with permission? If it takes payments, holds customer records or grants access to anything sensitive, and the answer is no, that is the next purchase. Third, if a tester ran a quiet credential attack against your VPN or your Microsoft 365 tenant tonight, would anything alert a person? Until the answer is yes, the first five rows of the table are where your money belongs.
- No recent scan results and no confident asset list: start with a vulnerability scan plus a cloud configuration review. They are the fastest routes to a defensible baseline.
- Hygiene under control and custom software in production: a web application pentest or secure code review on the system whose compromise would hurt most.
- Manual tests coming back clean and monitoring that pages a human: purple team to tune detection, then red team to test the whole organisation.
Sequencing assessments over a year
A single assessment chosen well beats several chosen badly, but organisations that take this seriously end up with a rhythm rather than an annual event, because each assessment type feeds the next. A workable first-year sequence looks like this:
The sequence bends to external dates. If an auditor, a customer contract or a certification deadline is driving the work, plan backwards from the date the evidence is due and leave room for remediation and a retest before it. A report full of open critical findings delivered the week before an audit helps nobody. And if two assessments compete for the same budget, prefer the one whose findings you have the capacity to fix this quarter, because an unread report has no security value at all.
- 1. Breadth first: scan everything internet-facing and review the cloud tenant configuration. Fix what surfaces before paying for anything manual.
- 2. Depth second: a penetration test of the application or network segment whose compromise would do the most damage.
- 3. Remediate, then retest. Proving that fixed findings stay fixed is worth more than fresh findings on a new target.
- 4. Keep the scanner running on a schedule between engagements. Scanning works as a standing control; manual testing works as a periodic event.
- 5. Once manual testing stops producing severe findings and detection exists, add a purple team exercise, and consider a red team the following cycle.
Why starting with a red team is usually wrong
A red team simulates a realistic adversary pursuing an agreed objective, quietly, over weeks, against people and process as well as technology. It answers one question with high confidence: can an attacker of this profile reach that objective without being caught? That is a valuable question with a condition attached. The answer has to be in genuine doubt.
Against an environment that has never had a penetration test, the answer is already known. The operators reach the objective early through whichever door was left open, the engagement ends with a narrative about one path, and the weaknesses either side of that path go unexamined, because stealth work does not produce coverage. You pay for the deepest assessment on the table and receive less breadth than the shallowest. Worse, the headline finding is usually something a scan or a short pentest would have caught, and the exercise tests nothing about your response, because there was no detection in place to respond.
A red team earns its place when penetration tests have stopped finding severe issues, when detection exists and a named person is on the hook to respond, and when the thing under test is the organisation's reaction as much as its perimeter. If any of those are missing, buy the assessment that builds them first. There is one deliberate exception: a board sometimes commissions an early red team to make the risk vivid before funding a program. That can be legitimate, provided everyone signs off knowing it will demonstrate a weakness that is already suspected, and the budget for what comes next already exists.
How to read a proposal
Proposals for the same engagement can look interchangeable while describing entirely different work. The differences worth finding hide in a handful of places, and checking them takes less than an hour.
Three things should make you pause: a guaranteed clean report, a finding count promised in advance, and a penetration test quoted at scan-level effort with no named humans attached. Each one signals that the deliverable is being manufactured before the work has begun.
- Method: the proposal should say what humans will do that tools do not. If every listed activity could be performed by software, you are reading a scan proposal, whatever the title says.
- People: ask who will do the work and what they have tested that resembles your stack. Certifications set a floor; a redacted sample report tells you more.
- Scope: named assets, roles and environments, with exclusions written down. A scope defined only in hours is a warning sign.
- Deliverable: reproduction steps, business impact in plain English, and a specific fix per finding. Ask whether a retest of remediated findings is included and how it is evidenced.
- Safety: written rules of engagement, an emergency contact on both sides, and clarity about testing production versus a test environment.
- Independence: if a regime such as CPS 234 applies to you, confirm the testers are functionally independent of whoever built or operates the controls being tested.
Where this sits in our work
We run every assessment type in this guide, and in our experience the scoping conversation is where most of the value gets decided. The table tells you what to ask for and what to expect back; it deliberately stops short of how each assessment is executed, because that depth is what an engagement delivers. Our approach page describes how one runs from start to finish. The short version is that findings arrive in plain English with concrete fixes, and a retest proves the holes are closed.
If you are not sure which row you need, that is a normal place to be. A short conversation about your environment, your obligations and what you can realistically fix this quarter settles it faster than a tender document.