What a penetration test attestation letter has to say, and what stays in the report

Start with scope, stated as named assets. A usable report lists the hosts, domains, applications, API base paths, mobile builds and cloud tenants in scope, and the environment they sat in, whether production, staging or a production-like clone. It gives the dates the window opened and closed and names the build, release tag or commit tested, because a report against an application that ships weekly describes a version that may not exist by the time anyone reads it. It records the source addresses the testers worked from, which lets your own team match test traffic against what their monitoring did or did not raise.

Each finding needs a stable identifier that still makes sense in a ticket six months later, the affected asset, a severity with the method that produced it, reproduction steps a developer can follow without asking a question, the supporting evidence, and remediation advice specific enough to act on. Remediation advice that stops at a generic instruction to validate input marks a weak report; the useful version names the parameter, the layer where the control belongs and what a correct fix looks like in that codebase. Weak reports also carry scanner output with its default descriptions intact. The tell is that the affected asset is a hostname rather than a feature, and the remediation text would read identically in a report written about any other company.

The section most reports omit is what was not tested, and it decides whether the document can be relied on later. Coverage gaps are ordinary and disclosing them costs the testing firm nothing: hosts the client excluded, functionality unreachable because the test account lacked a role, flows blocked by rate limiting or a filtering layer left enabled at the client's request, denial of service testing deliberately not run, an environment unavailable for a day of the window. A report listing no gaps at all came either from an unusually clean engagement or from a firm that does not track coverage, and you cannot tell which by reading it.

Severity usually arrives as a CVSS score computed against the vulnerability in the abstract. A base score answers how bad the flaw would be under a worst-plausible deployment. It does not know that the affected service sits on an internal subnet reachable only through a jump host with phishing-resistant multi-factor authentication, holds no personal information and is being decommissioned in six weeks. CVSS carries an environmental metric group in both the 3.1 and 4.0 versions for exactly that adjustment, and it is the part of the standard most remediation plans never touch.

Applied honestly, the ordering inverts more often than people expect. A medium-rated broken access control on an internet-facing endpoint, one that lets any authenticated customer read another customer's records, outranks a critical-rated remote code execution on a management interface that is not routable from outside and needs a credential no attacker holds. Three inputs move a rating in practice: whether an attacker can reach the flaw, how sensitive the data or privilege behind it is, and whether working exploit code is circulating. The Exploit Prediction Scoring System and the catalogue of known exploited vulnerabilities that CISA publishes both speak to the third input, and a base score never looks at it.

Australian control expectations run on the same logic. The Essential Eight maturity model puts its shortest patching window, 48 hours, on services exposed to the internet once a working exploit exists, and allows two weeks for the same assets where none does. That is a reachability and exploitability judgement expressed as a compliance deadline. A remediation queue ordered by base score alone spends the fortnight after a test on work that barely moves your exposure while an authorisation flaw on a public endpoint waits its turn. Re-rank the list against your own environment before anyone commits to dates, and record the reasoning beside each change, because that reasoning is what an assessor asks for when a medium has sat open for a quarter.

A finding is closed when somebody other than the person who wrote the fix has confirmed that the original attack path no longer works and that no new one opened in its place. That is a retest, and it needs the same access, accounts and environment as the original engagement, plus the identifier of the build carrying the fix. A retest run from a written description of the change, or from a developer's screenshot of the patched code, will not stand up when a reviewer asks how closure was verified.

Three end states need to stay separate in the finding register: fixed and verified, mitigated by a compensating control, and accepted. A compensating control has to be named with its failure mode understood, because a filtering rule in front of a vulnerable endpoint stops helping the moment traffic reaches that endpoint by another route. An accepted risk needs a named person with the authority to accept it, a stated reason, an expiry date and a trigger that forces review. An acceptance missing any of those shows as resolved in the register while the finding is still live in the system.

The retest window is the constraint that should drive the release plan. Work backwards from the date it closes and batch fixes into one release where you can, because a retest fired at a half-fixed environment produces a document recording highs still open, and that is the version a customer eventually asks to see. Ask for the letter to be issued or reissued after the retest so it describes the state you are in rather than the state when testing stopped, and check the engagement terms treat that reissue as included rather than as a separate piece of work.

The letter is short, and every line has to be checkable by someone who has never spoken to you. It sits on the testing firm's letterhead, carries the signature and contact details of a named person there, and states the legal entity tested, what was tested named as assets, the window dates, the methodology followed, the access level the testers held, the findings by severity, and the current remediation state with the date closure was verified. With less than that, the reader is being asked to take the testing firm's word for the parts that matter most.

The access level line carries the most weight and is the one weak letters leave out. A letter stating that unauthenticated testing was performed against the web application tells a reader that credentials were never issued, so the authorisation surface behind login, where most serious business logic flaws live, went unexercised. A letter saying only that penetration testing was performed leaves that question open, and an experienced reviewer will ask it anyway. Name the roles credentials were issued for, say whether testing was authenticated, unauthenticated or both, and say whether the testers had source code.

Two further items separate a strong letter from a weak one. The first is a sentence naming what the test did not cover, because a letter with no stated boundary implies coverage the engagement never had, and that is the sentence read closely after an incident. The second is the version or period the letter is scoped to, so a reader can judge staleness without guessing. A letter covering a build from March, offered in November by an organisation that deploys weekly, describes software that is no longer running, and a reviewer who notices will ask for a current one.

Parts of a report amount to a working attack path into systems you are still fixing, and they should never travel beyond the people doing the fixing. Redaction decisions are not a judgement about how much technical detail a reader could cope with; the test is whether the material shortens the distance between opening the document and reproducing the finding.

There is an Australian legal dimension worth planning for rather than discovering later. A report is a dated record of what your organisation knew about a weakness and when it knew it. Australian Privacy Principle 11 requires reasonable steps to protect personal information from misuse, interference, loss and unauthorised access, so where a breach follows a flaw that sat unremediated in a report you held, the report becomes evidence about the steps you took. It cuts both ways, because it also shows you were testing and tracking. Some organisations commission testing through external counsel with the aim of bringing the report within legal professional privilege, which is a decision to take with a lawyer before the engagement starts rather than after an incident.

Distribution control is the other half of the job. Decide before the report arrives who receives it, in what form, and whether copies are watermarked per recipient so a leaked file can be traced. Keep it out of general-access storage, including the project wiki page where someone attaches every document from an engagement, and put a non-disclosure agreement in place before any copy reaches a third party even in redacted form. Ask how long the testing firm retains their own copy and where it is held, because their retention is now part of your exposure. Whoever ends up on the distribution list, five categories of material come out of the file before a copy leaves the remediation team.

The same logic runs in reverse when your own questionnaire asks a supplier for their penetration test. Ask for the letter plus a current remediation status. The full report tells you how to attack a company whose systems you have no standing to touch, and holding it is a liability you gain nothing from. The letter establishes whether an independent party tested the product you actually buy, at what depth, and whether what they found was fixed.

Read the letter for four things. Does the scope name the product you use rather than a different one from the same vendor. Is the window inside the last twelve months and after the release you depend on. Does it state an access level. Does the remediation line carry a verification date rather than an intention. If it passes those and you still need more, ask for the executive summary under a non-disclosure agreement, a supervised read in a data room with download disabled, or a call where their engineer answers your questions with the report open in front of them.

Work out what the framework behind the request needs before demanding paper. An ISO/IEC 27001 auditor wants evidence that technical testing happens on a defined cadence and that findings feed a tracked remediation process, so the letter plus your finding register plus the retest evidence answers better than the report would. A government buyer may be asking about assessment against the Information Security Manual, which is assessed under the Australian Signals Directorate's Infosec Registered Assessors Program rather than by a commercial penetration test. Entities regulated by APRA work to CPS 234, which requires them to test the effectiveness of their information security controls through a systematic testing programme, so the cadence and the closure evidence carry more weight than any single document.

Nearly every problem here is cheaper to fix in the statement of work than after the report lands. Name the report and the letter as separate deliverables and say who signs each. Ask whether you get a draft for factual review before the report is finalised, because the fastest route to a wrong severity is a tester making a reasonable assumption about your architecture that nobody corrected. Ask whether findings can be delivered as structured data alongside the PDF, so they import into your tracker with an identifier, an owner and a due date instead of being retyped by hand.

The assurance questions come next. Ask whether retesting is included, what the window is, and whether the letter is reissued once retesting is done. Ask to see a redacted sample attestation letter, since a firm that writes strong ones will send one immediately. Ask what severity method is used and whether findings will be rated with your environment considered. Ask whether the report will record what was not tested, and whether the testers will speak to a customer's reviewer directly when a question comes back that only they can answer.

Black Shard runs offensive work alongside the defensive and advisory side of the practice, and we build the two deliverables for their two audiences from the start: a report an engineering team can act on line by line, and a letter a customer's security reviewer can rely on without ever seeing the report. Inside vCISO engagements the same discipline gets applied to reports written by other firms, because the finding register, the retest evidence and the acceptance record are what an assessor or a buyer will test. Our trust page works on the same principle, publishing our own certifications with the link a reader can verify them through.