Continuous penetration testing
Continuous penetration testing, without the marketing
A vendor-neutral reference for anyone deciding whether to buy a testing subscription: what the label guarantees on its own, what a credible program has to include, and when one scoped test a year is still enough.
- Primary sources on every claim
- EU and PCI obligations
- Includes when not to buy
- Production changes Every month, if you ship weekly or monthly.
- One test a year One month with dated test evidence. Eleven without.
- A continuous program Dated evidence every month, plus retest of what you fixed.
This is coverage of evidence, not coverage of risk. No testing program, continuous or otherwise, finds everything.
Definition
What “penetration testing as a service” actually means
PTaaS is a packaging term. Two providers can sell the same word and deliver very different work, so it is worth separating the parts that are defined somewhere from the parts that are not.
What a continuous program is
- A scoped, authorized assessment repeated on a rhythm. NIST defines penetration testing as a method in which assessors, “typically working under specific constraints”, attempt to circumvent the security features of a system. Repeating it does not change that definition.
- Manual work by named, competent people. CREST’s specification requires the provider to hold documented policies, procedures and methodologies, and every individual involved to have appropriate skills, experience and competency.
- Testing tied to change as well as to the calendar. OWASP puts a verification step after every approved change reaches production, inside the change management process.
- Retest of what you fixed. DORA requires validation that identified gaps are “fully addressed”; PCI DSS v4.0.1 requires that “penetration testing is repeated to verify the corrections”.
- Evidence that outlives the engagement: the type, scope, time and results of each test, with the criticality of every finding and the action taken against it.
What it is not
- Not a scanner subscription. NIST records high false positive and high false negative rates for signature-based scanners, and notes that they cover surface vulnerabilities only.
- Not a bug bounty. A bounty pays per valid finding to researchers who choose to look; a test is a contracted engagement with agreed rules of engagement and a report you commissioned.
- Not automation. NIST is explicit that “continuous” means a frequency sufficient to support risk decisions, and that data collection “no matter how frequent, is performed at discrete intervals”.
- Not a transfer of responsibility. NCSC: vulnerability risk assessment and mitigation “is a business process and should not be wholly outsourced to the test team”.
- Not a coverage guarantee. A penetration test identifies “a small representative sample of all possible security risks in a system”, whether you buy one a year or one a month.
The decay problem
Why one test a year stopped describing anything
Nothing about penetration testing broke. What broke is the assumption underneath the annual cadence: that the system under test is roughly the same system twelve months later.
It’s not uncommon for a year or more to elapse between penetration tests. So, vulnerabilities could exist for long periods of time without you knowing about them if this is your only means of validating security.
National Cyber Security Centre (UK), Penetration testing
-
48,244
CVE records published in 2025
Up from 18,375 in 2020: a 2.6× increase in five years. The 2026 second quarter alone (20,709) exceeded all of 2021.
-
1,709
Vulnerabilities known to be exploited
CISA’s catalog at version 2026.09.11, of which 225 entries were added since January 2026 and 360 are flagged for ransomware use.
-
41%
Teams deploying at least weekly
Elite (19%) plus high (22%) performers in a 2024 survey of nearly 3,000 practitioners. Each share carries an 89% uncertainty interval.
Deployment frequency, as measured
- Elite On demand: multiple deploys per day 19%
- High Between once per day and once per week 22%
- Medium Between once per week and once per month 35%
- Low Between once per month and once every six months 25%
Performance levels and shares from the 2024 Accelerate State of DevOps Report, page 13. The levels are derived by cluster analysis of the survey responses rather than set in advance. This research is published by the DevOps Research and Assessment program at Google Cloud and is unrelated to the EU Digital Operational Resilience Act discussed further down this page.
Boundaries
Four things that get sold as each other
All four are legitimate. Each answers a different question. The expensive mistake is buying one and believing you bought another.
-
Vulnerability scanning
- Answers
- Which known, published weaknesses are present on the assets I pointed the scanner at?
- Who does it
- A tool on a schedule, with someone competent reading the output.
- What you get
- A dated finding list, refreshed as often as you run it.
- What it does not do
- NIST records high false positive and high false negative rates for signature-based scanners and notes they cover surface vulnerabilities only. A scanner will not chain two low findings into one critical path.
-
Point-in-time penetration test
- Answers
- Could a skilled attacker break this specific scope, on this specific date?
- Who does it
- Contracted testers working under agreed rules of engagement.
- What you get
- A scoped report with a severity rating per finding and a method of resolving each one.
- What it does not do
- In NCSC’s words, it “can only validate that your organisation’s IT systems are not vulnerable to known issues on the day of the test”.
-
Continuous program
- Answers
- Is the system still standing up to attack after everything we changed since the last cycle?
- Who does it
- The same contracted testers on a rhythm, with tooling underneath them rather than instead of them.
- What you get
- Dated evidence at planned intervals, a finding register with states, and verification that the fixes held.
- What it does not do
- It does not make coverage complete and it is not automation. More frequent testing is not the same thing as deeper testing.
-
Bug bounty
- Answers
- Will anyone outside my organization find something we have not?
- Who does it
- Researchers who choose to look. CISA notes a program “may be offered to the general public or may only be offered to select researchers or those who meet certain criteria”.
- What you get
- Reports that arrive when someone finds something, paid per valid and impactful finding.
- What it does not do
- You do not decide what gets looked at, or when. OWASP places the difference between a bounty and a commissioned test in “authorization, rules of engagement, and how results are reported and remediated”.
These are complements, not substitutes. Most mature programs run scanning underneath, commissioned testing on a rhythm, and a disclosure route for anything the outside world finds. The failure is a contract that promises one and delivers another.
Obligations
What the rules actually say about frequency
Almost none of them name an interval. Read together they ask for the same three things: test according to risk, test again after significant change, and prove that findings were closed.
| Instrument | What the text requires | Interval named? | After change? |
|---|---|---|---|
| NIS2, Art. 21(2) | Measures for “security in network and information systems acquisition, development and maintenance, including vulnerability handling and disclosure” and “policies and procedures to assess the effectiveness of cybersecurity risk-management measures”. | No | Not in the Directive |
| Reg. (EU) 2024/2690, Annex 6.5 | The entity establishes “the need, scope, frequency and type of security tests” from its own risk assessment, tests to a documented methodology, and documents “the type, scope, time and results of the tests”. | No: you set it | Yes, recital 46 |
| DORA, Art. 24 | A “sound and comprehensive digital operational resilience testing programme”, tests by independent parties, and validation that identified gaps are “fully addressed”. | At least yearly on systems supporting critical or important functions | Risk-based |
| DORA, Art. 26 | Threat-led penetration testing on live production systems, for financial entities identified by their competent authority. | At least every 3 years | Authority may adjust |
| ISO/IEC 27001:2022 | Annex A control 8.8, Management of technical vulnerabilities, and control 8.29, Security testing in development and acceptance. | No | Not in these controls |
| PCI DSS v4.0.1, 11.4.2 and 11.4.3 | Internal and external penetration testing per a defined methodology, by a tester with organizational independence. | At least once every 12 months | Yes: “after any significant infrastructure or application upgrade or change” |
| PCI DSS v4.0.1, 11.4.4 | Exploitable findings corrected in line with your own risk assessment, and “penetration testing is repeated to verify the corrections”. | Follows the finding | Follows the fix |
Finding lifecycle
A finding is not closed until someone attacked it again
The difference between a testing subscription and a testing program is what happens after the finding is written down. Four states, and only the last one is green.
- Found
Found
A tester or a tool identifies an issue inside the agreed scope. Nothing is proven yet. Given the false positive rates NIST records for signature-based tooling, an unvalidated finding is a candidate, not a fact.
Evidence produced A dated entry naming the asset, the method and who found it.
- Triaged
Triaged
The finding is validated, rated and assigned. NCSC expects a severity rating for every issue found, together with an assessment of the risk it exposes the organization to. That rating is yours, not the scanner’s.
Evidence produced A severity rating, a named owner and a target date, set against your own risk criteria.
- Fixed
Fixed
Someone changes the code, the configuration or the control. At this point the finding is closed in your tracker and still open in reality: a fix is a claim until it has been tested.
Evidence produced A change record that links the remediation back to the finding.
- Retested
Retested
The tester attacks the same path again and confirms it no longer works. DORA requires validation methodologies that “ascertain that all identified weaknesses, deficiencies or gaps are fully addressed”. PCI DSS requires the penetration test to be repeated to verify the corrections.
Evidence produced A dated retest record: the only artifact that shows the risk actually went away.
A contract that measures discovery but never verification will report a growing number forever. Before signing, ask what the retest turnaround is, who pays for it, and whether it sits inside the cycle or is billed as new work.
Cadence designer
Design a testing rhythm you can defend
Four questions. The output is a rhythm split into continuous, periodic and annual work, the evidence each layer produces for an auditor, and the cases where a single dated test is still the right answer. It will tell you when you do not need a subscription.
Recommendation
A continuous program is the right shape
You change production faster than a yearly report can describe it. Split the work into three layers so each one answers a different question, and make sure the contract covers retest of closed findings, not only discovery of new ones.
A continuous program is the right shape. 3 of 3 layers apply for roughly monthly change across 2 asset types.
Twelve-month rhythm
- Continuous Always on, reported monthly
- Periodic manual testing Every two months, and after every significant change
- Annual deep-dive Once a year, full scope, independently reported
Cells are months. The pattern is a shape, not a schedule: your own risk assessment sets the real frequency, and every framework on this page expects you to be able to explain how you set it.
The three layers
-
Continuous
Always on, reported monthly
Covers
- External surface discovery: new hosts, subdomains, forgotten staging instances
- Authenticated and unauthenticated scanning of the published estate
- Retest of every finding that was reported as fixed
- Configuration drift against your own hardening baseline
- Public exposure of storage, databases and management planes
Evidence
Dated scan and discovery records at planned intervals, plus a finding register that shows each item moving from found to triaged to fixed to retested.
Implementing Regulation (EU) 2024/2690, Annex 6.10.2(b)
-
Periodic manual testing
Every two months, and after every significant change
Covers
- Manual testing of the features that changed since the last cycle
- Access control and authorization logic across roles and tenants
- Session handling, password reset and account recovery flows
- Identity and permission review for new roles and service principals
- Network path testing between environments after infrastructure changes
Evidence
A record per cycle of the type, scope, time and results of the test, the criticality of each finding, and the mitigating action taken.
Implementing Regulation (EU) 2024/2690, Annex 6.5.2(c)
-
Annual deep-dive
Once a year, full scope, independently reported
Covers
- Full manual application test against the current OWASP testing methodology
- Business logic abuse across complete user journeys
- Chained findings: how low-severity issues combine into a real attack path
- Privilege escalation paths from a compromised workload to the control plane
- Tenant and account isolation under a realistic attacker model
Evidence
An independent report with a severity rating per finding, a remediation plan, and a validation record showing the plan was completed.
Regulation (EU) 2022/2554 Art. 24(4)–(6); NCSC, Penetration testing
What your drivers actually require
- NIS2
- NIS2 names no test interval. Article 21(2) requires measures for “security in network and information systems acquisition, development and maintenance” and “policies and procedures to assess the effectiveness of cybersecurity risk-management measures”. Where the Commission implementing regulation applies, the entity itself must establish the need, scope, frequency and type of security tests from its own risk assessment, and document the type, scope, time and results of each test.Directive (EU) 2022/2555 Art. 21(2)(e)–(f); Implementing Regulation (EU) 2024/2690 Annex 6.5
Where a point-in-time test is still the right answer
- A customer, insurer or procurement team asks for a dated report from a named independent party for a defined scope.
- A system is about to go live for the first time and there is no history to test against.
This produces a starting shape for a conversation. It is not an assessment of your environment and not a compliance determination. Answers stay in the page, are not saved to your browser and are never sent anywhere.
Interactive mode is not available. You can read the full reference content below. No answers are assessed and no result is calculated.
Interactive mode is unavailable, so the full reference is set out below: the three layers of a testing program and the evidence each produces, what each obligation actually requires, and the cases where a single dated test remains the right answer.
-
L1 · Continuous
Always on, reported monthly. Covers: External surface discovery: new hosts, subdomains, forgotten staging instances; Authenticated and unauthenticated scanning of the published estate; Retest of every finding that was reported as fixed; Configuration drift against your own hardening baseline; Public exposure of storage, databases and management planes.
Evidence: Dated scan and discovery records at planned intervals, plus a finding register that shows each item moving from found to triaged to fixed to retested. (Implementing Regulation (EU) 2024/2690, Annex 6.10.2(b))
-
L2 · Periodic manual testing
Every two months, and after every significant change. Covers: Manual testing of the features that changed since the last cycle; Access control and authorization logic across roles and tenants; Session handling, password reset and account recovery flows; Identity and permission review for new roles and service principals; Network path testing between environments after infrastructure changes.
Evidence: A record per cycle of the type, scope, time and results of the test, the criticality of each finding, and the mitigating action taken. (Implementing Regulation (EU) 2024/2690, Annex 6.5.2(c))
-
L3 · Annual deep-dive
Once a year, full scope, independently reported. Covers: Full manual application test against the current OWASP testing methodology; Business logic abuse across complete user journeys; Chained findings: how low-severity issues combine into a real attack path; Privilege escalation paths from a compromised workload to the control plane; Tenant and account isolation under a realistic attacker model.
Evidence: An independent report with a severity rating per finding, a remediation plan, and a validation record showing the plan was completed. (Regulation (EU) 2022/2554 Art. 24(4)–(6); NCSC, Penetration testing)
-
D1 · NIS2
Essential or important entity under Directive (EU) 2022/2555.
NIS2 names no test interval. Article 21(2) requires measures for “security in network and information systems acquisition, development and maintenance” and “policies and procedures to assess the effectiveness of cybersecurity risk-management measures”. Where the Commission implementing regulation applies, the entity itself must establish the need, scope, frequency and type of security tests from its own risk assessment, and document the type, scope, time and results of each test.
Directive (EU) 2022/2555 Art. 21(2)(e)–(f); Implementing Regulation (EU) 2024/2690 Annex 6.5
-
D2 · DORA
Financial entity under Regulation (EU) 2022/2554.
DORA requires a testing program, not a test. Article 24 requires a “sound and comprehensive digital operational resilience testing programme”, tests by independent parties, validation that findings are fully addressed, and, at least yearly, appropriate tests on all ICT systems supporting critical or important functions. Article 25 lists the test types the program must provide for; identified entities also carry out threat-led penetration testing at least every three years.
Regulation (EU) 2022/2554 Art. 24–26
-
D3 · ISO/IEC 27001
Certified, or preparing for certification, against ISO/IEC 27001:2022.
Annex A control 8.8 is “Management of technical vulnerabilities” and 8.29 is “Security testing in development and acceptance”. Neither names an interval. An auditor looks for a defined process, dated records that it ran, and evidence that findings were closed, so a documented cadence with dated results is worth more to a certification audit than one undated report.
ISO/IEC 27001:2022 Annex A controls 8.8 and 8.29 (control titles)
-
D4 · PCI DSS
You store, process or transmit cardholder data.
PCI DSS v4.0.1 requires internal and external penetration testing at least once every 12 months and, as a separate condition, after any significant infrastructure or application upgrade or change. Requirement 11.4.4 then requires the penetration test to be repeated to verify the corrections, and 11.4.1 requires results and remediation records to be retained for at least 12 months. Internal vulnerability scans run at least every three months and again after any significant change.
PCI DSS v4.0.1 requirements 11.4.1–11.4.4 and 11.3.1, 11.3.1.3
-
D5 · Customer assurance
Prospects and clients ask for a test report during procurement.
Procurement reviews ask for a dated, scoped report from a named independent party. A continuous program satisfies this only if it produces a periodic signed report as well as a live finding register, so keep at least one reportable cycle a year even if the rest of the work is rolling.
NCSC, Penetration testing: report contents and scoping output
-
PIT · Where a point-in-time test is still the right answer
A customer, insurer or procurement team asks for a dated report from a named independent party for a defined scope. A system is about to go live for the first time and there is no history to test against.
This produces a starting shape for a conversation. It is not an assessment of your environment and not a compliance determination. Answers stay in the page, are not saved to your browser and are never sent anywhere.
Honest limits
What continuous testing does not fix
A subscription changes how often the evidence is refreshed. It does not change the nature of the work, and there are three things it will not do for you.
The terms “continuous” and “ongoing” in this context mean that security controls and organizational risks are assessed and analyzed at a frequency sufficient to support risk-based security decisions to adequately protect organization information. Data collection, no matter how frequent, is performed at discrete intervals.
NIST SP 800-137, Information Security Continuous Monitoring
-
It does not make coverage complete
OWASP, quoting Gary McGraw: “In practice, a penetration test can only identify a small representative sample of all possible security risks in a system.” Twelve samples a year is still sampling.
-
It does not replace the work around it
OWASP’s position is a balanced program: threat modeling early, secure code review, automated scanning at scale, and penetration testing to prove exploitability. Buying one of those five layers twelve times does not produce the other four.
-
It does not move the risk decision
NCSC is blunt about this: vulnerability risk assessment and mitigation “is a business process and should not be wholly outsourced to the test team”. A tester tells you what is broken. Deciding what to do about it stays with you.
Guides
Four guides, in the order you need them
Vendor-neutral and sourced from the instruments and standards bodies themselves, not from vendor marketing.
- What continuous testing is The activity has a precise definition. The delivery model does not. Here is what is actually defined, what changes when testing repeats, and what the PTaaS label guarantees on its own.
- Testing vs scanning vs bounty Three controls, three different questions, three different contracts. What each one is genuinely good at, what NIST and CISA record about their limits, and how to stop buying one while believing you bought another.
- Testing after change Four frameworks require testing after significant change, and none of them defines “significant” for you. How to build the trigger list, decide what each trigger tests, and record the decision when you decide not to test.
- Evidence for auditors Testing that leaves no record is an expense, not a control. The nine artifacts a program should produce, which framework asks for each one, and why the retest record is the one most programs are missing.
Questions
Frequently asked questions
Short answers with the source named. Longer treatments are in the guides.
What is continuous penetration testing?
Is PTaaS the same as continuous penetration testing?
How is continuous testing different from vulnerability scanning?
How is it different from a bug bounty?
Does NIS2 require annual penetration testing?
How often does DORA require testing?
Does PCI DSS require testing after every change?
Do we still need one large annual test if we test continuously?
When should we not buy continuous penetration testing?
What should a continuous testing contract include?
Sources
- Penetration testing Definition, the day-of-the-test limitation, report contents and the ownership of risk decisions.
- SP 800-115: Technical Guide to Information Security Testing and Assessment Penetration testing phases, scanner limitations, and the original statement that an annual cadence “may be sufficient”.
- SP 800-137: Information Security Continuous Monitoring What “continuous” means, and the statement that data collection is performed at discrete intervals.
- Web Security Testing Guide: Introduction and testing framework The balanced approach, periodic health checks, and change verification after deployment.
- Directive (EU) 2022/2555 (NIS2) Article 21(2), points (e) and (f).
- Commission Implementing Regulation (EU) 2024/2690 Recital 46 and Annex points 6.5 and 6.10 on security testing and vulnerability handling.
- Regulation (EU) 2022/2554 (DORA) Articles 24 to 26 on digital operational resilience testing.
- CREST Defensible Penetration Test The three elements that make an engagement commercially defensible.
- BOD 20-01: Develop and Publish a Vulnerability Disclosure Policy How a bug bounty differs from a disclosure policy, and what triage timelines should cover.
- PCI DSS v4.0.1 document library Requirements 11.3.1, 11.4.1 to 11.4.5 and 6.4.2, as reproduced in the Report on Compliance template.
- CVE Program metrics Published CVE records by year.
- Known Exploited Vulnerabilities Catalog Catalog version 2026.09.11.
- 2024 Accelerate State of DevOps Report Deployment frequency performance levels, page 13.