AI Pentesting

Continuous vs Annual Penetration Testing: How Often Should You Test in 2026?

Amartya | CodeAnt AI Code Review Platform

Sonali Sood

Founding GTM, CodeAnt AI

Table of Content
No headings found on page
Find Vulnerabilities. Prove They're Exploitable.

Validate real-world vulnerabilities with AI-powered penetration testing.

How often should you run a penetration test? Most teams still run penetration testing once a year. But their applications don’t change once a year. They change every week, new endpoints, updated authentication flows, third-party integrations, infrastructure changes.

That mismatch creates a structural problem. Security is being tested at one cadence, while risk is being introduced at another.

The result is what we can call the deployment velocity gap, the time between when a vulnerability enters the system and when it is actually detected.

In an annual testing model, that gap can stretch for months. A system may be β€œsecure” at the moment of testing, but every change that follows creates new, untested surface area. By the time the next test arrives, the application has already evolved far beyond what was originally evaluated.

This is not a failure of pentesting itself. It’s a mismatch between how often systems change and how often they are tested.

To understand why this gap exists, and why it continues to grow in modern engineering teams, we need to look at how annual penetration testing actually works in practice, and what it does (and does not) cover.

For a broader introduction to AI-driven security testing, see our AI penetration testing guide, which covers how AI pentesting works, its methodology, and how it differs from vulnerability scanning.

What Is Continuous Penetration Testing?

Continuous penetration testing is the practice of testing an application's security repeatedly as its attack surface changes, rather than relying on a single point-in-time penetration test each year. The goal is to reduce the time between a security-relevant change entering production and that change being evaluated through penetration testing.

Unlike annual penetration testing, which evaluates the application during a defined engagement window, continuous penetration testing aligns security testing more closely with development and deployment activity. Depending on the organization's risk profile and release cadence, this can mean monthly, sprint-based, or targeted testing after significant changes.

Why Annual Pentesting Made Sense (And Where It Falls Short)

Annual penetration testing made sense when codebases and production environments changed less frequently.

A company might deploy new code on a slower cadence, with the same services and infrastructure remaining relatively stable between assessments. An annual test could therefore provide a useful snapshot of the application's security posture for a meaningful period of time.

Modern SaaS environments are different.

Engineering teams can ship code multiple times per week. New API endpoints get added. Authentication flows get updated. Third-party integrations get introduced. Infrastructure gets reconfigured. Each of these changes can alter the application's attack surface.

The problem is not that annual pentesting is bad security work. The problem is that it evaluates the application during a defined window, while the application continues to evolve after that assessment.

Annual pentesting therefore has two practical constraints:

It tests a moment in time. The engagement captures a snapshot of the application and infrastructure. The system that is tested during the engagement can change substantially after the assessment ends.

It tests a defined scope. A penetration test has a defined scope, testing window, and set of techniques. Rarely used workflows, newly introduced endpoints, configuration changes, and other attack paths may fall outside the assessment depending on the engagement.

As deployment velocity increases, both constraints become more important.

The question is therefore not whether annual penetration testing has value. It does. The question is whether the testing cadence matches the rate at which the application changes.

What Continuous Pentesting Actually Means

Continuous pentesting is the practice of running security assessments at a cadence that reflects how frequently your application and attack surface change, rather than relying exclusively on a single annual assessment.

What it does not mean: running a scanner on every commit. Fully automated testing does not automatically become penetration testing. A mature program still requires defined scope, authorization, controlled exploitation, validated findings, appropriate reporting, and human oversight.

What it does mean in practice:

  • Monthly testing: A recurring assessment evaluates the current production environment at a shorter interval than an annual engagement.

  • Sprint-cadence testing: Testing can be aligned with development cycles when the application changes frequently.

  • Targeted testing: Major features, authentication changes, new APIs, integrations, and other high-risk changes can trigger focused security assessments.

  • Integrated security lifecycle: Security testing can complement controls at the IDE, pull request, CI/CD, production, and remediation stages.

The key operational insight is that continuous pentesting is not defined by one specific frequency. The appropriate cadence depends on deployment velocity, application risk, attack surface, regulatory requirements, and the organization's ability to remediate findings.

For teams evaluating AI-driven approaches, CodeAnt AI's penetration testing platform supports black-box, white-box, and gray-box assessment models.

How Often Should a SaaS Company Run a Penetration Test?

Test at least once a year, and more often when your attack surface changes faster than that. Your compliance framework sets the minimum. How often you ship, and what a missed change would expose, sets the right cadence.

What the standards say about pentest frequency:

Framework

What it requires

Source

PCI DSS v4

Internal and external penetration tests at least once every 12 months and after any significant infrastructure or application upgrade or change (Requirements 11.4.2 and 11.4.3). If segmentation isolates the cardholder data environment, segmentation tests at least once every 12 months and after changes to segmentation controls (11.4.5).

PCI DSS Requirement 11

SOC 2

No fixed frequency. Criterion CC4.1 names penetration testing as one of the separate evaluations management uses, and the scope and frequency of separate evaluations follow risk.

AICPA 2017 Trust Services Criteria

HIPAA

The current Security Rule sets no pentest frequency. HHS's proposed update, issued December 27, 2024, would require penetration testing at least once every 12 months and vulnerability scanning at least every six months. It is not final, and the current rule remains in effect.

HHS NPRM fact sheet

How often to test, by how fast you change:

  • You ship a few times a year: one full pentest a year, plus a test after any significant change. That meets PCI DSS and most SOC 2 audits.

  • You ship every week or two: a full pentest every quarter, with attack surface monitoring every day in between. Over a quarter the attack surface usually changes significantly but not drastically, which is why quarterly is the cadence finance and insurance teams on our calls pick most often.

  • You deploy daily, or you hold card or health data: quarterly full-scope tests, plus a scoped test before each major release and whenever monitoring finds a new exposed service, login or API.

Whatever your cadence, run an extra test when you:

  • launch a new product, API or login flow

  • change authentication, roles or tenant isolation

  • move cloud provider or add an integration that touches customer data

  • acquire a company or take on new domains

  • face a large customer's security review

Exposure between tests is not hypothetical. In industry-level scans CodeAnt runs for data intelligence between August 4 and October 30, 2026, about two in three externally scanned domains had at least one high-severity finding reachable from the internet, and up to 9.3% were critically compromised.

The question is not how often your auditor needs a report. It is how long a new risk can stay untested.

Continuous Penetration Testing vs Annual Pentesting


Annual penetration testing

Continuous penetration testing

Testing frequency

Typically once per year

Monthly, sprint-based, or aligned to major releases

Average exposure window

Can extend for months between assessments

Reduced by shortening the interval between assessments

Cost model

Usually priced per engagement

Can be subscription, recurring engagement, or usage-based

Compliance coverage

Provides point-in-time assessment evidence

Can provide recurring evidence, but does not automatically replace compliance-specific testing requirements

Untested deployments/year

Potentially many, depending on release cadence

Fewer, depending on the testing cadence and scope

Which Pentest Platforms Offer Continuous Testing?

Every vendor below says continuous, but they mean different things: a platform your team runs whenever it likes, a fixed number of expert tests a year, a test on every deploy, or daily monitoring with a deep test when something changes. Compare what triggers a test, what you get at the end of it and how the bill grows when you test more often. Every competitor fact was checked on the vendor's own website.

Provider

What "continuous" means

Model

Time to report

Pricing model

Report and compliance mapping

Best for

Main tradeoff

CodeAnt AI

Attack surface monitoring every day; a full pentest each quarter, before major releases or when monitoring flags a new exposure; on-demand runs any time

AI pentest with exploit-proven findings, reviewed by CodeAnt security researchers; black, grey and white box across web apps, APIs, cloud, Kubernetes, Active Directory and networks

Within 48 hours

Priced on the attack surface you have, with very generous pricing for fast-growing companies; you pay only when a working PoC exploit is shipped

Every finding mapped to SOC 2 and ISO 27001; free, unlimited re-scans after a fix

Security leads who need audit evidence and testing that keeps pace with releases

Each pentest is launched on a schedule or on demand, not as a blocking step on every deploy

Pentera

Tests launched on demand, on a schedule or through an AI prompt whenever the environment changes

Automated security validation platform your team runs; an expert-run service (SECTOR11) is offered separately

Not stated

Not published

Executive summaries and technical reports; no compliance mapping described

Enterprises validating internal and external controls at scale

You define, launch and tune the tests yourself; no price or report deadline published

Horizon3 NodeZero

Unlimited pentests on annual per-asset plans, with recurring tests you schedule on Core and above

Autonomous pentest platform your team runs, plus a WebApp product

Not stated on its pricing page

Per asset: Flex $15 per test; Core $50, Pro $65, Elite $85 per asset a year with unlimited testing; WebApp Flex $10,000 per app per test; WebApp Continuous custom

Reports on every tier; separate PCI and NIS 2 pages

Infrastructure teams testing networks on a schedule

Continuous web app testing is quote only; no report deadline

Cobalt

A Continuous Pentesting Program and an Autonomous Pentest; the Enterprise tier is for raising test frequency

PTaaS with human testers plus an Autonomous Pentest

Autonomous findings in 24 hours; human pentests start in 1 to 3 business days by tier

Credits (each one is 8 hours of testing) in annual packages; Autonomous Pentest $3,500 per test if started and completed by Dec 31, 2026

Structured report at engagement close; unlimited on-demand retesting during the contract

Teams that want human-led tests booked through a platform

Credits buy testing time whether or not anything is exploitable, and do not roll over

Intruder

Test on every release instead of once a year

AI web app pentest plus vulnerability scanning

Same-day reports

$3,500 per test, or 4 for $12,000 used within a year; testing on every release is quoted on a call

Reports usable as ISO 27001 and SOC 2 evidence; full refund if your auditor rejects the report; unlimited retesting

Teams that want a fixed price per test and an auditor guarantee

Release-based testing has no listed price; each test is paid whatever it finds

Aikido

Runs on every deploy for the branches you pick, or on a daily, weekly or monthly schedule, after one full assessment

AI pentest inside a developer security suite

Not stated in its docs

10 credits ($10) for each agent launched on a code change; a standard pentest is 4,000 credits

Compliance-ready report and retesting with every pentest

Developer teams already using Aikido

Cost grows with commits, and its no-findings waiver does not cover continuous runs

Astra Security

Autonomous pentesting described as continuous offensive security testing, on waitlist; annual PTaaS plans plus scanners

PTaaS plus DAST and cloud scanners

Compliance reports mapped within days

Pentest $5,999 a year for one target; Pentest Plus $9,999 a year for two; scanners from $69 a month

SOC 2, ISO 27001 and PCI DSS reports; free rescans after remediation

Smaller teams that want a scanner plus a yearly expert test

The always-on part today is scanning; the autonomous pentest is waitlist only

How AI Makes Continuous Penetration Testing More Practical

AI can change the economics and operational model of recurring penetration testing by automating parts of the assessment that traditionally require substantial manual effort.

Depending on the platform, AI can assist with:

  • Large-scale attack-surface discovery

  • Application and API mapping

  • Source-code comprehension

  • Test-case generation and adaptation

  • Vulnerability investigation

  • Exploit validation

  • Correlation between findings

  • Attack-path analysis

  • Evidence collection and reporting

  • Retesting after remediation

The important distinction is that automation does not automatically make an assessment a penetration test. A mature program still needs defined scope, authorization, controlled exploitation, human oversight, validated findings, and appropriate reporting.

For teams evaluating AI-driven approaches, CodeAnt AI's penetration testing platform combines automated testing with black-box, white-box, and gray-box assessment models.

How Does CodeAnt AI Run Continuous Pentesting?

CodeAnt AI has run 12,000 pentests and is trusted by more than 1,000 companies, including Fortune 500 companies like NRG, cybersecurity leaders like Commvault, and fast-growing companies like 11x and Sanas.

CodeAnt AI splits continuous testing into a daily loop and a deep loop, so you are not paying for a full pentest every day on a perimeter that barely moved.

  • Every day: attack surface monitoring watches your domains, subdomains, IP addresses and open services, and tells you when something new or risky appears.

  • Every quarter, before major releases, or when monitoring flags a new exposure: a full pentest. AI agents map what is reachable and chain findings into attack paths for hours, then CodeAnt security researchers review the results before the report goes out.

  • After every fix: free, unlimited re-scans, so you can prove a fix without waiting for the next test.

What you get from each pentest:

  • A full report within 48 hours, with every finding mapped to SOC 2 and ISO 27001: an executive summary you can share with auditors and customers, and step-by-step reproduction for engineers.

  • Exploit proof: you pay only when CodeAnt ships a working proof-of-concept exploit. If nothing exploitable is found, you do not pay.

  • Black box, grey box and white box testing for web apps, APIs, cloud, Kubernetes, Active Directory, and internal and external networks.

Pricing: CodeAnt prices each pentest on the attack surface you have, and gives very generous pricing to fast-growing companies.

Proof: CodeAnt is SOC 2 Type II and a Gartner Cool Vendor 2026. As a VulnCheck CNA partner, its researchers have published more than 100 CVEs, including CVE-2026-29000 in pac4j-jwt (CVSS 10.0) and CVE-2026-28292 in simple-git (CVSS 9.8). Phunware called it "the most advanced and thorough penetration test we've run to date."

Where CodeAnt is not the whole answer: the pentest runs on a schedule or on demand, not as a blocking step on every deploy. For checks on every pull request, pair it with CodeAnt's code review and quality gates in CI/CD.

Run a 48-hour AI pentest

How Much Does Continuous Pentesting Cost Compared With an Annual Test?

Compare the billing unit, not the headline price, because the unit decides what happens to your bill when you test more often. These are the models vendors publish (checked November 3, 2026):

  • Per asset per year, unlimited testing: Horizon3 NodeZero Core $50, Pro $65 and Elite $85 per asset a year.

  • Per test: Intruder $3,500 per AI pentest, or $12,000 for four; Cobalt's Autonomous Pentest $3,500 per test if completed by Dec 31, 2026; NodeZero WebApp Flex $10,000 per app per test.

  • Per hour of testing: Cobalt credits, each worth 8 hours of testing, sold in annual packages.

  • Per agent run: Aikido's continuous mode, 10 credits ($10) for each agent it launches on a code change.

  • Per target per year: Astra's Pentest plan, $5,999 a year for one target.

  • CodeAnt AI prices each pentest on the attack surface you have, gives very generous pricing to fast-growing companies, and charges only when it ships a working proof-of-concept exploit.

Three things decide whether continuous testing costs more than one annual test:

  1. Retests. Ask whether re-verification after a fix is included and how often you can run it. Intruder, Cobalt (during the contract) and CodeAnt include unlimited retesting.

  2. What you pay when nothing is found. Per-test and per-credit models bill for the test whatever it finds. CodeAnt bills only when it ships a working exploit. Aikido waives your first full assessment if it finds no high or critical issue, but not continuous runs.

  3. How cost grows with releases. Per-run and per-agent models grow with every deploy. Per-asset and per-engagement pricing stays flat across the year, so it is easier to budget.

If you already pay separately for attack surface monitoring and a yearly pentest, compare a quarterly program against those two lines together, not against the pentest invoice alone.

These are illustrative figures for a company with $10M ARR, weekly releases, 15 engineers at $100/hr, 12% breach probability, $500K average breach cost.

For a deeper look at the factors that influence pentesting costs, see our penetration testing cost guide.

Recurring penetration testing also requires clear expectations around scope, response times, reporting, retesting, and finding remediation. See our guide to PTaaS provider SLAs for the operational questions to ask before choosing a recurring testing provider.

What If Our Budget Only Covers One Annual Pentest?

This is the most common reason teams stay on annual testing: the pentest budget was set from last year's invoice, and the auditor only asks for one report. Three ways to move without a new budget line:

  • Run one full test now, timed to your audit window or your quiet period, and move to a quarterly cadence at renewal.

  • Spend the single test on what changed most: new APIs, login flows, roles and tenant boundaries, rather than retesting a perimeter that barely moved.

  • Use free re-verification to prove fixes between tests, so the next test starts from a clean baseline.

Two questions buyers ask us at this point:

  • Our auditor only needs one test a year. Why do more? For most frameworks that is true; the table above shows the minimums. The annual report satisfies the auditor. The extra tests cover the releases you ship after it, and give enterprise customers a recent report when they ask.

  • If we can log in to the platform every day, why only four full tests a year? Monitoring runs every day. A full pentest runs agents for hours and adds researcher review, so it runs on a cadence, before big releases, or when monitoring finds something worth attacking. Re-verifying a fix is unlimited.

Continuous Pentesting in the CI/CD Lifecycle

Continuous penetration testing works best when it complements, rather than replaces, the security controls already operating in the development lifecycle.

A mature security workflow can layer controls across multiple stages:

  • IDE: identify security and code-quality issues while developers are writing code

  • Pull request: review changes before they merge

  • CI/CD: run automated security checks before deployment

  • Production: monitor the live attack surface

  • Penetration testing: actively attempt to exploit weaknesses in the deployed system

  • Retesting: verify that remediation actually closed the attack path

This layered model matters because penetration testing and automated security scanning solve different problems. Scanners can identify known patterns at high volume, while penetration testing is designed to investigate how vulnerabilities can be exploited in the context of a real application.

For more on integrating automated security testing into development workflows, see our guide to automated pentesting tools for DevSecOps.

The Maturity Model: Which Testing Cadence Fits Your Organization

The Security Testing Maturity Framework

Not every organization needs the same testing cadence. A practical security testing program can mature from periodic assessment toward more frequent, development-aligned validation as deployment velocity, risk, and security resources increase.

The following model is a practical way to think about that progression, rather than a formal industry standard.

Maturity Level

Description

Deployment Velocity

Testing Model

Minimum Frequency

Level 0

No structured security testing

Any

Annual minimum

Annual

Level 1

Compliance-driven testing

Monthly or less

Annual + automated scanning

Annual

Level 2

Risk-aware testing

Bi-weekly

Quarterly + sprint-aware

Quarterly

Level 3

DevSecOps-integrated testing

Weekly

Sprint-cadence + monthly deep

Per-sprint

Level 4

Continuous security program

Daily

Continuous all layers

Ongoing

Level 5

Security-native development

Continuous

Embedded, automated + weekly deep

Real-time

Decision Framework: Annual vs Continuous

Choosing the right testing model depends on several factors:

  • how fast your system changes

  • how sensitive your data is

  • what your compliance requirements demand

  • how quickly your team can respond to findings

Instead of a single answer, use this decision framework.

1. How Often Do You Deploy?

Your deployment frequency directly determines how quickly risk accumulates.

Deployment Frequency

Recommended Model

Why It Matters

What to Invest In

Less than monthly

Annual or semi-annual

Attack surface changes slowly

Strong pre-deployment security reviews

Monthly to bi-weekly

Quarterly (minimum)

New risk accumulates faster than annual coverage

Quarterly external tests + automated regression

Weekly or more

Continuous or sprint-based

Annual testing covers <10% of deployments

Security program aligned with release cadence

2. What Data Do You Handle?

Data sensitivity changes both risk tolerance and testing frequency requirements.

Data Type

Recommended Approach

Why

PII (>10K users), payment data, health data

Quarterly or continuous (minimum annual for compliance)

Breach impact + regulatory exposure is high

Business confidential, moderate PII

Annual minimum, quarterly if deploying frequently

Risk grows with deployment velocity

Internal tools, low sensitivity

Annual may be sufficient

Lower impact if compromised

πŸ‘‰ In high-risk environments, economics shift, breach cost often justifies continuous testing.

3. What Is Your Regulatory Environment?

Compliance should influence your testing strategy, but compliance and continuous security testing are not interchangeable.

Different frameworks and contractual requirements have different expectations around penetration testing, vulnerability management, risk assessment, and evidence.

An organization should therefore determine its required testing activities from the applicable standard, scope, contractual obligations, and auditor guidance.

Framework or requirement

What to consider

PCI DSS

Determine the penetration testing activities required for the applicable scope and assessment

SOC 2

Use the applicable trust services criteria and auditor expectations to determine evidence requirements

HIPAA

Consider the organization's risk analysis, safeguards, and applicable security testing practices

ISO 27001

Align testing with the organization's risk treatment, controls, and applicable assessment requirements

Continuous penetration testing can complement these requirements by providing more frequent security validation and additional evidence between formal assessments. It should not automatically be presented as a replacement for a specific compliance-mandated penetration test.

The practical goal is to satisfy the required baseline while reducing the amount of time that newly introduced attack surface remains untested.

4. Do You Have a Security Team?

Your ability to act on findings determines how continuous your model can be.

Team Setup

Recommended Model

Why

Dedicated security team (even 1 person)

Continuous testing

Can triage and respond in real time

No dedicated team (shared responsibility)

Sprint-based / monthly cadence

Prevents alert overload

No team + no plans

Quarterly testing

Continuous model will fail operationally

Final Recommendation

If you simplify everything above, the decision comes down to this:

Scenario

Recommended Model

High velocity (weekly+) + sensitive data + budget

Continuous

High velocity (weekly+) + sensitive data + limited budget

Quarterly

Moderate velocity (monthly) + sensitive data

Quarterly

Moderate velocity + low sensitivity

Semi-annual

Low velocity (monthly or less)

Annual

That said, the right testing model is not about preference. It’s about alignment. If your system changes faster than your testing cycle, risk accumulates faster than it is detected.

The Finding SLA Matrix for Continuous Programs

Continuous testing requires clear SLAs, because findings arrive continuously, the team needs defined timelines for each severity:

Severity

CVSS Range

Acknowledgment SLA

Remediation SLA

Retest SLA

Escalation

Critical

9.0–10.0

4 hours

48 hours

Within 24h of fix

C-suite notification

High

7.0–8.9

24 hours

7 days

Within 48h of fix

Security team lead

Medium

4.0–6.9

72 hours

30 days

Within sprint

Engineering manager

Low

0.1–3.9

1 week

90 days

Next quarterly

Backlog

Informational

N/A

2 weeks

Next roadmap

N/A

None

Common Failure Modes in Continuous Testing Programs

Why Continuous Programs Fail After 6 Months

Organizations that start continuous testing programs often abandon them within 6–12 months. The failure patterns are consistent:

Failure Mode 1: Finding Fatigue Without Triage




Failure Mode 2: Testing Doesn't Track Deployment Changes




Failure Mode 3: Surface Monitoring Without Action




Failure Mode 4: Compliance-Minimum Thinking




Metrics That Define a Successful Continuous Program

The KPI Stack for Continuous Security Testing

A continuous penetration testing program should measure whether security validation is actually keeping pace with application change.

Useful metrics include:

Metric

What It Measures

Why It Matters

Mean Time to Detection

Time between a security-relevant change and its detection

Shows how quickly new risk is identified

Mean Time to Remediation

Time between finding discovery and remediation

Measures how long known risk remains open

Security Validation Coverage

Percentage of relevant releases or changes evaluated

Shows whether testing tracks application change

Retest Completion Rate

Percentage of remediated findings that are retested

Confirms whether fixes were actually validated

Findings Reopened

Findings that reappear after remediation

Identifies incomplete or ineffective fixes

Attack Surface Coverage

Portion of the known attack surface included in testing

Helps identify areas that may remain untested

The objective is not to optimize every metric independently. The goal is to shorten the time between meaningful application changes, security validation, remediation, and verification.

class ContinuousSecurityProgramMetrics:
    """Track and report continuous security testing program effectiveness"""

    def calculate_program_kpis(self, program_data: dict) -> dict:

        findings_data = program_data['findings']
        test_events = program_data['test_events']
        deployments = program_data['deployments']

        # KPI 1: Mean Time to Detection (MTTD)
        # How long from vulnerability introduction to detection?
        mttd_values = []
        for finding in findings_data:
            if finding.get('introduction_date') and finding.get('detection_date'):
                days = (finding['detection_date'] - finding['introduction_date']).days
                mttd_values.append(days)

        mttd = sum(mttd_values) / len(mttd_values) if mttd_values else None

        # KPI 2: Mean Time to Remediation (MTTR)
        # How long from detection to confirmed fix?
        mttr_values = []
        for finding in findings_data:
            if finding.get('detection_date') and finding.get('remediation_date'):
                days = (finding['remediation_date'] - finding['detection_date']).days
                mttr_values.append(days)

        mttr = sum(mttr_values) / len(mttr_values) if mttr_values else None

        # KPI 3: Vulnerability Introduction Rate
        # New security findings per 100 deployments
        total_findings = len(findings_data)
        total_deployments = len(deployments)
        vuln_rate = (total_findings / total_deployments * 100) if total_deployments else 0

        # KPI 4: Escape Rate
        # Percentage of vulnerabilities NOT caught before production
        # (Found by external researchers or incident response, not internal testing)
        external_discoveries = sum(
            1 for f in findings_data
            if f.get('discovered_by') == 'external'
        )
        escape_rate = (external_discoveries / total_findings * 100) if total_findings else 0

        # KPI 5: SLA Compliance Rate
        # Percentage of findings remediated within defined SLAs
        sla_compliant = sum(
            1 for f in findings_data
            if f.get('remediated_within_sla') == True
        )
        sla_rate = (sla_compliant / total_findings * 100) if total_findings else 0

        # KPI 6: CVSS Trend
        # Is the average CVSS of findings going up or down over time?
        monthly_avg_cvss = {}
        for finding in findings_data:
            month = finding['detection_date'].strftime('%Y-%m')
            if month not in monthly_avg_cvss:
                monthly_avg_cvss[month] = []
            monthly_avg_cvss[month].append(finding['cvss'])

        cvss_trend = {
            month: sum(scores) / len(scores)
            for month, scores in monthly_avg_cvss.items()
        }

        # KPI 7: Attack Surface Growth Rate
        # How fast is the untested attack surface growing?
        surface_snapshots = program_data.get('surface_snapshots', [])
        if len(surface_snapshots) >= 2:
            first = surface_snapshots[0]
            last = surface_snapshots[-1]
            surface_growth = (
                (len(last['endpoints']) - len(first['endpoints'])) /
                len(first['endpoints']) * 100
            )
        else:
            surface_growth = None

        return {
            'mean_time_to_detection_days': round(mttd, 1) if mttd else 'N/A',
            'mean_time_to_remediation_days': round(mttr, 1) if mttr else 'N/A',
            'vulnerability_introduction_rate_per_100_deployments': round(vuln_rate, 2),
            'escape_rate_percent': round(escape_rate, 1),
            'sla_compliance_rate_percent': round(sla_rate, 1),
            'cvss_trend_by_month': cvss_trend,
            'attack_surface_growth_percent': round(surface_growth, 1) if surface_growth else 'N/A',

            'program_health': self.assess_program_health(mttd, mttr, escape_rate, sla_rate),

            'benchmarks': {
                'mttd_industry_annual': 180,  # days
                'mttd_industry_continuous': 14,
                'mttd_your_program': mttd,
                'mttr_pci_requirement_critical': 1,  # day
                'sla_compliance_target': 95,  # percent
            }
        }

    def assess_program_health(self, mttd, mttr, escape_rate, sla_rate) -> str:
        score = 0

        if mttd and mttd < 14: score += 2
        elif mttd and mttd < 30: score += 1

        if mttr and mttr < 7: score += 2
        elif mttr and mttr < 30: score += 1

        if escape_rate < 5: score += 2
        elif escape_rate < 15: score += 1

        if sla_rate > 95: score += 2
        elif sla_rate > 80: score += 1

        if score >= 7: return 'EXCELLENT'
        elif score >= 5: return 'GOOD'
        elif score >= 3: return 'IMPROVING'
        else: return 'NEEDS_ATTENTION'
class ContinuousSecurityProgramMetrics:
    """Track and report continuous security testing program effectiveness"""

    def calculate_program_kpis(self, program_data: dict) -> dict:

        findings_data = program_data['findings']
        test_events = program_data['test_events']
        deployments = program_data['deployments']

        # KPI 1: Mean Time to Detection (MTTD)
        # How long from vulnerability introduction to detection?
        mttd_values = []
        for finding in findings_data:
            if finding.get('introduction_date') and finding.get('detection_date'):
                days = (finding['detection_date'] - finding['introduction_date']).days
                mttd_values.append(days)

        mttd = sum(mttd_values) / len(mttd_values) if mttd_values else None

        # KPI 2: Mean Time to Remediation (MTTR)
        # How long from detection to confirmed fix?
        mttr_values = []
        for finding in findings_data:
            if finding.get('detection_date') and finding.get('remediation_date'):
                days = (finding['remediation_date'] - finding['detection_date']).days
                mttr_values.append(days)

        mttr = sum(mttr_values) / len(mttr_values) if mttr_values else None

        # KPI 3: Vulnerability Introduction Rate
        # New security findings per 100 deployments
        total_findings = len(findings_data)
        total_deployments = len(deployments)
        vuln_rate = (total_findings / total_deployments * 100) if total_deployments else 0

        # KPI 4: Escape Rate
        # Percentage of vulnerabilities NOT caught before production
        # (Found by external researchers or incident response, not internal testing)
        external_discoveries = sum(
            1 for f in findings_data
            if f.get('discovered_by') == 'external'
        )
        escape_rate = (external_discoveries / total_findings * 100) if total_findings else 0

        # KPI 5: SLA Compliance Rate
        # Percentage of findings remediated within defined SLAs
        sla_compliant = sum(
            1 for f in findings_data
            if f.get('remediated_within_sla') == True
        )
        sla_rate = (sla_compliant / total_findings * 100) if total_findings else 0

        # KPI 6: CVSS Trend
        # Is the average CVSS of findings going up or down over time?
        monthly_avg_cvss = {}
        for finding in findings_data:
            month = finding['detection_date'].strftime('%Y-%m')
            if month not in monthly_avg_cvss:
                monthly_avg_cvss[month] = []
            monthly_avg_cvss[month].append(finding['cvss'])

        cvss_trend = {
            month: sum(scores) / len(scores)
            for month, scores in monthly_avg_cvss.items()
        }

        # KPI 7: Attack Surface Growth Rate
        # How fast is the untested attack surface growing?
        surface_snapshots = program_data.get('surface_snapshots', [])
        if len(surface_snapshots) >= 2:
            first = surface_snapshots[0]
            last = surface_snapshots[-1]
            surface_growth = (
                (len(last['endpoints']) - len(first['endpoints'])) /
                len(first['endpoints']) * 100
            )
        else:
            surface_growth = None

        return {
            'mean_time_to_detection_days': round(mttd, 1) if mttd else 'N/A',
            'mean_time_to_remediation_days': round(mttr, 1) if mttr else 'N/A',
            'vulnerability_introduction_rate_per_100_deployments': round(vuln_rate, 2),
            'escape_rate_percent': round(escape_rate, 1),
            'sla_compliance_rate_percent': round(sla_rate, 1),
            'cvss_trend_by_month': cvss_trend,
            'attack_surface_growth_percent': round(surface_growth, 1) if surface_growth else 'N/A',

            'program_health': self.assess_program_health(mttd, mttr, escape_rate, sla_rate),

            'benchmarks': {
                'mttd_industry_annual': 180,  # days
                'mttd_industry_continuous': 14,
                'mttd_your_program': mttd,
                'mttr_pci_requirement_critical': 1,  # day
                'sla_compliance_target': 95,  # percent
            }
        }

    def assess_program_health(self, mttd, mttr, escape_rate, sla_rate) -> str:
        score = 0

        if mttd and mttd < 14: score += 2
        elif mttd and mttd < 30: score += 1

        if mttr and mttr < 7: score += 2
        elif mttr and mttr < 30: score += 1

        if escape_rate < 5: score += 2
        elif escape_rate < 15: score += 1

        if sla_rate > 95: score += 2
        elif sla_rate > 80: score += 1

        if score >= 7: return 'EXCELLENT'
        elif score >= 5: return 'GOOD'
        elif score >= 3: return 'IMPROVING'
        else: return 'NEEDS_ATTENTION'
class ContinuousSecurityProgramMetrics:
    """Track and report continuous security testing program effectiveness"""

    def calculate_program_kpis(self, program_data: dict) -> dict:

        findings_data = program_data['findings']
        test_events = program_data['test_events']
        deployments = program_data['deployments']

        # KPI 1: Mean Time to Detection (MTTD)
        # How long from vulnerability introduction to detection?
        mttd_values = []
        for finding in findings_data:
            if finding.get('introduction_date') and finding.get('detection_date'):
                days = (finding['detection_date'] - finding['introduction_date']).days
                mttd_values.append(days)

        mttd = sum(mttd_values) / len(mttd_values) if mttd_values else None

        # KPI 2: Mean Time to Remediation (MTTR)
        # How long from detection to confirmed fix?
        mttr_values = []
        for finding in findings_data:
            if finding.get('detection_date') and finding.get('remediation_date'):
                days = (finding['remediation_date'] - finding['detection_date']).days
                mttr_values.append(days)

        mttr = sum(mttr_values) / len(mttr_values) if mttr_values else None

        # KPI 3: Vulnerability Introduction Rate
        # New security findings per 100 deployments
        total_findings = len(findings_data)
        total_deployments = len(deployments)
        vuln_rate = (total_findings / total_deployments * 100) if total_deployments else 0

        # KPI 4: Escape Rate
        # Percentage of vulnerabilities NOT caught before production
        # (Found by external researchers or incident response, not internal testing)
        external_discoveries = sum(
            1 for f in findings_data
            if f.get('discovered_by') == 'external'
        )
        escape_rate = (external_discoveries / total_findings * 100) if total_findings else 0

        # KPI 5: SLA Compliance Rate
        # Percentage of findings remediated within defined SLAs
        sla_compliant = sum(
            1 for f in findings_data
            if f.get('remediated_within_sla') == True
        )
        sla_rate = (sla_compliant / total_findings * 100) if total_findings else 0

        # KPI 6: CVSS Trend
        # Is the average CVSS of findings going up or down over time?
        monthly_avg_cvss = {}
        for finding in findings_data:
            month = finding['detection_date'].strftime('%Y-%m')
            if month not in monthly_avg_cvss:
                monthly_avg_cvss[month] = []
            monthly_avg_cvss[month].append(finding['cvss'])

        cvss_trend = {
            month: sum(scores) / len(scores)
            for month, scores in monthly_avg_cvss.items()
        }

        # KPI 7: Attack Surface Growth Rate
        # How fast is the untested attack surface growing?
        surface_snapshots = program_data.get('surface_snapshots', [])
        if len(surface_snapshots) >= 2:
            first = surface_snapshots[0]
            last = surface_snapshots[-1]
            surface_growth = (
                (len(last['endpoints']) - len(first['endpoints'])) /
                len(first['endpoints']) * 100
            )
        else:
            surface_growth = None

        return {
            'mean_time_to_detection_days': round(mttd, 1) if mttd else 'N/A',
            'mean_time_to_remediation_days': round(mttr, 1) if mttr else 'N/A',
            'vulnerability_introduction_rate_per_100_deployments': round(vuln_rate, 2),
            'escape_rate_percent': round(escape_rate, 1),
            'sla_compliance_rate_percent': round(sla_rate, 1),
            'cvss_trend_by_month': cvss_trend,
            'attack_surface_growth_percent': round(surface_growth, 1) if surface_growth else 'N/A',

            'program_health': self.assess_program_health(mttd, mttr, escape_rate, sla_rate),

            'benchmarks': {
                'mttd_industry_annual': 180,  # days
                'mttd_industry_continuous': 14,
                'mttd_your_program': mttd,
                'mttr_pci_requirement_critical': 1,  # day
                'sla_compliance_target': 95,  # percent
            }
        }

    def assess_program_health(self, mttd, mttr, escape_rate, sla_rate) -> str:
        score = 0

        if mttd and mttd < 14: score += 2
        elif mttd and mttd < 30: score += 1

        if mttr and mttr < 7: score += 2
        elif mttr and mttr < 30: score += 1

        if escape_rate < 5: score += 2
        elif escape_rate < 15: score += 1

        if sla_rate > 95: score += 2
        elif sla_rate > 80: score += 1

        if score >= 7: return 'EXCELLENT'
        elif score >= 5: return 'GOOD'
        elif score >= 3: return 'IMPROVING'
        else: return 'NEEDS_ATTENTION'

The Security Posture Dashboard

Metric

Annual Model Baseline

Continuous Program Target

Why It Matters

Mean Time to Detection

~180 days

<14 days

Determines breach window

Mean Time to Remediation

~45 days (batch quarterly)

<7 days (continuous pipeline)

Reduces risk-open duration

Vulnerability Escape Rate

~25% (found by others first)

<5%

Measures program effectiveness

SLA Compliance Rate

~60%

>95%

Audit evidence quality

False Positive Rate

~40%

<10%

Engineering team trust

Attack Surface Coverage

~70% (tested version drifts)

>95% (weekly updates)

Completeness of protection

CVSS Trend (target: declining)

Uncorrelated

Measurable decline

Program impact evidence

The Gap Closes When Testing Matches Change

Annual penetration testing provides a valuable point-in-time assessment. But for SaaS companies that deploy continuously, the security state of the application can change substantially between assessments.

Continuous penetration testing addresses that gap by bringing security validation closer to the cadence at which the application changes.

The right model depends on your deployment frequency, attack surface, risk profile, compliance requirements, and ability to act on findings. For some organizations, annual testing may remain appropriate. For others, quarterly, monthly, or release-aligned testing can provide substantially more timely validation.

The important thing is to make the testing cadence a deliberate security decision rather than an artifact of the calendar.

Want to see how continuous penetration testing could fit into your security workflow?

Explore CodeAnt AI Penetration Testing β†’

Continue reading:

Frequently Asked
Questions
How often should a SaaS company run a penetration test?

At least once a year and after any significant change, which is the floor PCI DSS sets in Requirements 11.4.2 and 11.4.3. If you ship every week, run a full pentest every quarter, add a test before major releases or after changes to login, roles or tenant isolation, and monitor your attack surface every day in between.

We pass our annual SOC 2 audit. Why do we need more than one pentest a year?
Can continuous testing replace our annual compliance pentest?
How long does it take to stand up a continuous penetration testing program?
How do vendors price continuous pentesting: per app, per asset or per test?
Go Beyond Detection.
Prove What's Exploitable.