Continuous penetration testing: close the gap between compliance and real security

Summary

Annual penetration tests and noisy scanners no longer cut it. To address these challenges, organisations are increasingly turning to continuous penetration testing. Horizon3.ai’s analysis of over 50,000 production pentests shows attackers are chaining exposures at machine speed, credential-based attacks are surging, and organisations that test only once per year expose themselves for months at a time.

Australian organisations should adopt continuous, offensive validation focused on credential exposure, cloud IAM, and rapid remediation.

Key evidence from the report: NodeZero reached critical impacts in an average of 14 hours, and the median customer testing interval is 6 days for those who adopt continuous models.

The problem: compliance checklists are not the same as readiness

Traditional approaches rely on periodic scans and point-in-time pen tests. The Horizon3.ai dataset shows these approaches create predictable windows for attackers to exploit.

The report found that only 26% of organisations conduct pentests more than once a year, and 84% experienced an incident in the past year. Attackers exploit this predictability quickly.

Vulnerability scanners produce volume, not always context. Nearly 98% of organisations use scanning, but only 34% find it highly effective; 36% said they are overloaded with false positives. That noise delays remediation and lets real risks slip through.

What the data actually shows: exploitable weaknesses at scale

From over 50,000 NodeZero pentests in 2024, Horizon3 recorded:

• 415,315 compromised hosts and 174,499 sensitive data exposures.
• 28,866 credential dumping instances discovered, with 95% remediated on retest.
• NodeZero exploited 229 CVEs 99,924 times, 170 of which were on CISA’s KEV list.

Time to impact matters. The fastest path to domain admin in the NodeZero dataset was 60 seconds, and the average time to reach a critical impact was 14 hours.

Manual pentests commonly take days or weeks and will miss the continuous drift in real-world environments.

Why continuous penetration testing is a higher-return approach

Continuous penetration testing and validation aligns detection, remediation and verification to real operational tempo.

Benefits observed:

• Faster detection of attack paths that scanners miss, particularly credential-based lateral movement and chained exploits.
• Measurable MTTR improvement: NodeZero customers routinely re-test and verify fixes, achieving 95% remediation confirmation for credential dumping.
• Better prioritisation via proven exploit paths and business-impact scoring, reflecting CISA KEV and NIST SP 800-115 guidance.

Australian context: mapping to Essential Eight and regulatory drivers

For Australian organisations, continuous pentesting is not just best practice — it helps demonstrate control effectiveness under ASD/ACSC frameworks such as the Essential Eight maturity model.

ACSC guidance emphasises verifying control implementation and effectiveness. Continuous offensive testing provides tangible evidence of this.

Regulatory pressure is also increasing under frameworks like NIS2 and SOCI. Continuous testing reduces exposure windows that could lead to breaches, downtime, and compliance penalties.

Practical playbook: how to shift from annual checks to continual readiness

1. Scope for risk, not volume
   Focus on crown-jewel assets: identity stores (AD/Azure AD), cloud roles, and external attack surfaces.
   Horizon3 data highlights that Microsoft Entra/Azure AD compromises are dominant cloud risk vectors.
2. Adopt frequent, automated ops plus periodic manual red-teams
   Run autonomous ops daily or weekly, complemented by quarterly manual red-team engagements.
   NodeZero customers’ median testing interval was 6 days.
3. Measure MTTR for critical weaknesses
   Set measurable targets — for example, verify remediation of critical impacts within 7 days.
   Follow the report’s remediation verification model.
4. Prioritise KEV / actively exploited CVEs
   Integrate CISA KEV data into vulnerability management workflows. Many exploited CVEs in the report were KEV-listed.
5. Focus on credential hygiene and IAM
   Credential dumping and privilege escalation drive most breaches. Continuous testing emulates dump-to-DA paths for realistic exposure data.
6. Governance and reporting
   Provide executives and boards with evidence: exploit paths, remediation proof, and business impact summaries.

Case highlights from the Horizon3 report

• National materials supplier case: Uncredentialed internal pentest found domain compromise, 2 million sensitive resources exposed, and access to 23% of user accounts. Subsequent testing validated significant remediation progress.
• Credential dumping detection: NodeZero identified 28,866 instances and verified 95% remediations on retest — clear evidence of continuous improvement.

Is autonomous pentesting safe for production systems?

Yes. When operated by an experienced provider with clear engagement rules, continuous pentesting is safe and minimally disruptive.

Horizon3’s report confirms NodeZero is used in production with real-time exploit notification and rollback control, ensuring rapid remediation without operational risk.

Recommendations

• Move from annual to monthly validation cycles, prioritising cloud and identity assets.
• Use CISA KEV data to prioritise patching of known-exploited vulnerabilities.
• Track and reduce MTTR for critical findings, verifying fixes via retesting.

Next Steps

If you’re responsible for security assurance, request a short readiness assessment from CyberPulse.
We’ll map your crown jewels, measure current MTTR, and propose a continuous penetration testing cadence aligned with Essential Eight and KEV priorities.

Useful links

Download: Horizon3 Insights Report

Related Services: CyberPulse Penetration Testing Service