Why Continuous Compliance Demands Structured Evidence Management
Regulatory compliance is no longer a once-a-year exercise. Frameworks such as DORA, NIS2, ISO 27001, SOC 2 Type II, and PCI DSS increasingly expect organisations to demonstrate ongoing adherence to controls — not just a snapshot taken days before an audit. That shift has a direct consequence for evidence management: the volume, velocity, and variety of compliance evidence now far exceed what manual processes can handle.
A well-governed evidence repository is the backbone of any continuous compliance programme. It is the single source of truth that auditors, compliance officers, and regulators rely on to determine whether controls are designed effectively and operating consistently over time. Without one, organisations revert to the costly, error-prone scramble that characterises ad-hoc evidence collection.
The Problem with Ad-Hoc Evidence Collection
Most compliance teams are familiar with the following scenario: an audit is announced, and a frantic evidence-gathering exercise begins. Spreadsheets are circulated, screenshots are taken, emails are forwarded, and teams are asked to “prove” that controls were operating six months ago. The results are predictable:
- Missing artifacts: Evidence that was never collected at the time of the control activity is reconstructed from memory or secondary sources, reducing its reliability.
- Inconsistent formats: Different teams produce evidence in different formats — PDFs, screenshots, CSV exports, verbal confirmations — making comparison and verification difficult.
- Incomplete coverage: Some control domains have robust evidence while others have virtually none, creating audit findings that could have been avoided.
- Questionable integrity: When evidence is assembled after the fact, auditors rightly question whether it accurately reflects what actually happened.
- Resource drain: The scramble diverts compliance and operational staff from their primary responsibilities for weeks or even months.
These problems are not merely inconvenient — they represent governance failures. An organisation that cannot produce timely, reliable evidence of its controls is, from a regulatory perspective, an organisation that cannot demonstrate compliance.
Evidence Repository Architecture: A Conceptual Overview
An effective evidence repository is not a single tool or database. It is a governed capability comprising four interconnected layers, each with distinct responsibilities and ownership.
Automated Collection Layer
The foundation of any evidence repository is the ability to capture compliance-relevant artifacts automatically, at the point where control activities occur. In a CI/CD-governed environment, this includes pipeline execution logs, approval and authorisation records, security scan results, deployment traces, and configuration change records. The key principle is that evidence should be a by-product of controlled processes, not a separate manual activity.
Organisation Layer
Raw evidence is of limited value without structure. The organisation layer ensures that every piece of evidence is classified and indexed according to:
- Control domain: Which control objective does this evidence support (e.g., access control, change management, security testing)?
- Regulation or framework: Which regulatory requirement does this evidence map to (e.g., DORA Article 9, ISO 27001 Annex A.12)?
- Time period: What audit period does this evidence cover?
- System or service: Which application, pipeline, or infrastructure component generated this evidence?
Retention Layer
Evidence must be retained for periods that satisfy all applicable regulatory requirements. The retention layer enforces policies that govern how long evidence is stored, when it is archived, and when it is securely disposed of. Retention periods must be aligned with the most stringent applicable regulation — not the least.
Access Layer
Not all evidence should be accessible to all personnel. The access layer governs who can view evidence, who can export it, and — critically — maintains an audit trail of all evidence access. This is essential for maintaining chain of custody and for demonstrating to auditors that the evidence repository itself is subject to appropriate controls.
Evidence Categories and Sources
The following table provides a reference framework for the types of evidence a well-governed repository should contain. Compliance teams should use this as a starting point and adapt it to their specific regulatory obligations and organisational context.
| Category | Specific Artifacts | Source System | Format | Retention Period |
|---|---|---|---|---|
| Access Control Evidence | User access reviews, privilege assignments, authentication logs, role change records | Identity provider, access management platform, pipeline access controls | Structured logs, review sign-off records | Per regulation (see below) |
| Change Management Evidence | Change requests, approval records, deployment logs, rollback records | Pipeline orchestrator, change management system, version control | Pipeline execution logs, approval audit trails | Per regulation (see below) |
| Security Testing Evidence | SAST/DAST scan results, dependency vulnerability reports, penetration test reports | Security scanning tools, vulnerability management platform | Scan reports (structured), remediation tracking records | Per regulation (see below) |
| Incident Evidence | Incident reports, root cause analyses, remediation actions, communication records | Incident management system, communication platforms | Incident records, timeline reconstructions | Per regulation (see below) |
| Third-Party Evidence | SBOMs, vendor risk assessments, third-party audit reports, licence compliance records | Dependency management tools, vendor management platform | SBOM files (CycloneDX/SPDX), assessment reports | Per regulation (see below) |
| Policy Evidence | Policy documents, policy acknowledgements, training completion records, exception approvals | Policy management system, learning management system | Versioned documents, sign-off records | Per regulation (see below) |
Retention Requirements by Regulation
Different regulatory frameworks impose different retention requirements. Organisations subject to multiple frameworks must align to the most stringent applicable period.
| Regulation / Framework | Retention Requirement | Key Consideration |
|---|---|---|
| DORA | 5 years minimum | Applies to ICT-related records; competent authorities may request historical evidence going back five years |
| NIS2 | Varies by EU member state transposition | Check national implementing legislation; some member states specify 3–5 years |
| ISO 27001 | Defined by the organisation’s ISMS | The organisation must define and justify its own retention periods; auditors will verify consistency |
| SOC 2 | Audit period plus retention | Evidence must cover the full audit period (typically 6–12 months) and be available for the audit engagement |
| PCI DSS | 1 year minimum | Audit logs must be retained for at least one year, with a minimum of three months immediately available |
Evidence Integrity Principles
The value of evidence depends entirely on its integrity. Auditors assess not just what evidence exists, but how it was produced, stored, and protected. The following principles should guide evidence management:
- System-generated evidence is preferred over manual evidence. A pipeline execution log is inherently more reliable than a screenshot taken by an individual. Where possible, evidence should be produced automatically by the systems that enforce controls.
- Immutable storage is essential. Evidence must be stored in a manner that prevents modification after the fact. If evidence can be altered, its value as proof of control operation is severely diminished.
- Timestamps must be authoritative. Every piece of evidence should carry a reliable timestamp from a trusted source. Manual timestamps or timestamps that can be edited are a red flag for auditors.
- Chain of custody must be maintained. The evidence repository must record who created, accessed, exported, or modified evidence, and when. This is the evidence about the evidence — and auditors will look for it.
Governance of the Evidence Repository
An evidence repository without governance is simply a document store. Effective governance requires clear answers to the following questions:
- Ownership: Who is accountable for the completeness, accuracy, and availability of the evidence repository? This should be a named role — typically within the compliance or risk management function — not a shared responsibility with no single owner.
- Review cadence: How frequently is the evidence repository reviewed for completeness? Monthly reviews are recommended; quarterly reviews are the minimum acceptable frequency.
- Completeness checks: Is there a defined process for verifying that all required evidence categories are being collected, and that no gaps exist? Completeness checks should be documented and the results retained as evidence themselves.
- Quality assurance: Is collected evidence reviewed for accuracy and relevance, or is it simply accumulated? A repository full of irrelevant artifacts is nearly as problematic as an empty one.
What Auditors Should Verify
When assessing an organisation’s evidence management capability, auditors should verify the following:
- An evidence repository exists and is actively maintained — not merely planned or partially implemented.
- Retention policies are documented, aligned with regulatory requirements, and enforced (not just defined).
- Evidence is system-generated where possible, with a clear rationale for any categories that rely on manual collection.
- Access to the evidence repository is controlled, with role-based permissions and a complete audit trail of access events.
- Evidence is organised by control domain, regulation, and time period, enabling efficient retrieval during audits.
- Completeness reviews are conducted regularly and documented.
- Evidence integrity mechanisms are in place (immutable storage, authoritative timestamps, chain of custody records).
Red Flags
The following indicators should raise immediate concerns about the maturity and reliability of an organisation’s evidence management:
- Evidence collected only before audits: If evidence collection ramps up in the weeks preceding an audit, the organisation does not have continuous compliance — it has periodic compliance theatre.
- Manual evidence creation: Screenshots, manually assembled reports, and hand-written logs suggest that controls are not integrated into automated processes.
- No retention policy: The absence of a documented, enforced retention policy is a governance failure that will result in audit findings.
- Evidence stored in editable formats: Word documents, unlocked spreadsheets, and other easily modified formats undermine evidence integrity.
- No access controls on evidence: If anyone in the organisation can view, modify, or delete evidence, the chain of custody is broken.
- Evidence repository with no audit trail: If there is no record of who accessed or exported evidence, the repository itself lacks a fundamental control.
Related Resources
For further guidance on audit governance and continuous compliance, see:
Related for Auditors
- Glossary — Plain-language definitions of technical terms
- Continuous Compliance via CI/CD
- Audit Day Playbook
- How Auditors Review CI/CD
New to CI/CD auditing? Start with our Auditor’s Guide.
