Audit Trail Generation and Management

Audit Trail Generation and Management

IaC environments generate rich audit trails through version control, deployment logs, and configuration history. However, these trails must be properly collected, preserved, and made accessible for compliance audits. Automated audit trail generation ensures completeness while reducing manual effort.

Git repositories provide natural audit trails for infrastructure changes. Every commit records who changed what, when, and why (through commit messages). Pull request discussions capture approval processes and security reviews. However, Git history alone doesn't satisfy all compliance requirements - additional context about deployments, validations, and runtime effects must be captured.

# Audit Trail Collection Pipeline
apiVersion: v1
kind: ConfigMap
metadata:
  name: audit-trail-config
  namespace: compliance
data:
  fluent-bit.conf: |
    [SERVICE]
        Flush         5
        Log_Level     info
        Daemon        off

    # Collect Kubernetes audit logs
    [INPUT]
        Name              tail
        Path              /var/log/kubernetes/audit.log
        Parser            json
        Tag               k8s.audit
        Refresh_Interval  5

    # Collect Git webhook events
    [INPUT]
        Name              http
        Host              0.0.0.0
        Port              9880
        Tag               git.events

    # Collect IaC deployment logs
    [INPUT]
        Name              tail
        Path              /var/log/iac-deployments/*.log
        Parser            json
        Tag               iac.deployment
        
    # Enrich with compliance context
    [FILTER]
        Name              lua
        Match             *
        script            compliance_enrichment.lua
        call              add_compliance_context

    # Output to compliant storage
    [OUTPUT]
        Name              s3
        Match             *
        bucket            compliance-audit-trails
        region            us-east-1
        use_put_object    On
        total_file_size   100M
        compression       gzip
        s3_key_format     /%{compliance_framework}/%{year}/%{month}/%{day}/%{hour}_%{hostname}_%{uuid}.gz

    # Also send to SIEM
    [OUTPUT]
        Name              splunk
        Match             *
        Host              splunk.internal.com
        Port              8088
        TLS               On
        TLS.Verify        On
        Splunk_Token      ${SPLUNK_HEC_TOKEN}
        Splunk_Source     iac_compliance
        Splunk_Sourcetype _json

  compliance_enrichment.lua: |
    function add_compliance_context(tag, timestamp, record)
        -- Add compliance framework context
        if string.match(record["path"], "pci%-") then
            record["compliance_framework"] = "PCI-DSS"
        elseif string.match(record["path"], "hipaa%-") then
            record["compliance_framework"] = "HIPAA"
        else
            record["compliance_framework"] = "SOC2"
        end
        
        -- Add data classification
        if record["resource_type"] == "aws_s3_bucket" then
            if string.match(record["resource_name"], "pii") then
                record["data_classification"] = "PII"
            elseif string.match(record["resource_name"], "public") then
                record["data_classification"] = "PUBLIC"
            else
                record["data_classification"] = "INTERNAL"
            end
        end
        
        -- Add retention requirements
        record["retention_days"] = get_retention_requirement(record["compliance_framework"])
        
        return 2, timestamp, record
    end
    
    function get_retention_requirement(framework)
        local retention = {
            ["PCI-DSS"] = 365,    -- 1 year
            ["HIPAA"] = 2190,     -- 6 years
            ["SOC2"] = 1095,      -- 3 years
            ["GDPR"] = 1095       -- 3 years
        }
        return retention[framework] or 2555  -- Default 7 years
    end

Evidence collection automation reduces audit preparation time from weeks to hours. Scripts automatically gather IaC configurations, scan results, deployment logs, and runtime assessments. This evidence is organized according to control requirements and packaged for auditor review.