Documentation

Bundle Format Specification

Complete technical specification of evidence bundle format, structure, and verification.

Version: 1.0

Status: Stable Effective: 2026-01-01 Previous versions: None (initial release)


Design Goals

  1. Inspectable - Industry-standard formats (tar, gzip, JSON)
  2. Tamper-evident - SHA-256 checksums + Ed25519 signatures
  3. Self-contained - All evidence and metadata in one file
  4. Auditor-friendly - No proprietary tools required
  5. Verifiable - Independent verification without evidence SDK

Bundle Structure

File Hierarchy

evidence-bundle-20260109-123456.tar.gz
└── (tar.gz extraction)
    ├── manifest.json          # Bundle metadata and table of contents
    ├── run.json               # Collection execution context
    ├── checksums.sha256       # SHA-256 checksums of all files
    ├── signature.sig          # Ed25519 signature of checksums file
    ├── sources/               # Raw evidence artifacts
    │   ├── github/
    │   │   ├── org_settings.json
    │   │   ├── repo_acme_backend_branch_protection.json
    │   │   └── repo_acme_backend_codeowners.json
    │   ├── aws/
    │   │   ├── iam_password_policy.json
    │   │   ├── cloudtrail_trail_production_status.json
    │   │   └── cloudwatch_log_group_aws_lambda_api.json
    │   └── google_workspace/
    │       ├── 2sv_enforcement.json
    │       ├── admin_roles.json
    │       └── user_lifecycle.json
    └── derived/               # SDK-generated analysis
        ├── normalized.json    # Normalized control mappings
        └── hints.json         # Compliance recommendations

File Specifications

1. manifest.json

Purpose: Bundle metadata, artifact inventory, signer information

Schema:

{
  "bundle_version": "1.0",
  "created_at": "2026-01-09T12:34:56Z",
  "framework": "soc2_type1",
  "controls": ["CC6.1", "CC6.6", "CC7.2"],
  "sources": ["github", "aws", "google_workspace"],
  "artifacts": [
    {
      "filename": "sources/github/org_settings.json",
      "source": "github",
      "controls": ["CC6.1"],
      "collected_at": "2026-01-09T12:35:00Z",
      "checksum": "abc123def456..."
    }
  ],
  "signer": {
    "algorithm": "Ed25519",
    "public_key": "def456789abc...",
    "key_id": "evidence-sdk-v1"
  },
  "sdk_version": "0.1.0"
}

Field descriptions:

FieldTypeRequiredDescription
bundle_versionstringYesBundle format version (currently 1.0)
created_atISO 8601YesBundle creation timestamp (UTC)
frameworkstringYesCompliance framework (e.g., soc2_type1)
controlsarrayYesSOC 2 controls collected for
sourcesarrayYesList of connectors used
artifactsarrayYesComplete inventory of evidence files
artifacts[].filenamestringYesRelative path within bundle
artifacts[].sourcestringYesSource connector (github, aws, etc.)
artifacts[].controlsarrayYesControls this artifact satisfies
artifacts[].collected_atISO 8601YesWhen artifact was collected
artifacts[].checksumstringYesSHA-256 checksum (hex)
signerobjectYesSignature metadata
signer.algorithmstringYesSignature algorithm (Ed25519)
signer.public_keystringYesPublic key (hex encoded)
signer.key_idstringYesKey identifier for rotation tracking
sdk_versionstringYesevidence SDK version that created bundle

2. run.json

Purpose: Collection execution context and environment metadata

Schema:

{
  "collection_started_at": "2026-01-09T12:34:56Z",
  "collection_completed_at": "2026-01-09T12:35:12Z",
  "duration_seconds": 16,
  "sdk_version": "0.1.0",
  "config_file": "evidence.yaml",
  "host": {
    "hostname": "github-actions-runner-1",
    "platform": "linux",
    "arch": "x64"
  },
  "environment": {
    "NODE_VERSION": "20.10.0",
    "RUNNER_OS": "ubuntu-22.04",
    "CI": "true"
  },
  "artifacts_collected": 9,
  "total_size_bytes": 35678,
  "errors": [],
  "warnings": [
    {
      "source": "github",
      "message": "Branch protection not configured for repository: acme/sandbox",
      "severity": "low"
    }
  ]
}

Field descriptions:

FieldTypeRequiredDescription
collection_started_atISO 8601YesCollection start time (UTC)
collection_completed_atISO 8601YesCollection end time (UTC)
duration_secondsnumberYesTotal collection duration
sdk_versionstringYesevidence SDK version
config_filestringNoConfig file path
hostobjectYesHost system information
host.hostnamestringYesHostname of collector
host.platformstringYesOS platform (linux, darwin, win32)
host.archstringYesCPU architecture (x64, arm64)
environmentobjectNoEnvironment variables (redacted)
artifacts_collectednumberYesTotal artifacts in bundle
total_size_bytesnumberYesTotal bundle size (uncompressed)
errorsarrayNoCollection errors (if any)
warningsarrayNoCollection warnings (if any)

3. checksums.sha256

Purpose: SHA-256 checksums of all files in bundle

Format: BSD-style checksum format (compatible with sha256sum -c)

Example:

abc123def456...  manifest.json
def456ghi789...  run.json
ghi789jkl012...  sources/github/org_settings.json
jkl012mno345...  sources/github/repo_acme_backend_branch_protection.json
mno345pqr678...  sources/github/repo_acme_backend_codeowners.json
pqr678stu901...  sources/aws/iam_password_policy.json
stu901vwx234...  sources/aws/cloudtrail_trail_production_status.json
vwx234yz0567...  sources/google_workspace/2sv_enforcement.json
yz0567abc890...  derived/normalized.json
abc890def123...  derived/hints.json

Rules:

  • One line per file
  • Format: {checksum} {filename} (two spaces separator)
  • Checksums are lowercase hexadecimal (64 characters for SHA-256)
  • Filenames are relative paths within bundle
  • File itself (checksums.sha256) is NOT included in checksums
  • signature.sig is NOT included in checksums

Verification:

# Standard Unix verification
sha256sum -c checksums.sha256

# macOS verification
shasum -a 256 -c checksums.sha256

4. signature.sig

Purpose: Ed25519 signature of checksums file

Format: Binary Ed25519 signature (64 bytes)

Generation process:

  1. Compute SHA-256 of checksums.sha256 file
  2. Sign SHA-256 hash with private Ed25519 key
  3. Write 64-byte signature to signature.sig

Verification process:

  1. Read signature.sig (64 bytes)
  2. Read checksums.sha256 content
  3. Extract public key from manifest.json (signer.public_key)
  4. Verify signature using Ed25519 algorithm

Example (hex representation):

0a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b0c1d2e3f4a5b6c7d8e9f0a1b
2c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b0c1d2e3f4a5b6c7d8e9f0a1b2c3d

Verification (using OpenSSL and evidence CLI):

# Using evidence CLI (recommended)
evidence verify bundle.tar.gz

# Manual verification (advanced)
# Requires Ed25519 library (libsodium, noble-ed25519, etc.)

5. sources/ Directory

Purpose: Raw evidence artifacts collected from sources

Structure:

sources/
├── github/
│   ├── org_settings.json
│   ├── repo_{owner}_{repo}_branch_protection.json
│   └── repo_{owner}_{repo}_codeowners.json
├── aws/
│   ├── iam_password_policy.json
│   ├── cloudtrail_trail_{name}_status.json
│   ├── cloudtrail_trail_{name}_config.json
│   └── cloudwatch_log_group_{name}.json
└── google_workspace/
    ├── 2sv_enforcement.json
    ├── admin_roles.json
    ├── user_lifecycle.json
    └── domain_security.json

Naming conventions:

  • GitHub: repo_{owner}_{repo}_{artifact}.json
  • AWS: {service}_{resource}_{name}.json
  • Google Workspace: {feature}.json

File format:

  • All files are valid JSON
  • Pretty-printed with 2-space indentation
  • UTF-8 encoding
  • Line endings: LF (\n)

Content:

  • Raw API responses (minimal processing)
  • Sensitive data redacted (secrets, tokens, passwords)
  • Timestamps in ISO 8601 format
  • No proprietary encoding

6. derived/ Directory

Purpose: SDK-generated analysis and recommendations

normalized.json

Purpose: Normalize evidence artifacts to control mappings

Schema:

{
  "controls": {
    "CC6.1": {
      "satisfied": true,
      "evidence": [
        {
          "artifact": "sources/github/org_settings.json",
          "field": "two_factor_requirement_enabled",
          "value": true,
          "requirement": "2FA must be enforced",
          "status": "pass"
        },
        {
          "artifact": "sources/aws/iam_password_policy.json",
          "field": "MinimumPasswordLength",
          "value": 14,
          "requirement": "Password length >= 12",
          "status": "pass"
        }
      ]
    },
    "CC6.6": {
      "satisfied": true,
      "evidence": [
        {
          "artifact": "sources/github/repo_acme_backend_codeowners.json",
          "field": "content",
          "value": "present",
          "requirement": "CODEOWNERS file exists",
          "status": "pass"
        }
      ]
    },
    "CC7.2": {
      "satisfied": true,
      "evidence": [
        {
          "artifact": "sources/github/repo_acme_backend_branch_protection.json",
          "field": "required_pull_request_reviews.required_approving_review_count",
          "value": 2,
          "requirement": "Required reviewers >= 1",
          "status": "pass"
        },
        {
          "artifact": "sources/aws/cloudtrail_trail_production_status.json",
          "field": "IsLogging",
          "value": true,
          "requirement": "CloudTrail must be logging",
          "status": "pass"
        }
      ]
    }
  },
  "summary": {
    "controls_satisfied": 3,
    "controls_total": 3,
    "evidence_count": 6,
    "overall_status": "pass"
  }
}

hints.json

Purpose: Compliance recommendations and remediation guidance

Schema:

{
  "recommendations": [
    {
      "severity": "low",
      "control": "CC7.2",
      "title": "Branch protection not configured for acme/sandbox",
      "description": "Repository 'acme/sandbox' does not have branch protection enabled for the main branch.",
      "remediation": {
        "steps": [
          "Go to Repository → Settings → Branches",
          "Add branch protection rule for 'main'",
          "Enable 'Require a pull request before merging'",
          "Set required approvers to 2"
        ],
        "documentation": "https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches"
      }
    }
  ],
  "summary": {
    "total_recommendations": 1,
    "severity_breakdown": {
      "critical": 0,
      "high": 0,
      "medium": 0,
      "low": 1
    }
  }
}

Compression & Archiving

Tar Format

Type: POSIX tar (ustar format) Tool: GNU tar or compatible

Creation:

tar -czf evidence-bundle-20260109-123456.tar.gz \
  -C /tmp/bundle \
  .

Extraction:

tar -xzf evidence-bundle-20260109-123456.tar.gz -C /output/dir

Gzip Compression

Algorithm: DEFLATE (RFC 1951) Default level: 6 (balanced) Configurable: 0-9 (via bundle.compression_level)

Compression ratios:

  • JSON artifacts: 70-90% compression
  • Total bundle: ~60-80% compression

Example:

  • Uncompressed: 50 KB
  • Compressed (level 6): 10-20 KB
  • Compressed (level 9): 8-15 KB

Integrity Verification

Checksum Verification

Algorithm: SHA-256 (256-bit hash)

Process:

  1. Extract bundle (tar -xzf)
  2. Compute SHA-256 of each file
  3. Compare with checksums in checksums.sha256
  4. All checksums must match

Command:

cd extracted-bundle/
sha256sum -c checksums.sha256

Expected output:

manifest.json: OK
run.json: OK
sources/github/org_settings.json: OK
...

Failure indicates:

  • File tampered with
  • File corrupted
  • Incomplete extraction

Signature Verification

Algorithm: Ed25519 (EdDSA with Curve25519)

Key sizes:

  • Private key: 32 bytes (256 bits)
  • Public key: 32 bytes (256 bits)
  • Signature: 64 bytes (512 bits)

Process:

  1. Read signature.sig (64 bytes)
  2. Read checksums.sha256 file content
  3. Extract public key from manifest.json
  4. Verify Ed25519 signature

Why Ed25519:

  • Fast verification (<1ms)
  • Small signature size (64 bytes)
  • Strong security (128-bit security level)
  • Deterministic signing (same input → same signature)
  • Industry standard (RFC 8032)

Verification tools:

  • evidence CLI: evidence verify bundle.tar.gz
  • OpenSSL: Complex, not recommended
  • libsodium: crypto_sign_verify_detached()
  • noble-ed25519 (JavaScript): ed.verify()

Bundle Size Limits

Organization SizeTypical Bundle SizeMax Recommended
Small (1-10 repos)5-50 KB10 MB
Medium (10-50 repos)50-500 KB25 MB
Large (50-200 repos)500 KB - 5 MB50 MB
Enterprise (200+ repos)5-50 MB100 MB

Size Contributors

Fixed overhead:

  • manifest.json: ~2-10 KB
  • run.json: ~1-2 KB
  • checksums.sha256: ~1-5 KB
  • signature.sig: 64 bytes
  • derived/: ~5-20 KB

Variable (per artifact):

  • GitHub org settings: ~2-5 KB
  • GitHub branch protection: ~1-3 KB per repo
  • AWS IAM policy: ~2-5 KB
  • CloudTrail config: ~1-3 KB per trail
  • CloudWatch log metadata: ~1 KB per log group
  • Google Workspace 2SV: ~1-2 KB
  • Google Workspace admin roles: ~5-20 KB

Large size causes:

  • Many repositories (100+)
  • Extensive CloudWatch log groups
  • Verbose manifest mode
  • Large admin role lists (Google Workspace)

Mitigation:

  • Filter repositories (not wildcard *)
  • Limit log groups to relevant services
  • Disable verbose manifest
  • Increase max_size_mb if needed

Version History

Version 1.0 (Current)

Release: 2026-01-01 Status: Stable

Format:

  • Tar + Gzip archive
  • SHA-256 checksums
  • Ed25519 signatures
  • JSON artifacts

Supported sources:

  • GitHub (token auth)
  • AWS (env + assume_role)
  • Google Workspace (service account)

Supported controls:

  • CC6.1, CC6.6, CC7.2

Breaking changes from previous versions: None (initial release)


Future Versions

Version 1.1 (Planned)

Target: Q2 2026

New features:

  • Additional source connectors (Okta, Azure AD)
  • More SOC 2 controls (CC6.x, CC7.x, CC8.x)
  • Policy evaluation results in bundle

Backward compatibility: Full (can read v1.0 bundles)


Version 2.0 (Future)

Target: Q4 2026

Major changes:

  • Support for SOC 2 Type II (continuous collection)
  • Timeline bundles (multi-period evidence)
  • Additional frameworks (ISO 27001, HIPAA)

Backward compatibility: Read-only (cannot write v1.x bundles)


Appendix A: Complete Example

Minimal Bundle (GitHub only)

Uncompressed size: ~15 KB Compressed size: ~3 KB Controls: CC6.1 only Sources: GitHub (1 repo)

Structure:

evidence-bundle-20260109-120000.tar.gz
├── manifest.json           (2 KB)
├── run.json                (1 KB)
├── checksums.sha256        (500 bytes)
├── signature.sig           (64 bytes)
├── sources/
│   └── github/
│       └── org_settings.json (3 KB)
└── derived/
    ├── normalized.json     (2 KB)
    └── hints.json          (1 KB)

Standard Bundle (Multi-source)

Uncompressed size: ~35 KB Compressed size: ~8 KB Controls: CC6.1, CC6.6, CC7.2 Sources: GitHub (3 repos), AWS, Google Workspace

Structure:

evidence-bundle-20260109-123456.tar.gz
├── manifest.json           (5 KB)
├── run.json                (2 KB)
├── checksums.sha256        (2 KB)
├── signature.sig           (64 bytes)
├── sources/
│   ├── github/
│   │   ├── org_settings.json (3 KB)
│   │   ├── repo_acme_backend_branch_protection.json (2 KB)
│   │   ├── repo_acme_backend_codeowners.json (1 KB)
│   │   ├── repo_acme_frontend_branch_protection.json (2 KB)
│   │   └── repo_acme_api_branch_protection.json (2 KB)
│   ├── aws/
│   │   ├── iam_password_policy.json (3 KB)
│   │   └── cloudtrail_trail_production_status.json (2 KB)
│   └── google_workspace/
│       ├── 2sv_enforcement.json (1 KB)
│       ├── admin_roles.json (5 KB)
│       └── user_lifecycle.json (3 KB)
└── derived/
    ├── normalized.json     (3 KB)
    └── hints.json          (2 KB)

Appendix B: Verification Script

Bash script for manual verification:

#!/bin/bash
# verify-bundle.sh - Verify evidence bundle integrity

BUNDLE=$1

if [ -z "$BUNDLE" ]; then
  echo "Usage: verify-bundle.sh <bundle.tar.gz>"
  exit 1
fi

# Extract bundle
echo "Extracting bundle..."
TEMP_DIR=$(mktemp -d)
tar -xzf "$BUNDLE" -C "$TEMP_DIR"

cd "$TEMP_DIR"

# Verify checksums
echo "Verifying checksums..."
if sha256sum -c checksums.sha256; then
  echo "✓ Checksums valid"
else
  echo "✗ Checksum verification failed"
  exit 1
fi

# Verify signature (requires evidence CLI)
echo "Verifying signature..."
if command -v evidence &> /dev/null; then
  evidence verify "$BUNDLE"
else
  echo "⚠ evidence CLI not found, skipping signature verification"
  echo "  Install with: npm install -g @evidence-oss/cli"
fi

# Cleanup
cd /
rm -rf "$TEMP_DIR"

echo "✓ Bundle verification complete"

See Also