shopdb-flask/CERT-AUTH-IMPLEMENTATION.md

Per-Endpoint Certificate Authentication - Implementation Guide

Overview

Implement client certificate authentication for ShopDB endpoint agents that works across multiple deployment types:

• WJ Production: IIS 2019 on Windows Server
• Other Sites: Docker Compose with Nginx reverse proxy

Architecture

Agent (has cert) → Server (IIS or Nginx) → Flask
                        ↓
                   Extracts cert
                   Passes to Flask via header/environ
                        ↓
                   Flask validates fingerprint
                   against database

Phase 1: Database Schema

Add client certificate columns to assets table.

Migration: plugins/endpoints/migrations/versions/0021_client_certificates.py

"""Add client certificate authentication columns

Revision ID: endpoints0021clientcerts
Revises: endpoints0020appfingerprints
Create Date: 2026-09-14
"""

from alembic import op
import sqlalchemy as sa

revision = 'endpoints0021clientcerts'
down_revision = 'endpoints0020appfingerprints'
branch_labels = None
depends_on = None
plugin_name = 'endpoints'


def upgrade():
    # Add client cert columns to assets table
    op.add_column('assets',
        sa.Column('client_cert_fingerprint', sa.String(64), nullable=True))
    
    op.add_column('assets',
        sa.Column('cert_issued_at', sa.DateTime, nullable=True))
    
    op.add_column('assets',
        sa.Column('cert_expires_at', sa.DateTime, nullable=True))
    
    # Add index for fast fingerprint lookup during authentication
    op.create_index(
        'idx_assets_cert_fingerprint',
        'assets',
        ['client_cert_fingerprint'],
        unique=False
    )


def downgrade():
    op.drop_index('idx_assets_cert_fingerprint', 'assets')
    op.drop_column('assets', 'cert_expires_at')
    op.drop_column('assets', 'cert_issued_at')
    op.drop_column('assets', 'client_cert_fingerprint')

Run migration:

docker compose exec api flask db upgrade

---

Phase 2: Flask Code (Universal)

Add client certificate authentication support that works with both IIS and Nginx.

File: shopdb/core/auth/client_cert.py (new file)

"""Client certificate authentication helpers.

Extracts client certificate fingerprints from various deployment types:
- IIS 2019: SSL_CLIENT_CERT environ (PEM format)
- IIS with ARR: X-ARR-ClientCert header (base64 DER)
- Nginx: X-Client-Cert-Fingerprint header (pre-computed)
"""

import hashlib
import base64
from flask import request, current_app
from cryptography import x509
from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives import serialization


def get_client_cert_fingerprint():
    """Extract client certificate SHA256 fingerprint from request.
    
    Returns:
        str: 64-character lowercase hex fingerprint, or None if no cert
    
    Supports multiple deployment types:
    - Nginx: reads X-Client-Cert-Fingerprint header (SHA256, colon-separated)
    - IIS direct: reads SSL_CLIENT_CERT environ (PEM format)
    - IIS with ARR: reads X-ARR-ClientCert header (base64 DER)
    """
    
    # Option 1: Nginx passes pre-computed fingerprint
    # Format: "AB:CD:EF:..." (colon-separated uppercase hex)
    nginx_fp = request.headers.get('X-Client-Cert-Fingerprint')
    if nginx_fp:
        # Normalize: remove colons, lowercase
        return nginx_fp.replace(':', '').lower()
    
    # Option 2: IIS passes full cert in PEM format via WSGI environ
    # This is the standard IIS behavior when client certs are enabled
    iis_cert_pem = request.environ.get('SSL_CLIENT_CERT')
    if iis_cert_pem:
        try:
            # Parse PEM certificate
            if isinstance(iis_cert_pem, str):
                iis_cert_pem = iis_cert_pem.encode('utf-8')
            
            cert = x509.load_pem_x509_certificate(iis_cert_pem, default_backend())
            
            # Convert to DER and hash
            cert_der = cert.public_bytes(serialization.Encoding.DER)
            fingerprint = hashlib.sha256(cert_der).hexdigest()
            
            current_app.logger.debug(f'[client_cert] Extracted from IIS SSL_CLIENT_CERT: {fingerprint[:16]}...')
            return fingerprint
        except Exception as e:
            current_app.logger.warning(f'[client_cert] Failed to parse IIS cert: {e}')
            return None
    
    # Option 3: IIS with ARR (Application Request Routing) passes base64-encoded DER
    arr_cert_b64 = request.headers.get('X-ARR-ClientCert')
    if arr_cert_b64:
        try:
            # Decode base64 to get DER bytes
            cert_der = base64.b64decode(arr_cert_b64)
            fingerprint = hashlib.sha256(cert_der).hexdigest()
            
            current_app.logger.debug(f'[client_cert] Extracted from ARR header: {fingerprint[:16]}...')
            return fingerprint
        except Exception as e:
            current_app.logger.warning(f'[client_cert] Failed to parse ARR cert: {e}')
            return None
    
    # No client certificate present
    return None


def authenticate_by_client_cert():
    """Authenticate request by client certificate.
    
    Returns:
        Asset: The authenticated asset, or None if cert auth fails
    
    Checks:
    1. Client cert present in request?
    2. Fingerprint matches an asset in database?
    3. Certificate not expired?
    
    Does NOT raise exceptions - returns None on any failure.
    Caller decides whether to fall back to other auth methods.
    """
    from shopdb.api import Asset
    from datetime import datetime, timezone
    
    fingerprint = get_client_cert_fingerprint()
    if not fingerprint:
        return None
    
    # Look up asset by cert fingerprint
    asset = Asset.query.filter_by(client_cert_fingerprint=fingerprint).first()
    if not asset:
        current_app.logger.warning(
            f'[client_cert] Unknown fingerprint: {fingerprint[:16]}... '
            f'(not associated with any asset)'
        )
        return None
    
    # Check certificate expiration
    if asset.cert_expires_at:
        now = datetime.now(timezone.utc).replace(tzinfo=None)
        if asset.cert_expires_at < now:
            current_app.logger.warning(
                f'[client_cert] Expired certificate for asset {asset.assetid} '
                f'(expired {asset.cert_expires_at})'
            )
            return None
    
    current_app.logger.info(
        f'[client_cert] Authenticated asset {asset.assetid} '
        f'via certificate {fingerprint[:16]}...'
    )
    
    return asset

File: plugins/endpoints/plugin.py (modify apply_collector_payload)

# Add import at top
from shopdb.core.auth.client_cert import authenticate_by_client_cert

# In apply_collector_payload method, BEFORE existing auth check:
def apply_collector_payload(self, payload: Dict) -> Dict:
    """Process endpoint collector payload (ADR-006 contract)."""
    
    # CERT AUTH FIRST: Check for client certificate
    cert_asset = authenticate_by_client_cert()
    if cert_asset:
        # Client cert authenticated - use this asset
        assetid = cert_asset.assetid
        hostname = payload.get('hostname', 'UNKNOWN')
        
        current_app.logger.info(
            f'[collector] asset_id={assetid} authenticated via client certificate'
        )
        
        # Update cert metadata if agent provides new expiration
        if 'cert_expires_at' in payload:
            cert_asset.cert_expires_at = payload['cert_expires_at']
            db.session.flush()
    else:
        # FALLBACK: Existing CollectorKey authentication
        # (your existing _collector_auth_or_error logic here)
        pass
    
    # Rest of method unchanged...

---

Phase 3: IIS 2019 Configuration (WJ Production)

Step 1: Enable Client Certificates in IIS

1. Open IIS Manager
2. Select your ShopDB site (not server level)
3. Open SSL Settings
4. Check "Accept" (not "Require" - we want dual-auth during rollout)
5. Click Apply

Step 2: Verify Certificate Mapping (Optional)

IIS will automatically pass client certs to WSGI via SSL_CLIENT_CERT environ variable - no additional config needed.

If using ARR (Application Request Routing):

1. Open URL Rewrite for your site
2. Add Server Variable: HTTP_X_ARR_ClientCert
3. Set value to: {HTTP_SSL_CLIENT_CERT}

Step 3: Test with curl

# Generate test client cert (PowerShell)
$cert = New-SelfSignedCertificate `
    -Subject "CN=TEST-ENDPOINT" `
    -CertStoreLocation "Cert:\CurrentUser\My" `
    -KeyExportPolicy Exportable `
    -NotAfter (Get-Date).AddYears(2)

# Export to PFX
$pwd = ConvertTo-SecureString -String "test123" -Force -AsPlainText
Export-PfxCertificate `
    -Cert $cert `
    -FilePath "C:\temp\test-client.pfx" `
    -Password $pwd

# Test with curl (Git Bash on Windows)
curl -k --cert-type P12 --cert C:/temp/test-client.pfx:test123 \
    https://localhost/api/endpoints/test/client-cert

Expected response:

{
  "data": {
    "message": "Client cert test endpoint",
    "ssl_client_cert_present": true,
    "fingerprint_sha256": "325e42ef..."
  }
}

Step 4: Configure Agent to Use Certificate

Agent change (C# .NET):

// ShopDbEndpointAgent.Core/EndpointCollector.cs

private static HttpClient CreateHttpClient(string certThumbprint = null)
{
    var handler = new HttpClientHandler
    {
        ServerCertificateCustomValidationCallback = (msg, cert, chain, errors) =>
        {
            // Accept self-signed server cert (existing behavior)
            return true;
        }
    };

    // If client cert configured, attach it
    if (!string.IsNullOrEmpty(certThumbprint))
    {
        var cert = GetClientCertFromStore(certThumbprint);
        if (cert != null)
        {
            handler.ClientCertificates.Add(cert);
            Log.Information($"Using client certificate: {cert.Subject}");
        }
    }

    return new HttpClient(handler)
    {
        Timeout = TimeSpan.FromSeconds(90)
    };
}

private static X509Certificate2 GetClientCertFromStore(string thumbprint)
{
    using (var store = new X509Store(StoreName.My, StoreLocation.LocalMachine))
    {
        store.Open(OpenFlags.ReadOnly);
        var certs = store.Certificates.Find(
            X509FindType.FindByThumbprint,
            thumbprint,
            validOnly: false
        );
        return certs.Count > 0 ? certs[0] : null;
    }
}

Config file: C:\ProgramData\ShopDB\config.json

{
  "apiUrl": "https://shopdb.wj.local/api",
  "clientCertThumbprint": "325E42EFA401CDA4B75BB7B478CFE01FBD5EE956C629ADD3A075A74847D71F77"
}

---

Phase 4: Docker + Nginx Configuration (Other Sites)

Step 1: Add nginx.conf

File: nginx.conf (create in repo root)

events {
    worker_connections 2048;
}

http {
    upstream flask_backend {
        server api:5001;
    }

    server {
        listen 443 ssl;
        server_name _;

        # Server certificate (HTTPS)
        ssl_certificate /etc/nginx/certs/server.crt;
        ssl_certificate_key /etc/nginx/certs/server.key;

        # Client certificate validation (optional during rollout)
        # optional_no_ca: accept cert but don't validate against CA
        # Flask validates fingerprint against database
        ssl_verify_client optional_no_ca;

        # Increase timeouts for long-polling agents
        proxy_connect_timeout 90s;
        proxy_send_timeout 90s;
        proxy_read_timeout 90s;

        # Health check endpoint
        location /api/dashboard/health {
            proxy_pass http://flask_backend;
            proxy_http_version 1.1;
            proxy_set_header Connection "";
        }

        # All other routes
        location / {
            proxy_pass http://flask_backend;
            proxy_http_version 1.1;
            proxy_set_header Connection "";
            
            # Pass real client IP (not nginx IP)
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
            proxy_set_header Host $host;
            
            # Pass client certificate fingerprint to Flask
            # $ssl_client_fingerprint format: "AB:CD:EF:..." (colon-separated hex)
            proxy_set_header X-Client-Cert-Fingerprint $ssl_client_fingerprint;
            proxy_set_header X-Client-Cert-S-DN $ssl_client_s_dn;
        }

        # Access log includes client cert status
        access_log /dev/stdout combined;
        error_log /dev/stderr warn;
    }
}

Step 2: Update docker-compose.yml

services:
  nginx:
    image: nginx:1.27-alpine
    restart: unless-stopped
    depends_on:
      - api
    ports:
      - "5001:443"
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
      - ./certs:/etc/nginx/certs:ro
    healthcheck:
      test: ["CMD", "wget", "--spider", "-q", "--no-check-certificate", "https://localhost/api/dashboard/health"]
      interval: 30s
      timeout: 5s
      retries: 3

  api:
    # ... existing config ...
    ports:
      # Expose on localhost only (for debugging)
      - "127.0.0.1:5002:5001"
    command:
      - gunicorn
      - --bind
      - 0.0.0.0:5001
      # REMOVE: --certfile (nginx does TLS now)
      # REMOVE: --keyfile
      # REMOVE: --cert-reqs
      - --worker-class
      - gevent
      - --workers
      - "4"
      - --worker-connections
      - "2000"
      - --timeout
      - "60"
      - --access-logfile
      - "-"
      - --access-logformat
      - '%(h)s %(l)s %(u)s %(t)s "%(r)s" %(s)s %(b)s %(D)sus "%(f)s" "%(a)s"'
      - --error-logfile
      - "-"
      - wsgi:app
    # Remove healthcheck (nginx handles it now)

Step 3: Rebuild and Deploy

# Stop current stack
docker compose down

# Start with nginx
docker compose up -d

# Verify nginx is proxying
curl -k https://localhost:5001/api/dashboard/health

# Test with client cert (same cert from earlier test)
openssl s_client -connect localhost:5001 \
    -cert certs/test-client.crt \
    -key certs/test-client.key \
    -quiet <<< "GET /api/endpoints/test/client-cert HTTP/1.1
Host: localhost:5001
Connection: close

"

---

Phase 5: Agent Certificate Generation

PowerShell Script (Windows agents)

File: scripts/generate-agent-cert.ps1

param(
    [Parameter(Mandatory=$true)]
    [string]$Hostname,
    
    [Parameter(Mandatory=$false)]
    [int]$ValidityYears = 2
)

# Generate self-signed cert for this endpoint
$cert = New-SelfSignedCertificate `
    -Subject "CN=$Hostname" `
    -DnsName $Hostname `
    -CertStoreLocation "Cert:\LocalMachine\My" `
    -KeyExportPolicy NonExportable `
    -KeyUsage DigitalSignature, KeyEncipherment `
    -NotAfter (Get-Date).AddYears($ValidityYears) `
    -HashAlgorithm SHA256

Write-Host "Generated certificate:"
Write-Host "  Thumbprint: $($cert.Thumbprint)"
Write-Host "  Subject: $($cert.Subject)"
Write-Host "  Expires: $($cert.NotAfter)"

# Compute SHA256 fingerprint (for database registration)
$certBytes = $cert.Export([System.Security.Cryptography.X509Certificates.X509ContentType]::Cert)
$sha256 = [System.Security.Cryptography.SHA256]::Create()
$fingerprintBytes = $sha256.ComputeHash($certBytes)
$fingerprint = ($fingerprintBytes | ForEach-Object { $_.ToString("x2") }) -join ""

Write-Host ""
Write-Host "Register in ShopDB with:"
Write-Host "  UPDATE assets"
Write-Host "  SET client_cert_fingerprint = '$fingerprint',"
Write-Host "      cert_issued_at = CURRENT_TIMESTAMP,"
Write-Host "      cert_expires_at = '$($cert.NotAfter.ToString("yyyy-MM-dd HH:mm:ss"))'"
Write-Host "  WHERE hostname = '$Hostname';"

# Update agent config
$configPath = "C:\ProgramData\ShopDB\config.json"
if (Test-Path $configPath) {
    $config = Get-Content $configPath | ConvertFrom-Json
    $config.clientCertThumbprint = $cert.Thumbprint
    $config | ConvertTo-Json | Set-Content $configPath
    Write-Host ""
    Write-Host "Updated $configPath with thumbprint"
}

return $cert.Thumbprint

Usage:

# On endpoint VM
.\generate-agent-cert.ps1 -Hostname "DESKTOP-HF1K9M4"

# Output:
# Generated certificate:
#   Thumbprint: 325E42EFA401CDA4B75BB7B478CFE01FBD5EE956C629ADD3A075A74847D71F77
#   Subject: CN=DESKTOP-HF1K9M4
#   Expires: 9/14/2028 12:00:00 PM
#
# Register in ShopDB with:
#   UPDATE assets
#   SET client_cert_fingerprint = '325e42efa401cda4b75bb7b478cfe01fbd5ee956c629add3a075a74847d71f77',
#       cert_issued_at = CURRENT_TIMESTAMP,
#       cert_expires_at = '2028-09-14 12:00:00'
#   WHERE hostname = 'DESKTOP-HF1K9M4';

---

Rollout Sequence

Week 1: WJ Production (IIS)

1. Deploy Flask code (cert auth + CollectorKey fallback)
2. Configure IIS SSL Settings → "Accept" client certs
3. Generate cert for 1 test endpoint
4. Register cert fingerprint in database
5. Update agent config with thumbprint
6. Verify dual-auth works (cert + fallback)

Week 2: Remaining WJ Fleet

1. Generate certs for remaining endpoints (PowerShell script)
2. Register fingerprints in database
3. Deploy updated agent configs

Week 3: Docker Sites

1. Add nginx container to docker-compose.yml
2. Deploy updated bundle to 1 pilot site
3. Generate certs for that site's endpoints
4. Register fingerprints
5. Verify nginx proxy works

Month 2: Remaining Sites

1. Roll out nginx updates to remaining Docker sites
2. Generate certs per-site
3. After 100% cert adoption: change ssl_verify_client optional to require

---

Testing Checklist

Flask Tests

• get_client_cert_fingerprint() returns None when no cert
• get_client_cert_fingerprint() parses nginx header (X-Client-Cert-Fingerprint)
• get_client_cert_fingerprint() parses IIS environ (SSL_CLIENT_CERT)
• authenticate_by_client_cert() returns asset when fingerprint matches
• authenticate_by_client_cert() returns None when cert expired
• Collector falls back to CollectorKey when cert auth fails

IIS Tests

• IIS passes SSL_CLIENT_CERT to Flask
• Agent with cert authenticates successfully
• Agent without cert falls back to CollectorKey
• Expired cert rejected

Docker/Nginx Tests

• Nginx forwards requests to Gunicorn
• Nginx adds X-Client-Cert-Fingerprint header
• Agent with cert authenticates via nginx
• Agent without cert uses CollectorKey
• Long-polling works (25+ second holds)

---

Security Considerations

1. Cert Storage: LocalMachine\My store (admin-only access)
2. Private Key: Non-exportable (can't be stolen)
3. Expiration: 2 years (balance security vs. rotation overhead)
4. Revocation: Remove fingerprint from database (immediate)
5. Self-Signed: No CA needed (air-gapped sites)

---

Rollback Plan

If Flask code breaks:

git revert <commit>
docker compose restart api  # or iisreset

If IIS config breaks:

1. IIS Manager → SSL Settings → Uncheck "Accept"
2. Click Apply
3. Agents fall back to CollectorKey immediately

If nginx breaks:

# Restore direct Gunicorn access
docker compose down nginx
# Re-add --certfile/--keyfile to api service
docker compose up -d api

---

Monitoring

Metrics to Track

• % of agents using cert auth vs. CollectorKey
• Cert expiration warnings (30 days before)
• Failed cert authentications (unknown fingerprint)
• Expired cert attempts

Logs to Watch

[client_cert] Authenticated asset 2528 via certificate 325e42ef...
[client_cert] Unknown fingerprint: 325e42ef... (not associated with any asset)
[client_cert] Expired certificate for asset 2528 (expired 2026-09-14 12:00:00)

---

Next Steps

1. Create migration (0021_client_certificates.py)
2. Add Flask auth code (client_cert.py)
3. Test on IIS (your WJ production)
4. Add nginx config (for Docker sites)
5. Generate first cert (test endpoint)
6. Document in wiki (deployment runbook)

Estimated effort:

• Flask code: 2 hours
• IIS config + test: 1 hour
• Docker/nginx: 1 hour
• Agent changes: 2 hours
• Total: 6 hours

Ready to start with Flask code?