shopdb-flask/CERT-AUTH-SECURITY.md

# Client Certificate Authentication Security Documentation

## Overview

ShopDB endpoint agent uses mTLS (mutual TLS) for per-endpoint authentication. This document covers security considerations, deployment requirements, and operational procedures.

**Last Updated:** 2026-09-14  
**Status:** Production-ready with documented limitations

---

## Security Model

### Authentication Flow

1. **Agent Startup**: Loads client certificate from LocalMachine\My by thumbprint
2. **HTTPS Connection**: Presents certificate to nginx during TLS handshake
3. **Nginx Validation**: Accepts certificate (no CA validation - self-signed OK)
4. **Fingerprint Extraction**: Nginx computes SHA1, passes to Flask via header
5. **Flask Authentication**: Looks up asset by fingerprint, checks revocation/expiration
6. **Authorization**: Verifies cert's asset matches payload's hostname asset
7. **Fallback**: If cert auth fails, falls back to collector API key

### Dual-Auth During Rollout

- **Phase 1-3**: Both cert AND API key accepted (backward compatible)
- **Phase 4**: API key disabled via `plugin.endpoints.envkeyfallback=false`
- **Rollback**: Re-enable API key fallback instantly if cert auth breaks

---

## Known Limitations

### 1. SHA1 Fingerprints (nginx limitation)

**Issue:** Nginx `$ssl_client_fingerprint` outputs SHA1 (40 hex chars), not SHA256.

**Why SHA1?**
- Nginx does not expose SHA256 fingerprint via built-in variables
- Would require custom nginx module or full certificate parsing in Flask

**Is SHA1 Acceptable?**
- ✅ YES for certificate fingerprinting (not signing)
- ✅ Acceptable in air-gapped environments with:
  - Short-lived certificates (2 years max)
  - Certificate revocation support (cert_revoked_at column)
  - Self-signed certificates only (no external CA trust)

**SHA1 Collision Risk:**
- Computationally expensive to generate collision (estimated $100K+ in 2026)
- Would require attacker to generate cert with same SHA1 as legitimate endpoint
- Air-gapped environment prevents external certificate injection
- Revocation support allows instant kill switch if compromised

**Migration to SHA256:**
```
Option 1: Custom nginx module to expose SHA256
Option 2: Migrate to IIS (exposes full PEM cert for server-side hashing)
Option 3: Pass full certificate to Flask for SHA256 computation (bandwidth cost)
```

### 2. Private Key Permissions (tray access)

**Issue:** PowerShell script grants `BUILTIN\Users` read access to private key.

**Why?** Tray app (running as interactive user) needs cert access for manual agent runs.

**Risk:** ANY interactive user on the machine can extract the private key.

**Mitigations:**
1. **Best:** Remove tray dependency (tray calls service API, service uses cert)
2. **Acceptable:** Grant only to specific user account running tray
3. **Current:** Document that tray deployment requires trusted console users only

**When Acceptable:**
- Shop floor PCs with single trusted operator per machine
- Kiosks where all users are authorized operators
- Environments where console access = trusted

**NOT Acceptable:**
- Multi-user workstations with untrusted users
- Shared administrator workstations
- Development machines

### 3. No Certificate Revocation List (CRL)

**Issue:** No automated CRL distribution. Revocation is manual SQL UPDATE.

**Workaround:**
```sql
-- Revoke certificate (takes effect immediately)
UPDATE assets
SET cert_revoked_at = CURRENT_TIMESTAMP,
    cert_revocation_reason = 'Compromised - laptop stolen - TICKET-456'
WHERE client_cert_fingerprint = 'abc123...';
```

**Future:** Web UI for certificate management (provision, revoke, monitor expiration).

---

## Security Features Implemented

### ✅ Constant-Time Fingerprint Comparison

Prevents timing attacks on fingerprint validation:

```python
if not hmac.compare_digest(asset.client_cert_fingerprint, fingerprint):
    return None  # Constant time, no early exit
```

### ✅ Format Validation

Rejects malformed fingerprints before database lookup:

```python
# Must be 40 hex chars (SHA1) or 64 hex chars (SHA256)
if len(normalized) not in (40, 64):
    return None
if not re.match(r'^[0-9a-f]+$', normalized):
    return None
```

### ✅ Certificate Revocation Support

Admin can revoke compromised certificates instantly:

```sql
-- Revoke
UPDATE assets SET cert_revoked_at = NOW(), cert_revocation_reason = '...' WHERE ...;
-- Un-revoke (if cert was revoked by mistake)
UPDATE assets SET cert_revoked_at = NULL, cert_revocation_reason = NULL WHERE ...;
```

### ✅ Expiration Enforcement

Expired certificates rejected automatically:

```python
if asset.cert_expires_at < now:
    return None  # Must regenerate and re-register
```

### ✅ Authorization Check

Certificate authenticates asset X, but payload reports for asset Y → DENIED:

```python
if g.cert_authenticated_asset_id != assetid:
    raise PermissionError(...)  # Prevents cross-asset impersonation
```

### ✅ Audit Logging

All authentication attempts logged to AuditLog:

- `auth.cert.success` - Successful authentication
- `auth.cert.unknown` - Unknown fingerprint
- `auth.cert.revoked` - Attempt to use revoked cert
- `auth.cert.expired` - Attempt to use expired cert

---

## Deployment Requirements

### Phase 1: Server Deployment

1. **Deploy nginx with mTLS support:**
   ```yaml
   # docker-compose.yml
   services:
     nginx:
       image: nginx:alpine
       volumes:
         - ./nginx.conf:/etc/nginx/nginx.conf
         - ./certs:/etc/nginx/certs
   ```

2. **Run database migrations:**
   ```bash
   # Adds client_cert_fingerprint, cert_issued_at, cert_expires_at, 
   # cert_revoked_at, cert_revocation_reason columns
   flask db upgrade
   ```

3. **Verify fallback still works:**
   ```bash
   # Existing agents continue using API key
   curl -H "X-API-Key: $COLLECTOR_API_KEY" https://shopdb/api/collector/endpoints
   ```

### Phase 2: Certificate Generation (per endpoint)

1. **Run PowerShell script on endpoint:**
   ```powershell
   .\generate-agent-cert.ps1 -ValidityYears 2
   ```

2. **Script outputs SQL to register certificate:**
   ```sql
   UPDATE assets
   SET client_cert_fingerprint = 'abc123...',
       cert_issued_at = CURRENT_TIMESTAMP,
       cert_expires_at = '2028-09-14 12:00:00',
       cert_revoked_at = NULL,
       cert_revocation_reason = NULL
   WHERE assetnumber = 'G5QX1GT3ESF';
   ```

3. **Verify agent config updated:**
   ```json
   {
     "baseurl": "https://shopdb.company.com",
     "apikey": "collector_key_here",
     "clientCertThumbprint": "ABC123...",
     "cadenceseconds": 900
   }
   ```

4. **Restart agent service:**
   ```powershell
   Restart-Service ShopDbEndpointAgent
   ```

5. **Verify cert auth in logs:**
   ```
   [client_cert] Authenticated asset 1234 (number=G5QX1GT3ESF) via certificate abc123...
   ```

### Phase 3: Fleet Rollout

**Automation Required:**
- PowerShell script to generate certs across fleet
- CSV export of asset_number,fingerprint,expires_at
- SQL bulk INSERT script
- Group Policy or deployment tool to push agent config updates

**Validation:**
- Monitor `auth.cert.success` audit log entries
- Dashboard showing cert expiration dates
- Alert on endpoints still using API key after N days

### Phase 4: Retire API Key Fallback

**Only after 100% cert coverage verified:**

```sql
UPDATE settings
SET value = 'false'
WHERE key = 'plugin.endpoints.envkeyfallback';
```

**Revoke old collector API key tokens:**
```python
# In Flask admin UI or API
revoke_service_tokens('collector.ingest.endpoints')
```

**CRITICAL:** Test rollback plan before this step!

---

## Operational Procedures

### Certificate Rotation

**When to rotate:**
- 30 days before expiration (automated alert recommended)
- After suspected compromise
- After employee termination (if they had console access)

**How to rotate:**
1. Generate new certificate (keep old one installed temporarily)
2. Update database with new fingerprint
3. Update agent config with new thumbprint
4. Restart agent
5. Verify new cert working
6. Remove old cert from certificate store

### Certificate Revocation

**Immediate revocation (compromised/lost endpoint):**

```sql
UPDATE assets
SET cert_revoked_at = CURRENT_TIMESTAMP,
    cert_revocation_reason = 'Laptop stolen - INCIDENT-789'
WHERE assetnumber = 'G5QX1GT3ESF';
```

**Effect:** Agent's next request denied, falls back to API key if enabled.

### Monitoring

**Key metrics to monitor:**

1. **Cert Expiration:**
   ```sql
   SELECT assetnumber, cert_expires_at,
          DATEDIFF(cert_expires_at, NOW()) AS days_remaining
   FROM assets
   WHERE cert_expires_at IS NOT NULL
     AND cert_expires_at < DATE_ADD(NOW(), INTERVAL 30 DAY)
   ORDER BY cert_expires_at;
   ```

2. **Auth Failure Rate:**
   ```sql
   SELECT action, COUNT(*) AS count
   FROM auditlog
   WHERE action LIKE 'auth.cert.%'
     AND createdat > NOW() - INTERVAL 24 HOUR
   GROUP BY action;
   ```

3. **Endpoints Still Using API Key:**
   ```sql
   SELECT e.hostname, e.lastreportedat
   FROM endpointstates e
   JOIN assets a ON a.assetid = e.assetid
   WHERE a.client_cert_fingerprint IS NULL
     AND e.lastreportedat > NOW() - INTERVAL 1 HOUR;
   ```

---

## Troubleshooting

### Agent reports but logs show API key auth (not cert)

**Cause:** Client certificate not configured or not found.

**Fix:**
```powershell
# Verify cert exists
Get-ChildItem Cert:\LocalMachine\My | Where-Object { $_.Subject -like "*$env:COMPUTERNAME*" }

# Verify config
Get-Content C:\ProgramData\ShopDB\config.json | ConvertFrom-Json

# Verify cert has private key
$cert = Get-Item "Cert:\LocalMachine\My\$thumbprint"
$cert.HasPrivateKey  # Should be True
```

### "Revoked certificate" in logs but cert not revoked

**Cause:** Database cert_revoked_at column set incorrectly.

**Fix:**
```sql
UPDATE assets
SET cert_revoked_at = NULL,
    cert_revocation_reason = NULL
WHERE assetnumber = 'G5QX1GT3ESF';
```

### "Unknown fingerprint" in logs

**Cause:** Certificate fingerprint not registered in database.

**Fix:**
```powershell
# Get fingerprint from cert
$cert = Get-Item "Cert:\LocalMachine\My\$thumbprint"
$certBytes = $cert.Export([System.Security.Cryptography.X509Certificates.X509ContentType]::Cert)
$sha1 = [System.Security.Cryptography.SHA1]::Create()
$fp = ($sha1.ComputeHash($certBytes) | ForEach-Object { $_.ToString("x2") }) -join ""

# Register in database
UPDATE assets SET client_cert_fingerprint = '$fp' WHERE assetnumber = 'G5QX1GT3ESF';
```

### Agent fails after cert rotation

**Cause:** Old cert removed before new cert registered in database.

**Fix:**
1. Reinstall old cert temporarily
2. Register new cert fingerprint in database
3. Update agent config to new thumbprint
4. Restart agent
5. Remove old cert

---

## Security Best Practices

### ✅ DO

- Generate certificates with 2-year maximum validity
- Rotate certificates 30 days before expiration
- Revoke certificates immediately when endpoint compromised
- Monitor audit logs for revoked/expired cert attempts
- Document certificate issuance and revocation in tickets
- Test rollback to API key before disabling fallback
- Use short-lived certificates in untrusted environments

### ❌ DON'T

- Grant `BUILTIN\Users` access to private keys on multi-user machines
- Skip hostname validation in cert generation script
- Deploy to production without testing cert revocation
- Disable API key fallback without verified 100% cert coverage
- Use expired certificates (regenerate immediately)
- Share certificates between endpoints
- Export private keys to network shares

---

## Future Enhancements

1. **Web UI for Certificate Management**
   - Upload CSRs or paste fingerprints
   - View expiration dashboard
   - One-click revocation
   - Automated renewal reminders

2. **SHA256 Migration**
   - Custom nginx module for SHA256
   - Or migrate to IIS deployment
   - Dual-column support (SHA1 + SHA256) during migration

3. **Automated Certificate Rotation**
   - Agent auto-generates new cert 30 days before expiration
   - Server auto-accepts new fingerprint after validating old cert
   - Zero-touch renewal

4. **Per-User Certificate ACLs**
   - Grant tray access to specific user account only
   - Service creates runas token for tray HTTP requests

---

## References

- **ADR-006:** Collector Contract (dual-auth model)
- **ADR-016:** Credential Delivery (provisioning workflow)
- **AGENT-CONTRACT.md:** Wire protocol and authentication
- **nginx.conf:** mTLS configuration and SHA1 limitation
- **generate-agent-cert.ps1:** Certificate generation script

---

## Contact

For security questions or to report vulnerabilities:
- **Internal:** File ticket in JIRA under "Security" component
- **Emergency:** Page on-call security engineer via PagerDuty