This commit adds: - Database migrations for app_delta fingerprints and client certificates - Scripts for certificate generation and fingerprint backfilling - Comprehensive security audit reports and findings tracking - Implementation documentation for certificate authentication - Test utilities for certificate renewal and fingerprinting - GE-ENFORCE collector fix documentation Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
160 lines
4.6 KiB
Markdown
160 lines
4.6 KiB
Markdown
# Client Certificate Authentication - WSGI Server Comparison
|
|
|
|
## Current Setup
|
|
- **Gunicorn 26.2.0** with gevent workers
|
|
- 4 workers x 2000 greenlets = 8000 concurrent connections
|
|
- Long-polling agents hold connections for 25+ seconds
|
|
- TLS termination in Gunicorn with --certfile/--keyfile
|
|
- **Problem**: Gunicorn doesn't expose client cert to Flask via WSGI environ
|
|
|
|
## Option 1: Hypercorn (ASGI)
|
|
|
|
**Pros:**
|
|
- Native client cert support via `ssl.SSLContext.wrap_socket()`
|
|
- Exposes client cert via ASGI scope['extensions']['tls']['client_cert_der']
|
|
- Modern async server (Trio/asyncio)
|
|
- Supports HTTP/2 and WebSockets
|
|
|
|
**Cons:**
|
|
- **ASGI, not WSGI** - Flask is WSGI, need asgiref adapter
|
|
- Different async model than gevent (asyncio instead of greenlets)
|
|
- Less mature for Flask production deployments
|
|
- Breaking change: entire stack needs rework
|
|
|
|
**Verdict:** ❌ Too invasive - requires migrating from WSGI to ASGI
|
|
|
|
---
|
|
|
|
## Option 2: Waitress (Pure Python WSGI)
|
|
|
|
**Pros:**
|
|
- ✅ Already in requirements.txt (line 698: waitress==3.0.2)
|
|
- ✅ Pure Python WSGI server (drop-in Gunicorn replacement)
|
|
- ✅ Production-ready (powers PyPI, Pyramid, etc.)
|
|
- ✅ Native Windows support
|
|
- Thread-based concurrency (not gevent, but configurable)
|
|
|
|
**Cons:**
|
|
- ❌ **No client certificate support** - waitress doesn't expose client certs to WSGI apps
|
|
- Uses threads instead of gevent greenlets (heavier for long-polling)
|
|
- Lower connection ceiling than gevent (4 threads vs 8000 greenlets)
|
|
|
|
**Verdict:** ❌ Doesn't solve the problem - no client cert support
|
|
|
|
---
|
|
|
|
## Option 3: Nginx Reverse Proxy (Current Recommendation)
|
|
|
|
**Pros:**
|
|
- ✅ Battle-tested client cert validation
|
|
- ✅ Keep Gunicorn + gevent (no code changes)
|
|
- ✅ Extracts cert fingerprint and passes via X-Client-Cert-Fingerprint header
|
|
- ✅ Optional validation with `ssl_verify_client optional_no_ca`
|
|
- ✅ Standard pattern for mTLS in Docker
|
|
|
|
**Cons:**
|
|
- Adds one more container (nginx:alpine ~30MB)
|
|
- Slightly more complex docker-compose.yml
|
|
|
|
**Verdict:** ✅ Best option - proven, minimal changes, keeps gevent
|
|
|
|
---
|
|
|
|
## Option 4: uWSGI
|
|
|
|
**Pros:**
|
|
- Full-featured WSGI server
|
|
- Supports client certs via `uwsgi_param SSL_CLIENT_CERT`
|
|
- Gevent plugin available
|
|
|
|
**Cons:**
|
|
- Complex configuration
|
|
- Deprecated (project in maintenance mode as of 2023)
|
|
- No active development
|
|
|
|
**Verdict:** ❌ Deprecated, avoid
|
|
|
|
---
|
|
|
|
## Option 5: Custom Gunicorn Worker (SSLGeventWorker)
|
|
|
|
**Approach**: Extend gevent worker to extract client cert from SSLSocket and inject into WSGI environ
|
|
|
|
**Pros:**
|
|
- Keep Gunicorn + gevent
|
|
- No reverse proxy needed
|
|
- ~50 lines of Python
|
|
|
|
**Cons:**
|
|
- Custom code to maintain
|
|
- Accesses Gunicorn internals (may break on upgrades)
|
|
- Less tested than Nginx approach
|
|
|
|
**Example:**
|
|
```python
|
|
# custom_worker.py
|
|
from gunicorn.workers.ggevent import GeventWorker
|
|
import ssl
|
|
|
|
class SSLGeventWorker(GeventWorker):
|
|
def handle_request(self, listener, req, client, addr):
|
|
# Extract client cert from SSLSocket
|
|
if isinstance(client, ssl.SSLSocket):
|
|
try:
|
|
cert_der = client.getpeercert(binary_form=True)
|
|
if cert_der:
|
|
req.environ['peercert'] = cert_der
|
|
except Exception:
|
|
pass
|
|
return super().handle_request(listener, req, client, addr)
|
|
```
|
|
|
|
```yaml
|
|
# docker-compose.yml
|
|
command:
|
|
- gunicorn
|
|
- --worker-class
|
|
- custom_worker.SSLGeventWorker
|
|
- --cert-reqs
|
|
- "1"
|
|
- wsgi:app
|
|
```
|
|
|
|
**Verdict:** ⚠️ Possible but fragile - use if Nginx is not acceptable
|
|
|
|
---
|
|
|
|
## Recommendation
|
|
|
|
**Go with Option 3: Nginx reverse proxy**
|
|
|
|
### Why:
|
|
1. **Proven pattern** - industry standard for mTLS in Docker
|
|
2. **Minimal risk** - no changes to Flask or Gunicorn config
|
|
3. **Keep gevent** - preserves your 8000 concurrent connection capacity
|
|
4. **Optional validation** - `ssl_verify_client optional_no_ca` supports dual-auth during rollout
|
|
5. **Easy rollback** - just remove nginx container
|
|
|
|
### Implementation:
|
|
- Add nginx:alpine container (30MB)
|
|
- 10-line nginx.conf
|
|
- No Python code changes (Flask reads `X-Client-Cert-Fingerprint` header)
|
|
- 5 minutes to implement
|
|
- Test immediately with same test cert
|
|
|
|
### Alternative (if Nginx rejected):
|
|
- Option 5: Custom Gunicorn worker (50 lines, fragile)
|
|
- Option 1: Migrate to Hypercorn + ASGI (days of work, high risk)
|
|
|
|
---
|
|
|
|
## Research Summary
|
|
|
|
**Tested:** Gunicorn with --cert-reqs 1 accepts client cert but doesn't expose to Flask
|
|
|
|
**Researched:** Waitress (no client cert), uWSGI (deprecated), Hypercorn (ASGI only)
|
|
|
|
**Validated:** Nginx reverse proxy is standard solution for mTLS with WSGI apps
|
|
|
|
**Decision Point:** Add nginx container or write custom Gunicorn worker?
|