Caddy2 Configuration System Documentation

Overview

This document details the corrected Caddy configuration system for do.dev infrastructure. Unlike the previous API-based approach documented in CADDY.md, this system uses Caddyfile-based configuration with Docker deployment.

Critical Discovery (July 2025)

Previous Issue: The sync API was using Caddy's JSON API to configure routes, but the Docker Caddy instance was running with a Caddyfile that overrode API configurations on restart.

Solution: Modified the sync system to write Caddyfile snippets to /config/*.caddy files that are imported by the main Caddyfile.

System Architecture

Docker Caddy Container

Container Name: caddy-reverse-proxy
Host: 10.3.3.3
Location: /root/local/caddy
Ports: 80:80, 443:443, 2019:2019
Configuration Mode: Caddyfile + imports

Main Caddyfile Structure

# Global options
{
    # Admin API configuration
    admin 0.0.0.0:2019
    
    # ACME configuration for Let's Encrypt
    email {$ACME_EMAIL}
    
    # DNS challenge using Cloudflare
    acme_dns cloudflare {$CLOUDFLARE_API_TOKEN}
    
    # Global logging
    log {
        output file /logs/caddy.log {
            roll_size 10MB
            roll_keep 5
            roll_keep_for 720h
        }
        format json
        level {$CADDY_LOG_LEVEL}
    }
}

# Health check endpoint on admin API
http://localhost:2019 {
    handle /health {
        respond "OK" 200
    }
}

# Default HTTP server - redirect to HTTPS
http:// {
    redir https://{host}{uri} permanent
}

# Import additional configurations
import /config/*.caddy

Environment Variables

<<<<<<< HEAD
CLOUDFLARE_API_TOKEN=gMrkoq9k9kWldyKWSRWtIqlZNDd3dibYZ2he4L9Z
=======
CLOUDFLARE_API_TOKEN=your_cloudflare_api_token_here  # Replace with your actual Cloudflare API token
>>>>>>> fcd62685a9d930d1de5a3007c43332340dff891c
ACME_EMAIL=tim@do.dev
CADDY_LOG_LEVEL=INFO

SSL Certificate Management

Automatic HTTPS via Cloudflare DNS

Method: ACME DNS challenge using Cloudflare API
Issuer: Let's Encrypt (E5/E6 intermediate)
Wildcard Support: Yes, via DNS-01 challenge
Auto-renewal: Handled by Caddy automatically

Certificate Validation

# Check certificate issuer
echo | openssl s_client -connect do.dev:443 -servername do.dev 2>/dev/null | openssl x509 -noout -issuer -subject

Expected output:

issuer=C=US, O=Let's Encrypt, CN=E6
subject=CN=do.dev

Current Domain Configuration

Active Domains (11 total)

All configured in /config/domains.caddy:

# Auto-generated domain configurations from sync

dns.local.dev {
    reverse_proxy {
        to 10.3.3.3:5380
        health_uri /
        health_interval 30s
        health_timeout 5s
    }
}

local.dev {
    reverse_proxy {
        to 10.1.0.33:3010 10.3.0.33:3010
        health_uri /
        health_interval 30s
        health_timeout 5s
        lb_policy first
    }
}

do.dev {
    reverse_proxy {
        to 10.1.0.33:3005 10.3.0.33:3005
        health_uri /
        health_interval 30s
        health_timeout 5s
        lb_policy first
    }
}

contacts.dev {
    reverse_proxy {
        to 10.1.0.33:3014 10.3.0.33:3014
        health_uri /
        health_interval 30s
        health_timeout 5s
        lb_policy first
    }
}

doc.dev {
    reverse_proxy {
        to 10.1.0.33:3016 10.3.0.33:3016
        health_uri /
        health_interval 30s
        health_timeout 5s
        lb_policy first
    }
}

biturl.dev {
    reverse_proxy {
        to 10.1.0.33:3013 10.3.0.33:3013
        health_uri /
        health_interval 30s
        health_timeout 5s
        lb_policy first
    }
}

isup.dev {
    reverse_proxy {
        to 10.1.0.33:3018 10.3.0.33:3018
        health_uri /
        health_interval 30s
        health_timeout 5s
        lb_policy first
    }
}

homepage.dev {
    reverse_proxy {
        to 10.1.0.33:3017 10.3.0.33:3017
        health_uri /
        health_interval 30s
        health_timeout 5s
        lb_policy first
    }
}

customers.dev {
    reverse_proxy {
        to 10.1.0.33:3015 10.3.0.33:3015
        health_uri /
        health_interval 30s
        health_timeout 5s
        lb_policy first
    }
}

sell.dev {
    reverse_proxy {
        to 10.1.0.33:3019 10.3.0.33:3019
        health_uri /
        health_interval 30s
        health_timeout 5s
        lb_policy first
    }
}

talk.dev {
    reverse_proxy {
        to 10.1.0.33:3012 10.3.0.33:3012
        health_uri /
        health_interval 30s
        health_timeout 5s
        lb_policy first
    }
}

Multi-Upstream Failover Configuration

Load Balancing Policy

Policy: first - Always use the first healthy upstream
Primary: 10.1.0.33 (primary infrastructure)
Secondary: 10.3.0.33 (failover infrastructure)

Health Checks

Path: / (root endpoint)
Interval: 30 seconds
Timeout: 5 seconds
Expected Status: 200 OK

Sync System Integration

Current Status (Needs Update)

The sync API (/api/caddy/sync) still uses the old JSON API approach. It needs to be updated to:

Generate Caddyfile syntax instead of JSON
Write to /config/domains.caddy instead of API endpoints
Trigger Caddy reload via caddy reload --config /etc/caddy/Caddyfile

Required Sync API Changes

// Instead of this (old API approach):
await fetch(`${this.apiUrl}/config/apps/http/servers/srv2/routes`, {
  method: 'PUT',
  headers: this.headers,
  body: JSON.stringify(httpsServer.routes),
})

// Do this (new Caddyfile approach):
const caddyfileContent = generateCaddyfileFromRoutes(routes)
await writeToContainer('/config/domains.caddy', caddyfileContent)
await execInContainer('caddy reload --config /etc/caddy/Caddyfile')

Management Operations

Manual Configuration Update

# SSH to Caddy host
ssh root@10.3.3.3

# Edit domain configuration
docker exec caddy-reverse-proxy sh -c 'cat > /config/domains.caddy << EOF
# Updated configuration
EOF'

# Reload Caddy
docker exec caddy-reverse-proxy caddy reload --config /etc/caddy/Caddyfile

Container Management

# Restart Caddy container
docker restart caddy-reverse-proxy

# Check container status
docker ps | grep caddy

# View logs
docker logs caddy-reverse-proxy
docker exec caddy-reverse-proxy tail -f /logs/caddy.log

SSL Certificate Troubleshooting

# Check recent certificate acquisition
docker exec caddy-reverse-proxy grep -i "certificate.*obtained" /logs/caddy.log | tail -5

# Force certificate renewal (restart container)
docker restart caddy-reverse-proxy

# Check certificate details
echo | openssl s_client -connect [domain]:443 -servername [domain] 2>/dev/null | openssl x509 -noout -text

Testing and Validation

Domain Testing

# Test HTTP redirect
curl -I http://do.dev

# Test HTTPS
curl -I https://do.dev

# Test all domains
for domain in do.dev contacts.dev doc.dev biturl.dev isup.dev homepage.dev customers.dev sell.dev talk.dev; do
  echo "Testing $domain:"
  curl -I https://$domain --connect-timeout 3
  echo "---"
done

Expected Response Headers

HTTP/2 200 
alt-svc: h3=":443"; ma=2592000
via: 1.1 Caddy
x-powered-by: Next.js

SSL Validation

All domains should have:

✅ Valid Let's Encrypt certificates
✅ HTTP/2 support
✅ No certificate warnings
✅ Automatic HTTP→HTTPS redirect

Troubleshooting

Common Issues

Issue: Domains not working after configuration change

Solution: Reload Caddy configuration

docker exec caddy-reverse-proxy caddy reload --config /etc/caddy/Caddyfile

Issue: SSL certificate errors

Symptoms: SSL certificate problem or handshake failures Solution:

Check Cloudflare API token is valid
Restart container to force certificate renewal
Wait 30-60 seconds for certificate generation

Issue: API sync not applying changes

Root Cause: Sync API uses old JSON API approach Temporary Solution: Manually update /config/domains.caddy Permanent Solution: Update sync API to use Caddyfile generation

Debugging Commands

# Check Caddy configuration syntax
docker exec caddy-reverse-proxy caddy fmt --diff /etc/caddy/Caddyfile

# Validate configuration
docker exec caddy-reverse-proxy caddy validate --config /etc/caddy/Caddyfile

# Check domain resolution
nslookup [domain]

# Test specific upstream
curl -I http://10.1.0.33:[port]

Performance Monitoring

Health Check Status

All upstreams are continuously monitored:

Healthy: 10.1.0.33 infrastructure (primary)
Unknown: 10.3.0.33 infrastructure (may timeout during monitoring)

Log Monitoring

# Monitor real-time health checks
docker exec caddy-reverse-proxy tail -f /logs/caddy.log | grep health_checker

# Monitor certificate renewals
docker exec caddy-reverse-proxy tail -f /logs/caddy.log | grep -i certificate

Migration Notes

From API-based (CADDY.md) to Caddyfile-based (CADDY2.md)

Configuration Source: Database → Caddyfile snippets
Update Method: JSON API → File write + reload
SSL Management: Same (Cloudflare DNS challenge)
Load Balancing: Same (first policy with failover)

Backwards Compatibility

Admin API still accessible on port 2019
Health check endpoint: http://10.3.3.3:2019/health
Monitoring tools can still query Caddy API for status

Future Improvements

Update Sync API: Modify /api/caddy/sync to generate Caddyfile syntax
Automated Deployment: CI/CD integration for configuration updates
Monitoring Dashboard: Real-time health status for all domains
Certificate Alerting: Notification system for certificate renewals

CADDY.md - Previous API-based approach (deprecated)
VERCEL_DEPLOYMENT_GUIDE.md - Application deployment
DNS_MANAGEMENT.md - DNS configuration

Last Updated: July 25, 2025
Status: All 11 domains operational with valid SSL certificates
Next Action: Update sync API to use Caddyfile generation instead of JSON API

On this page