Convex Multi-Project Architecture

🚨 CRITICAL: This document is REQUIRED READING for anyone working with Convex in this monorepo. All Convex-related work must follow this architecture.

Overview

The do.dev stack uses a sophisticated multi-project Convex architecture to separate concerns, enable scalability, and provide isolation between different system components. This architecture is designed to support multiple applications while maintaining clear separation of authentication, local infrastructure, and application-specific data.

🔍 Deployment Verification

Always verify deployments before making changes:

pnpm convex:verify

This runs a verification script that checks all Convex deployments are correctly configured and warns about any misconfigurations that could cause data loss.

Architecture Diagram

┌─────────────────────────────────────────────────────────────────┐
│                    do.dev Convex Architecture                  │
├─────────────────────────────────────────────────────────────────┤
│  Auth Project (dependable-pika-747)                           │
│  ├─ Unified Authentication                                      │
│  ├─ User Management (usr_ prefixed IDs)                        │
│  ├─ Multi-app Authorization                                     │
│  └─ Clerk Integration Bridge                                    │
├─────────────────────────────────────────────────────────────────┤
│  Local Infrastructure Project (agile-cardinal-285)             │
│  ├─ DNS Management                                              │
│  ├─ Caddy Proxy Configuration                                   │
│  ├─ Local Development Tools                                     │
│  └─ Infrastructure Monitoring                                   │
├─────────────────────────────────────────────────────────────────┤
│  Main Data Project (do-dev-all)                               │
│  ├─ Business Logic                                              │
│  ├─ Application Data                                            │
│  ├─ Legacy Domain Tables                                        │
│  └─ [Being migrated to appropriate projects]                   │
├─────────────────────────────────────────────────────────────────┤
│  Individual App Projects                                        │
│  ├─ promptnow (specialized AI prompts)                         │
│  ├─ groktalk-com (communication features)                      │
│  ├─ local-test (development testing)                           │
│  └─ [Future app projects]                                       │
└─────────────────────────────────────────────────────────────────┘

Project Breakdown

1. Auth Project (`dependable-pika-747`)

Purpose: Unified authentication service for all do.dev applications

Location: apps/webs/auth/

Deployment URL: https://dependable-pika-747.convex.cloud

Key Responsibilities:

User Management: Central user store with usr_ prefixed IDs
Customer Management: Customer profiles, billing, subscriptions
Multi-App Authorization: Role-based access across all do.dev apps
Convex Auth Integration: Email OTP, Google OAuth, GitHub OAuth
Clerk Bridge: Synchronization with Clerk satellite authentication
Session Management: Cross-app session handling and refresh tokens

Schema Highlights:

users: {
  userId: "usr_xxxxx",           // Custom ID generation
  apps: [                        // Multi-app access control
    {
      appId: "dodev" | "promptnow" | "groktalk",
      roles: ["user", "admin"],
      firstAccess: timestamp,
      lastAccess: timestamp
    }
  ],
  providers: ["email", "google", "github", "clerk"]
}

customers: {                    // Customer management
  userId: Id<"users">,
  customerId: "cus_xxxxx",      // Custom customer ID
  billingEmail: string,
  subscription: {
    plan: "free" | "pro" | "enterprise",
    status: "active" | "cancelled" | "past_due",
    currentPeriodEnd: timestamp
  },
  paymentMethods: [...],
  invoices: [...]
}

clerkUserMapping: {             // Bridge to Clerk
  clerkId: string,
  userId: Id<"users">,
  userIdStr: string             // The usr_ prefixed string
}

Environment Variables:

# In apps requiring auth project access
NEXT_PUBLIC_CONVEX_DEPLOYMENT_AUTH=https://dependable-pika-747.convex.cloud

2. Local Infrastructure Project (`agile-cardinal-285`)

Purpose: Local development infrastructure and proxy management

Location: packages/convex-local/

Deployment URL: https://agile-cardinal-285.convex.cloud

Key Responsibilities:

DNS Management: Local DNS server configuration and record management
Caddy Proxy: Dynamic reverse proxy configuration and health monitoring
Infrastructure Monitoring: Service health checks and status reporting
Development Tools: Local development environment support

Schema Highlights:

proxy_routes: {
  domain: string,
  upstreamId: Id<"proxy_upstreams">,
  priority: number,
  isActive: boolean
}

proxy_upstreams: {
  name: string,
  addresses: [                  // Multiple upstream addresses
    { host: string, port: number, priority: number }
  ]
}

dns_records: {
  zone: string,
  name: string,
  type: "A" | "CNAME" | "MX",
  value: string
}

Environment Variables:

# In dodev app
CONVEX_DEPLOYMENT_LOCAL=dev:agile-cardinal-285
NEXT_PUBLIC_CONVEX_DEPLOYMENT_LOCAL=https://agile-cardinal-285.convex.cloud

3. Main Data Project (`do-dev-all`)

Purpose: Primary business logic and application data (legacy/migration target)

Note: This project contains legacy domain tables that should be migrated to the local infrastructure project.

Key Responsibilities:

Business Logic: Core application functionality (legacy)
Legacy Infrastructure: Domain management (to be migrated to local infrastructure)
Application State: Legacy application data storage
Migration Target: Being phased out as data moves to appropriate projects

Migration Plan:

Domain-related tables → Local Infrastructure Project
Customer data → Auth Project (already migrated)
User/auth data → Auth Project (already migrated)
App-specific logic → Individual app projects

4. Individual App Projects

promptnow (`apps/webs/promptnow/`)

Purpose: AI prompt management and optimization
Schema: Prompts, templates, AI configurations
Authentication: Integrates with auth project

groktalk-com (`apps/webs/groktalk-com/`)

Purpose: Communication and messaging features
Schema: Messages, conversations, integrations
Authentication: Integrates with auth project

local-test (`apps/webs/local-test/`)

Purpose: Development testing and experimentation
Schema: Test data, metrics, development artifacts
Authentication: Optional, for testing auth flows

# Primary business logic (legacy - being migrated)
CONVEX_DEPLOYMENT=dev:agile-cardinal-285  # team: timmeade, project: local-5d640
NEXT_PUBLIC_CONVEX_URL=https://agile-cardinal-285.convex.cloud

# Local Infrastructure (DNS, Caddy, Dev Tools)
CONVEX_DEPLOYMENT_LOCAL=dev:agile-cardinal-285
NEXT_PUBLIC_CONVEX_DEPLOYMENT_LOCAL=https://agile-cardinal-285.convex.cloud

# Unified Authentication
NEXT_PUBLIC_CONVEX_DEPLOYMENT_AUTH=https://dependable-pika-747.convex.cloud

Individual App Configuration

# App-specific deployment
CONVEX_DEPLOYMENT=dev:your-app-deployment
NEXT_PUBLIC_CONVEX_URL=https://your-app-deployment.convex.cloud

# Shared auth service
NEXT_PUBLIC_CONVEX_DEPLOYMENT_AUTH=https://dependable-pika-747.convex.cloud

Configuration Patterns

1. Multi-Client Setup (dodev app)

// Local infrastructure client
const localConvex = new ConvexHttpClient(
  process.env.NEXT_PUBLIC_CONVEX_DEPLOYMENT_LOCAL!
);

// Auth service client  
const authConvex = new ConvexHttpClient(
  process.env.NEXT_PUBLIC_CONVEX_DEPLOYMENT_AUTH!
);

// Usage examples:
const domains = await localConvex.query("proxy.listRoutes");
const user = await authConvex.mutation("clerkSync:syncClerkUser", {...});

2. Single Client Setup (individual apps)

// Main app client
const convex = new ConvexReactClient(process.env.NEXT_PUBLIC_CONVEX_URL!);

// Auth service client (for user management)
const authConvex = new ConvexHttpClient(
  process.env.NEXT_PUBLIC_CONVEX_DEPLOYMENT_AUTH!
);

Development Workflow

1. Setting Up New Convex Project

# Navigate to your app directory
cd apps/webs/your-app

# Initialize Convex
npx convex dev

# Configure environment variables
echo "CONVEX_DEPLOYMENT=dev:your-deployment-name" >> .env.local
echo "NEXT_PUBLIC_CONVEX_URL=https://your-deployment.convex.cloud" >> .env.local
echo "NEXT_PUBLIC_CONVEX_DEPLOYMENT_AUTH=https://dependable-pika-747.convex.cloud" >> .env.local

2. Authentication Integration

All apps MUST integrate with the auth project:

// User sync on login
const syncResult = await authConvex.mutation("clerkSync:syncClerkUser", {
  clerkUserId,
  email: user.email,
  name: user.name,
  appId: "your-app-id", // Add your app to auth/convex/schema.ts appIds
});

// Check user permissions
const authUser = await authConvex.query("users:getByClerkId", {
  clerkUserId
});

3. Local Infrastructure Access

Only dodev app should access local infrastructure:

// DNS management
const zones = await localConvex.query("dns.listZones");

// Proxy configuration
const routes = await localConvex.query("proxy.listRoutes");

// Health monitoring
const status = await localConvex.query("health.getSystemStatus");

Cross-Project Communication

Authentication Flow

1. User logs in via Clerk (any app)
2. App calls authConvex.mutation("clerkSync:syncClerkUser")
3. Auth project creates/updates user with usr_ prefix
4. App receives user ID for local operations
5. App can query auth project for permissions

Infrastructure Management Flow

1. dodev app manages local infrastructure
2. DNS changes → localConvex.mutation("dns.updateRecord")
3. Proxy config → localConvex.mutation("proxy.updateRoute")
4. Other apps benefit from infrastructure without direct access

✅ Allowed Cross-Project Access

Any app → Auth project (user management)
dodev app → Local infrastructure (admin functions)
Auth project ← Any app (user sync, permissions)

❌ Prohibited Cross-Project Access

Individual apps → Local infrastructure (security isolation)
Apps → Other apps' data (data isolation)
Direct database access (use proper Convex functions)

Security & Permissions

Project Isolation

Each project has separate schemas and cannot access others' data directly
Authentication is centralized but permissions are project-specific
Local infrastructure access is restricted to admin applications

User Role Management

// In auth project schema
apps: [{
  appId: "dodev" | "promptnow" | "groktalk",
  roles: ["user", "admin", "super_admin", "do-root"],
  firstAccess: timestamp,
  lastAccess: timestamp
}]

Environment Security

Never hardcode deployment URLs in source code
Use environment variables for all Convex endpoints
Separate development and production deployments
Rotate secrets regularly and update environment configs

Deployment Guide

Production Deployment Checklist

Environment Variables:
- NEXT_PUBLIC_CONVEX_DEPLOYMENT_AUTH configured
- App-specific NEXT_PUBLIC_CONVEX_URL configured
- Local infrastructure URLs (dodev only)
Authentication Setup:
- App ID added to auth project schema
- User sync endpoint implemented
- Role-based access configured
Cross-Project Testing:
- User authentication flow tested
- Permissions verified across projects
- Infrastructure access tested (dodev only)
Monitoring Setup:
- Health checks configured
- Error logging implemented
- Performance monitoring enabled

// Problem: User sync failing
// Solution: Check auth project deployment URL
const authConvex = new ConvexHttpClient(
  process.env.NEXT_PUBLIC_CONVEX_DEPLOYMENT_AUTH! // Must be correct
);

2. Missing Environment Variables

# Check all required variables are set
echo $NEXT_PUBLIC_CONVEX_DEPLOYMENT_AUTH
echo $NEXT_PUBLIC_CONVEX_URL
echo $NEXT_PUBLIC_CONVEX_DEPLOYMENT_LOCAL  # dodev only

3. Project Permission Issues

// Ensure app ID is registered in auth project
// Check apps/webs/auth/convex/schema.ts appIds union

Debug Commands

# Check auth project functions
cd apps/webs/auth
npx convex run users:list

# Check local infrastructure
cd packages/convex-local  
npx convex run proxy:listRoutes

# Check individual app
cd apps/webs/your-app
npx convex run your-function

Migration Guide

Moving from Single to Multi-Project

Identify Data Boundaries:
- Authentication data → Auth project
- Infrastructure data → Local infrastructure project
- App-specific data → Individual app projects

Create New Projects:

npx convex dev  # In new project directory

Update Environment Variables:
- Add auth project URLs
- Update infrastructure URLs (if applicable)
- Test cross-project communication
Migrate Functions:
- Move auth functions → Auth project
- Move infrastructure functions → Local project
- Update client calls to use appropriate projects
Test Integration:
- User authentication flows
- Cross-project data access
- Permission enforcement

Best Practices

1. Project Responsibility Separation

Auth project: Authentication, user management, and customer data
Infrastructure project: Only local development tools and DNS/proxy
App projects: Only app-specific business logic
Main project (do-dev-all): Legacy data being migrated
Never mix concerns across project boundaries

2. Environment Management

Use consistent naming for environment variables
Document all required variables for each app
Test with missing variables to ensure graceful degradation

3. Error Handling

// Always handle cross-project failures gracefully
try {
  const user = await authConvex.mutation("clerkSync:syncClerkUser", data);
} catch (error) {
  console.error("Auth sync failed:", error);
  // Implement fallback behavior
}

4. Performance Optimization

Cache cross-project calls when appropriate
Batch operations to reduce network calls
Use appropriate timeouts for cross-project communication

5. Development Workflow

Start with auth integration for new apps
Test cross-project communication early
Document app-specific configuration requirements
Use consistent patterns across all applications

Future Considerations

Planned Improvements

Production Environment: Separate production deployments for each project
Automated Deployment: CI/CD pipelines for each Convex project
Enhanced Monitoring: Cross-project observability and alerting
Schema Validation: Automated validation of cross-project interfaces

Scaling Patterns

Regional Deployments: Geo-distributed Convex projects
Load Balancing: Multiple instances of high-traffic projects
Data Partitioning: Horizontal scaling strategies
Caching Layer: Redis integration for frequently accessed data

Summary

This multi-project Convex architecture provides:

Clear separation of concerns between authentication/customers, infrastructure, and applications
Scalable foundation for multiple do.dev applications
Security isolation between different system components
Unified authentication and customer management across all applications
Centralized infrastructure management for local development
Phased migration path from legacy monolithic structure

Remember: This architecture is designed for long-term scalability and maintainability. Follow the patterns established here for all new Convex development.

On this page