Staff-Only Production to Development Sync Feature

Overview

This feature allows staff developers to reset their development environment by syncing all data from their production dashboard. This is particularly useful for:

• Testing with real production data
• Recovering from a broken development state
• Testing migration scripts and updates safely
• Maintaining a consistent testing environment

How It Works

Architecture

The sync feature consists of three main components:

1. Data Export (exportUserData): Exports all user data from the source environment
2. Data Clear (clearDevData): Removes all existing data in development
3. Data Import (importProductionData): Imports the exported data into development

Data Included in Sync

The sync process copies the following data:

• User preferences and settings
• Organizations and memberships
• Homepages and their configurations
• Pages and page settings
• Tabs and tab layouts
• Apps (blocks) and their positions
• App types/templates
• Shared resources

Security

• Staff-Only Access: Only emails listed in STAFF_EMAILS array can access this feature
• Environment Check: Import/clear operations only work in development environment
• Confirmation Required: Users must confirm the destructive action before proceeding
• Authentication: Cross-deployment sync requires secure token authentication

Setup Instructions

1. Add Staff Emails

Edit /apps/homepage/convex/staffSync.ts and add staff developer emails:

const STAFF_EMAILS = [
  "tim@dodot.dev",
  "jane@dodot.dev",  // Add your staff emails here
  // Add more staff emails as needed
]

2. Configure Environment Variables (Optional)

For cross-deployment sync, add to your .env.local:

# Secret for cross-deployment data export
EXPORT_SECRET=your-secure-secret-here

# Production Convex deployment URL (if different from development)
PRODUCTION_CONVEX_URL=https://your-production-deployment.convex.cloud

3. Deploy to Convex

npx convex dev

Usage

Accessing the Feature

1. Open your development dashboard
2. Click on the user menu or settings icon
3. Open "Preferences" modal
4. If you're a staff member, you'll see "Staff Developer Tools" section
5. Click "Reset Development from Production"

What Happens During Sync

1. Confirmation: A warning dialog explains the destructive nature of the operation
2. Export: All your production data is exported
3. Clear: Your development database is cleared
4. Import: Production data is imported with new IDs
5. Reload: Page automatically reloads to show new data

Important Notes

⚠️ WARNING: This operation is destructive and cannot be undone!

• All current development data will be permanently deleted
• The sync is one-way (production → development only)
• New IDs are generated for all imported data
• Relationships between entities are preserved

Technical Implementation

Files Modified/Created

1. /apps/homepage/convex/staffSync.ts

   • Main sync logic and queries/mutations
   • Staff authentication
   • Data export/import functions

2. /apps/homepage/convex/http.ts

   • HTTP endpoint for cross-deployment communication
   • Secure token authentication
   • Health check endpoint

3. /apps/homepage/components/dashboard/UserPreferencesModal.tsx

   • UI component for triggering sync
   • Staff-only section with warnings
   • Progress feedback and error handling

Key Functions

isStaff

Query to check if current user is a staff member.

exportUserData / exportUserDataByEmail

Exports all user data including organizations, homepages, pages, tabs, apps, and resources.

clearDevData

Safely removes all user data in development environment only.

importProductionData

Imports previously exported data, creating new IDs and maintaining relationships.

syncProductionToDevelopment

Main action that orchestrates the entire sync process.

Cross-Deployment Setup (Advanced)

For separate production and development Convex deployments:

Production Setup

1. Deploy the HTTP endpoint to production
2. Set EXPORT_SECRET environment variable
3. Ensure staff emails are configured

Development Setup

1. Update syncProductionToDevelopment action with production URL:

const result = await syncProductionToDev({
  productionUrl: "https://your-production-url.com",
  productionDeploymentName: "production:your-app-name",
})

2. Implement HTTP client to fetch from production endpoint:

const response = await fetch(`${productionUrl}/export-user-data`, {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({
    authToken: process.env.EXPORT_SECRET,
    userEmail: identity.email,
  }),
})

Troubleshooting

"Unauthorized: Staff access only"

• Ensure your email is in the STAFF_EMAILS array
• Check that you're logged in with the correct account

"This operation is only allowed in development environment"

• Verify process.env.CONVEX_DEPLOYMENT === "development"
• Check you're not trying to run this in production

Data Not Appearing After Sync

• Check browser console for errors
• Verify all data was exported successfully
• Ensure import completed without errors
• Try manual page refresh

Missing Relationships

• Check ID mapping logic in import function
• Verify all related entities were exported
• Ensure import order is correct (organizations → homepages → pages → tabs → apps)

Future Enhancements

Potential improvements for this feature:

1. Selective Sync: Choose specific pages/tabs to sync
2. Backup Before Clear: Automatic backup of development data
3. Sync History: Track when and what was synced
4. Incremental Sync: Only sync changes since last sync
5. Two-Way Sync: Optionally push development changes to production
6. Team Sync: Allow syncing between team members' environments
7. Scheduled Sync: Automatic nightly sync for fresh data

Security Considerations

• Never expose the EXPORT_SECRET in client code
• Regularly rotate the export secret
• Audit staff email list regularly
• Consider implementing audit logs for sync operations
• Use HTTPS for all cross-deployment communication
• Implement rate limiting to prevent abuse