Testing Plan - customers.dev

Comprehensive testing strategy for the customers.dev platform covering all layers from unit tests to end-to-end testing.

Table of Contents

  1. Overview
  2. Testing Strategy
  3. Test Coverage Requirements
  4. Unit Testing
  5. Integration Testing
  6. API Testing
  7. Security Testing
  8. Performance Testing
  9. E2E Testing
  10. Test Data Management
  11. CI/CD Integration

Overview

Testing Objectives

  1. Functional Correctness: Verify all features work as specified
  2. Security: Ensure RBAC, authentication, and data isolation work correctly
  3. Performance: Validate response times and rate limiting
  4. Reliability: Ensure system handles errors gracefully
  5. Regression Prevention: Catch breaking changes early

Testing Pyramid

           ┌─────────────┐
           │   E2E (5%)  │  ← Browser-based user workflows
           └─────────────┘
         ┌─────────────────┐
         │ Integration (20%)│ ← API endpoints, auth flows
         └─────────────────┘
     ┌───────────────────────┐
     │   Unit Tests (75%)    │ ← Functions, utilities, RBAC
     └───────────────────────┘

Testing Strategy

Phase 1: Foundation (Week 1-2)

  • ✅ RBAC unit tests
  • ✅ Convex function tests
  • 🔄 HTTP Action tests
  • 🔄 API endpoint integration tests

Phase 2: API Coverage (Week 2-4)

  • REST API endpoint tests
  • Authentication flow tests
  • Rate limiting tests
  • Error handling tests

Phase 3: Dashboard (Week 4-6)

  • Component tests
  • Integration tests
  • E2E user workflows
  • Accessibility tests

Phase 4: Production Readiness (Week 6-12)

  • Performance benchmarks
  • Load testing
  • Security audits
  • Chaos engineering

Test Coverage Requirements

Minimum Coverage Targets

LayerTargetCritical Paths
Unit Tests≥80%≥95%
Integration Tests≥70%≥90%
API Tests≥90%100%
E2E TestsCritical flows100%

Critical Paths Defined

Must Have 100% Coverage:

  • Authentication and authorization
  • API key validation
  • Rate limiting
  • Payment processing (future)
  • Data deletion/privacy operations

Unit Testing

RBAC System Tests

File: apps/web/convex/rbac.test.ts

import { describe, it, expect } from "vitest"
import { hasPermission, PERMISSIONS } from "./rbac"

describe("RBAC System", () => {
  describe("hasPermission", () => {
    it("should allow viewer to READ", () => {
      expect(hasPermission("viewer", PERMISSIONS.READ)).toBe(true)
    })

    it("should deny viewer WRITE permission", () => {
      expect(hasPermission("viewer", PERMISSIONS.WRITE)).toBe(false)
    })

    it("should allow member to READ and WRITE", () => {
      expect(hasPermission("member", PERMISSIONS.READ)).toBe(true)
      expect(hasPermission("member", PERMISSIONS.WRITE)).toBe(true)
    })

    it("should deny member DELETE permission", () => {
      expect(hasPermission("member", PERMISSIONS.DELETE)).toBe(false)
    })

    it("should allow admin all permissions except MANAGE_ORG", () => {
      expect(hasPermission("admin", PERMISSIONS.READ)).toBe(true)
      expect(hasPermission("admin", PERMISSIONS.WRITE)).toBe(true)
      expect(hasPermission("admin", PERMISSIONS.DELETE)).toBe(true)
      expect(hasPermission("admin", PERMISSIONS.MANAGE_MEMBERS)).toBe(true)
      expect(hasPermission("admin", PERMISSIONS.MANAGE_API_KEYS)).toBe(true)
      expect(hasPermission("admin", PERMISSIONS.MANAGE_ORG)).toBe(false)
    })

    it("should allow owner all permissions", () => {
      expect(hasPermission("owner", PERMISSIONS.READ)).toBe(true)
      expect(hasPermission("owner", PERMISSIONS.WRITE)).toBe(true)
      expect(hasPermission("owner", PERMISSIONS.DELETE)).toBe(true)
      expect(hasPermission("owner", PERMISSIONS.MANAGE_MEMBERS)).toBe(true)
      expect(hasPermission("owner", PERMISSIONS.MANAGE_API_KEYS)).toBe(true)
      expect(hasPermission("owner", PERMISSIONS.MANAGE_ORG)).toBe(true)
    })
  })
})

Convex Function Tests

File: apps/web/convex/customers.test.ts

import { convexTest } from "convex-test"
import { describe, it, expect, beforeEach } from "vitest"
import schema from "./schema"
import { api } from "./_generated/api"

describe("Customer Functions", () => {
  let t: ConvexTestingHelper

  beforeEach(async () => {
    t = convexTest(schema)

    // Seed test data
    const orgId = await t.run(async (ctx) => {
      return await ctx.db.insert("organizations", {
        name: "Test Org",
        ownerId: "user_123",
        plan: "free",
        settings: {
          allowPublicProfiles: false,
          requireTwoFactor: false,
          maxMembers: 5,
        },
        createdAt: Date.now(),
        updatedAt: Date.now(),
      })
    })

    // Create user and membership
    await t.run(async (ctx) => {
      const userId = await ctx.db.insert("users", {
        clerkId: "user_123",
        email: "test@example.com",
        firstName: "Test",
        lastName: "User",
        createdAt: Date.now(),
      })

      await ctx.db.insert("organizationMembers", {
        orgId,
        userId,
        role: "owner",
        status: "active",
        joinedAt: Date.now(),
      })
    })
  })

  describe("createCustomer", () => {
    it("should create customer with valid permissions", async () => {
      const customer = await t.mutation(api.customers.createCustomer, {
        orgId: t.orgId,
        name: "Acme Corp",
        domain: "acme.com",
        status: "customer",
      })

      expect(customer).toBeDefined()
      expect(customer.name).toBe("Acme Corp")
    })

    it("should require WRITE permission", async () => {
      // Change user role to viewer
      await t.run(async (ctx) => {
        const membership = await ctx.db
          .query("organizationMembers")
          .withIndex("by_user")
          .first()
        await ctx.db.patch(membership._id, { role: "viewer" })
      })

      await expect(
        t.mutation(api.customers.createCustomer, {
          orgId: t.orgId,
          name: "Acme Corp",
          status: "customer",
        })
      ).rejects.toThrow("Insufficient permissions")
    })

    it("should enforce organization isolation", async () => {
      const otherOrgId = await t.run(async (ctx) => {
        return await ctx.db.insert("organizations", {
          name: "Other Org",
          ownerId: "user_456",
          plan: "free",
          settings: { allowPublicProfiles: false, requireTwoFactor: false, maxMembers: 5 },
          createdAt: Date.now(),
          updatedAt: Date.now(),
        })
      })

      await expect(
        t.mutation(api.customers.createCustomer, {
          orgId: otherOrgId,
          name: "Acme Corp",
          status: "customer",
        })
      ).rejects.toThrow("User is not a member of this organization")
    })
  })

  describe("getCustomers", () => {
    it("should return customers for organization", async () => {
      // Create test customers
      await t.mutation(api.customers.createCustomer, {
        orgId: t.orgId,
        name: "Acme Corp",
        status: "customer",
      })

      await t.mutation(api.customers.createCustomer, {
        orgId: t.orgId,
        name: "TechStart",
        status: "lead",
      })

      const customers = await t.query(api.customers.getCustomers, {
        orgId: t.orgId,
      })

      expect(customers).toHaveLength(2)
      expect(customers[0].name).toBe("TechStart") // Most recent first
      expect(customers[1].name).toBe("Acme Corp")
    })

    it("should filter by status", async () => {
      await t.mutation(api.customers.createCustomer, {
        orgId: t.orgId,
        name: "Customer A",
        status: "customer",
      })

      await t.mutation(api.customers.createCustomer, {
        orgId: t.orgId,
        name: "Lead B",
        status: "lead",
      })

      const customers = await t.query(api.customers.getCustomers, {
        orgId: t.orgId,
        status: "customer",
      })

      expect(customers).toHaveLength(1)
      expect(customers[0].name).toBe("Customer A")
    })

    it("should not return other org's customers", async () => {
      const otherOrgId = await t.run(async (ctx) => {
        return await ctx.db.insert("organizations", {
          name: "Other Org",
          ownerId: "user_456",
          plan: "free",
          settings: { allowPublicProfiles: false, requireTwoFactor: false, maxMembers: 5 },
          createdAt: Date.now(),
          updatedAt: Date.now(),
        })
      })

      // Create customer in other org
      await t.run(async (ctx) => {
        await ctx.db.insert("customers", {
          orgId: otherOrgId,
          name: "Other Customer",
          status: "customer",
          createdBy: "user_456",
          updatedBy: "user_456",
          createdAt: Date.now(),
          updatedAt: Date.now(),
        })
      })

      const customers = await t.query(api.customers.getCustomers, {
        orgId: t.orgId,
      })

      expect(customers).toHaveLength(0)
    })
  })
})

API Key Validation Tests

File: apps/web/convex/apiKeys.test.ts

import { convexTest } from "convex-test"
import { describe, it, expect, beforeEach } from "vitest"
import { api } from "./_generated/api"

describe("API Key Functions", () => {
  let t: ConvexTestingHelper
  let apiKey: string

  beforeEach(async () => {
    t = convexTest(schema)
    // Setup org, user, membership...

    // Create API key
    const result = await t.mutation(api.apiKeys.createApiKey, {
      orgId: t.orgId,
      name: "Test Key",
      permissions: ["customers:read", "customers:write"],
      requestsPerMinute: 60,
    })

    apiKey = result.apiKey
  })

  describe("validateApiKey", () => {
    it("should validate correct API key", async () => {
      const result = await t.query(api.apiKeys.validateApiKey, {
        apiKey,
      })

      expect(result.valid).toBe(true)
      expect(result.permissions).toEqual(["customers:read", "customers:write"])
    })

    it("should reject invalid API key", async () => {
      const result = await t.query(api.apiKeys.validateApiKey, {
        apiKey: "sk_test_invalid",
      })

      expect(result.valid).toBe(false)
      expect(result.error).toBe("Invalid API key")
    })

    it("should reject revoked API key", async () => {
      // Revoke the key
      await t.mutation(api.apiKeys.revokeApiKey, {
        id: result.keyId,
      })

      const validation = await t.query(api.apiKeys.validateApiKey, {
        apiKey,
      })

      expect(validation.valid).toBe(false)
      expect(validation.error).toBe("API key is not active")
    })

    it("should reject expired API key", async () => {
      // Create key with past expiration
      const expiredResult = await t.run(async (ctx) => {
        const yesterday = Date.now() - 24 * 60 * 60 * 1000
        return await ctx.db.insert("apiKeys", {
          orgId: t.orgId,
          name: "Expired Key",
          keyHash: "hash",
          keyPrefix: "sk_test_exp",
          permissions: ["customers:read"],
          rateLimit: { requestsPerMinute: 60, requestsPerHour: 1000, requestsPerDay: 10000 },
          status: "active",
          expiresAt: yesterday,
          createdBy: t.userId,
          createdAt: Date.now(),
          updatedAt: Date.now(),
        })
      })

      const validation = await t.query(api.apiKeys.validateApiKey, {
        apiKey: "sk_test_exp_fake",
      })

      expect(validation.valid).toBe(false)
      expect(validation.error).toBe("API key has expired")
    })
  })

  describe("checkRateLimit", () => {
    it("should allow requests under limit", async () => {
      const result = await t.query(api.apiKeys.checkRateLimit, {
        apiKeyId: t.apiKeyId,
      })

      expect(result.allowed).toBe(true)
      expect(result.remaining.perMinute).toBe(60)
    })

    it("should reject when rate limit exceeded", async () => {
      // Track 60 requests
      for (let i = 0; i < 60; i++) {
        await t.mutation(api.apiKeys.trackApiKeyUsage, {
          apiKeyId: t.apiKeyId,
          orgId: t.orgId,
          endpoint: "/customers",
          method: "GET",
          statusCode: 200,
        })
      }

      const result = await t.query(api.apiKeys.checkRateLimit, {
        apiKeyId: t.apiKeyId,
      })

      expect(result.allowed).toBe(false)
      expect(result.error).toContain("Rate limit exceeded")
      expect(result.resetAt).toBeDefined()
    })
  })
})

Integration Testing

API Endpoint Tests

File: tests/integration/api.test.ts

import { describe, it, expect, beforeAll, afterAll } from "vitest"

describe("REST API Endpoints", () => {
  let apiKey: string
  const baseURL = "http://localhost:3000/api"

  beforeAll(async () => {
    // Setup test organization and API key
    apiKey = await setupTestAPIKey()
  })

  afterAll(async () => {
    // Cleanup test data
    await cleanupTestData()
  })

  describe("GET /customers", () => {
    it("should return customers with valid API key", async () => {
      const response = await fetch(`${baseURL}/customers`, {
        headers: {
          "Authorization": `Bearer ${apiKey}`,
        },
      })

      expect(response.status).toBe(200)
      expect(response.headers.get("X-RateLimit-Limit")).toBe("60")

      const data = await response.json()
      expect(data).toHaveProperty("data")
      expect(Array.isArray(data.data)).toBe(true)
    })

    it("should return 401 without API key", async () => {
      const response = await fetch(`${baseURL}/customers`)

      expect(response.status).toBe(401)
      const data = await response.json()
      expect(data.error).toBe("Missing API key")
    })

    it("should return 401 with invalid API key", async () => {
      const response = await fetch(`${baseURL}/customers`, {
        headers: {
          "Authorization": "Bearer sk_test_invalid",
        },
      })

      expect(response.status).toBe(401)
      const data = await response.json()
      expect(data.error).toBe("Invalid API key")
    })

    it("should support pagination", async () => {
      const response = await fetch(`${baseURL}/customers?limit=10`, {
        headers: {
          "Authorization": `Bearer ${apiKey}`,
        },
      })

      const data = await response.json()
      expect(data).toHaveProperty("cursor")
      expect(data).toHaveProperty("hasMore")
    })
  })

  describe("POST /customers", () => {
    it("should create customer with valid data", async () => {
      const response = await fetch(`${baseURL}/customers`, {
        method: "POST",
        headers: {
          "Authorization": `Bearer ${apiKey}`,
          "Content-Type": "application/json",
        },
        body: JSON.stringify({
          name: "Test Corp",
          domain: "test.com",
          status: "lead",
        }),
      })

      expect(response.status).toBe(201)
      const data = await response.json()
      expect(data.id).toBeDefined()
      expect(data.name).toBe("Test Corp")
    })

    it("should return 400 with invalid data", async () => {
      const response = await fetch(`${baseURL}/customers`, {
        method: "POST",
        headers: {
          "Authorization": `Bearer ${apiKey}`,
          "Content-Type": "application/json",
        },
        body: JSON.stringify({
          // Missing required 'name' field
          status: "lead",
        }),
      })

      expect(response.status).toBe(400)
      const data = await response.json()
      expect(data.error).toContain("validation")
    })

    it("should return 403 without write permission", async () => {
      const readOnlyKey = await createReadOnlyAPIKey()

      const response = await fetch(`${baseURL}/customers`, {
        method: "POST",
        headers: {
          "Authorization": `Bearer ${readOnlyKey}`,
          "Content-Type": "application/json",
        },
        body: JSON.stringify({
          name: "Test Corp",
          status: "lead",
        }),
      })

      expect(response.status).toBe(403)
      const data = await response.json()
      expect(data.error).toContain("Insufficient permissions")
    })

    it("should return 429 when rate limited", async () => {
      // Make 61 requests (exceeds 60/minute limit)
      const requests = []
      for (let i = 0; i < 61; i++) {
        requests.push(
          fetch(`${baseURL}/customers`, {
            headers: { "Authorization": `Bearer ${apiKey}` },
          })
        )
      }

      const responses = await Promise.all(requests)
      const lastResponse = responses[responses.length - 1]

      expect(lastResponse.status).toBe(429)
      expect(lastResponse.headers.get("Retry-After")).toBeDefined()
    })
  })
})

API Testing

Authentication Flow Tests

File: tests/api/auth.test.ts

describe("Authentication Flows", () => {
  describe("API Key Authentication", () => {
    it("should accept valid Bearer token", async () => {
      const response = await makeAuthenticatedRequest()
      expect(response.status).not.toBe(401)
    })

    it("should reject missing Authorization header", async () => {
      const response = await fetch(endpoint)
      expect(response.status).toBe(401)
    })

    it("should reject malformed Authorization header", async () => {
      const response = await fetch(endpoint, {
        headers: { "Authorization": "InvalidFormat" },
      })
      expect(response.status).toBe(401)
    })

    it("should reject expired API key", async () => {
      const expiredKey = await createExpiredAPIKey()
      const response = await fetch(endpoint, {
        headers: { "Authorization": `Bearer ${expiredKey}` },
      })
      expect(response.status).toBe(401)
      expect(await response.json()).toMatchObject({
        error: "API key has expired",
      })
    })
  })

  describe("Permission Enforcement", () => {
    it("should enforce customers:read permission", async () => {
      const writeOnlyKey = await createAPIKey({ permissions: ["customers:write"] })
      const response = await fetch(`${baseURL}/customers`, {
        headers: { "Authorization": `Bearer ${writeOnlyKey}` },
      })
      expect(response.status).toBe(403)
    })

    it("should enforce customers:write permission", async () => {
      const readOnlyKey = await createAPIKey({ permissions: ["customers:read"] })
      const response = await fetch(`${baseURL}/customers`, {
        method: "POST",
        headers: {
          "Authorization": `Bearer ${readOnlyKey}`,
          "Content-Type": "application/json",
        },
        body: JSON.stringify({ name: "Test" }),
      })
      expect(response.status).toBe(403)
    })
  })
})

Rate Limiting Tests

File: tests/api/rate-limiting.test.ts

describe("Rate Limiting", () => {
  it("should enforce per-minute rate limit", async () => {
    const apiKey = await createAPIKey({ requestsPerMinute: 5 })

    // Make 5 successful requests
    for (let i = 0; i < 5; i++) {
      const response = await makeRequest(apiKey)
      expect(response.status).toBe(200)
    }

    // 6th request should be rate limited
    const response = await makeRequest(apiKey)
    expect(response.status).toBe(429)
    expect(response.headers.get("X-RateLimit-Limit")).toBe("5")
    expect(response.headers.get("X-RateLimit-Used")).toBe("5")
    expect(response.headers.get("X-RateLimit-Reset")).toBeDefined()
  })

  it("should reset rate limit after window", async () => {
    const apiKey = await createAPIKey({ requestsPerMinute: 2 })

    // Exhaust limit
    await makeRequest(apiKey)
    await makeRequest(apiKey)

    let response = await makeRequest(apiKey)
    expect(response.status).toBe(429)

    // Wait for window to reset
    await new Promise(resolve => setTimeout(resolve, 61000))

    // Should work again
    response = await makeRequest(apiKey)
    expect(response.status).toBe(200)
  })

  it("should track per-hour rate limit separately", async () => {
    const apiKey = await createAPIKey({
      requestsPerMinute: 100,
      requestsPerHour: 10,
    })

    // Make 10 requests over 2 minutes (within hour limit)
    for (let i = 0; i < 10; i++) {
      if (i === 5) await new Promise(resolve => setTimeout(resolve, 61000))
      const response = await makeRequest(apiKey)
      expect(response.status).toBe(200)
    }

    // 11th request in same hour should fail
    const response = await makeRequest(apiKey)
    expect(response.status).toBe(429)
    expect(await response.json()).toMatchObject({
      error: "Rate limit exceeded for requests per hour",
    })
  })
})

Security Testing

RBAC Security Tests

File: tests/security/rbac.test.ts

describe("RBAC Security", () => {
  describe("Organization Isolation", () => {
    it("should prevent cross-org data access", async () => {
      const org1 = await createOrganization("Org 1")
      const org2 = await createOrganization("Org 2")

      const customer = await createCustomer(org1.id, "Secret Customer")

      // User from org2 should not see org1's customer
      const org2User = await createUser(org2.id, "member")
      const customers = await getCustomersAs(org2User, org1.id)

      expect(customers).toHaveLength(0) // Or throw error
    })

    it("should prevent cross-org updates", async () => {
      const org1 = await createOrganization("Org 1")
      const org2 = await createOrganization("Org 2")

      const customer = await createCustomer(org1.id, "Customer")

      const org2User = await createUser(org2.id, "admin")

      await expect(
        updateCustomerAs(org2User, customer.id, { name: "Hacked" })
      ).rejects.toThrow("User is not a member of this organization")
    })
  })

  describe("Permission Enforcement", () => {
    it("should prevent viewer from creating records", async () => {
      const viewer = await createUser(org.id, "viewer")

      await expect(
        createCustomerAs(viewer, { name: "Test" })
      ).rejects.toThrow("Insufficient permissions")
    })

    it("should prevent member from deleting records", async () => {
      const member = await createUser(org.id, "member")
      const customer = await createCustomer(org.id, "Test")

      await expect(
        deleteCustomerAs(member, customer.id)
      ).rejects.toThrow("Insufficient permissions")
    })

    it("should prevent admin from changing org settings", async () => {
      const admin = await createUser(org.id, "admin")

      await expect(
        updateOrganizationAs(admin, org.id, { plan: "enterprise" })
      ).rejects.toThrow("Insufficient permissions")
    })
  })

  describe("API Key Security", () => {
    it("should not return full API key after creation", async () => {
      const keys = await getAPIKeys(org.id)
      expect(keys.every(k => !k.keyHash)).toBe(true)
      expect(keys.every(k => k.keyPrefix)).toBe(true)
    })

    it("should hash API keys in database", async () => {
      const { apiKey, keyId } = await createAPIKey()
      const storedKey = await getStoredAPIKey(keyId)

      expect(storedKey.keyHash).not.toBe(apiKey)
      expect(storedKey.keyHash).toMatch(/^[A-Za-z0-9+/=]+$/) // Base64
    })

    it("should validate API key permissions", async () => {
      const readOnlyKey = await createAPIKey({
        permissions: ["customers:read"],
      })

      const result = await validateAPIKey(readOnlyKey)
      expect(result.permissions).toEqual(["customers:read"])
      expect(result.permissions).not.toContain("customers:write")
    })
  })
})

Input Validation Tests

File: tests/security/validation.test.ts

describe("Input Validation", () => {
  describe("Customer Creation", () => {
    it("should reject invalid email", async () => {
      await expect(
        createCustomer({ name: "Test", email: "invalid-email" })
      ).rejects.toThrow("Invalid email format")
    })

    it("should reject XSS attempts in name", async () => {
      const maliciousName = '<script>alert("XSS")</script>'
      const customer = await createCustomer({ name: maliciousName })

      // Should be sanitized
      expect(customer.name).not.toContain("<script>")
    })

    it("should reject SQL injection attempts", async () => {
      const maliciousInput = "Test'; DROP TABLE customers; --"
      const customer = await createCustomer({ name: maliciousInput })

      // Should be safely stored (Convex handles this)
      expect(customer.name).toBe(maliciousInput) // Stored as-is, safe in Convex
    })
  })

  describe("API Key Creation", () => {
    it("should reject invalid permission values", async () => {
      await expect(
        createAPIKey({ permissions: ["invalid:permission"] })
      ).rejects.toThrow("Invalid permission")
    })

    it("should reject negative rate limits", async () => {
      await expect(
        createAPIKey({ requestsPerMinute: -1 })
      ).rejects.toThrow("Rate limit must be positive")
    })
  })
})

Performance Testing

Load Testing

File: tests/performance/load.test.ts

import { check } from "k6"
import http from "k6/http"

export const options = {
  stages: [
    { duration: "2m", target: 100 },  // Ramp up to 100 users
    { duration: "5m", target: 100 },  // Stay at 100 users
    { duration: "2m", target: 200 },  // Ramp up to 200 users
    { duration: "5m", target: 200 },  // Stay at 200 users
    { duration: "2m", target: 0 },    // Ramp down
  ],
  thresholds: {
    http_req_duration: ["p(95)<500"], // 95% of requests under 500ms
    http_req_failed: ["rate<0.01"],   // Less than 1% errors
  },
}

export default function () {
  const response = http.get("https://api.customers.dev/v1/customers", {
    headers: {
      "Authorization": `Bearer ${__ENV.API_KEY}`,
    },
  })

  check(response, {
    "status is 200": (r) => r.status === 200,
    "response time < 500ms": (r) => r.timings.duration < 500,
  })
}

Response Time Benchmarks

File: tests/performance/benchmarks.test.ts

describe("Response Time Benchmarks", () => {
  it("GET /customers should respond within 200ms", async () => {
    const start = Date.now()
    await fetch(`${baseURL}/customers`, {
      headers: { "Authorization": `Bearer ${apiKey}` },
    })
    const duration = Date.now() - start

    expect(duration).toBeLessThan(200)
  })

  it("POST /customers should respond within 300ms", async () => {
    const start = Date.now()
    await fetch(`${baseURL}/customers`, {
      method: "POST",
      headers: {
        "Authorization": `Bearer ${apiKey}`,
        "Content-Type": "application/json",
      },
      body: JSON.stringify({ name: "Test" }),
    })
    const duration = Date.now() - start

    expect(duration).toBeLessThan(300)
  })

  it("should handle 1000 concurrent requests", async () => {
    const requests = Array(1000).fill(null).map(() =>
      fetch(`${baseURL}/customers`, {
        headers: { "Authorization": `Bearer ${apiKey}` },
      })
    )

    const start = Date.now()
    const responses = await Promise.all(requests)
    const duration = Date.now() - start

    expect(responses.every(r => r.ok)).toBe(true)
    expect(duration).toBeLessThan(5000) // 5 seconds for 1000 requests
  })
})

E2E Testing

User Workflows

File: tests/e2e/workflows.spec.ts (Playwright)

import { test, expect } from "@playwright/test"

test.describe("Customer Management Workflow", () => {
  test("should complete full customer lifecycle", async ({ page }) => {
    // 1. Login
    await page.goto("https://customers.dev/sign-in")
    await page.fill('[name="email"]', "test@example.com")
    await page.fill('[name="password"]', "password123")
    await page.click('button[type="submit"]')

    // 2. Navigate to customers
    await page.click('a[href="/customers"]')
    await expect(page).toHaveURL(/.*\/customers/)

    // 3. Create new customer
    await page.click('button:has-text("Add Customer")')
    await page.fill('[name="name"]', "Acme Corp")
    await page.fill('[name="domain"]', "acme.com")
    await page.selectOption('[name="status"]', "customer")
    await page.click('button:has-text("Save")')

    // 4. Verify customer created
    await expect(page.locator('text=Acme Corp')).toBeVisible()

    // 5. Add contact
    await page.click('text=Acme Corp')
    await page.click('button:has-text("Add Contact")')
    await page.fill('[name="firstName"]', "John")
    await page.fill('[name="lastName"]', "Doe")
    await page.fill('[name="email"]', "john@acme.com")
    await page.click('button:has-text("Save")')

    // 6. Verify contact created
    await expect(page.locator('text=John Doe')).toBeVisible()

    // 7. Update customer status
    await page.click('button:has-text("Edit")')
    await page.selectOption('[name="status"]', "churned")
    await page.click('button:has-text("Save")')

    // 8. Verify status updated
    await expect(page.locator('text=Churned')).toBeVisible()
  })

  test("should enforce viewer permissions", async ({ page }) => {
    // Login as viewer
    await loginAs(page, "viewer@example.com")

    // Navigate to customers
    await page.goto("/customers")

    // Should see customers
    await expect(page.locator('[data-testid="customer-list"]')).toBeVisible()

    // Should NOT see "Add Customer" button
    await expect(page.locator('button:has-text("Add Customer")')).not.toBeVisible()

    // Should NOT see "Delete" buttons
    await expect(page.locator('button:has-text("Delete")')).not.toBeVisible()
  })
})

test.describe("API Key Management", () => {
  test("should create and use API key", async ({ page, context }) => {
    // 1. Login as admin
    await loginAs(page, "admin@example.com")

    // 2. Navigate to API keys
    await page.goto("/settings/api-keys")

    // 3. Create API key
    await page.click('button:has-text("Create API Key")')
    await page.fill('[name="name"]', "Test Key")
    await page.check('[name="permissions"][value="customers:read"]')
    await page.check('[name="permissions"][value="customers:write"]')
    await page.click('button:has-text("Create")')

    // 4. Copy API key (shown only once)
    const apiKey = await page.locator('[data-testid="api-key-value"]').textContent()
    expect(apiKey).toMatch(/^sk_test_/)

    // 5. Use API key to make request
    const response = await context.request.get(
      "https://api.customers.dev/v1/customers",
      {
        headers: { "Authorization": `Bearer ${apiKey}` },
      }
    )
    expect(response.ok()).toBeTruthy()
  })
})

Test Data Management

Test Data Factory

File: tests/utils/factories.ts

import { faker } from "@faker-js/faker"

export const factories = {
  organization: (overrides = {}) => ({
    name: faker.company.name(),
    plan: "free",
    settings: {
      allowPublicProfiles: false,
      requireTwoFactor: false,
      maxMembers: 5,
    },
    ...overrides,
  }),

  user: (overrides = {}) => ({
    email: faker.internet.email(),
    firstName: faker.person.firstName(),
    lastName: faker.person.lastName(),
    ...overrides,
  }),

  customer: (orgId: string, overrides = {}) => ({
    orgId,
    name: faker.company.name(),
    domain: faker.internet.domainName(),
    status: "customer" as const,
    industry: faker.commerce.department(),
    employeeCount: faker.number.int({ min: 1, max: 10000 }),
    ...overrides,
  }),

  contact: (customerId: string, overrides = {}) => ({
    customerId,
    firstName: faker.person.firstName(),
    lastName: faker.person.lastName(),
    email: faker.internet.email(),
    phone: faker.phone.number(),
    title: faker.person.jobTitle(),
    ...overrides,
  }),

  apiKey: (orgId: string, overrides = {}) => ({
    orgId,
    name: `Test Key ${faker.string.alphanumeric(4)}`,
    permissions: ["customers:read", "customers:write"],
    requestsPerMinute: 60,
    requestsPerHour: 1000,
    requestsPerDay: 10000,
    ...overrides,
  }),
}

Test Database Seeding

File: tests/utils/seed.ts

export async function seedTestData() {
  // Create organizations
  const org1 = await factories.organization({ name: "Test Org 1" })
  const org2 = await factories.organization({ name: "Test Org 2" })

  // Create users
  const owner = await factories.user({ role: "owner", orgId: org1.id })
  const admin = await factories.user({ role: "admin", orgId: org1.id })
  const member = await factories.user({ role: "member", orgId: org1.id })
  const viewer = await factories.user({ role: "viewer", orgId: org1.id })

  // Create customers
  const customers = await Promise.all([
    factories.customer(org1.id, { name: "Customer A", status: "customer" }),
    factories.customer(org1.id, { name: "Customer B", status: "lead" }),
    factories.customer(org1.id, { name: "Customer C", status: "churned" }),
  ])

  // Create contacts
  for (const customer of customers) {
    await factories.contact(customer.id, { isPrimaryContact: true })
    await factories.contact(customer.id)
  }

  return { org1, org2, owner, admin, member, viewer, customers }
}

CI/CD Integration

GitHub Actions Workflow

File: .github/workflows/test.yml

name: Test Suite

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main, develop]

jobs:
  unit-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: pnpm/action-setup@v2
      - uses: actions/setup-node@v3
        with:
          node-version: 20
          cache: "pnpm"

      - name: Install dependencies
        run: pnpm install

      - name: Run unit tests
        run: pnpm test:unit

      - name: Upload coverage
        uses: codecov/codecov-action@v3

  integration-tests:
    runs-on: ubuntu-latest
    needs: unit-tests
    steps:
      - uses: actions/checkout@v3
      - uses: pnpm/action-setup@v2
      - uses: actions/setup-node@v3

      - name: Install dependencies
        run: pnpm install

      - name: Start test server
        run: pnpm dev &
        env:
          CONVEX_DEPLOYMENT: test

      - name: Run integration tests
        run: pnpm test:integration

  e2e-tests:
    runs-on: ubuntu-latest
    needs: integration-tests
    steps:
      - uses: actions/checkout@v3
      - uses: pnpm/action-setup@v2
      - uses: actions/setup-node@v3

      - name: Install dependencies
        run: pnpm install

      - name: Install Playwright
        run: npx playwright install --with-deps

      - name: Run E2E tests
        run: pnpm test:e2e

      - name: Upload test results
        if: always()
        uses: actions/upload-artifact@v3
        with:
          name: playwright-report
          path: playwright-report/

  security-scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Run security audit
        run: pnpm audit --prod

      - name: Check dependencies
        run: pnpm outdated

  performance-test:
    runs-on: ubuntu-latest
    if: github.ref == 'refs/heads/main'
    needs: [unit-tests, integration-tests]
    steps:
      - uses: actions/checkout@v3

      - name: Run load tests
        run: |
          docker run --rm -v $PWD:/tests \
            grafana/k6 run /tests/tests/performance/load.test.ts

Test Scripts

File: package.json

{
  "scripts": {
    "test": "vitest",
    "test:unit": "vitest run --coverage",
    "test:integration": "vitest run tests/integration",
    "test:e2e": "playwright test",
    "test:security": "pnpm audit && pnpm outdated",
    "test:performance": "k6 run tests/performance/load.test.ts",
    "test:watch": "vitest watch",
    "test:ui": "vitest --ui"
  }
}

Test Execution Schedule

Development

  • Pre-commit: Unit tests for changed files
  • Pre-push: Full unit test suite
  • Local: On-demand integration and E2E tests

CI/CD

  • Pull Request: Unit + Integration tests
  • Main Branch: Full test suite including E2E
  • Nightly: Performance and load tests
  • Weekly: Security audits

Production

  • Continuous: API health checks
  • Hourly: Synthetic monitoring
  • Daily: Performance benchmarks

Success Criteria

Test Coverage

  • ✅ Unit tests: ≥80% coverage
  • ✅ Integration tests: ≥70% coverage
  • ✅ API tests: ≥90% coverage
  • ✅ E2E tests: All critical workflows

Performance

  • ✅ API response time: p95 <500ms
  • ✅ Page load time: <3s on 3G
  • ✅ Error rate: <0.1%
  • ✅ Uptime: ≥99.9%

Security

  • ✅ Zero critical vulnerabilities
  • ✅ All RBAC tests passing
  • ✅ API key validation: 100% coverage
  • ✅ Rate limiting enforced

Last Updated: January 2025 Version: 1.0.0

On this page