Testing Plan - customers.dev
Comprehensive testing strategy for the customers.dev platform covering all layers from unit tests to end-to-end testing.
Table of Contents
- Overview
- Testing Strategy
- Test Coverage Requirements
- Unit Testing
- Integration Testing
- API Testing
- Security Testing
- Performance Testing
- E2E Testing
- Test Data Management
- CI/CD Integration
Overview
Testing Objectives
- Functional Correctness: Verify all features work as specified
- Security: Ensure RBAC, authentication, and data isolation work correctly
- Performance: Validate response times and rate limiting
- Reliability: Ensure system handles errors gracefully
- Regression Prevention: Catch breaking changes early
Testing Pyramid
┌─────────────┐
│ E2E (5%) │ ← Browser-based user workflows
└─────────────┘
┌─────────────────┐
│ Integration (20%)│ ← API endpoints, auth flows
└─────────────────┘
┌───────────────────────┐
│ Unit Tests (75%) │ ← Functions, utilities, RBAC
└───────────────────────┘Testing Strategy
Phase 1: Foundation (Week 1-2)
- ✅ RBAC unit tests
- ✅ Convex function tests
- 🔄 HTTP Action tests
- 🔄 API endpoint integration tests
Phase 2: API Coverage (Week 2-4)
- REST API endpoint tests
- Authentication flow tests
- Rate limiting tests
- Error handling tests
Phase 3: Dashboard (Week 4-6)
- Component tests
- Integration tests
- E2E user workflows
- Accessibility tests
Phase 4: Production Readiness (Week 6-12)
- Performance benchmarks
- Load testing
- Security audits
- Chaos engineering
Test Coverage Requirements
Minimum Coverage Targets
| Layer | Target | Critical Paths |
|---|---|---|
| Unit Tests | ≥80% | ≥95% |
| Integration Tests | ≥70% | ≥90% |
| API Tests | ≥90% | 100% |
| E2E Tests | Critical flows | 100% |
Critical Paths Defined
Must Have 100% Coverage:
- Authentication and authorization
- API key validation
- Rate limiting
- Payment processing (future)
- Data deletion/privacy operations
Unit Testing
RBAC System Tests
File: apps/web/convex/rbac.test.ts
import { describe, it, expect } from "vitest"
import { hasPermission, PERMISSIONS } from "./rbac"
describe("RBAC System", () => {
describe("hasPermission", () => {
it("should allow viewer to READ", () => {
expect(hasPermission("viewer", PERMISSIONS.READ)).toBe(true)
})
it("should deny viewer WRITE permission", () => {
expect(hasPermission("viewer", PERMISSIONS.WRITE)).toBe(false)
})
it("should allow member to READ and WRITE", () => {
expect(hasPermission("member", PERMISSIONS.READ)).toBe(true)
expect(hasPermission("member", PERMISSIONS.WRITE)).toBe(true)
})
it("should deny member DELETE permission", () => {
expect(hasPermission("member", PERMISSIONS.DELETE)).toBe(false)
})
it("should allow admin all permissions except MANAGE_ORG", () => {
expect(hasPermission("admin", PERMISSIONS.READ)).toBe(true)
expect(hasPermission("admin", PERMISSIONS.WRITE)).toBe(true)
expect(hasPermission("admin", PERMISSIONS.DELETE)).toBe(true)
expect(hasPermission("admin", PERMISSIONS.MANAGE_MEMBERS)).toBe(true)
expect(hasPermission("admin", PERMISSIONS.MANAGE_API_KEYS)).toBe(true)
expect(hasPermission("admin", PERMISSIONS.MANAGE_ORG)).toBe(false)
})
it("should allow owner all permissions", () => {
expect(hasPermission("owner", PERMISSIONS.READ)).toBe(true)
expect(hasPermission("owner", PERMISSIONS.WRITE)).toBe(true)
expect(hasPermission("owner", PERMISSIONS.DELETE)).toBe(true)
expect(hasPermission("owner", PERMISSIONS.MANAGE_MEMBERS)).toBe(true)
expect(hasPermission("owner", PERMISSIONS.MANAGE_API_KEYS)).toBe(true)
expect(hasPermission("owner", PERMISSIONS.MANAGE_ORG)).toBe(true)
})
})
})Convex Function Tests
File: apps/web/convex/customers.test.ts
import { convexTest } from "convex-test"
import { describe, it, expect, beforeEach } from "vitest"
import schema from "./schema"
import { api } from "./_generated/api"
describe("Customer Functions", () => {
let t: ConvexTestingHelper
beforeEach(async () => {
t = convexTest(schema)
// Seed test data
const orgId = await t.run(async (ctx) => {
return await ctx.db.insert("organizations", {
name: "Test Org",
ownerId: "user_123",
plan: "free",
settings: {
allowPublicProfiles: false,
requireTwoFactor: false,
maxMembers: 5,
},
createdAt: Date.now(),
updatedAt: Date.now(),
})
})
// Create user and membership
await t.run(async (ctx) => {
const userId = await ctx.db.insert("users", {
clerkId: "user_123",
email: "test@example.com",
firstName: "Test",
lastName: "User",
createdAt: Date.now(),
})
await ctx.db.insert("organizationMembers", {
orgId,
userId,
role: "owner",
status: "active",
joinedAt: Date.now(),
})
})
})
describe("createCustomer", () => {
it("should create customer with valid permissions", async () => {
const customer = await t.mutation(api.customers.createCustomer, {
orgId: t.orgId,
name: "Acme Corp",
domain: "acme.com",
status: "customer",
})
expect(customer).toBeDefined()
expect(customer.name).toBe("Acme Corp")
})
it("should require WRITE permission", async () => {
// Change user role to viewer
await t.run(async (ctx) => {
const membership = await ctx.db
.query("organizationMembers")
.withIndex("by_user")
.first()
await ctx.db.patch(membership._id, { role: "viewer" })
})
await expect(
t.mutation(api.customers.createCustomer, {
orgId: t.orgId,
name: "Acme Corp",
status: "customer",
})
).rejects.toThrow("Insufficient permissions")
})
it("should enforce organization isolation", async () => {
const otherOrgId = await t.run(async (ctx) => {
return await ctx.db.insert("organizations", {
name: "Other Org",
ownerId: "user_456",
plan: "free",
settings: { allowPublicProfiles: false, requireTwoFactor: false, maxMembers: 5 },
createdAt: Date.now(),
updatedAt: Date.now(),
})
})
await expect(
t.mutation(api.customers.createCustomer, {
orgId: otherOrgId,
name: "Acme Corp",
status: "customer",
})
).rejects.toThrow("User is not a member of this organization")
})
})
describe("getCustomers", () => {
it("should return customers for organization", async () => {
// Create test customers
await t.mutation(api.customers.createCustomer, {
orgId: t.orgId,
name: "Acme Corp",
status: "customer",
})
await t.mutation(api.customers.createCustomer, {
orgId: t.orgId,
name: "TechStart",
status: "lead",
})
const customers = await t.query(api.customers.getCustomers, {
orgId: t.orgId,
})
expect(customers).toHaveLength(2)
expect(customers[0].name).toBe("TechStart") // Most recent first
expect(customers[1].name).toBe("Acme Corp")
})
it("should filter by status", async () => {
await t.mutation(api.customers.createCustomer, {
orgId: t.orgId,
name: "Customer A",
status: "customer",
})
await t.mutation(api.customers.createCustomer, {
orgId: t.orgId,
name: "Lead B",
status: "lead",
})
const customers = await t.query(api.customers.getCustomers, {
orgId: t.orgId,
status: "customer",
})
expect(customers).toHaveLength(1)
expect(customers[0].name).toBe("Customer A")
})
it("should not return other org's customers", async () => {
const otherOrgId = await t.run(async (ctx) => {
return await ctx.db.insert("organizations", {
name: "Other Org",
ownerId: "user_456",
plan: "free",
settings: { allowPublicProfiles: false, requireTwoFactor: false, maxMembers: 5 },
createdAt: Date.now(),
updatedAt: Date.now(),
})
})
// Create customer in other org
await t.run(async (ctx) => {
await ctx.db.insert("customers", {
orgId: otherOrgId,
name: "Other Customer",
status: "customer",
createdBy: "user_456",
updatedBy: "user_456",
createdAt: Date.now(),
updatedAt: Date.now(),
})
})
const customers = await t.query(api.customers.getCustomers, {
orgId: t.orgId,
})
expect(customers).toHaveLength(0)
})
})
})API Key Validation Tests
File: apps/web/convex/apiKeys.test.ts
import { convexTest } from "convex-test"
import { describe, it, expect, beforeEach } from "vitest"
import { api } from "./_generated/api"
describe("API Key Functions", () => {
let t: ConvexTestingHelper
let apiKey: string
beforeEach(async () => {
t = convexTest(schema)
// Setup org, user, membership...
// Create API key
const result = await t.mutation(api.apiKeys.createApiKey, {
orgId: t.orgId,
name: "Test Key",
permissions: ["customers:read", "customers:write"],
requestsPerMinute: 60,
})
apiKey = result.apiKey
})
describe("validateApiKey", () => {
it("should validate correct API key", async () => {
const result = await t.query(api.apiKeys.validateApiKey, {
apiKey,
})
expect(result.valid).toBe(true)
expect(result.permissions).toEqual(["customers:read", "customers:write"])
})
it("should reject invalid API key", async () => {
const result = await t.query(api.apiKeys.validateApiKey, {
apiKey: "sk_test_invalid",
})
expect(result.valid).toBe(false)
expect(result.error).toBe("Invalid API key")
})
it("should reject revoked API key", async () => {
// Revoke the key
await t.mutation(api.apiKeys.revokeApiKey, {
id: result.keyId,
})
const validation = await t.query(api.apiKeys.validateApiKey, {
apiKey,
})
expect(validation.valid).toBe(false)
expect(validation.error).toBe("API key is not active")
})
it("should reject expired API key", async () => {
// Create key with past expiration
const expiredResult = await t.run(async (ctx) => {
const yesterday = Date.now() - 24 * 60 * 60 * 1000
return await ctx.db.insert("apiKeys", {
orgId: t.orgId,
name: "Expired Key",
keyHash: "hash",
keyPrefix: "sk_test_exp",
permissions: ["customers:read"],
rateLimit: { requestsPerMinute: 60, requestsPerHour: 1000, requestsPerDay: 10000 },
status: "active",
expiresAt: yesterday,
createdBy: t.userId,
createdAt: Date.now(),
updatedAt: Date.now(),
})
})
const validation = await t.query(api.apiKeys.validateApiKey, {
apiKey: "sk_test_exp_fake",
})
expect(validation.valid).toBe(false)
expect(validation.error).toBe("API key has expired")
})
})
describe("checkRateLimit", () => {
it("should allow requests under limit", async () => {
const result = await t.query(api.apiKeys.checkRateLimit, {
apiKeyId: t.apiKeyId,
})
expect(result.allowed).toBe(true)
expect(result.remaining.perMinute).toBe(60)
})
it("should reject when rate limit exceeded", async () => {
// Track 60 requests
for (let i = 0; i < 60; i++) {
await t.mutation(api.apiKeys.trackApiKeyUsage, {
apiKeyId: t.apiKeyId,
orgId: t.orgId,
endpoint: "/customers",
method: "GET",
statusCode: 200,
})
}
const result = await t.query(api.apiKeys.checkRateLimit, {
apiKeyId: t.apiKeyId,
})
expect(result.allowed).toBe(false)
expect(result.error).toContain("Rate limit exceeded")
expect(result.resetAt).toBeDefined()
})
})
})Integration Testing
API Endpoint Tests
File: tests/integration/api.test.ts
import { describe, it, expect, beforeAll, afterAll } from "vitest"
describe("REST API Endpoints", () => {
let apiKey: string
const baseURL = "http://localhost:3000/api"
beforeAll(async () => {
// Setup test organization and API key
apiKey = await setupTestAPIKey()
})
afterAll(async () => {
// Cleanup test data
await cleanupTestData()
})
describe("GET /customers", () => {
it("should return customers with valid API key", async () => {
const response = await fetch(`${baseURL}/customers`, {
headers: {
"Authorization": `Bearer ${apiKey}`,
},
})
expect(response.status).toBe(200)
expect(response.headers.get("X-RateLimit-Limit")).toBe("60")
const data = await response.json()
expect(data).toHaveProperty("data")
expect(Array.isArray(data.data)).toBe(true)
})
it("should return 401 without API key", async () => {
const response = await fetch(`${baseURL}/customers`)
expect(response.status).toBe(401)
const data = await response.json()
expect(data.error).toBe("Missing API key")
})
it("should return 401 with invalid API key", async () => {
const response = await fetch(`${baseURL}/customers`, {
headers: {
"Authorization": "Bearer sk_test_invalid",
},
})
expect(response.status).toBe(401)
const data = await response.json()
expect(data.error).toBe("Invalid API key")
})
it("should support pagination", async () => {
const response = await fetch(`${baseURL}/customers?limit=10`, {
headers: {
"Authorization": `Bearer ${apiKey}`,
},
})
const data = await response.json()
expect(data).toHaveProperty("cursor")
expect(data).toHaveProperty("hasMore")
})
})
describe("POST /customers", () => {
it("should create customer with valid data", async () => {
const response = await fetch(`${baseURL}/customers`, {
method: "POST",
headers: {
"Authorization": `Bearer ${apiKey}`,
"Content-Type": "application/json",
},
body: JSON.stringify({
name: "Test Corp",
domain: "test.com",
status: "lead",
}),
})
expect(response.status).toBe(201)
const data = await response.json()
expect(data.id).toBeDefined()
expect(data.name).toBe("Test Corp")
})
it("should return 400 with invalid data", async () => {
const response = await fetch(`${baseURL}/customers`, {
method: "POST",
headers: {
"Authorization": `Bearer ${apiKey}`,
"Content-Type": "application/json",
},
body: JSON.stringify({
// Missing required 'name' field
status: "lead",
}),
})
expect(response.status).toBe(400)
const data = await response.json()
expect(data.error).toContain("validation")
})
it("should return 403 without write permission", async () => {
const readOnlyKey = await createReadOnlyAPIKey()
const response = await fetch(`${baseURL}/customers`, {
method: "POST",
headers: {
"Authorization": `Bearer ${readOnlyKey}`,
"Content-Type": "application/json",
},
body: JSON.stringify({
name: "Test Corp",
status: "lead",
}),
})
expect(response.status).toBe(403)
const data = await response.json()
expect(data.error).toContain("Insufficient permissions")
})
it("should return 429 when rate limited", async () => {
// Make 61 requests (exceeds 60/minute limit)
const requests = []
for (let i = 0; i < 61; i++) {
requests.push(
fetch(`${baseURL}/customers`, {
headers: { "Authorization": `Bearer ${apiKey}` },
})
)
}
const responses = await Promise.all(requests)
const lastResponse = responses[responses.length - 1]
expect(lastResponse.status).toBe(429)
expect(lastResponse.headers.get("Retry-After")).toBeDefined()
})
})
})API Testing
Authentication Flow Tests
File: tests/api/auth.test.ts
describe("Authentication Flows", () => {
describe("API Key Authentication", () => {
it("should accept valid Bearer token", async () => {
const response = await makeAuthenticatedRequest()
expect(response.status).not.toBe(401)
})
it("should reject missing Authorization header", async () => {
const response = await fetch(endpoint)
expect(response.status).toBe(401)
})
it("should reject malformed Authorization header", async () => {
const response = await fetch(endpoint, {
headers: { "Authorization": "InvalidFormat" },
})
expect(response.status).toBe(401)
})
it("should reject expired API key", async () => {
const expiredKey = await createExpiredAPIKey()
const response = await fetch(endpoint, {
headers: { "Authorization": `Bearer ${expiredKey}` },
})
expect(response.status).toBe(401)
expect(await response.json()).toMatchObject({
error: "API key has expired",
})
})
})
describe("Permission Enforcement", () => {
it("should enforce customers:read permission", async () => {
const writeOnlyKey = await createAPIKey({ permissions: ["customers:write"] })
const response = await fetch(`${baseURL}/customers`, {
headers: { "Authorization": `Bearer ${writeOnlyKey}` },
})
expect(response.status).toBe(403)
})
it("should enforce customers:write permission", async () => {
const readOnlyKey = await createAPIKey({ permissions: ["customers:read"] })
const response = await fetch(`${baseURL}/customers`, {
method: "POST",
headers: {
"Authorization": `Bearer ${readOnlyKey}`,
"Content-Type": "application/json",
},
body: JSON.stringify({ name: "Test" }),
})
expect(response.status).toBe(403)
})
})
})Rate Limiting Tests
File: tests/api/rate-limiting.test.ts
describe("Rate Limiting", () => {
it("should enforce per-minute rate limit", async () => {
const apiKey = await createAPIKey({ requestsPerMinute: 5 })
// Make 5 successful requests
for (let i = 0; i < 5; i++) {
const response = await makeRequest(apiKey)
expect(response.status).toBe(200)
}
// 6th request should be rate limited
const response = await makeRequest(apiKey)
expect(response.status).toBe(429)
expect(response.headers.get("X-RateLimit-Limit")).toBe("5")
expect(response.headers.get("X-RateLimit-Used")).toBe("5")
expect(response.headers.get("X-RateLimit-Reset")).toBeDefined()
})
it("should reset rate limit after window", async () => {
const apiKey = await createAPIKey({ requestsPerMinute: 2 })
// Exhaust limit
await makeRequest(apiKey)
await makeRequest(apiKey)
let response = await makeRequest(apiKey)
expect(response.status).toBe(429)
// Wait for window to reset
await new Promise(resolve => setTimeout(resolve, 61000))
// Should work again
response = await makeRequest(apiKey)
expect(response.status).toBe(200)
})
it("should track per-hour rate limit separately", async () => {
const apiKey = await createAPIKey({
requestsPerMinute: 100,
requestsPerHour: 10,
})
// Make 10 requests over 2 minutes (within hour limit)
for (let i = 0; i < 10; i++) {
if (i === 5) await new Promise(resolve => setTimeout(resolve, 61000))
const response = await makeRequest(apiKey)
expect(response.status).toBe(200)
}
// 11th request in same hour should fail
const response = await makeRequest(apiKey)
expect(response.status).toBe(429)
expect(await response.json()).toMatchObject({
error: "Rate limit exceeded for requests per hour",
})
})
})Security Testing
RBAC Security Tests
File: tests/security/rbac.test.ts
describe("RBAC Security", () => {
describe("Organization Isolation", () => {
it("should prevent cross-org data access", async () => {
const org1 = await createOrganization("Org 1")
const org2 = await createOrganization("Org 2")
const customer = await createCustomer(org1.id, "Secret Customer")
// User from org2 should not see org1's customer
const org2User = await createUser(org2.id, "member")
const customers = await getCustomersAs(org2User, org1.id)
expect(customers).toHaveLength(0) // Or throw error
})
it("should prevent cross-org updates", async () => {
const org1 = await createOrganization("Org 1")
const org2 = await createOrganization("Org 2")
const customer = await createCustomer(org1.id, "Customer")
const org2User = await createUser(org2.id, "admin")
await expect(
updateCustomerAs(org2User, customer.id, { name: "Hacked" })
).rejects.toThrow("User is not a member of this organization")
})
})
describe("Permission Enforcement", () => {
it("should prevent viewer from creating records", async () => {
const viewer = await createUser(org.id, "viewer")
await expect(
createCustomerAs(viewer, { name: "Test" })
).rejects.toThrow("Insufficient permissions")
})
it("should prevent member from deleting records", async () => {
const member = await createUser(org.id, "member")
const customer = await createCustomer(org.id, "Test")
await expect(
deleteCustomerAs(member, customer.id)
).rejects.toThrow("Insufficient permissions")
})
it("should prevent admin from changing org settings", async () => {
const admin = await createUser(org.id, "admin")
await expect(
updateOrganizationAs(admin, org.id, { plan: "enterprise" })
).rejects.toThrow("Insufficient permissions")
})
})
describe("API Key Security", () => {
it("should not return full API key after creation", async () => {
const keys = await getAPIKeys(org.id)
expect(keys.every(k => !k.keyHash)).toBe(true)
expect(keys.every(k => k.keyPrefix)).toBe(true)
})
it("should hash API keys in database", async () => {
const { apiKey, keyId } = await createAPIKey()
const storedKey = await getStoredAPIKey(keyId)
expect(storedKey.keyHash).not.toBe(apiKey)
expect(storedKey.keyHash).toMatch(/^[A-Za-z0-9+/=]+$/) // Base64
})
it("should validate API key permissions", async () => {
const readOnlyKey = await createAPIKey({
permissions: ["customers:read"],
})
const result = await validateAPIKey(readOnlyKey)
expect(result.permissions).toEqual(["customers:read"])
expect(result.permissions).not.toContain("customers:write")
})
})
})Input Validation Tests
File: tests/security/validation.test.ts
describe("Input Validation", () => {
describe("Customer Creation", () => {
it("should reject invalid email", async () => {
await expect(
createCustomer({ name: "Test", email: "invalid-email" })
).rejects.toThrow("Invalid email format")
})
it("should reject XSS attempts in name", async () => {
const maliciousName = '<script>alert("XSS")</script>'
const customer = await createCustomer({ name: maliciousName })
// Should be sanitized
expect(customer.name).not.toContain("<script>")
})
it("should reject SQL injection attempts", async () => {
const maliciousInput = "Test'; DROP TABLE customers; --"
const customer = await createCustomer({ name: maliciousInput })
// Should be safely stored (Convex handles this)
expect(customer.name).toBe(maliciousInput) // Stored as-is, safe in Convex
})
})
describe("API Key Creation", () => {
it("should reject invalid permission values", async () => {
await expect(
createAPIKey({ permissions: ["invalid:permission"] })
).rejects.toThrow("Invalid permission")
})
it("should reject negative rate limits", async () => {
await expect(
createAPIKey({ requestsPerMinute: -1 })
).rejects.toThrow("Rate limit must be positive")
})
})
})Performance Testing
Load Testing
File: tests/performance/load.test.ts
import { check } from "k6"
import http from "k6/http"
export const options = {
stages: [
{ duration: "2m", target: 100 }, // Ramp up to 100 users
{ duration: "5m", target: 100 }, // Stay at 100 users
{ duration: "2m", target: 200 }, // Ramp up to 200 users
{ duration: "5m", target: 200 }, // Stay at 200 users
{ duration: "2m", target: 0 }, // Ramp down
],
thresholds: {
http_req_duration: ["p(95)<500"], // 95% of requests under 500ms
http_req_failed: ["rate<0.01"], // Less than 1% errors
},
}
export default function () {
const response = http.get("https://api.customers.dev/v1/customers", {
headers: {
"Authorization": `Bearer ${__ENV.API_KEY}`,
},
})
check(response, {
"status is 200": (r) => r.status === 200,
"response time < 500ms": (r) => r.timings.duration < 500,
})
}Response Time Benchmarks
File: tests/performance/benchmarks.test.ts
describe("Response Time Benchmarks", () => {
it("GET /customers should respond within 200ms", async () => {
const start = Date.now()
await fetch(`${baseURL}/customers`, {
headers: { "Authorization": `Bearer ${apiKey}` },
})
const duration = Date.now() - start
expect(duration).toBeLessThan(200)
})
it("POST /customers should respond within 300ms", async () => {
const start = Date.now()
await fetch(`${baseURL}/customers`, {
method: "POST",
headers: {
"Authorization": `Bearer ${apiKey}`,
"Content-Type": "application/json",
},
body: JSON.stringify({ name: "Test" }),
})
const duration = Date.now() - start
expect(duration).toBeLessThan(300)
})
it("should handle 1000 concurrent requests", async () => {
const requests = Array(1000).fill(null).map(() =>
fetch(`${baseURL}/customers`, {
headers: { "Authorization": `Bearer ${apiKey}` },
})
)
const start = Date.now()
const responses = await Promise.all(requests)
const duration = Date.now() - start
expect(responses.every(r => r.ok)).toBe(true)
expect(duration).toBeLessThan(5000) // 5 seconds for 1000 requests
})
})E2E Testing
User Workflows
File: tests/e2e/workflows.spec.ts (Playwright)
import { test, expect } from "@playwright/test"
test.describe("Customer Management Workflow", () => {
test("should complete full customer lifecycle", async ({ page }) => {
// 1. Login
await page.goto("https://customers.dev/sign-in")
await page.fill('[name="email"]', "test@example.com")
await page.fill('[name="password"]', "password123")
await page.click('button[type="submit"]')
// 2. Navigate to customers
await page.click('a[href="/customers"]')
await expect(page).toHaveURL(/.*\/customers/)
// 3. Create new customer
await page.click('button:has-text("Add Customer")')
await page.fill('[name="name"]', "Acme Corp")
await page.fill('[name="domain"]', "acme.com")
await page.selectOption('[name="status"]', "customer")
await page.click('button:has-text("Save")')
// 4. Verify customer created
await expect(page.locator('text=Acme Corp')).toBeVisible()
// 5. Add contact
await page.click('text=Acme Corp')
await page.click('button:has-text("Add Contact")')
await page.fill('[name="firstName"]', "John")
await page.fill('[name="lastName"]', "Doe")
await page.fill('[name="email"]', "john@acme.com")
await page.click('button:has-text("Save")')
// 6. Verify contact created
await expect(page.locator('text=John Doe')).toBeVisible()
// 7. Update customer status
await page.click('button:has-text("Edit")')
await page.selectOption('[name="status"]', "churned")
await page.click('button:has-text("Save")')
// 8. Verify status updated
await expect(page.locator('text=Churned')).toBeVisible()
})
test("should enforce viewer permissions", async ({ page }) => {
// Login as viewer
await loginAs(page, "viewer@example.com")
// Navigate to customers
await page.goto("/customers")
// Should see customers
await expect(page.locator('[data-testid="customer-list"]')).toBeVisible()
// Should NOT see "Add Customer" button
await expect(page.locator('button:has-text("Add Customer")')).not.toBeVisible()
// Should NOT see "Delete" buttons
await expect(page.locator('button:has-text("Delete")')).not.toBeVisible()
})
})
test.describe("API Key Management", () => {
test("should create and use API key", async ({ page, context }) => {
// 1. Login as admin
await loginAs(page, "admin@example.com")
// 2. Navigate to API keys
await page.goto("/settings/api-keys")
// 3. Create API key
await page.click('button:has-text("Create API Key")')
await page.fill('[name="name"]', "Test Key")
await page.check('[name="permissions"][value="customers:read"]')
await page.check('[name="permissions"][value="customers:write"]')
await page.click('button:has-text("Create")')
// 4. Copy API key (shown only once)
const apiKey = await page.locator('[data-testid="api-key-value"]').textContent()
expect(apiKey).toMatch(/^sk_test_/)
// 5. Use API key to make request
const response = await context.request.get(
"https://api.customers.dev/v1/customers",
{
headers: { "Authorization": `Bearer ${apiKey}` },
}
)
expect(response.ok()).toBeTruthy()
})
})Test Data Management
Test Data Factory
File: tests/utils/factories.ts
import { faker } from "@faker-js/faker"
export const factories = {
organization: (overrides = {}) => ({
name: faker.company.name(),
plan: "free",
settings: {
allowPublicProfiles: false,
requireTwoFactor: false,
maxMembers: 5,
},
...overrides,
}),
user: (overrides = {}) => ({
email: faker.internet.email(),
firstName: faker.person.firstName(),
lastName: faker.person.lastName(),
...overrides,
}),
customer: (orgId: string, overrides = {}) => ({
orgId,
name: faker.company.name(),
domain: faker.internet.domainName(),
status: "customer" as const,
industry: faker.commerce.department(),
employeeCount: faker.number.int({ min: 1, max: 10000 }),
...overrides,
}),
contact: (customerId: string, overrides = {}) => ({
customerId,
firstName: faker.person.firstName(),
lastName: faker.person.lastName(),
email: faker.internet.email(),
phone: faker.phone.number(),
title: faker.person.jobTitle(),
...overrides,
}),
apiKey: (orgId: string, overrides = {}) => ({
orgId,
name: `Test Key ${faker.string.alphanumeric(4)}`,
permissions: ["customers:read", "customers:write"],
requestsPerMinute: 60,
requestsPerHour: 1000,
requestsPerDay: 10000,
...overrides,
}),
}Test Database Seeding
File: tests/utils/seed.ts
export async function seedTestData() {
// Create organizations
const org1 = await factories.organization({ name: "Test Org 1" })
const org2 = await factories.organization({ name: "Test Org 2" })
// Create users
const owner = await factories.user({ role: "owner", orgId: org1.id })
const admin = await factories.user({ role: "admin", orgId: org1.id })
const member = await factories.user({ role: "member", orgId: org1.id })
const viewer = await factories.user({ role: "viewer", orgId: org1.id })
// Create customers
const customers = await Promise.all([
factories.customer(org1.id, { name: "Customer A", status: "customer" }),
factories.customer(org1.id, { name: "Customer B", status: "lead" }),
factories.customer(org1.id, { name: "Customer C", status: "churned" }),
])
// Create contacts
for (const customer of customers) {
await factories.contact(customer.id, { isPrimaryContact: true })
await factories.contact(customer.id)
}
return { org1, org2, owner, admin, member, viewer, customers }
}CI/CD Integration
GitHub Actions Workflow
File: .github/workflows/test.yml
name: Test Suite
on:
push:
branches: [main, develop]
pull_request:
branches: [main, develop]
jobs:
unit-tests:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: pnpm/action-setup@v2
- uses: actions/setup-node@v3
with:
node-version: 20
cache: "pnpm"
- name: Install dependencies
run: pnpm install
- name: Run unit tests
run: pnpm test:unit
- name: Upload coverage
uses: codecov/codecov-action@v3
integration-tests:
runs-on: ubuntu-latest
needs: unit-tests
steps:
- uses: actions/checkout@v3
- uses: pnpm/action-setup@v2
- uses: actions/setup-node@v3
- name: Install dependencies
run: pnpm install
- name: Start test server
run: pnpm dev &
env:
CONVEX_DEPLOYMENT: test
- name: Run integration tests
run: pnpm test:integration
e2e-tests:
runs-on: ubuntu-latest
needs: integration-tests
steps:
- uses: actions/checkout@v3
- uses: pnpm/action-setup@v2
- uses: actions/setup-node@v3
- name: Install dependencies
run: pnpm install
- name: Install Playwright
run: npx playwright install --with-deps
- name: Run E2E tests
run: pnpm test:e2e
- name: Upload test results
if: always()
uses: actions/upload-artifact@v3
with:
name: playwright-report
path: playwright-report/
security-scan:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Run security audit
run: pnpm audit --prod
- name: Check dependencies
run: pnpm outdated
performance-test:
runs-on: ubuntu-latest
if: github.ref == 'refs/heads/main'
needs: [unit-tests, integration-tests]
steps:
- uses: actions/checkout@v3
- name: Run load tests
run: |
docker run --rm -v $PWD:/tests \
grafana/k6 run /tests/tests/performance/load.test.tsTest Scripts
File: package.json
{
"scripts": {
"test": "vitest",
"test:unit": "vitest run --coverage",
"test:integration": "vitest run tests/integration",
"test:e2e": "playwright test",
"test:security": "pnpm audit && pnpm outdated",
"test:performance": "k6 run tests/performance/load.test.ts",
"test:watch": "vitest watch",
"test:ui": "vitest --ui"
}
}Test Execution Schedule
Development
- Pre-commit: Unit tests for changed files
- Pre-push: Full unit test suite
- Local: On-demand integration and E2E tests
CI/CD
- Pull Request: Unit + Integration tests
- Main Branch: Full test suite including E2E
- Nightly: Performance and load tests
- Weekly: Security audits
Production
- Continuous: API health checks
- Hourly: Synthetic monitoring
- Daily: Performance benchmarks
Success Criteria
Test Coverage
- ✅ Unit tests: ≥80% coverage
- ✅ Integration tests: ≥70% coverage
- ✅ API tests: ≥90% coverage
- ✅ E2E tests: All critical workflows
Performance
- ✅ API response time: p95 <500ms
- ✅ Page load time: <3s on 3G
- ✅ Error rate: <0.1%
- ✅ Uptime: ≥99.9%
Security
- ✅ Zero critical vulnerabilities
- ✅ All RBAC tests passing
- ✅ API key validation: 100% coverage
- ✅ Rate limiting enforced
Last Updated: January 2025 Version: 1.0.0