Integration Patterns

Learn proven patterns for integrating external services, APIs, and third-party systems with your Orchestre projects. These patterns ensure reliable, maintainable, and scalable integrations.

Integration Principles

1. Isolation - Keep external dependencies isolated
2. Abstraction - Hide implementation details
3. Resilience - Handle failures gracefully
4. Testability - Make integrations easy to test
5. Documentation - Keep integration details clear

---

Service Integration Patterns

Pattern: Adapter Pattern

Context: Integrating with external services while maintaining flexibility

Implementation:

/execute-task "Create adapter pattern for [service]:
1. Define interface for service operations
2. Implement adapter for [provider]
3. Add configuration management
4. Include error handling and retries
5. Create mock adapter for testing"

Example: Email Service Integration

/execute-task "Create adapter pattern for email service:
1. Define EmailService interface (send, sendBulk, getStatus)
2. Implement SendGridAdapter
3. Add configuration for API keys and settings
4. Include retry logic and error handling
5. Create MockEmailAdapter for testing"

Benefits:

• Provider independence
• Easy to switch services
• Simplified testing
• Consistent interface

Pattern: Circuit Breaker

Context: Protecting against cascading failures from external services

Implementation:

/execute-task "Implement circuit breaker for [service]:
- Closed state: Normal operation
- Open state: Fail fast after threshold
- Half-open state: Test recovery
- Monitoring and alerting"

Example: Payment Gateway

/execute-task "Implement circuit breaker for payment gateway:
- Track failure rate over sliding window
- Open circuit after 5 failures in 60 seconds
- Half-open after 30 seconds to test recovery
- Alert ops team when circuit opens"

Benefits:

• Prevents cascade failures
• Faster failure response
• Automatic recovery
• System resilience

---

API Integration Patterns

Pattern: API Gateway

Context: Centralizing external API access

Implementation:

/orchestrate "Design API gateway for external services:
- Central configuration
- Authentication handling
- Rate limiting
- Response caching
- Error transformation"

Example: Multi-Service Integration

/execute-task "Create API gateway for:
- Weather API (rate limit: 100/hour)
- Maps API (auth: API key)
- Analytics API (auth: OAuth2)
With unified error handling and caching"

Pattern: Webhook Handler

Context: Receiving events from external services

Implementation:

/execute-task "Create webhook handler for [service]:
1. Endpoint with signature verification
2. Event queue for processing
3. Idempotency handling
4. Retry mechanism
5. Event log and monitoring"

Example: Stripe Webhooks

/execute-task "Create Stripe webhook handler:
1. POST /webhooks/stripe with signature verification
2. Queue events in Redis for async processing
3. Track processed events to prevent duplicates
4. Implement exponential backoff for retries
5. Log all events and monitor processing time"

---

Database Integration Patterns

Pattern: Repository Pattern

Context: Abstracting database operations

Implementation:

/execute-task "Implement repository pattern for [entity]:
- Interface defining operations
- Implementation for [database]
- Query builder abstraction
- Transaction support
- Caching layer"

Example: User Repository

/execute-task "Implement repository pattern for User:
- UserRepository interface (find, create, update, delete)
- PostgreSQL implementation
- Query builder for complex filters
- Transaction support for multi-step operations
- Redis caching with TTL"

Pattern: Database Migration

Context: Managing database schema changes

Implementation:

/execute-task "Set up database migration system:
- Migration files with up/down methods
- Version tracking
- Rollback capability
- Seed data management
- CI/CD integration"

---

Message Queue Integration

Pattern: Producer-Consumer

Context: Asynchronous processing with message queues

Implementation:

/execute-task "Implement message queue integration:
Producer:
- Message serialization
- Partition/routing key strategy
- Error handling
Consumer:
- Message deserialization  
- Processing logic
- Acknowledgment strategy
- Dead letter queue handling"

Example: Order Processing

/execute-task "Implement order processing queue:
Producer (API):
- Publish OrderCreated events to 'orders' queue
- Include order data and metadata
- Handle publish failures
Consumer (Worker):
- Process orders from queue
- Update inventory
- Send confirmation emails
- Move failed orders to DLQ"

Pattern: Event Sourcing

Context: Building event-driven architectures

Implementation:

/orchestrate "Implement event sourcing for [domain]:
- Event store design
- Event types and schemas
- Projection handlers
- Snapshot strategy
- Event replay capability"

---

Authentication Integration

Pattern: OAuth2 Integration

Context: Integrating with OAuth2 providers

Implementation:

/execute-task "Integrate OAuth2 with [provider]:
1. Authorization flow implementation
2. Token management (access/refresh)
3. User profile mapping
4. Session handling
5. Logout flow"

Example: Google OAuth

/execute-task "Integrate Google OAuth2:
1. Implement authorization code flow
2. Store tokens securely with encryption
3. Map Google profile to user model
4. Create or update user sessions
5. Handle revocation and logout"

Pattern: JWT Federation

Context: Integrating multiple services with shared authentication

Implementation:

/execute-task "Implement JWT federation:
- Shared public key infrastructure
- Token validation middleware
- Claims mapping
- Token refresh strategy
- Revocation handling"

---

File Storage Integration

Pattern: Storage Abstraction

Context: Working with multiple file storage providers

Implementation:

/execute-task "Create storage abstraction layer:
- Storage interface (upload, download, delete, list)
- Provider implementations (S3, GCS, local)
- Streaming support
- Metadata handling
- URL generation"

Example: Multi-Cloud Storage

/execute-task "Create storage abstraction for:
- AWS S3 for production
- Google Cloud Storage for backups
- Local filesystem for development
With streaming uploads and signed URLs"

---

Third-Party SDK Integration

Pattern: SDK Wrapper

Context: Integrating third-party SDKs safely

Implementation:

/execute-task "Create wrapper for [SDK]:
- Facade pattern for common operations
- Version isolation
- Error normalization
- Logging and monitoring
- Graceful degradation"

Example: Analytics SDK

/execute-task "Create wrapper for analytics SDK:
- Facade for track, identify, page methods
- Isolate SDK version changes
- Normalize errors to app error types
- Add debug logging
- Queue events if SDK fails"

---

Testing Integration Patterns

Pattern: Integration Test Harness

Context: Testing external integrations reliably

Implementation:

/execute-task "Create integration test harness:
- Mock servers for external APIs
- Fixture data management
- Environment configuration
- Test data cleanup
- Performance benchmarks"

Pattern: Contract Testing

Context: Ensuring API compatibility

Implementation:

/execute-task "Implement contract tests for [API]:
- Define API contracts
- Provider verification tests
- Consumer contract tests
- Contract versioning
- Breaking change detection"

---

Monitoring Integration

Pattern: Integration Health Checks

Context: Monitoring external service health

Implementation:

/execute-task "Create health check system:
- Periodic service checks
- Dependency status endpoint
- Alert thresholds
- Circuit breaker integration
- Status page updates"

Example: Multi-Service Health

/execute-task "Create health checks for:
- Database connectivity
- Redis availability
- External API status
- Message queue health
With /health endpoint and PagerDuty alerts"

---

Anti-Patterns to Avoid

Anti-Pattern: Direct Integration

❌ Avoid:

// Directly calling external service
const result = await stripe.charges.create({...});

✅ Better:

// Through abstraction layer
const result = await paymentService.charge({...});

Anti-Pattern: Synchronous Everything

❌ Avoid:

// Blocking on external calls
await emailService.send(welcome);
await analytics.track(signup);
await crm.createContact(user);

✅ Better:

// Async with queues
await eventQueue.publish('user.signup', { user });

Anti-Pattern: No Fallbacks

❌ Avoid:

// No fallback strategy
const weather = await weatherAPI.get(location);

✅ Better:

// With fallback
const weather = await weatherService.get(location) 
  || await cache.get(location) 
  || defaultWeather;

---

Integration Checklist

Before integrating any external service:

• [ ] Abstraction: Is the integration abstracted?
• [ ] Testing: Can it be tested in isolation?
• [ ] Failure: How does it handle failures?
• [ ] Monitoring: How will you know if it breaks?
• [ ] Documentation: Is the integration documented?
• [ ] Security: Are credentials managed safely?
• [ ] Performance: What's the performance impact?
• [ ] Costs: What are the usage costs?

---

Common Integration Scenarios

Scenario: Payment Processing

/orchestrate "Integrate payment processing:
- Multiple payment methods (card, bank, wallet)
- PCI compliance requirements
- Webhook handling for async events
- Refund and dispute handling
- Multi-currency support"

Scenario: Email Service

/orchestrate "Integrate email service:
- Transactional email sending
- Bulk email capabilities
- Template management
- Bounce and complaint handling
- Analytics tracking"

Scenario: Cloud Storage

/orchestrate "Integrate cloud storage:
- File upload/download
- Access control
- CDN integration
- Backup strategy
- Cost optimization"

---

Best Practices

1. Always Abstract: Never call external services directly
2. Plan for Failure: Every integration will fail eventually
3. Monitor Everything: You can't fix what you can't see
4. Test Thoroughly: Integration bugs are expensive
5. Document Clearly: Future you will thank present you

---

Next Steps

• Explore Security Patterns
• Review Testing Patterns
• Study Performance Patterns
• Check Integration Examples