openevidence-prod-checklist

# OpenEvidence Production Checklist ## Overview Comprehensive checklist for deploying OpenEvidence clinical AI integrations to production healthcare environments. ## Prerequisites - Completed development and testing phases - Signed BAA with OpenEvidence - Security review completed - Compliance team sign-off ## Production Readiness Checklist ### 1. Legal & Compliance - [ ] **Business Associate Agreement (BAA) signed** - Contact: compliance@openevidence.com - Required for any PHI handling - [ ] **API Terms of Service accepted** - Review: https://www.openevidence.com/policies/api - [ ] **Clinical use disclaimers in place** - All clinical outputs must include appropriate disclaimers - "For informational purposes only. Verify with current guidelines." - [ ] **Liability and malpractice considerations reviewed** - Legal review of clinical decision support usage - [ ] **Data processing agreement documented** - What data is sent, stored, processed ### 2. Security - [ ] **Production API keys generated** - Separate keys from development/staging - Stored in secret manager (not environment variables) - [ ] **Key rotation schedule established** - Recommend: 90-day rotation cycle - [ ] **IP allowlist configured** (if available) - Restrict API access to known IP ranges - [ ] **TLS 1.2+ enforced** - No fallback to older protocols - [ ] **PHI sanitization implemented** - Review `openevidence-security-basics` - [ ] **Webhook signature verification active** - All webhook endpoints validate signatures - [ ] **Audit logging enabled** - HIPAA-compliant access logging - Minimum 6-year retention ### 3. Infrastructure - [ ] **High availability configuration** ```typescript // Multiple API endpoints for failover const config = { primaryEndpoint: 'https://api.openevidence.com', fallbackEndpoint: 'https://api-fallback.openevidence.com', timeout: 30000, retries: 3, }; ``` - [ ] **Circuit breaker implemented** ```typescript import CircuitBreaker from 'opossum'; const breaker = new CircuitBreaker(clinicalQuery, { timeout: 30000, errorThresholdPercentage: 50, resetTimeout: 30000, }); ``` - [ ] **Rate limit handling configured** - Review `openevidence-rate-limits` - Request queuing for traffic spikes - [ ] **Timeout configuration appropriate** - Clinical queries: 15-30s - DeepConsult: 5-10 minutes - [ ] **Health check endpoint implemented** ```typescript app.get('/health/openevidence', async (req, res) => { try { await client.health.check(); res.json({ status: 'healthy' }); } catch (error) { res.status(503).json({ status: 'unhealthy', error: error.message }); } }); ``` ### 4. Monitoring & Alerting - [ ] **Metrics collection active** - Request count, latency, error rate - Review `openevidence-observability` - [ ] **Alert rules configured** | Alert | Threshold | Severity | |-------|-----------|----------| | Error rate > 5% | 5 min window | Warning | | Error rate > 20% | 5 min window | Critical | | P95 latency > 15s | 5 min window | Warning | | Health check failed | 2 consecutive | Critical | - [ ] **Dashboard created** - Real-time visibility into OpenEvidence integration health - [ ] **Log aggregation configured** - Centralized logging for debugging - PHI excluded from logs ### 5. Error Handling & Resilience - [ ] **Graceful degradation implemented** ```typescript async function clinicalQueryWithFallback(question: string) { try { return await openEvidenceQuery(question); } catch (error) { // Log error, alert, return cached or default response await alertService.send('OpenEvidence unavailable'); return { answer: 'Clinical evidence service temporarily unavailable. Please consult current guidelines directly.', fallback: true, }; } } ``` - [ ] **Retry logic tested** - Exponential backoff with jitter - Respects Retry-After headers - [ ] **Error classification implemented** - Review `openevidence-common-errors` ### 6. Performance - [ ] **Response caching strategy** - Cache frequently queried clinical information - Cache TTL appropriate for clinical data freshness - [ ] **Connection pooling configured** - Reuse HTTP connections - [ ] **Query optimization reviewed** - Clear, specific clinical questions - Appropriate context provided - [ ] **Load testing completed** - Test at 2x expected peak traffic - Verify rate limits not exceeded ### 7. Documentation & Training - [ ] **Runbook created** - Incident response procedures - Review `openevidence-incident-runbook` - [ ] **On-call training completed** - Team knows how to triage OpenEvidence issues - [ ] **User documentation updated** - How to use clinical decision support - Limitations and appropriate use - [ ] **Clinical staff training** - Understanding AI-assisted decision support - When to rely on vs. question AI recommendations ### 8. Testing & Validation - [ ] **Integration tests passing** - Against sandbox environment - [ ] **End-to-end tests passing** - Full workflow validation - [ ] **Clinical validation completed** - Sample queries reviewed by clinical team - Responses appropriate and accurate - [ ] **Performance benchmarks met** - P50 < 5s, P95 < 15s for clinical queries - P50 < 3min, P95 < 5min for DeepConsult ### 9. Deployment - [ ] **Blue-green or canary deployment planned** - Gradual rollout with monitoring - [ ] **Rollback procedure documented** - Quick revert if issues detected - [ ] **Feature flags configured** - Ability to disable OpenEvidence quickly - [ ] **Database migrations completed** - Audit log tables, cache tables ready ### 10. Post-Launch - [ ] **Monitoring dashboard reviewed daily** (first week) - [ ] **Error reports triaged immediately** - [ ] **User feedback collection active** - [ ] **Performance baseline established** ## Production Configuration Example ```typescript // config/openevidence.production.ts export const productionConfig = { // Credentials from secret manager credentials: { source: 'secret-manager', apiKeyPath: 'projects/prod/secrets/openevidence-api-key', orgIdPath: 'projects/prod/secrets/openevidence-org-id', }, // Endpoints api: { baseUrl: 'https://api.openevidence.com', timeout: 30000, retries: 3, }, // Rate limiting rateLimits: { enabled: true, maxConcurrent: 10, requestsPerMinute: 100, }, // Circuit breaker circuitBreaker: { enabled: true, errorThreshold: 50, resetTimeout: 30000, }, // Caching cache: { enabled: true, ttlSeconds: 3600, maxEntries: 10000, }, // Monitoring monitoring: { metricsEnabled: true, tracingEnabled: true, logLevel: 'info', }, }; ``` ## Output - Completed production readiness checklist - Configuration validated - All stakeholders signed off - Go-live approved ## Resources - [OpenEvidence](https://www.openevidence.com/) - [OpenEvidence Security](https://www.openevidence.com/security) ## Next Steps For version upgrades, see `openevidence-upgrade-migration`.