---
name: openevidence-prod-checklist
description: |
  Production readiness checklist for OpenEvidence clinical AI deployments.
  Use when preparing for production launch, conducting pre-deployment reviews,
  or auditing OpenEvidence integration for compliance.
  Trigger with phrases like "openevidence production", "openevidence go-live",
  "openevidence checklist", "deploy openevidence", "openevidence readiness".
allowed-tools: Read, Grep
version: 1.0.0
license: MIT
author: Jeremy Longshore <jeremy@intentsolutions.io>
---

# OpenEvidence Production Checklist

## Overview
Comprehensive checklist for deploying OpenEvidence clinical AI integrations to production healthcare environments.

## Prerequisites
- Completed development and testing phases
- Signed BAA with OpenEvidence
- Security review completed
- Compliance team sign-off

## Production Readiness Checklist

### 1. Legal & Compliance

- [ ] **Business Associate Agreement (BAA) signed**
  - Contact: compliance@openevidence.com
  - Required for any PHI handling

- [ ] **API Terms of Service accepted**
  - Review: https://www.openevidence.com/policies/api

- [ ] **Clinical use disclaimers in place**
  - All clinical outputs must include appropriate disclaimers
  - "For informational purposes only. Verify with current guidelines."

- [ ] **Liability and malpractice considerations reviewed**
  - Legal review of clinical decision support usage

- [ ] **Data processing agreement documented**
  - What data is sent, stored, processed

### 2. Security

- [ ] **Production API keys generated**
  - Separate keys from development/staging
  - Stored in secret manager (not environment variables)

- [ ] **Key rotation schedule established**
  - Recommend: 90-day rotation cycle

- [ ] **IP allowlist configured** (if available)
  - Restrict API access to known IP ranges

- [ ] **TLS 1.2+ enforced**
  - No fallback to older protocols

- [ ] **PHI sanitization implemented**
  - Review `openevidence-security-basics`

- [ ] **Webhook signature verification active**
  - All webhook endpoints validate signatures

- [ ] **Audit logging enabled**
  - HIPAA-compliant access logging
  - Minimum 6-year retention

### 3. Infrastructure

- [ ] **High availability configuration**
  ```typescript
  // Multiple API endpoints for failover
  const config = {
    primaryEndpoint: 'https://api.openevidence.com',
    fallbackEndpoint: 'https://api-fallback.openevidence.com',
    timeout: 30000,
    retries: 3,
  };
  ```

- [ ] **Circuit breaker implemented**
  ```typescript
  import CircuitBreaker from 'opossum';

  const breaker = new CircuitBreaker(clinicalQuery, {
    timeout: 30000,
    errorThresholdPercentage: 50,
    resetTimeout: 30000,
  });
  ```

- [ ] **Rate limit handling configured**
  - Review `openevidence-rate-limits`
  - Request queuing for traffic spikes

- [ ] **Timeout configuration appropriate**
  - Clinical queries: 15-30s
  - DeepConsult: 5-10 minutes

- [ ] **Health check endpoint implemented**
  ```typescript
  app.get('/health/openevidence', async (req, res) => {
    try {
      await client.health.check();
      res.json({ status: 'healthy' });
    } catch (error) {
      res.status(503).json({ status: 'unhealthy', error: error.message });
    }
  });
  ```

### 4. Monitoring & Alerting

- [ ] **Metrics collection active**
  - Request count, latency, error rate
  - Review `openevidence-observability`

- [ ] **Alert rules configured**
  | Alert | Threshold | Severity |
  |-------|-----------|----------|
  | Error rate > 5% | 5 min window | Warning |
  | Error rate > 20% | 5 min window | Critical |
  | P95 latency > 15s | 5 min window | Warning |
  | Health check failed | 2 consecutive | Critical |

- [ ] **Dashboard created**
  - Real-time visibility into OpenEvidence integration health

- [ ] **Log aggregation configured**
  - Centralized logging for debugging
  - PHI excluded from logs

### 5. Error Handling & Resilience

- [ ] **Graceful degradation implemented**
  ```typescript
  async function clinicalQueryWithFallback(question: string) {
    try {
      return await openEvidenceQuery(question);
    } catch (error) {
      // Log error, alert, return cached or default response
      await alertService.send('OpenEvidence unavailable');
      return {
        answer: 'Clinical evidence service temporarily unavailable. Please consult current guidelines directly.',
        fallback: true,
      };
    }
  }
  ```

- [ ] **Retry logic tested**
  - Exponential backoff with jitter
  - Respects Retry-After headers

- [ ] **Error classification implemented**
  - Review `openevidence-common-errors`

### 6. Performance

- [ ] **Response caching strategy**
  - Cache frequently queried clinical information
  - Cache TTL appropriate for clinical data freshness

- [ ] **Connection pooling configured**
  - Reuse HTTP connections

- [ ] **Query optimization reviewed**
  - Clear, specific clinical questions
  - Appropriate context provided

- [ ] **Load testing completed**
  - Test at 2x expected peak traffic
  - Verify rate limits not exceeded

### 7. Documentation & Training

- [ ] **Runbook created**
  - Incident response procedures
  - Review `openevidence-incident-runbook`

- [ ] **On-call training completed**
  - Team knows how to triage OpenEvidence issues

- [ ] **User documentation updated**
  - How to use clinical decision support
  - Limitations and appropriate use

- [ ] **Clinical staff training**
  - Understanding AI-assisted decision support
  - When to rely on vs. question AI recommendations

### 8. Testing & Validation

- [ ] **Integration tests passing**
  - Against sandbox environment

- [ ] **End-to-end tests passing**
  - Full workflow validation

- [ ] **Clinical validation completed**
  - Sample queries reviewed by clinical team
  - Responses appropriate and accurate

- [ ] **Performance benchmarks met**
  - P50 < 5s, P95 < 15s for clinical queries
  - P50 < 3min, P95 < 5min for DeepConsult

### 9. Deployment

- [ ] **Blue-green or canary deployment planned**
  - Gradual rollout with monitoring

- [ ] **Rollback procedure documented**
  - Quick revert if issues detected

- [ ] **Feature flags configured**
  - Ability to disable OpenEvidence quickly

- [ ] **Database migrations completed**
  - Audit log tables, cache tables ready

### 10. Post-Launch

- [ ] **Monitoring dashboard reviewed daily** (first week)

- [ ] **Error reports triaged immediately**

- [ ] **User feedback collection active**

- [ ] **Performance baseline established**

## Production Configuration Example

```typescript
// config/openevidence.production.ts
export const productionConfig = {
  // Credentials from secret manager
  credentials: {
    source: 'secret-manager',
    apiKeyPath: 'projects/prod/secrets/openevidence-api-key',
    orgIdPath: 'projects/prod/secrets/openevidence-org-id',
  },

  // Endpoints
  api: {
    baseUrl: 'https://api.openevidence.com',
    timeout: 30000,
    retries: 3,
  },

  // Rate limiting
  rateLimits: {
    enabled: true,
    maxConcurrent: 10,
    requestsPerMinute: 100,
  },

  // Circuit breaker
  circuitBreaker: {
    enabled: true,
    errorThreshold: 50,
    resetTimeout: 30000,
  },

  // Caching
  cache: {
    enabled: true,
    ttlSeconds: 3600,
    maxEntries: 10000,
  },

  // Monitoring
  monitoring: {
    metricsEnabled: true,
    tracingEnabled: true,
    logLevel: 'info',
  },
};
```

## Output
- Completed production readiness checklist
- Configuration validated
- All stakeholders signed off
- Go-live approved

## Resources
- [OpenEvidence](https://www.openevidence.com/)
- [OpenEvidence Security](https://www.openevidence.com/security)

## Next Steps
For version upgrades, see `openevidence-upgrade-migration`.
