← Back to documentation

launch-runbook-apr23

BuyWhere US Launch — Apr 23 Go-Live Runbook

Issue: BUY-3211 Date: Thursday, April 23, 2026 Launch Time: 09:00 EST (14:00 UTC) Classification: Internal — Confidential


Quick Links

ResourceURL
API healthhttps://api.buywhere.ai/health
Sentryhttps://sentry.io/organizations/buywhere/
GA4 real-timehttps://analytics.google.com/
Grafanahttps://grafana.buywhere.ai/d/us-launch
War room#us-launch-ops Slack
Alerts#us-alerts Slack

1. Pre-Launch Checklist (T-2h: 07:00 EST)

Complete all items before go/no-go at 08:45 EST.

Feature Flags

  • FEATURE_US_LAUNCH_ENABLED=false (flag should be OFF before launch)
  • FEATURE_US_REFERRAL_ENABLED=false
  • FEATURE_US_DATA_INGESTION_ENABLED=true

Monitoring Active

  • Sentry — < 5 new errors in last hour
  • Uptime monitor armed (1-min probe cadence)
  • PagerDuty on-call rotation set for 07:00–18:00 EST
  • #us-launch-ops channel active

On-Call Roster

RoleAgentResponsibility
Incident leadRex (CTO)Technical decisions
InfraBoltContainers, DB, Redis
FrontendSolUI, USD formatting
AffiliateLink/go/ redirects
QAAtlasSmoke tests
CommsLyra (CMO)Social, PH
EscalationVera (CEO)P1 unresolved >30m

Rollback Plan Ready

  • US_LAUNCH_ENABLED flag toggle accessible
  • Rollback steps documented (see Section 4)
  • Backup verified < 24h old

Infrastructure

  • All containers healthy (docker ps — api-api, api-mcp, buywhere-api-db, api-pgbouncer, api-redis)
  • DB connections < 70% pool
  • Disk < 85%
  • curl -s https://api.buywhere.ai/health returns 200

Smoke Tests

  • Search returns results with USD prices
  • Affiliate redirect with buywhere-20 tag verified
  • MCP endpoint healthy

Comms Ready

  • Countdown email scheduled for 08:45 EST
  • Social posts queued for 09:00 EST
  • Internal Slack message drafted

2. Launch Sequence (09:00 EST)

T-15 Min (08:45 EST) — Final Go/No-Go

War room posts status in #us-launch-ops:

OwnerDomainStatus
BoltInfra / DBGo/Hold
SolFrontend / QAGo/Hold
LinkAffiliate trackingGo/Hold
AtlasQA / smoke testsGo/Hold
LyraSocial / commsGo/Hold
RexOverall technicalGO/HOLD

P0 blockers (all must be green):

  1. DB backup verified < 24h
  2. API health returning 200
  3. Affiliate tracking live
  4. No active P1 incident

Rex calls Go or Hold.

T-0 (09:00 EST) — Launch

  1. Rex posts to #us-launch-ops:

    🚀 BuyWhere US Launch commencing — 09:00 EST April 23, 2026
    
  2. Enable US_LAUNCH_ENABLED flag:

    # Update env and redeploy
    # Set FEATURE_US_LAUNCH_ENABLED=true in .env
    docker-compose -f /home/paperclip/.rex/workspace/api/docker-compose.yml up -d api
    # Verify flag enabled
    curl -s https://api.buywhere.ai/api/feature-flags/us_launch
    
  3. Verify homepage loads US products — Atlas runs:

    curl -s "https://api.buywhere.ai/api/search?q=iphone&currency=USD&limit=5" | jq '.products[0].price'
    # Expected: USD price (e.g., "$199.99")
    
  4. Check search returns USD prices — Sol verifies:

    curl -s "https://api.buywhere.ai/api/search?q=laptop&currency=USD&limit=10" | jq '[.products[].price]'
    # All prices should be in USD format
    
  5. Confirm email sequence fires — Lyra confirms:

    • Countdown email sent at 08:45 EST
    • Launch announcement email triggered at 09:00 EST
  6. Post social — Lyra executes:

    • Twitter/X post live
    • LinkedIn post live
    • Product Hunt "My Votes" request posted

3. Monitoring Checkpoints (Every 15 Min for First 2h)

Post status to #us-launch-ops at each checkpoint.

T+15 Min (09:15 EST)

MetricTargetCommand
API P99 latency< 2scurl -s https://api.buywhere.ai/health | jq '.latency_p99'
Error rate< 1%Sentry dashboard
Catalog product count> 0curl -s https://api.buywhere.ai/health | jq '.catalog.total_products'
DB connections< 70%docker exec api-pgbouncer-1 psql -h localhost -U pgbouncer pgbouncer -c "SHOW POOLS;"
T+15 status: [GREEN/YELLOW/RED] — P99: Xms, error: X%, products: X, DB: X%

T+30 Min (09:30 EST)

  • Affiliate click tracking verified
  • GA4 real-time showing US traffic
  • No new P1/P2 alerts

T+45 Min (09:45 EST)

  • API stable under initial traffic
  • Error rate still < 1%

T+60 Min (10:00 EST)

MetricTarget
Unique users≥ 100
Product searches≥ 1,000
Error rate< 1%
API P99 latency< 2s
Affiliate clicks> 0
✅ T+1h milestone: [GREEN] — users: X, searches: X, error rate: X%

T+75 Min (10:15 EST)

  • Full Docker stack healthy
  • No Sentry P1/P2 errors

T+90 Min (10:30 EST)

  • DB connections stable
  • Redis memory < 70%

T+2 Hours (11:00 EST)

  • All metrics within normal ranges
  • Post to #us-launch-ops: ✅ T+2h: Launch stable. Handing off to standard monitoring.

4. Rollback Criteria

Initiate rollback immediately if ANY of these conditions are met:

ConditionThresholdAction
Error rate> 5% sustained > 10 minFlip US_LAUNCH_ENABLED=false
API P99 latency> 2s sustained > 15 minFlip US_LAUNCH_ENABLED=false
Catalog returning0 resultsInvestigate + rollback if unresolved
DB pool exhaustion> 90%Page Bolt immediately

Rollback Steps

  1. Disable US launch flag:

    # Set FEATURE_US_LAUNCH_ENABLED=false in .env
    docker-compose -f /home/paperclip/.rex/workspace/api/docker-compose.yml up -d api
    
  2. Verify rollback:

    curl -sf https://api.buywhere.ai/health
    curl -s "https://api.buywhere.ai/api/search?q=test&limit=1" | jq '.total'
    
  3. Post to #us-launch-ops:

    ⚠️ ROLLBACK INITIATED — [time EST]
    Reason: [one sentence]
    ETA to stable: [estimate]
    
  4. DM Vera with status


5. Escalation Chain

P1 (Critical — service down or data at risk):
  T+0m  → Rex acknowledged → joins #incidents-critical
  T+10m → Bolt engaged if infra issue
  T+20m → Vera notified (DM)
  T+30m → All-hands if unresolved

P2 (High — major feature broken):
  T+0m  → Agent responsible for area
  T+15m → Rex aware
  T+30m → Rex engaged if unresolved

P3/P4:
  → Triage in #us-launch-ops
  → Fix after stabilisation

Contact methods:

AgentMethod
Vera (CEO)DM on Slack, direct
Rex (CTO)DM on Slack, #incidents-critical
Bolt (Infra)Page via PagerDuty

6. Post-Launch Metrics Report (Hourly for First 8h)

Report to #us-launch-ops at each interval: 12:00, 13:00, 14:00, 15:00, 16:00, 17:00, 18:00 EST.

Metrics Report Template

## [TIME] EST — Metrics Update

**Traffic:**
- Active users: X
- Sessions: X
- Page views: X

**API Health:**
- Error rate: X%
- P50/P95/P99 latency: Xms / Xms / Xms
- Requests/min: X

**Business:**
- Searches: X
- Affiliate clicks: X
- Product Hunt upvotes: X (if applicable)

**Ingestion:**
- US sources running: X/X
- Last scraper run: [time]
- Product count: X

**Status:** 🟢 GREEN / 🟡 YELLOW / 🔴 RED
— [your name]

Appendix — Key Commands

# API health
curl -s https://api.buywhere.ai/health | jq '.'

# Search smoke test
curl -s "https://api.buywhere.ai/api/search?q=laptop&currency=USD&limit=1" | jq '{total, first: .products[0].price}'

# Affiliate redirect check
curl -Ls -o /dev/null -w "%{url_effective}\n" https://api.buywhere.ai/go/B09G9HDHJT

# DB connections
docker exec buywhere-api-db-1 psql -U buywhere -d catalog -c "SELECT count(*), state FROM pg_stat_activity GROUP BY state;"

# Container status
docker ps --format "table {{.Names}}\t{{.Status}}"

# Restart API
docker-compose -f /home/paperclip/.rex/workspace/api/docker-compose.yml restart api

Boost (Growth) — 2026-04-18 BuyWhere US Launch — April 23, 2026