diff --git a/DEPLOYMENT.md b/DEPLOYMENT.md new file mode 100644 index 0000000..0184483 --- /dev/null +++ b/DEPLOYMENT.md @@ -0,0 +1,682 @@ +# GRIMLOCK DEPLOYMENT CHECKLIST + +## PRE-DEPLOYMENT CHECKS + +### ✅ Backend Verification +```bash +cd grimlock + +# 1. Start backend +docker-compose up -d + +# 2. Check health +curl http://localhost:8000/api/health +# Expected: {"status":"healthy","context_loaded":true,...} + +# 3. Run tests +python test_api.py +# Expected: All tests pass + +# 4. Check logs +docker-compose logs backend +# Expected: No errors, "Application startup complete" +``` + +### ✅ Frontend Verification +```bash +cd grimlock/frontend + +# 1. Install dependencies +npm install + +# 2. Build check +npm run build +# Expected: ✓ Compiled successfully + +# 3. Start dev server +npm run dev +# Expected: Server running on http://localhost:3000 +``` + +### ✅ Environment Variables + +**Backend (.env)** +```env +# Required +ANTHROPIC_API_KEY=sk-ant-xxxxx # Your Claude API key +SECRET_KEY=xxxxx # Random secret for JWT +DATABASE_URL=postgresql://grimlock:grimlock@postgres/grimlock + +# Optional (already configured) +REDIS_URL=redis://redis:6379 +``` + +**Frontend (.env.local)** +```env +NEXT_PUBLIC_API_URL=http://localhost:8000 +NEXT_PUBLIC_WS_URL=http://localhost:8000 +``` + +--- + +## FIRST-TIME SETUP + +### Step 1: Clone Repository +```bash +git clone https://gittea.979labs.com/amitis55/grimlock.git +cd grimlock +``` + +### Step 2: Configure Backend +```bash +cd backend + +# Copy example env +cp .env.example .env + +# Edit .env and add: +# - Your ANTHROPIC_API_KEY +# - Generate SECRET_KEY: openssl rand -hex 32 +nano .env +``` + +### Step 3: Start Services +```bash +cd .. # back to grimlock/ + +# Start backend with Docker +docker-compose up -d + +# Wait 10 seconds for startup +sleep 10 + +# Verify backend +curl http://localhost:8000/api/health +``` + +### Step 4: Setup Frontend +```bash +cd frontend + +# Install dependencies +npm install + +# Create env file +cat > .env.local << EOF +NEXT_PUBLIC_API_URL=http://localhost:8000 +NEXT_PUBLIC_WS_URL=http://localhost:8000 +EOF + +# Start frontend +npm run dev +``` + +### Step 5: First User +```bash +# Open browser: http://localhost:3000 +# Click "Sign up" +# Fill in: +# Email: admin@vectorzulu.com +# Name: Admin +# Password: (secure password) +# Role: admin + +# This creates first user and channel +``` + +--- + +## PRODUCTION DEPLOYMENT + +### Option 1: Docker Compose (Recommended) + +**1. Production Environment** +```bash +# Create production .env +cd grimlock/backend +cp .env.example .env.production + +# Edit with production values: +# - DATABASE_URL (production PostgreSQL) +# - REDIS_URL (production Redis) +# - ANTHROPIC_API_KEY +# - Secure SECRET_KEY +``` + +**2. Frontend Build** +```bash +cd grimlock/frontend + +# Production build +npm run build + +# Test production build +npm start +``` + +**3. Deploy with Docker** +```bash +cd grimlock + +# Use production env +export ENV_FILE=backend/.env.production + +# Build and start +docker-compose -f docker-compose.prod.yml up -d + +# Note: You'll need to create docker-compose.prod.yml +# with production settings +``` + +### Option 2: Separate Services + +**Backend Deployment (e.g., AWS EC2, DigitalOcean)** +```bash +# On server +cd grimlock/backend + +# Install Python dependencies +pip install -r requirements.txt + +# Run with production server +uvicorn main:app --host 0.0.0.0 --port 8000 --workers 4 + +# Or with gunicorn +gunicorn main:app -w 4 -k uvicorn.workers.UvicornWorker -b 0.0.0.0:8000 +``` + +**Frontend Deployment (e.g., Vercel, Netlify)** +```bash +# Set environment variables in platform: +NEXT_PUBLIC_API_URL=https://api.yourdomain.com +NEXT_PUBLIC_WS_URL=https://api.yourdomain.com + +# Deploy +npm run build + +# Platform will auto-deploy from git +``` + +### Option 3: Kubernetes (Advanced) + +**Coming soon:** Helm charts and K8s manifests + +--- + +## DATABASE SETUP + +### Local Development +```bash +# PostgreSQL is included in docker-compose +# No additional setup needed + +# To access database: +docker-compose exec postgres psql -U grimlock -d grimlock +``` + +### Production Database + +**Option A: Managed Database (Recommended)** +- AWS RDS (PostgreSQL) +- DigitalOcean Managed Database +- Google Cloud SQL +- Heroku Postgres + +**Option B: Self-Hosted** +```bash +# Install PostgreSQL +sudo apt install postgresql postgresql-contrib + +# Create database +sudo -u postgres psql +CREATE DATABASE grimlock; +CREATE USER grimlock WITH PASSWORD 'secure_password'; +GRANT ALL PRIVILEGES ON DATABASE grimlock TO grimlock; + +# Update DATABASE_URL in .env +DATABASE_URL=postgresql://grimlock:secure_password@localhost/grimlock +``` + +--- + +## TESTING DEPLOYMENT + +### Automated Health Checks +```bash +#!/bin/bash +# health_check.sh + +# Backend health +HEALTH=$(curl -s http://localhost:8000/api/health) +if echo "$HEALTH" | grep -q '"status":"healthy"'; then + echo "✓ Backend healthy" +else + echo "✗ Backend unhealthy" + exit 1 +fi + +# Frontend health +FRONTEND=$(curl -s http://localhost:3000) +if [ $? -eq 0 ]; then + echo "✓ Frontend accessible" +else + echo "✗ Frontend not accessible" + exit 1 +fi + +# WebSocket test +# (would need more complex test) + +echo "✓ All services healthy" +``` + +### Manual Test Flow +1. **Registration** + - Go to /register + - Create account + - Should redirect to / + +2. **Channel Creation** + - Click "+" next to Channels + - Create "general" channel + - Should open channel page + +3. **Messaging** + - Send test message + - Should appear instantly + - Send "@grimlock hello" + - Should get AI response + +4. **Real-Time** + - Open second browser/incognito + - Login as different user + - Send message from one + - Should appear in both instantly + +5. **Direct Messages** + - Click on a user in sidebar + - Send DM + - Should work like channels + +--- + +## MONITORING + +### Backend Logs +```bash +# Docker logs +docker-compose logs -f backend + +# Look for: +# - "Application startup complete" +# - "Connected to database" +# - "WebSocket server started" +# - No error tracebacks +``` + +### Frontend Logs +```bash +# Terminal output +npm run dev + +# Browser console +# - No red errors +# - WebSocket connected +# - API calls succeeding +``` + +### Database Health +```bash +# Check connections +docker-compose exec postgres psql -U grimlock -d grimlock -c "SELECT count(*) FROM pg_stat_activity;" + +# Check table sizes +docker-compose exec postgres psql -U grimlock -d grimlock -c " +SELECT + schemaname, + tablename, + pg_size_pretty(pg_total_relation_size(schemaname||'.'||tablename)) AS size +FROM pg_tables +WHERE schemaname = 'public' +ORDER BY pg_total_relation_size(schemaname||'.'||tablename) DESC; +" +``` + +--- + +## PERFORMANCE TUNING + +### Backend +```python +# main.py - increase workers for production +if __name__ == "__main__": + import uvicorn + uvicorn.run( + "main:app", + host="0.0.0.0", + port=8000, + workers=4, # Adjust based on CPU cores + reload=False # Disable in production + ) +``` + +### Database +```sql +-- Add indexes for common queries +CREATE INDEX idx_messages_channel_id ON messages(channel_id); +CREATE INDEX idx_messages_created_at ON messages(created_at DESC); +CREATE INDEX idx_direct_messages_sender ON direct_messages(sender_id); +CREATE INDEX idx_direct_messages_recipient ON direct_messages(recipient_id); +``` + +### Frontend +```bash +# Production build is already optimized +npm run build + +# Features: +# - Code splitting +# - Minification +# - Tree shaking +# - Image optimization +``` + +--- + +## BACKUP & RECOVERY + +### Database Backup +```bash +# Automated daily backup +#!/bin/bash +# backup.sh + +DATE=$(date +%Y%m%d_%H%M%S) +BACKUP_FILE="grimlock_backup_$DATE.sql" + +docker-compose exec -T postgres pg_dump -U grimlock grimlock > "$BACKUP_FILE" + +# Compress +gzip "$BACKUP_FILE" + +# Upload to S3 (optional) +# aws s3 cp "$BACKUP_FILE.gz" s3://your-bucket/backups/ + +echo "Backup completed: $BACKUP_FILE.gz" +``` + +### Restore from Backup +```bash +# Stop services +docker-compose down + +# Restore database +gunzip grimlock_backup_20260213.sql.gz +docker-compose exec -T postgres psql -U grimlock grimlock < grimlock_backup_20260213.sql + +# Start services +docker-compose up -d +``` + +--- + +## SECURITY CHECKLIST + +### ✅ Before Production + +- [ ] Change default SECRET_KEY +- [ ] Use strong database password +- [ ] Enable HTTPS (SSL/TLS) +- [ ] Set CORS policies +- [ ] Rate limiting configured +- [ ] Input validation enabled +- [ ] SQL injection protection (using ORM) +- [ ] XSS protection (React auto-escapes) +- [ ] CSRF tokens (for forms) +- [ ] Regular security updates + +### ✅ Access Control + +- [ ] Database not publicly accessible +- [ ] Redis not publicly accessible +- [ ] Backend API behind firewall +- [ ] Only HTTPS endpoints exposed +- [ ] Admin routes protected +- [ ] File uploads size-limited +- [ ] File types validated + +--- + +## TROUBLESHOOTING + +### Backend Won't Start +```bash +# Check logs +docker-compose logs backend + +# Common issues: +# 1. Database not ready +# Solution: Wait 10 seconds after docker-compose up + +# 2. ANTHROPIC_API_KEY missing +# Solution: Add to .env file + +# 3. Port 8000 in use +# Solution: Stop other service or change port +``` + +### Frontend Won't Build +```bash +# Clear cache and rebuild +rm -rf .next node_modules package-lock.json +npm install +npm run build + +# Check for type errors +npm run build 2>&1 | grep "Type error" +``` + +### WebSocket Not Connecting +```bash +# Check backend WebSocket server +docker-compose logs backend | grep -i websocket + +# Check frontend .env.local +cat frontend/.env.local + +# Must have: +# NEXT_PUBLIC_WS_URL=http://localhost:8000 + +# In browser console, should see: +# "WebSocket connected" +``` + +### Database Connection Issues +```bash +# Test database connectivity +docker-compose exec postgres psql -U grimlock -d grimlock -c "SELECT 1;" + +# Check DATABASE_URL in .env +# Should be: postgresql://grimlock:grimlock@postgres/grimlock +``` + +--- + +## SCALING CONSIDERATIONS + +### Horizontal Scaling + +**Backend:** +- Stateless design allows multiple instances +- Load balancer distributes requests +- Shared PostgreSQL + Redis +- WebSocket sticky sessions required + +**Frontend:** +- Static files can be CDN-cached +- Multiple Next.js instances +- API calls to load balancer + +### Database Scaling + +**Read Replicas:** +- Master for writes +- Replicas for reads +- Update DATABASE_URL for reads + +**Connection Pooling:** +- Use PgBouncer +- Limit connections per instance +- Monitor connection usage + +### Caching + +**Redis:** +- Cache user sessions +- Cache channel lists +- Cache recent messages +- Set TTL appropriately + +--- + +## COST ESTIMATES + +### Development (Current) +- Backend: Local Docker (free) +- Frontend: Local development (free) +- Database: Local PostgreSQL (free) +- AI: ~$0.003/query + - 100 queries/day = $9/month + +### Production (Estimated) + +**Small Team (10 users)** +- VPS (2 GB RAM): $12/month +- Database (managed): $15/month +- AI (1000 queries/day): $90/month +- **Total: ~$120/month** + +**Medium Team (50 users)** +- VPS (4 GB RAM): $24/month +- Database (managed): $25/month +- AI (5000 queries/day): $450/month +- **Total: ~$500/month** + +**Enterprise (500 users)** +- Kubernetes cluster: $200/month +- Database (HA): $100/month +- AI (50K queries/day): $4,500/month +- CDN: $50/month +- **Total: ~$5,000/month** + +--- + +## SUCCESS METRICS + +### Technical Health +- [ ] Backend uptime > 99.9% +- [ ] API response time < 200ms +- [ ] WebSocket latency < 50ms +- [ ] Database queries < 100ms +- [ ] Zero critical errors + +### User Adoption +- [ ] 10+ daily active users +- [ ] 100+ messages per day +- [ ] 10+ channels created +- [ ] 50+ @grimlock queries/day +- [ ] Users prefer over Slack + +### Business Impact +- [ ] 5+ hours saved per week +- [ ] Reduced context switching +- [ ] Faster decision making +- [ ] Better knowledge retention +- [ ] Team satisfaction improved + +--- + +## NEXT STEPS AFTER DEPLOYMENT + +1. **Week 1: Internal Alpha** + - Deploy to staging + - Team of 5 tests + - Gather feedback + - Fix critical bugs + +2. **Week 2: Internal Beta** + - Deploy to production + - All Vector Zulu employees + - Monitor performance + - Iterate on UX + +3. **Week 3-4: Optimization** + - Add missing features + - Improve performance + - Polish UI/UX + - Write user docs + +4. **Month 2: Scale** + - Handle more users + - Add integrations + - External beta + - Prepare for launch + +--- + +## SUPPORT & MAINTENANCE + +### Daily +- Monitor logs for errors +- Check system health +- Review AI usage +- Respond to user issues + +### Weekly +- Database backup verification +- Security updates +- Performance review +- User feedback review + +### Monthly +- Dependency updates +- Feature planning +- Cost review +- Capacity planning + +--- + +## CONTACT & RESOURCES + +**Repository:** +https://gittea.979labs.com/amitis55/grimlock + +**Documentation:** +- FINAL.md - Backend API +- FRONTEND.md - Frontend deployment +- ARCHITECTURE.md - System design +- This file - Deployment guide + +**Questions?** +Create an issue in the repository or contact the dev team. + +--- + +# 🚀 READY TO DEPLOY + +You now have: +- ✅ Complete backend platform +- ✅ Complete frontend application +- ✅ Deployment documentation +- ✅ Testing procedures +- ✅ Monitoring setup +- ✅ Troubleshooting guide + +**Next command:** +```bash +cd grimlock && docker-compose up -d && cd frontend && npm run dev +``` + +**Then open:** http://localhost:3000 + +**Welcome to Grimlock.**