Documentation: Complete deployment and operations guide
This commit is contained in:
682
DEPLOYMENT.md
Normal file
682
DEPLOYMENT.md
Normal file
@@ -0,0 +1,682 @@
|
||||
# GRIMLOCK DEPLOYMENT CHECKLIST
|
||||
|
||||
## PRE-DEPLOYMENT CHECKS
|
||||
|
||||
### ✅ Backend Verification
|
||||
```bash
|
||||
cd grimlock
|
||||
|
||||
# 1. Start backend
|
||||
docker-compose up -d
|
||||
|
||||
# 2. Check health
|
||||
curl http://localhost:8000/api/health
|
||||
# Expected: {"status":"healthy","context_loaded":true,...}
|
||||
|
||||
# 3. Run tests
|
||||
python test_api.py
|
||||
# Expected: All tests pass
|
||||
|
||||
# 4. Check logs
|
||||
docker-compose logs backend
|
||||
# Expected: No errors, "Application startup complete"
|
||||
```
|
||||
|
||||
### ✅ Frontend Verification
|
||||
```bash
|
||||
cd grimlock/frontend
|
||||
|
||||
# 1. Install dependencies
|
||||
npm install
|
||||
|
||||
# 2. Build check
|
||||
npm run build
|
||||
# Expected: ✓ Compiled successfully
|
||||
|
||||
# 3. Start dev server
|
||||
npm run dev
|
||||
# Expected: Server running on http://localhost:3000
|
||||
```
|
||||
|
||||
### ✅ Environment Variables
|
||||
|
||||
**Backend (.env)**
|
||||
```env
|
||||
# Required
|
||||
ANTHROPIC_API_KEY=sk-ant-xxxxx # Your Claude API key
|
||||
SECRET_KEY=xxxxx # Random secret for JWT
|
||||
DATABASE_URL=postgresql://grimlock:grimlock@postgres/grimlock
|
||||
|
||||
# Optional (already configured)
|
||||
REDIS_URL=redis://redis:6379
|
||||
```
|
||||
|
||||
**Frontend (.env.local)**
|
||||
```env
|
||||
NEXT_PUBLIC_API_URL=http://localhost:8000
|
||||
NEXT_PUBLIC_WS_URL=http://localhost:8000
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## FIRST-TIME SETUP
|
||||
|
||||
### Step 1: Clone Repository
|
||||
```bash
|
||||
git clone https://gittea.979labs.com/amitis55/grimlock.git
|
||||
cd grimlock
|
||||
```
|
||||
|
||||
### Step 2: Configure Backend
|
||||
```bash
|
||||
cd backend
|
||||
|
||||
# Copy example env
|
||||
cp .env.example .env
|
||||
|
||||
# Edit .env and add:
|
||||
# - Your ANTHROPIC_API_KEY
|
||||
# - Generate SECRET_KEY: openssl rand -hex 32
|
||||
nano .env
|
||||
```
|
||||
|
||||
### Step 3: Start Services
|
||||
```bash
|
||||
cd .. # back to grimlock/
|
||||
|
||||
# Start backend with Docker
|
||||
docker-compose up -d
|
||||
|
||||
# Wait 10 seconds for startup
|
||||
sleep 10
|
||||
|
||||
# Verify backend
|
||||
curl http://localhost:8000/api/health
|
||||
```
|
||||
|
||||
### Step 4: Setup Frontend
|
||||
```bash
|
||||
cd frontend
|
||||
|
||||
# Install dependencies
|
||||
npm install
|
||||
|
||||
# Create env file
|
||||
cat > .env.local << EOF
|
||||
NEXT_PUBLIC_API_URL=http://localhost:8000
|
||||
NEXT_PUBLIC_WS_URL=http://localhost:8000
|
||||
EOF
|
||||
|
||||
# Start frontend
|
||||
npm run dev
|
||||
```
|
||||
|
||||
### Step 5: First User
|
||||
```bash
|
||||
# Open browser: http://localhost:3000
|
||||
# Click "Sign up"
|
||||
# Fill in:
|
||||
# Email: admin@vectorzulu.com
|
||||
# Name: Admin
|
||||
# Password: (secure password)
|
||||
# Role: admin
|
||||
|
||||
# This creates first user and channel
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## PRODUCTION DEPLOYMENT
|
||||
|
||||
### Option 1: Docker Compose (Recommended)
|
||||
|
||||
**1. Production Environment**
|
||||
```bash
|
||||
# Create production .env
|
||||
cd grimlock/backend
|
||||
cp .env.example .env.production
|
||||
|
||||
# Edit with production values:
|
||||
# - DATABASE_URL (production PostgreSQL)
|
||||
# - REDIS_URL (production Redis)
|
||||
# - ANTHROPIC_API_KEY
|
||||
# - Secure SECRET_KEY
|
||||
```
|
||||
|
||||
**2. Frontend Build**
|
||||
```bash
|
||||
cd grimlock/frontend
|
||||
|
||||
# Production build
|
||||
npm run build
|
||||
|
||||
# Test production build
|
||||
npm start
|
||||
```
|
||||
|
||||
**3. Deploy with Docker**
|
||||
```bash
|
||||
cd grimlock
|
||||
|
||||
# Use production env
|
||||
export ENV_FILE=backend/.env.production
|
||||
|
||||
# Build and start
|
||||
docker-compose -f docker-compose.prod.yml up -d
|
||||
|
||||
# Note: You'll need to create docker-compose.prod.yml
|
||||
# with production settings
|
||||
```
|
||||
|
||||
### Option 2: Separate Services
|
||||
|
||||
**Backend Deployment (e.g., AWS EC2, DigitalOcean)**
|
||||
```bash
|
||||
# On server
|
||||
cd grimlock/backend
|
||||
|
||||
# Install Python dependencies
|
||||
pip install -r requirements.txt
|
||||
|
||||
# Run with production server
|
||||
uvicorn main:app --host 0.0.0.0 --port 8000 --workers 4
|
||||
|
||||
# Or with gunicorn
|
||||
gunicorn main:app -w 4 -k uvicorn.workers.UvicornWorker -b 0.0.0.0:8000
|
||||
```
|
||||
|
||||
**Frontend Deployment (e.g., Vercel, Netlify)**
|
||||
```bash
|
||||
# Set environment variables in platform:
|
||||
NEXT_PUBLIC_API_URL=https://api.yourdomain.com
|
||||
NEXT_PUBLIC_WS_URL=https://api.yourdomain.com
|
||||
|
||||
# Deploy
|
||||
npm run build
|
||||
|
||||
# Platform will auto-deploy from git
|
||||
```
|
||||
|
||||
### Option 3: Kubernetes (Advanced)
|
||||
|
||||
**Coming soon:** Helm charts and K8s manifests
|
||||
|
||||
---
|
||||
|
||||
## DATABASE SETUP
|
||||
|
||||
### Local Development
|
||||
```bash
|
||||
# PostgreSQL is included in docker-compose
|
||||
# No additional setup needed
|
||||
|
||||
# To access database:
|
||||
docker-compose exec postgres psql -U grimlock -d grimlock
|
||||
```
|
||||
|
||||
### Production Database
|
||||
|
||||
**Option A: Managed Database (Recommended)**
|
||||
- AWS RDS (PostgreSQL)
|
||||
- DigitalOcean Managed Database
|
||||
- Google Cloud SQL
|
||||
- Heroku Postgres
|
||||
|
||||
**Option B: Self-Hosted**
|
||||
```bash
|
||||
# Install PostgreSQL
|
||||
sudo apt install postgresql postgresql-contrib
|
||||
|
||||
# Create database
|
||||
sudo -u postgres psql
|
||||
CREATE DATABASE grimlock;
|
||||
CREATE USER grimlock WITH PASSWORD 'secure_password';
|
||||
GRANT ALL PRIVILEGES ON DATABASE grimlock TO grimlock;
|
||||
|
||||
# Update DATABASE_URL in .env
|
||||
DATABASE_URL=postgresql://grimlock:secure_password@localhost/grimlock
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## TESTING DEPLOYMENT
|
||||
|
||||
### Automated Health Checks
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# health_check.sh
|
||||
|
||||
# Backend health
|
||||
HEALTH=$(curl -s http://localhost:8000/api/health)
|
||||
if echo "$HEALTH" | grep -q '"status":"healthy"'; then
|
||||
echo "✓ Backend healthy"
|
||||
else
|
||||
echo "✗ Backend unhealthy"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Frontend health
|
||||
FRONTEND=$(curl -s http://localhost:3000)
|
||||
if [ $? -eq 0 ]; then
|
||||
echo "✓ Frontend accessible"
|
||||
else
|
||||
echo "✗ Frontend not accessible"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# WebSocket test
|
||||
# (would need more complex test)
|
||||
|
||||
echo "✓ All services healthy"
|
||||
```
|
||||
|
||||
### Manual Test Flow
|
||||
1. **Registration**
|
||||
- Go to /register
|
||||
- Create account
|
||||
- Should redirect to /
|
||||
|
||||
2. **Channel Creation**
|
||||
- Click "+" next to Channels
|
||||
- Create "general" channel
|
||||
- Should open channel page
|
||||
|
||||
3. **Messaging**
|
||||
- Send test message
|
||||
- Should appear instantly
|
||||
- Send "@grimlock hello"
|
||||
- Should get AI response
|
||||
|
||||
4. **Real-Time**
|
||||
- Open second browser/incognito
|
||||
- Login as different user
|
||||
- Send message from one
|
||||
- Should appear in both instantly
|
||||
|
||||
5. **Direct Messages**
|
||||
- Click on a user in sidebar
|
||||
- Send DM
|
||||
- Should work like channels
|
||||
|
||||
---
|
||||
|
||||
## MONITORING
|
||||
|
||||
### Backend Logs
|
||||
```bash
|
||||
# Docker logs
|
||||
docker-compose logs -f backend
|
||||
|
||||
# Look for:
|
||||
# - "Application startup complete"
|
||||
# - "Connected to database"
|
||||
# - "WebSocket server started"
|
||||
# - No error tracebacks
|
||||
```
|
||||
|
||||
### Frontend Logs
|
||||
```bash
|
||||
# Terminal output
|
||||
npm run dev
|
||||
|
||||
# Browser console
|
||||
# - No red errors
|
||||
# - WebSocket connected
|
||||
# - API calls succeeding
|
||||
```
|
||||
|
||||
### Database Health
|
||||
```bash
|
||||
# Check connections
|
||||
docker-compose exec postgres psql -U grimlock -d grimlock -c "SELECT count(*) FROM pg_stat_activity;"
|
||||
|
||||
# Check table sizes
|
||||
docker-compose exec postgres psql -U grimlock -d grimlock -c "
|
||||
SELECT
|
||||
schemaname,
|
||||
tablename,
|
||||
pg_size_pretty(pg_total_relation_size(schemaname||'.'||tablename)) AS size
|
||||
FROM pg_tables
|
||||
WHERE schemaname = 'public'
|
||||
ORDER BY pg_total_relation_size(schemaname||'.'||tablename) DESC;
|
||||
"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## PERFORMANCE TUNING
|
||||
|
||||
### Backend
|
||||
```python
|
||||
# main.py - increase workers for production
|
||||
if __name__ == "__main__":
|
||||
import uvicorn
|
||||
uvicorn.run(
|
||||
"main:app",
|
||||
host="0.0.0.0",
|
||||
port=8000,
|
||||
workers=4, # Adjust based on CPU cores
|
||||
reload=False # Disable in production
|
||||
)
|
||||
```
|
||||
|
||||
### Database
|
||||
```sql
|
||||
-- Add indexes for common queries
|
||||
CREATE INDEX idx_messages_channel_id ON messages(channel_id);
|
||||
CREATE INDEX idx_messages_created_at ON messages(created_at DESC);
|
||||
CREATE INDEX idx_direct_messages_sender ON direct_messages(sender_id);
|
||||
CREATE INDEX idx_direct_messages_recipient ON direct_messages(recipient_id);
|
||||
```
|
||||
|
||||
### Frontend
|
||||
```bash
|
||||
# Production build is already optimized
|
||||
npm run build
|
||||
|
||||
# Features:
|
||||
# - Code splitting
|
||||
# - Minification
|
||||
# - Tree shaking
|
||||
# - Image optimization
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## BACKUP & RECOVERY
|
||||
|
||||
### Database Backup
|
||||
```bash
|
||||
# Automated daily backup
|
||||
#!/bin/bash
|
||||
# backup.sh
|
||||
|
||||
DATE=$(date +%Y%m%d_%H%M%S)
|
||||
BACKUP_FILE="grimlock_backup_$DATE.sql"
|
||||
|
||||
docker-compose exec -T postgres pg_dump -U grimlock grimlock > "$BACKUP_FILE"
|
||||
|
||||
# Compress
|
||||
gzip "$BACKUP_FILE"
|
||||
|
||||
# Upload to S3 (optional)
|
||||
# aws s3 cp "$BACKUP_FILE.gz" s3://your-bucket/backups/
|
||||
|
||||
echo "Backup completed: $BACKUP_FILE.gz"
|
||||
```
|
||||
|
||||
### Restore from Backup
|
||||
```bash
|
||||
# Stop services
|
||||
docker-compose down
|
||||
|
||||
# Restore database
|
||||
gunzip grimlock_backup_20260213.sql.gz
|
||||
docker-compose exec -T postgres psql -U grimlock grimlock < grimlock_backup_20260213.sql
|
||||
|
||||
# Start services
|
||||
docker-compose up -d
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## SECURITY CHECKLIST
|
||||
|
||||
### ✅ Before Production
|
||||
|
||||
- [ ] Change default SECRET_KEY
|
||||
- [ ] Use strong database password
|
||||
- [ ] Enable HTTPS (SSL/TLS)
|
||||
- [ ] Set CORS policies
|
||||
- [ ] Rate limiting configured
|
||||
- [ ] Input validation enabled
|
||||
- [ ] SQL injection protection (using ORM)
|
||||
- [ ] XSS protection (React auto-escapes)
|
||||
- [ ] CSRF tokens (for forms)
|
||||
- [ ] Regular security updates
|
||||
|
||||
### ✅ Access Control
|
||||
|
||||
- [ ] Database not publicly accessible
|
||||
- [ ] Redis not publicly accessible
|
||||
- [ ] Backend API behind firewall
|
||||
- [ ] Only HTTPS endpoints exposed
|
||||
- [ ] Admin routes protected
|
||||
- [ ] File uploads size-limited
|
||||
- [ ] File types validated
|
||||
|
||||
---
|
||||
|
||||
## TROUBLESHOOTING
|
||||
|
||||
### Backend Won't Start
|
||||
```bash
|
||||
# Check logs
|
||||
docker-compose logs backend
|
||||
|
||||
# Common issues:
|
||||
# 1. Database not ready
|
||||
# Solution: Wait 10 seconds after docker-compose up
|
||||
|
||||
# 2. ANTHROPIC_API_KEY missing
|
||||
# Solution: Add to .env file
|
||||
|
||||
# 3. Port 8000 in use
|
||||
# Solution: Stop other service or change port
|
||||
```
|
||||
|
||||
### Frontend Won't Build
|
||||
```bash
|
||||
# Clear cache and rebuild
|
||||
rm -rf .next node_modules package-lock.json
|
||||
npm install
|
||||
npm run build
|
||||
|
||||
# Check for type errors
|
||||
npm run build 2>&1 | grep "Type error"
|
||||
```
|
||||
|
||||
### WebSocket Not Connecting
|
||||
```bash
|
||||
# Check backend WebSocket server
|
||||
docker-compose logs backend | grep -i websocket
|
||||
|
||||
# Check frontend .env.local
|
||||
cat frontend/.env.local
|
||||
|
||||
# Must have:
|
||||
# NEXT_PUBLIC_WS_URL=http://localhost:8000
|
||||
|
||||
# In browser console, should see:
|
||||
# "WebSocket connected"
|
||||
```
|
||||
|
||||
### Database Connection Issues
|
||||
```bash
|
||||
# Test database connectivity
|
||||
docker-compose exec postgres psql -U grimlock -d grimlock -c "SELECT 1;"
|
||||
|
||||
# Check DATABASE_URL in .env
|
||||
# Should be: postgresql://grimlock:grimlock@postgres/grimlock
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## SCALING CONSIDERATIONS
|
||||
|
||||
### Horizontal Scaling
|
||||
|
||||
**Backend:**
|
||||
- Stateless design allows multiple instances
|
||||
- Load balancer distributes requests
|
||||
- Shared PostgreSQL + Redis
|
||||
- WebSocket sticky sessions required
|
||||
|
||||
**Frontend:**
|
||||
- Static files can be CDN-cached
|
||||
- Multiple Next.js instances
|
||||
- API calls to load balancer
|
||||
|
||||
### Database Scaling
|
||||
|
||||
**Read Replicas:**
|
||||
- Master for writes
|
||||
- Replicas for reads
|
||||
- Update DATABASE_URL for reads
|
||||
|
||||
**Connection Pooling:**
|
||||
- Use PgBouncer
|
||||
- Limit connections per instance
|
||||
- Monitor connection usage
|
||||
|
||||
### Caching
|
||||
|
||||
**Redis:**
|
||||
- Cache user sessions
|
||||
- Cache channel lists
|
||||
- Cache recent messages
|
||||
- Set TTL appropriately
|
||||
|
||||
---
|
||||
|
||||
## COST ESTIMATES
|
||||
|
||||
### Development (Current)
|
||||
- Backend: Local Docker (free)
|
||||
- Frontend: Local development (free)
|
||||
- Database: Local PostgreSQL (free)
|
||||
- AI: ~$0.003/query
|
||||
- 100 queries/day = $9/month
|
||||
|
||||
### Production (Estimated)
|
||||
|
||||
**Small Team (10 users)**
|
||||
- VPS (2 GB RAM): $12/month
|
||||
- Database (managed): $15/month
|
||||
- AI (1000 queries/day): $90/month
|
||||
- **Total: ~$120/month**
|
||||
|
||||
**Medium Team (50 users)**
|
||||
- VPS (4 GB RAM): $24/month
|
||||
- Database (managed): $25/month
|
||||
- AI (5000 queries/day): $450/month
|
||||
- **Total: ~$500/month**
|
||||
|
||||
**Enterprise (500 users)**
|
||||
- Kubernetes cluster: $200/month
|
||||
- Database (HA): $100/month
|
||||
- AI (50K queries/day): $4,500/month
|
||||
- CDN: $50/month
|
||||
- **Total: ~$5,000/month**
|
||||
|
||||
---
|
||||
|
||||
## SUCCESS METRICS
|
||||
|
||||
### Technical Health
|
||||
- [ ] Backend uptime > 99.9%
|
||||
- [ ] API response time < 200ms
|
||||
- [ ] WebSocket latency < 50ms
|
||||
- [ ] Database queries < 100ms
|
||||
- [ ] Zero critical errors
|
||||
|
||||
### User Adoption
|
||||
- [ ] 10+ daily active users
|
||||
- [ ] 100+ messages per day
|
||||
- [ ] 10+ channels created
|
||||
- [ ] 50+ @grimlock queries/day
|
||||
- [ ] Users prefer over Slack
|
||||
|
||||
### Business Impact
|
||||
- [ ] 5+ hours saved per week
|
||||
- [ ] Reduced context switching
|
||||
- [ ] Faster decision making
|
||||
- [ ] Better knowledge retention
|
||||
- [ ] Team satisfaction improved
|
||||
|
||||
---
|
||||
|
||||
## NEXT STEPS AFTER DEPLOYMENT
|
||||
|
||||
1. **Week 1: Internal Alpha**
|
||||
- Deploy to staging
|
||||
- Team of 5 tests
|
||||
- Gather feedback
|
||||
- Fix critical bugs
|
||||
|
||||
2. **Week 2: Internal Beta**
|
||||
- Deploy to production
|
||||
- All Vector Zulu employees
|
||||
- Monitor performance
|
||||
- Iterate on UX
|
||||
|
||||
3. **Week 3-4: Optimization**
|
||||
- Add missing features
|
||||
- Improve performance
|
||||
- Polish UI/UX
|
||||
- Write user docs
|
||||
|
||||
4. **Month 2: Scale**
|
||||
- Handle more users
|
||||
- Add integrations
|
||||
- External beta
|
||||
- Prepare for launch
|
||||
|
||||
---
|
||||
|
||||
## SUPPORT & MAINTENANCE
|
||||
|
||||
### Daily
|
||||
- Monitor logs for errors
|
||||
- Check system health
|
||||
- Review AI usage
|
||||
- Respond to user issues
|
||||
|
||||
### Weekly
|
||||
- Database backup verification
|
||||
- Security updates
|
||||
- Performance review
|
||||
- User feedback review
|
||||
|
||||
### Monthly
|
||||
- Dependency updates
|
||||
- Feature planning
|
||||
- Cost review
|
||||
- Capacity planning
|
||||
|
||||
---
|
||||
|
||||
## CONTACT & RESOURCES
|
||||
|
||||
**Repository:**
|
||||
https://gittea.979labs.com/amitis55/grimlock
|
||||
|
||||
**Documentation:**
|
||||
- FINAL.md - Backend API
|
||||
- FRONTEND.md - Frontend deployment
|
||||
- ARCHITECTURE.md - System design
|
||||
- This file - Deployment guide
|
||||
|
||||
**Questions?**
|
||||
Create an issue in the repository or contact the dev team.
|
||||
|
||||
---
|
||||
|
||||
# 🚀 READY TO DEPLOY
|
||||
|
||||
You now have:
|
||||
- ✅ Complete backend platform
|
||||
- ✅ Complete frontend application
|
||||
- ✅ Deployment documentation
|
||||
- ✅ Testing procedures
|
||||
- ✅ Monitoring setup
|
||||
- ✅ Troubleshooting guide
|
||||
|
||||
**Next command:**
|
||||
```bash
|
||||
cd grimlock && docker-compose up -d && cd frontend && npm run dev
|
||||
```
|
||||
|
||||
**Then open:** http://localhost:3000
|
||||
|
||||
**Welcome to Grimlock.**
|
||||
Reference in New Issue
Block a user