deployment-heroku-troubleshooting

Application crashes on Heroku with H10, H12, H13, H14 errors

Heroku Troubleshooting

Scope: Debugging Heroku apps, log analysis, crashes, performance issues, scaling strategies Lines: ~320 Last Updated: 2025-10-18

When to Use This Skill

Activate this skill when:

Core Concepts

Heroku Error Codes

HTTP Status Errors (H-codes):

Runtime Errors (R-codes):

Log Aggregation

Log sources:

Log structure (Heroku format):

timestamp source[process]: message
2025-10-18T12:34:56.789Z app[web.1]: Starting application...
2025-10-18T12:34:57.123Z heroku[web.1]: State changed from starting to up
2025-10-18T12:35:00.456Z heroku[router]: at=info method=GET path="/" host=myapp.herokuapp.com

Performance Metrics

Response time components:

Key metrics:

Patterns

Pattern 1: Debugging Application Crashes (H10)

View recent logs:

# Tail logs (follow)
heroku logs --tail --app myapp

# Last 1000 lines
heroku logs -n 1000

# Filter by process
heroku logs --dyno=web.1

# Search for errors
heroku logs --tail | grep -i error

Common H10 causes and fixes:

1. Missing dependencies:

# Error in logs:
# ModuleNotFoundError: No module named 'requests'

# Fix: Add to requirements.txt
echo "requests==2.31.0" >> requirements.txt
git commit -am "Add missing dependency"
git push heroku main

2. Port binding error:

# ❌ Wrong: Hardcoded port
app.run(host='0.0.0.0', port=5000)

# ✅ Correct: Use $PORT environment variable
import os
port = int(os.environ.get('PORT', 5000))
app.run(host='0.0.0.0', port=port)

3. Syntax error or import error:

# Test locally first
heroku local web

# Or run bash on dyno to debug
heroku run bash
python manage.py check

4. Database connection failure:

# Check DATABASE_URL is set
heroku config:get DATABASE_URL

# Test connection
heroku run python manage.py dbshell

Pattern 2: Fixing Request Timeouts (H12)

Identify slow requests (router logs):

# Filter for slow requests (>5000ms)
heroku logs --tail | grep 'service=[5-9][0-9][0-9][0-9]ms'

# Example output:
# at=info method=GET path="/slow-endpoint" service=12000ms status=200

Common timeout causes:

1. Slow database queries:

# Diagnose with query logging
import logging
logging.getLogger('django.db.backends').setLevel(logging.DEBUG)

# Or use Django Debug Toolbar locally
# Install: pip install django-debug-toolbar

2. Missing database indexes:

-- Check slow queries (Postgres)
SELECT * FROM pg_stat_statements
ORDER BY total_time DESC
LIMIT 10;

-- Add index
CREATE INDEX idx_users_email ON users(email);

3. External API calls:

# ❌ Wrong: Blocking synchronous call
response = requests.get('https://slow-api.com/data')

# ✅ Correct: Timeout + async processing
import requests

try:
    response = requests.get('https://api.com/data', timeout=5)
except requests.Timeout:
    # Fall back or retry later
    logger.error("API timeout")
    return default_response

# Or move to background worker
from celery import shared_task

@shared_task
def fetch_api_data():
    response = requests.get('https://slow-api.com/data', timeout=30)
    # Process response

4. Heavy computation in request cycle:

# ❌ Wrong: CPU-intensive in request
def report_view(request):
    data = generate_complex_report()  # Takes 45 seconds
    return JsonResponse(data)

# ✅ Correct: Async job with status polling
from celery import shared_task

@shared_task
def generate_report(report_id):
    data = generate_complex_report()
    Report.objects.filter(id=report_id).update(data=data, status='complete')

def report_view(request):
    report = Report.objects.create(status='pending')
    generate_report.delay(report.id)
    return JsonResponse({'status': 'pending', 'report_id': report.id})

def report_status_view(request, report_id):
    report = Report.objects.get(id=report_id)
    return JsonResponse({'status': report.status, 'data': report.data})

Pattern 3: Memory Issues (R14/R15)

Check dyno memory usage:

# View metrics in dashboard
heroku logs --tail | grep 'R14\|R15'

# Example output:
# Error R14 (Memory quota exceeded)
# Error R15 (Memory quota vastly exceeded)

Diagnose memory leaks:

Python (memory_profiler):

# Install: pip install memory_profiler

from memory_profiler import profile

@profile
def problematic_function():
    # This decorator logs memory usage line-by-line
    large_list = [i for i in range(10000000)]
    return process_data(large_list)

# Run locally to identify leaks
python -m memory_profiler script.py

Common memory issues:

1. Loading large datasets into memory:

# ❌ Wrong: Load entire dataset
users = User.objects.all()  # Fetches all rows
for user in users:
    process_user(user)

# ✅ Correct: Use iterator
users = User.objects.all().iterator(chunk_size=1000)
for user in users:
    process_user(user)

2. Not closing database connections:

# ❌ Wrong: Connection leak
conn = psycopg2.connect(DATABASE_URL)
cursor = conn.cursor()
cursor.execute("SELECT * FROM users")
# Missing conn.close()

# ✅ Correct: Context manager
import psycopg2
from contextlib import closing

with closing(psycopg2.connect(DATABASE_URL)) as conn:
    with conn.cursor() as cursor:
        cursor.execute("SELECT * FROM users")
        results = cursor.fetchall()

3. Caching too much data:

# ❌ Wrong: Unbounded cache
cache = {}
def get_user(user_id):
    if user_id not in cache:
        cache[user_id] = fetch_user(user_id)
    return cache[user_id]

# ✅ Correct: Use Redis or LRU cache
from functools import lru_cache

@lru_cache(maxsize=1000)
def get_user(user_id):
    return fetch_user(user_id)

Fix: Upgrade dyno type:

# Check current dyno type
heroku ps

# Upgrade to more memory
# Basic: 512MB → Standard-1X: 512MB → Standard-2X: 1GB
heroku ps:type standard-2x

Pattern 4: Boot Timeout (R10)

Common causes:

1. Not binding to $PORT:

// ❌ Wrong: Hardcoded port
app.listen(3000);

// ✅ Correct: Use PORT environment variable
const PORT = process.env.PORT || 3000;
app.listen(PORT, () => {
  console.log(`Server running on port ${PORT}`);
});

2. Slow initialization:

# ❌ Wrong: Database migrations on startup
# This can exceed 60s boot timeout
def app_startup():
    run_migrations()  # 90 seconds
    app.run()

# ✅ Correct: Use release phase in Procfile
# Procfile:
# release: python manage.py migrate
# web: gunicorn myapp.wsgi

3. Cold start with large dependencies:

# Optimize build time
# Cache dependencies, use pre-built wheels

# Python: Use specific versions (faster resolution)
# requirements.txt:
Django==4.2.7  # Not Django>=4.0
psycopg2-binary==2.9.9  # Use binary (not source)

Pattern 5: Database Connection Pool Exhaustion

Symptom:

# Error in logs:
# OperationalError: FATAL: remaining connection slots are reserved
# Or: psycopg2.pool.PoolError: connection pool exhausted

Diagnose:

# Check current connections
heroku pg:info

# Connections: 18/20 (Essential plan has 20 max)

# Find long-running queries
heroku pg:ps

Fixes:

1. Enable connection pooling:

# Increases effective connection limit
heroku pg:connection-pooling:attach DATABASE --as DATABASE_CONNECTION_POOL

# Update DATABASE_URL to use pooled connection
heroku config:set DATABASE_URL=$(heroku config:get DATABASE_CONNECTION_POOL_URL)

2. Reduce connection timeout:

# Django: Close connections after request
DATABASES = {
    'default': {
        # ... other settings
        'CONN_MAX_AGE': 60,  # Close idle connections after 60s
    }
}

3. Use connection pooler in app:

from sqlalchemy.pool import QueuePool

engine = create_engine(
    DATABASE_URL,
    poolclass=QueuePool,
    pool_size=5,              # Max connections per worker
    max_overflow=10,          # Extra connections when pool full
    pool_recycle=3600,        # Recycle connections after 1 hour
    pool_pre_ping=True        # Verify connections before use
)

Pattern 6: Failed Migrations

Symptom:

# Release phase failed
# Error: relation "new_table" already exists

Rollback failed release:

# View releases
heroku releases

# Rollback to previous version
heroku rollback v123

Fix migration:

# Run migration manually to diagnose
heroku run python manage.py migrate --fake-initial

# Or reset migration
heroku run python manage.py migrate app_name zero
heroku run python manage.py migrate

Prevent future issues:

# Use idempotent migrations
from django.db import migrations

class Migration(migrations.Migration):
    def forwards(self, apps, schema_editor):
        # Check if table exists before creating
        if not schema_editor.table_exists('new_table'):
            # Create table
            pass

Pattern 7: High Dyno Load (H23)

Symptom:

# Error H23 (Endpoint exhaustion)
# All dynos busy, requests queuing

Diagnose:

# Check dyno metrics (requires app metrics add-on or dashboard)
heroku logs --tail | grep 'at=info'

# Look for high service times and queue times
# at=info method=GET path="/" service=5000ms queue=2000ms

Fixes:

1. Scale dynos horizontally:

# Add more web dynos
heroku ps:scale web=3

# Or enable autoscaling (requires Performance dynos)
heroku ps:autoscale:enable web --min=2 --max=5 --p95=200

2. Optimize slow endpoints:

# Add caching
from django.views.decorators.cache import cache_page

@cache_page(60 * 5)  # Cache for 5 minutes
def homepage(request):
    return render(request, 'home.html')

3. Move work to background:

# Use Celery for async processing
from celery import shared_task

@shared_task
def send_notification_email(user_id):
    # Offload email sending to worker dyno
    user = User.objects.get(id=user_id)
    send_email(user.email, "Welcome!")

def signup_view(request):
    user = create_user(request.POST)
    send_notification_email.delay(user.id)  # Async
    return redirect('home')

Pattern 8: Log Analysis for Patterns

Extract key metrics:

# Average response time for endpoint
heroku logs -n 10000 | grep 'path="/api/users"' | \
  awk '{print $11}' | sed 's/service=//;s/ms//' | \
  awk '{sum+=$1; count++} END {print sum/count}'

# Count errors by type
heroku logs -n 10000 | grep -i error | \
  awk '{print $5}' | sort | uniq -c | sort -rn

# Top slowest endpoints
heroku logs -n 10000 | grep 'at=info' | \
  awk '{print $8, $11}' | sed 's/path=//;s/service=//' | \
  sort -t' ' -k2 -rn | head -20

Set up structured logging:

import structlog

logger = structlog.get_logger()

logger.info(
    "user_login",
    user_id=user.id,
    ip=request.META['REMOTE_ADDR'],
    duration_ms=elapsed_time * 1000
)

# Search in Papertrail: user_login user_id=123

Quick Reference

Error Code Quick Lookup

Code  | Meaning                        | Common Fix
------|--------------------------------|--------------------------------
H10   | App crashed                    | Check logs for error, fix code
H12   | Request timeout                | Optimize query, add timeout
H13   | Connection closed              | Check dyno health
H14   | No web dynos running           | heroku ps:scale web=1
H18   | Request interrupted            | Client issue, not server
R10   | Boot timeout                   | Bind to $PORT, speed up init
R14   | Memory quota exceeded          | Optimize memory, upgrade dyno
R15   | Memory killed immediately      | Fix memory leak urgently

Essential Debugging Commands

Command                              | Purpose
-------------------------------------|----------------------------------------
heroku logs --tail                   | Follow live logs
heroku logs -n 5000                  | Last 5000 log lines
heroku logs --dyno=web.1             | Filter by dyno
heroku logs --source=app             | Filter by source
heroku ps                            | Check dyno status
heroku ps:restart                    | Restart all dynos
heroku run bash                      | Open shell on dyno
heroku run python manage.py shell    | Django shell
heroku pg:ps                         | Active database queries
heroku pg:kill 1234                  | Kill query by PID
heroku releases                      | View deploy history
heroku rollback v123                 | Rollback to version
heroku maintenance:on                | Enable maintenance mode

Performance Optimization Checklist

✅ DO: Use database connection pooling
✅ DO: Add indexes for frequent queries
✅ DO: Cache expensive computations (Redis)
✅ DO: Move long tasks to worker dynos (Celery)
✅ DO: Set request timeouts on external APIs
✅ DO: Use CDN for static assets (Cloudflare)
✅ DO: Monitor memory usage over time
✅ DO: Scale horizontally for traffic spikes

❌ DON'T: Run CPU-intensive tasks in web dynos
❌ DON'T: Load large datasets into memory
❌ DON'T: Skip database indexes
❌ DON'T: Leave connections open (use context managers)
❌ DON'T: Ignore H12 timeout errors
❌ DON'T: Run migrations in app startup code
❌ DON'T: Deploy without testing locally first

Anti-Patterns

❌ Ignoring H12 timeouts: Accepting slow responses as normal ✅ Profile and optimize slow endpoints, move work to background

❌ No structured logging: Unstructured logs are hard to search/analyze ✅ Use JSON logging with context (user_id, request_id, duration)

❌ Not monitoring memory usage: Only notice when R15 kills app ✅ Track memory metrics, optimize before hitting limits

❌ Deploying on Friday afternoon: Risk of weekend outage ✅ Deploy early in week, monitor for 24h before weekend

❌ No rollback plan: Broken deploy with no quick fix ✅ Test deploys in staging, keep rollback command ready

❌ Running database queries in loops: N+1 query problem ✅ Use select_related/prefetch_related (Django) or JOIN queries

❌ Hardcoded timeouts: External API outages cascade ✅ Set aggressive timeouts (5-10s), fail fast with retries

❌ Single dyno for production: No redundancy, downtime on deploy ✅ Run 2+ web dynos for zero-downtime deploys

❌ No error tracking: Unaware of user-facing errors ✅ Add Sentry or similar for exception monitoring

❌ Guessing at problems: Trying random fixes ✅ Use logs, metrics, and profiling to diagnose root cause

Related Skills

Last Updated: 2025-10-18 Format Version: 1.0 (Atomic)