debugging-memory-leak-debugging

Comprehensive memory leak detection and debugging using heap profiling tools across multiple languages

Memory Leak Debugging

Scope: Heap profiling, memory leak detection patterns, and tools for Python, Go, Rust, C/C++, and .NET Lines: ~440 Last Updated: 2025-10-26 Format Version: 1.0 (Atomic)

When to Use This Skill

Activate this skill when:

Core Concepts

Concept 1: Memory Leak Types

Leak Categories:

Common Causes:

# ❌ Memory leak: cached data never evicted
cache = {}

def get_user(user_id):
    if user_id not in cache:
        cache[user_id] = fetch_user(user_id)  # Cache grows forever
    return cache[user_id]

# ✅ Fixed: use LRU cache with size limit
from functools import lru_cache

@lru_cache(maxsize=1000)
def get_user(user_id):
    return fetch_user(user_id)

Concept 2: Heap Profiling Workflow

Standard Process:

  1. Establish baseline: Measure initial memory usage
  2. Run workload: Execute representative operations
  3. Capture snapshot: Take heap dump at intervals
  4. Compare snapshots: Identify growing allocations
  5. Analyze backtraces: Find allocation call stacks
  6. Fix and verify: Patch leaks, re-profile to confirm

Sampling vs Tracking:

# Production-safe sampling (low overhead)
py-spy record -o profile.svg --duration 60 python app.py

# Development tracking (high overhead)
valgrind --leak-check=full --track-origins=yes ./binary

Concept 3: Leak Detection Tools by Language

Tool Selection Matrix:

| Language | Primary Tool | Secondary Tool | Production-Safe | |----------|--------------|----------------|-----------------| | C/C++ | Valgrind | AddressSanitizer | ASan only | | Python | tracemalloc | memory_profiler | tracemalloc yes | | Go | pprof heap | heapster | pprof yes | | Rust | MIRI | valgrind | No (dev only) | | .NET | dotnet-dump | PerfView | dotnet-dump yes | | Java | VisualVM | Eclipse MAT | Both yes |

Key Characteristics:

Patterns

Pattern 1: Valgrind Leak Detection (C/C++)

When to use:

# Full leak check with origins tracking
valgrind --leak-check=full \
         --show-leak-kinds=all \
         --track-origins=yes \
         --log-file=valgrind.log \
         ./myapp arg1 arg2

# Analyze log file
grep "definitely lost" valgrind.log
grep "indirectly lost" valgrind.log

# Example output interpretation:
# ==12345== LEAK SUMMARY:
# ==12345==    definitely lost: 4,096 bytes in 1 blocks
# ==12345==    indirectly lost: 8,192 bytes in 2 blocks
# ==12345==      possibly lost: 0 bytes in 0 blocks
# ==12345==    still reachable: 512 bytes in 4 blocks

Key Options:

Benefits:

Pattern 2: AddressSanitizer (ASan) for C/C++

When to use:

# Compile with ASan
g++ -fsanitize=address -g -O1 app.cpp -o app
# or
clang++ -fsanitize=address -g -O1 app.cpp -o app

# Run instrumented binary
ASAN_OPTIONS=detect_leaks=1:log_path=asan.log ./app

# Example ASan report:
# =================================================================
# ==12345==ERROR: LeakSanitizer: detected memory leaks
#
# Direct leak of 4096 byte(s) in 1 object(s) allocated from:
#     #0 0x7f1234567890 in malloc (/lib/x86_64-linux-gnu/libasan.so.5+0x...)
#     #1 0x5678901234ab in allocate_buffer app.cpp:42
#     #2 0x5678901234cd in main app.cpp:123

Configuration via ASAN_OPTIONS:

# Common ASan options
export ASAN_OPTIONS="detect_leaks=1:\
                     fast_unwind_on_malloc=0:\
                     malloc_context_size=30:\
                     log_path=asan.log"

Benefits:

Pattern 3: Python tracemalloc

When to use:

import tracemalloc
import linecache

def display_top(snapshot, key_type='lineno', limit=10):
    """Display top memory allocations."""
    snapshot = snapshot.filter_traces((
        tracemalloc.Filter(False, "<frozen importlib._bootstrap>"),
        tracemalloc.Filter(False, "<unknown>"),
    ))
    top_stats = snapshot.statistics(key_type)

    print(f"Top {limit} lines:")
    for index, stat in enumerate(top_stats[:limit], 1):
        frame = stat.traceback[0]
        filename = frame.filename
        lineno = frame.lineno
        print(f"#{index}: {filename}:{lineno}: {stat.size / 1024:.1f} KiB")
        line = linecache.getline(filename, lineno).strip()
        if line:
            print(f"    {line}")

# Start tracing
tracemalloc.start()

# Take snapshot before operation
snapshot1 = tracemalloc.take_snapshot()

# Run workload
run_application()

# Take snapshot after operation
snapshot2 = tracemalloc.take_snapshot()

# Compare snapshots
top_stats = snapshot2.compare_to(snapshot1, 'lineno')

print("[ Top 10 differences ]")
for stat in top_stats[:10]:
    print(stat)

# Stop tracing
tracemalloc.stop()

Snapshot Comparison:

# Identify memory growth between snapshots
for stat in top_stats[:10]:
    print(f"{stat.size_diff / 1024:.1f} KiB | {stat.count_diff:+} allocations")
    print(f"  {stat.traceback.format()[0]}")

Benefits:

Pattern 4: Python memory_profiler

When to use:

# Install: pip install memory_profiler

from memory_profiler import profile

@profile
def process_data(data):
    """Process large dataset."""
    result = []
    for item in data:
        # This line allocates heavily
        result.append(expensive_transform(item))
    return result

# Run with: python -m memory_profiler script.py
# Output:
# Line #    Mem usage    Increment  Occurrences   Line Contents
# ================================================================
#      5   38.816 MiB   38.816 MiB           1   @profile
#      6                                         def process_data(data):
#      7   38.816 MiB    0.000 MiB           1       result = []
#      8  156.234 MiB  117.418 MiB        1001       for item in data:
#      9  156.234 MiB    0.000 MiB        1000           result.append(expensive_transform(item))
#     10  156.234 MiB    0.000 MiB           1       return result

mprof Command-Line Tool:

# Record memory usage over time
mprof run python script.py

# Plot memory usage graph
mprof plot

# Show peak memory
mprof peak

Benefits:

Pattern 5: Go pprof Heap Profiling

When to use:

package main

import (
    "net/http"
    _ "net/http/pprof"  // Registers /debug/pprof/* handlers
    "runtime"
)

func main() {
    // Enable pprof HTTP endpoint
    go func() {
        http.ListenAndServe("localhost:6060", nil)
    }()

    // Your application code
    runServer()
}

// Programmatic heap profiling
func captureHeapProfile() {
    runtime.GC()  // Get up-to-date statistics
    f, _ := os.Create("heap.prof")
    defer f.Close()
    runtime.WriteHeapProfile(f)
}

Analyzing Heap Profiles:

# Capture heap profile from running service
curl http://localhost:6060/debug/pprof/heap > heap.prof

# Interactive analysis
go tool pprof heap.prof
# Commands in interactive mode:
# > top10        # Show top 10 memory consumers
# > list funcName  # Show source with allocations
# > web          # Open browser with call graph

# Generate flame graph
go tool pprof -http=:8080 heap.prof

# Compare two heap snapshots (find memory growth)
curl http://localhost:6060/debug/pprof/heap > heap1.prof
# ... wait and let app run ...
curl http://localhost:6060/debug/pprof/heap > heap2.prof
go tool pprof -base=heap1.prof heap2.prof

Key pprof Commands:

Benefits:

Pattern 6: Rust MIRI for Memory Debugging

When to use:

# Install MIRI
rustup +nightly component add miri

# Run tests with MIRI
cargo +nightly miri test

# Run specific binary with MIRI
cargo +nightly miri run

# Example MIRI output for use-after-free:
# error: Undefined Behavior: pointer to alloc123 was dereferenced after this allocation got freed
#  --> src/main.rs:10:13
#   |
# 10|     println!("{}", *ptr);
#   |     ^^^^^^^^^^^^^^^^^ pointer to alloc123 was dereferenced after this allocation got freed

MIRI Capabilities:

Limitations:

Pattern 7: .NET Memory Diagnostics

When to use:

# Install dotnet-dump
dotnet tool install --global dotnet-dump

# Capture memory dump from running process
dotnet-dump collect --process-id 12345

# Analyze dump
dotnet-dump analyze core_20251026_123456

# Inside analysis session:
> dumpheap -stat                    # Heap statistics by type
> dumpheap -mt <MethodTable>        # Objects of specific type
> gcroot <address>                  # Find roots keeping object alive
> eeheap -gc                        # GC heap statistics
> objsize <address>                 # Total size including referenced objects

Example Analysis:

# Find objects consuming most memory
> dumpheap -stat
# Output shows types sorted by total size:
# MT            Count    TotalSize Class Name
# 00007f1234567890  50000  12800000 System.String
# 00007f1234567891  10000  10240000 MyApp.LargeObject

# Investigate specific type
> dumpheap -mt 00007f1234567891
# Shows all instances of MyApp.LargeObject

# Find what's keeping them alive
> gcroot 00007f9876543210
# Shows reference chain from GC root to object

Eclipse MAT (Memory Analyzer Tool):

Benefits:

Pattern 8: Heaptrack for C/C++

When to use:

# Install heaptrack
sudo apt install heaptrack heaptrack-gui

# Record heap allocations
heaptrack ./myapp arg1 arg2
# Creates heaptrack.myapp.XXXXX.gz

# Analyze with GUI
heaptrack_gui heaptrack.myapp.12345.gz

# Command-line analysis
heaptrack_print heaptrack.myapp.12345.gz

Heaptrack Features:

Benefits:

Quick Reference

Tool Selection by Use Case

Use Case                    | Tool                      | Overhead  | Production-Safe
----------------------------|---------------------------|-----------|----------------
C/C++ leak detection        | Valgrind                  | 10-30x    | No
C/C++ faster detection      | AddressSanitizer          | 2x        | Maybe (staging)
C/C++ with GUI              | heaptrack                 | Low       | Yes
Python stdlib               | tracemalloc               | <5%       | Yes
Python line-by-line         | memory_profiler           | Medium    | No
Go live profiling           | pprof                     | <3%       | Yes
Rust unsafe validation      | MIRI                      | 100x+     | No (dev only)
.NET production dumps       | dotnet-dump               | N/A       | Yes
Java heap analysis          | Eclipse MAT, VisualVM     | N/A       | Yes (dumps)

Key Guidelines

✅ DO: Establish memory baseline before profiling
✅ DO: Use sampling profilers in production (pprof, tracemalloc)
✅ DO: Compare heap snapshots to identify growth
✅ DO: Run leak detectors in CI/CD (ASan, Valgrind in tests)
✅ DO: Set memory limits for caches and containers
✅ DO: Profile representative workloads (not synthetic tests)
✅ DO: Suppress known library leaks in Valgrind

❌ DON'T: Run Valgrind in production (too slow)
❌ DON'T: Ignore "still-reachable" leaks (may indicate issues)
❌ DON'T: Profile debug builds (optimizations affect allocations)
❌ DON'T: Assume GC languages never leak (reference cycles do leak)
❌ DON'T: Profile with insufficient workload (leaks may not appear)
❌ DON'T: Mix profiler data from different runs (inconsistent)

Anti-Patterns

Critical Violations

# ❌ NEVER: Unbounded cache growth
class UserService:
    def __init__(self):
        self.cache = {}  # Grows forever

    def get_user(self, user_id):
        if user_id not in self.cache:
            self.cache[user_id] = self.fetch_user(user_id)
        return self.cache[user_id]

# ✅ CORRECT: Use LRU cache with size limit
from functools import lru_cache

class UserService:
    @lru_cache(maxsize=1000)
    def get_user(self, user_id):
        return self.fetch_user(user_id)

❌ Unbounded caching: Cache grows indefinitely, causing OOM ✅ Correct approach: Use LRU cache with max size, TTL eviction, or weak references

Common Mistakes

// ❌ Don't: Forget to close resources in goroutines
func processFiles(files []string) {
    for _, file := range files {
        go func(f string) {
            data, _ := os.ReadFile(f)  // Leaks if goroutine never exits
            process(data)
        }(file)
    }
}

// ✅ Correct: Use context and cleanup
func processFiles(ctx context.Context, files []string) {
    var wg sync.WaitGroup
    for _, file := range files {
        wg.Add(1)
        go func(f string) {
            defer wg.Done()
            data, err := os.ReadFile(f)
            if err != nil {
                return
            }
            process(data)
        }(file)
    }
    wg.Wait()
}

❌ Unclosed resources in goroutines: File handles, connections leak ✅ Better: Use defer, WaitGroup, and context cancellation

// ❌ Don't: Raw pointers without ownership
class Service {
    std::vector<Connection*> connections;  // Who owns these?

    void add_connection(Connection* conn) {
        connections.push_back(conn);  // Leak risk
    }
};

// ✅ Correct: Use smart pointers with clear ownership
class Service {
    std::vector<std::unique_ptr<Connection>> connections;

    void add_connection(std::unique_ptr<Connection> conn) {
        connections.push_back(std::move(conn));  // Clear ownership
    }
};

❌ Raw pointer ownership unclear: Memory leaks from ambiguous ownership ✅ Better: Use unique_ptr/shared_ptr for automatic cleanup

# ❌ Don't: Create reference cycles without weak references
class Node:
    def __init__(self, value):
        self.value = value
        self.parent = None  # Strong reference
        self.children = []

    def add_child(self, child):
        child.parent = self  # Cycle: parent <-> child
        self.children.append(child)

# ✅ Correct: Use weak references to break cycles
import weakref

class Node:
    def __init__(self, value):
        self.value = value
        self.parent = None  # Will be weakref
        self.children = []

    def add_child(self, child):
        child.parent = weakref.ref(self)  # Weak reference
        self.children.append(child)

❌ Reference cycles: Python GC can't collect cyclic references efficiently ✅ Better: Use weakref to break cycles in parent-child relationships

# ❌ Don't: Run production with tracking profilers
valgrind ./production-server  # 10-30x slowdown!

# ✅ Correct: Use sampling profilers in production
# Go
curl http://localhost:6060/debug/pprof/heap > heap.prof

# Python
python -X tracemalloc=5 app.py  # Minimal overhead

❌ Tracking profilers in production: Valgrind, MIRI unusable in production ✅ Better: Use sampling profilers (pprof, tracemalloc, dotnet-dump)

Related Skills

Last Updated: 2025-10-26 Format Version: 1.0 (Atomic)