How do I profile Python code for performance?

Profiling Python code is essential for identifying performance bottlenecks and optimizing your applications. Various profiling tools can help you understand where your program spends most of its execution time. This guide covers some popular methods for profiling Python code.

Using cProfile

The built-in cProfile module is one of the most commonly used tools for profiling Python programs. It provides a way to measure where time is being spent in your Python application.

import cProfile

def my_function():
    # Sample function to profile
    total = 0
    for i in range(10000):
        total += i
    return total

if __name__ == '__main__':
    cProfile.run('my_function()')
    

Using timeit for small code snippets

For small snippets of Python code, the timeit module is a great choice. It provides a simple way to time small bits of Python code.

import timeit

code_to_test = """
a = []
for i in range(1000):
    a.append(i)
"""

execution_time = timeit.timeit(code_to_test, number=1000)
print(f"Execution time: {execution_time} seconds")
    

Visualizing Profiling Results

You can use visualization tools like SnakeViz to interpret profiling results from cProfile more easily. Installation can be done via pip:

pip install snakeviz
    

Run your profile and visualize with SnakeViz:

python -m cProfile -o output.prof myscript.py
snakeviz output.prof