The first aim of profiling is to test a representative system to identify what's slow, using too much RAM, causing too much disk I/O or network I/O. You should keep in mind that profiling typically adds an overhead to your code.
In this post, I will introduce tools you could use to profile your Python or Django projects, including: timer
, pycallgraph
, cProfile
, line-profiler
, memory-profiler
.
ref:
https://stackoverflow.com/questions/582336/how-can-you-profile-a-script
https://www.airpair.com/python/posts/optimizing-python-code
timer
The simplest way to profile a piece of code.
ref:
https://docs.python.org/3/library/timeit.html
pycallgraph
pycallgraph
is a Python module that creates call graph visualizations for Python applications.
ref:
https://pycallgraph.readthedocs.org/en/latest/
cProfile
cProfile
is a tool in Python's standard library to understand which functions in your code take the longest to run. It will give you a high-level view of the performance problem so you can direct your attention to the critical functions.
ref:
http://igor.kupczynski.info/2015/01/16/profiling-python-scripts.html
https://ymichael.com/2014/03/08/profiling-python-with-cprofile.html
cProfile with django-cprofile-middleware
Open any url with a ?prof
suffix to do the profiling, for instance, http://localhost:8000/foo/?prof
ref:
https://github.com/omarish/django-cprofile-middleware
cProfile with django-extension and kcachegrind
kcachegrind
is a profiling data visualization tool, used to determine the most time consuming execution parts of a program.
ref:
http://django-extensions.readthedocs.org/en/latest/runprofileserver.html
cProfile with django-debug-toolbar
You're only able to use django-debug-toolbar
if your view returns HTML, it needs a place to inject the debug panels into your DOM on the webpage.
ref:
https://github.com/django-debug-toolbar/django-debug-toolbar
line-profiler
line-profiler
is a module for doing line-by-line profiling of functions. One of my favorite tools.
ref:
https://github.com/rkern/line_profiler
ref:
https://djangosnippets.org/snippets/10483/
There is a pure Python alternative: pprofile
.
https://github.com/vpelletier/pprofile
line-profiler with django-devserver
ref:
https://github.com/dcramer/django-devserver
in settings.py
in your_app/views.py
line-profiler with django-debug-toolbar-line-profiler
ref:
http://django-debug-toolbar.readthedocs.org/en/latest/
https://github.com/dmclain/django-debug-toolbar-line-profiler
memory-profiler
This is a Python module for monitoring memory consumption of a process as well as line-by-line analysis of memory consumption for Python programs.
ref:
https://pypi.python.org/pypi/memory_profiler
There are other options:
http://stackoverflow.com/questions/110259/which-python-memory-profiler-is-recommended
dogslow
ref:
https://bitbucket.org/evzijst/dogslow
django-slow-tests
ref:
https://github.com/realpython/django-slow-tests