{"id":194,"date":"2015-08-31T03:27:25","date_gmt":"2015-08-30T19:27:25","guid":{"rendered":"https:\/\/vinta.ws\/code\/?p=194"},"modified":"2026-02-18T01:20:36","modified_gmt":"2026-02-17T17:20:36","slug":"tools-for-profiling-your-python-django-project","status":"publish","type":"post","link":"https:\/\/vinta.ws\/code\/tools-for-profiling-your-python-django-project.html","title":{"rendered":"Tools for Profiling your Python Projects"},"content":{"rendered":"

The first aim of profiling is to test a representative system to identify what's slow, using too much RAM, causing too much disk I\/O or network I\/O. You should keep in mind that profiling typically adds an overhead to your code.<\/p>\n

In this post, I will introduce tools you could use to profile your Python or Django projects, including: timer<\/code>, pycallgraph<\/code>, cProfile<\/code>, line-profiler<\/code>, memory-profiler<\/code>.<\/p>\n

ref:
\nhttps:\/\/stackoverflow.com\/questions\/582336\/how-can-you-profile-a-script<\/a>
\nhttps:\/\/www.airpair.com\/python\/posts\/optimizing-python-code<\/a><\/p>\n

timer<\/h2>\n

The simplest way to profile a piece of code.<\/p>\n

ref:
\nhttps:\/\/docs.python.org\/3\/library\/timeit.html<\/a><\/p>\n

pycallgraph<\/h2>\n

pycallgraph<\/code> is a Python module that creates call graph visualizations for Python applications.<\/p>\n

ref:
\nhttps:\/\/pycallgraph.readthedocs.org\/en\/latest\/<\/a><\/p>\n

$ sudo apt-get install graphviz\n$ pip install pycallgraph<\/code><\/pre>\n
# in your_app\/middlewares.py\nfrom pycallgraph import Config\nfrom pycallgraph import PyCallGraph\nfrom pycallgraph.globbing_filter import GlobbingFilter\nfrom pycallgraph.output import GraphvizOutput\nimport time\n\nclass PyCallGraphMiddleware(object):\n\n    def process_view(self, request, callback, callback_args, callback_kwargs):\n        if 'graph' in request.GET:\n            config = Config()\n            config.trace_filter = GlobbingFilter(include=['rest_framework.*', 'api.*', 'music.*'])\n            graphviz = GraphvizOutput(output_file='pycallgraph-{}.png'.format(time.time()))\n            pycallgraph = PyCallGraph(output=graphviz, config=config)\n            pycallgraph.start()\n\n            self.pycallgraph = pycallgraph\n\n    def process_response(self, request, response):\n        if 'graph' in request.GET:\n            self.pycallgraph.done()\n\n        return response<\/code><\/pre>\n
# in settings.py\nMIDDLEWARE_CLASSES = (\n    'your_app.middlewares.PyCallGraphMiddleware',\n    ...\n)<\/code><\/pre>\n
$ python manage.py runserver 0.0.0.0:8000\n$ open http:\/\/127.0.0.1:8000\/your_endpoint\/?graph=true<\/code><\/pre>\n
cProfile<\/h2>\n
cProfile<\/code> is a tool in Python's standard library to understand which functions in your code take the longest to run. It will give you a high-level view of the performance problem so you can direct your attention to the critical functions.<\/p>\n
ref:
\nhttp:\/\/igor.kupczynski.info\/2015\/01\/16\/profiling-python-scripts.html<\/a>
\nhttps:\/\/ymichael.com\/2014\/03\/08\/profiling-python-with-cprofile.html<\/a><\/p>\n
$ python -m cProfile manage.py test member\n$ python -m cProfile -o my-profile-data.out manage.py test --failtest\n$ python -m cProfile -o my-profile-data.out manage.py runserver 0.0.0.0:8000\n\n$ pip install cprofilev\n$ cprofilev -f my-profile-data.out -a 0.0.0.0 -p 4000\n$ open http:\/\/127.0.0.1:4000<\/code><\/pre>\n
cProfile with django-cprofile-middleware<\/h3>\n
$ pip install django-cprofile-middleware<\/code><\/pre>\n
# in settings.py\nMIDDLEWARE_CLASSES = (\n    ...\n    'django_cprofile_middleware.middleware.ProfilerMiddleware',\n)<\/code><\/pre>\n
Open any url with a ?prof<\/code> suffix to do the profiling, for instance, http:\/\/localhost:8000\/foo\/?prof<\/a><\/p>\n
ref:
\nhttps:\/\/github.com\/omarish\/django-cprofile-middleware<\/a><\/p>\n
cProfile with django-extension and kcachegrind<\/h3>\n
kcachegrind<\/code> is a profiling data visualization tool, used to determine the most time consuming execution parts of a program.<\/p>\n
ref:
\nhttp:\/\/django-extensions.readthedocs.org\/en\/latest\/runprofileserver.html<\/a><\/p>\n
$ pip install django-extensions<\/code><\/pre>\n
# in settings.py\nINSTALLED_APPS += (\n    'django_extensions',\n)<\/code><\/pre>\n
$ mkdir -p my-profile-data\n\n$ python manage.py runprofileserver \n--noreload \n--nomedia \n--nostatic \n--kcachegrind \n--prof-path=my-profile-data \n0.0.0.0:8000\n\n$ brew install qcachegrind --with-graphviz\n$ qcachegrind my-profile-data\/root.003563ms.1441992439.prof\n# or\n$ sudo apt-get install kcachegrind\n$ kcachegrind my-profile-data\/root.003563ms.1441992439.prof<\/code><\/pre>\n
cProfile with django-debug-toolbar<\/h3>\n
You're only able to use django-debug-toolbar<\/code> if your view returns HTML, it needs a place to inject the debug panels into your DOM on the webpage.<\/p>\n
ref:
\nhttps:\/\/github.com\/django-debug-toolbar\/django-debug-toolbar<\/a><\/p>\n
$ pip install django-debug-toolbar<\/code><\/pre>\n
# in settiangs.py\nINSTALLED_APPS += (\n    'debug_toolbar',\n)\n\nDEBUG_TOOLBAR_PANELS = [\n    ...\n    'debug_toolbar.panels.profiling.ProfilingPanel',\n    ...\n]<\/code><\/pre>\n
line-profiler<\/h2>\n
line-profiler<\/code> is a module for doing line-by-line profiling of functions. One of my favorite tools.<\/p>\n
ref:
\nhttps:\/\/github.com\/rkern\/line_profiler<\/a><\/p>\n
$ pip install line-profiler<\/code><\/pre>\n
# in your_app\/views.py\ndef do_line_profiler(view=None, extra_view=None):\n    import line_profiler\n\n    def wrapper(view):\n        def wrapped(*args, **kwargs):\n            prof = line_profiler.LineProfiler()\n            prof.add_function(view)\n            if extra_view:\n                [prof.add_function(v) for v in extra_view]\n            with prof:\n                resp = view(*args, **kwargs)\n            prof.print_stats()\n            return resp\n\n        return wrapped\n\n    if view:\n        return wrapper(view)\n\n    return wrapper\n\n@do_line_profiler\ndef your_view(request):\n    pass<\/code><\/pre>\n
ref:
\nhttps:\/\/djangosnippets.org\/snippets\/10483\/<\/a><\/p>\n
There is a pure Python alternative: pprofile<\/code>.
\nhttps:\/\/github.com\/vpelletier\/pprofile<\/a><\/p>\n
line-profiler with django-devserver<\/h3>\n
ref:
\nhttps:\/\/github.com\/dcramer\/django-devserver<\/a><\/p>\n
$ pip install git+git:\/\/github.com\/dcramer\/django-devserver#egg=django-devserver<\/code><\/pre>\n
in settings.py<\/p>\n
INSTALLED_APPS += (\n    'devserver',\n)\n\nDEVSERVER_MODULES = (\n    ...\n    'devserver.modules.profile.LineProfilerModule',\n    ...\n)\n\nDEVSERVER_AUTO_PROFILE = False<\/code><\/pre>\n
in your_app\/views.py<\/p>\n
from devserver.modules.profile import devserver_profile\n\n@devserver_profile()\ndef your_view(request):\n    pass<\/code><\/pre>\n
line-profiler with django-debug-toolbar-line-profiler<\/h3>\n
ref:
\nhttp:\/\/django-debug-toolbar.readthedocs.org\/en\/latest\/<\/a>
\nhttps:\/\/github.com\/dmclain\/django-debug-toolbar-line-profiler<\/a><\/p>\n
$ pip install django-debug-toolbar django-debug-toolbar-line-profiler<\/code><\/pre>\n
# in settings.py\nINSTALLED_APPS += (\n    'debug_toolbar',\n    'debug_toolbar_line_profiler',\n)\n\nDEBUG_TOOLBAR_PANELS = [\n    ...\n    'debug_toolbar_line_profiler.panel.ProfilingPanel',\n    ...\n]<\/code><\/pre>\n
memory-profiler<\/h2>\n
This is a Python module for monitoring memory consumption of a process as well as line-by-line analysis of memory consumption for Python programs.<\/p>\n
ref:
\nhttps:\/\/pypi.python.org\/pypi\/memory_profiler<\/a><\/p>\n
$ pip install memory-profiler psutil<\/code><\/pre>\n
# in your_app\/views.py\nfrom memory_profiler import profile\n\n@profile(precision=4)\ndef your_view(request):\n    pass<\/code><\/pre>\n
There are other options:
\nhttp:\/\/stackoverflow.com\/questions\/110259\/which-python-memory-profiler-is-recommended<\/a><\/p>\n
dogslow<\/h2>\n
ref:
\nhttps:\/\/bitbucket.org\/evzijst\/dogslow<\/a><\/p>\n
django-slow-tests<\/h2>\n
ref:
\nhttps:\/\/github.com\/realpython\/django-slow-tests<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"
Tools you could use to profile your Python, Django or Flask projects, including: timer, pycallgraph, cProfile, line-profiler, memory-profiler.<\/p>\n","protected":false},"author":1,"featured_media":294,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4,116],"tags":[29,13,81,2],"class_list":["post-194","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-about-python","category-about-web-development","tag-debug","tag-django","tag-profiling","tag-python"],"_links":{"self":[{"href":"https:\/\/vinta.ws\/code\/wp-json\/wp\/v2\/posts\/194","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/vinta.ws\/code\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/vinta.ws\/code\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/vinta.ws\/code\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/vinta.ws\/code\/wp-json\/wp\/v2\/comments?post=194"}],"version-history":[{"count":0,"href":"https:\/\/vinta.ws\/code\/wp-json\/wp\/v2\/posts\/194\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/vinta.ws\/code\/wp-json\/wp\/v2\/media\/294"}],"wp:attachment":[{"href":"https:\/\/vinta.ws\/code\/wp-json\/wp\/v2\/media?parent=194"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/vinta.ws\/code\/wp-json\/wp\/v2\/categories?post=194"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/vinta.ws\/code\/wp-json\/wp\/v2\/tags?post=194"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}