vscode - python debugger




How to debug in Django, the good way? (18)

So, I started learning to code in Python and later Django. The first times it was hard looking at tracebacks and actually figure out what I did wrong and where the syntax error was. Some time has passed now and some way along the way, I guess I got a routine in debugging my Django code. As this was done early in my coding experience, I sat down and wondered if how I was doing this was ineffective and could be done faster. I usually manage to find and correct the bugs in my code, but I wonder if I should be doing it faster?

I usually just use the debug info Django gives when enabled. When things do end up as I thought it would, I break the code flow a lot with a syntax error, and look at the variables at that point in the flow to figure out, where the code does something other than what I wanted.

But can this be improved? Are there some good tools or better ways to debug your Django code?


A little quickie for template tags:

@register.filter 
def pdb(element):
    import pdb; pdb.set_trace()
    return element

Now, inside a template you can do {{ template_var|pdb }} and enter a pdb session (given you're running the local devel server) where you can inspect element to your heart's content.

It's a very nice way to see what's happened to your object when it arrives at the template.


Add import pdb; pdb.set_trace() at the corresponding line in the Python code and execute it. The execution will stop with an interactive shell. In the shell you can execute Python code (i.e. print variables) or use commands such as:

  • c continue execution
  • n step to the next line within the same function
  • s step to the next line in this function or a called function
  • q quit the debugger/execution

Also see: https://poweruser.blog/setting-a-breakpoint-in-python-438e23fe6b28


An additional suggestion.

You can leverage nosetests and pdb together, rather injecting pdb.set_trace() in your views manually. The advantage is that you can observe error conditions when they first start, potentially in 3rd party code.

Here's an error for me today.

TypeError at /db/hcm91dmo/catalog/records/

render_option() argument after * must be a sequence, not int

....


Error during template rendering

In template /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/crispy_forms/templates/bootstrap3/field.html, error at line 28
render_option() argument after * must be a sequence, not int
18  
19          {% if field|is_checkboxselectmultiple %}
20              {% include 'bootstrap3/layout/checkboxselectmultiple.html' %}
21          {% endif %}
22  
23          {% if field|is_radioselect %}
24              {% include 'bootstrap3/layout/radioselect.html' %}
25          {% endif %}
26  
27          {% if not field|is_checkboxselectmultiple and not field|is_radioselect %}
28  

      {% if field|is_checkbox and form_show_labels %}

Now, I know this means that I goofed the constructor for the form, and I even have good idea of which field is a problem. But, can I use pdb to see what crispy forms is complaining about, within a template?

Yes, I can. Using the --pdb option on nosetests:

tests$ nosetests test_urls_catalog.py --pdb

As soon as I hit any exception (including ones handled gracefully), pdb stops where it happens and I can look around.

  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/forms/forms.py", line 537, in __str__
    return self.as_widget()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/forms/forms.py", line 593, in as_widget
    return force_text(widget.render(name, self.value(), attrs=attrs))
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/forms/widgets.py", line 513, in render
    options = self.render_options(choices, [value])
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/forms/widgets.py", line 543, in render_options
    output.append(self.render_option(selected_choices, *option))
TypeError: render_option() argument after * must be a sequence, not int
INFO lib.capture_middleware log write_to_index(http://localhost:8082/db/hcm91dmo/catalog/records.html)
INFO lib.capture_middleware log write_to_index:end
> /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/forms/widgets.py(543)render_options()
-> output.append(self.render_option(selected_choices, *option))
(Pdb) import pprint
(Pdb) pprint.PrettyPrinter(indent=4).pprint(self)
<django.forms.widgets.Select object at 0x115fe7d10>
(Pdb) pprint.PrettyPrinter(indent=4).pprint(vars(self))
{   'attrs': {   'class': 'select form-control'},
    'choices': [[('_', 'any type'), (7, (7, 'type 7', 'RECTYPE_TABLE'))]],
    'is_required': False}
(Pdb)         

Now, it's clear that my choices argument to the crispy field constructor was as it was a list within a list, rather than a list/tuple of tuples.

 'choices': [[('_', 'any type'), (7, (7, 'type 7', 'RECTYPE_TABLE'))]]

The neat thing is that this pdb is taking place within crispy's code, not mine and I didn't need to insert it manually.


As mentioned in other posts here - setting breakpoints in your code and walking thru the code to see if it behaves as you expected is a great way to learn something like Django until you have a good sense of how it all behaves - and what your code is doing.

To do this I would recommend using WingIde. Just like other mentioned IDEs nice and easy to use, nice layout and also easy to set breakpoints evaluate / modify the stack etc. Perfect for visualizing what your code is doing as you step through it. I'm a big fan of it.

Also I use PyCharm - it has excellent static code analysis and can help sometimes spot problems before you realize they are there.

As mentioned already django-debug-toolbar is essential - https://github.com/django-debug-toolbar/django-debug-toolbar

And while not explicitly a debug or analysis tool - one of my favorites is SQL Printing Middleware available from Django Snippets at https://djangosnippets.org/snippets/290/

This will display the SQL queries that your view has generated. This will give you a good sense of what the ORM is doing and if your queries are efficient or you need to rework your code (or add caching).

I find it invaluable for keeping an eye on query performance while developing and debugging my application.

Just one other tip - I modified it slightly for my own use to only show the summary and not the SQL statement.... So I always use it while developing and testing. I also added that if the len(connection.queries) is greater than a pre-defined threshold it displays an extra warning.

Then if I spot something bad (from a performance or number of queries perspective) is happening I turn back on the full display of the SQL statements to see exactly what is going on. Very handy when you are working on a large Django project with multiple developers.


For those that can accidentally add pdb into live commits, I can suggest this extension of #Koobz answer:

@register.filter 
def pdb(element):
    from django.conf import settings
    if settings.DEBUG:    
        import pdb
        pdb.set_trace()
    return element

From my own experience , there are two way:

  1. use ipdb,which is a enhanced debugger likes pdb.

    import ipdb;ipdb.set_trace()

  2. use django shell ,just use the command below. This is very helpfull when you are developing a new view.

    python manage.py shell


I find Visual Studio Code is awesome for debugging Django apps. The standard python launch.json parameters run python manage.py with the debugger attached, so you can set breakpoints and step through your code as you like.


I highly recommend epdb (Extended Python Debugger).

https://bitbucket.org/dugan/epdb

One thing I love about epdb for debugging Django or other Python webservers is the epdb.serve() command. This sets a trace and serves this on a local port that you can connect to. Typical use case:

I have a view that I want to go through step-by-step. I'll insert the following at the point I want to set the trace.

import epdb; epdb.serve()

Once this code gets executed, I open a Python interpreter and connect to the serving instance. I can analyze all the values and step through the code using the standard pdb commands like n, s, etc.

In [2]: import epdb; epdb.connect()
(Epdb) request
<WSGIRequest
path:/foo,
GET:<QueryDict: {}>, 
POST:<QuestDict: {}>,
...
>
(Epdb) request.session.session_key
'i31kq7lljj3up5v7hbw9cff0rga2vlq5'
(Epdb) list
 85         raise some_error.CustomError()
 86 
 87     # Example login view
 88     def login(request, username, password):
 89         import epdb; epdb.serve()
 90  ->     return my_login_method(username, password)
 91
 92     # Example view to show session key
 93     def get_session_key(request):
 94         return request.session.session_key
 95

And tons more that you can learn about typing epdb help at any time.

If you want to serve or connect to multiple epdb instances at the same time, you can specify the port to listen on (default is 8080). I.e.

import epdb; epdb.serve(4242)

>> import epdb; epdb.connect(host='192.168.3.2', port=4242)

host defaults to 'localhost' if not specified. I threw it in here to demonstrate how you can use this to debug something other than a local instance, like a development server on your local LAN. Obviously, if you do this be careful that the set trace never makes it onto your production server!

As a quick note, you can still do the same thing as the accepted answer with epdb (import epdb; epdb.set_trace()) but I wanted to highlight the serve functionality since I've found it so useful.


I really like Werkzeug's interactive debugger. It's similar to Django's debug page, except that you get an interactive shell on every level of the traceback. If you use the django-extensions, you get a runserver_plus managment command which starts the development server and gives you Werkzeug's debugger on exceptions.

Of course, you should only run this locally, as it gives anyone with a browser the rights to execute arbitrary python code in the context of the server.


I use PyCharm and different debug tools. Also have a nice articles set about easy set up those things for novices. You may start here. It tells about PDB and GUI debugging in general with Django projects. Hope someone would benefit from them.


I use PyCharm and stand by it all the way. It cost me a little but I have to say the advantage that I get out of it is priceless. I tried debugging from console and I do give people a lot of credit who can do that, but for me being able to visually debug my application(s) is great.

I have to say though, PyCharm does take a lot of memory. But then again, nothing good is free in life. They just came with their latest version 3. It also plays very well with Django, Flask and Google AppEngine. So, all in all, I'd say it's a great handy tool to have for any developer.

If you are not using it yet, I'd recommend to get the trial version for 30 days to take a look at the power of PyCharm. I'm sure there are other tools also available, such as Aptana. But I guess I just also like the way PyCharm looks. I feel very comfortable debugging my apps there.


I use pyDev with Eclipse really good, set break points, step into code, view values on any objects and variables, try it.



Most options are alredy mentioned. To print template context, I've created a simple library for that. See https://github.com/edoburu/django-debugtools

You can use it to print template context without any {% load %} construct:

{% print var %}   prints variable
{% print %}       prints all

It uses a customized pprint format to display the variables in a <pre> tag.


Sometimes when I wan to explore around in a particular method and summoning pdb is just too cumbersome, I would add:

import IPython; IPython.embed()

IPython.embed() starts an IPython shell which have access to the local variables from the point where you call it.


The easiest way to debug python - especially for programmers that are used to Visual Studio - is using PTVS (Python Tools for Visual Studio). The steps are simple:

  1. Download and install it from http://pytools.codeplex.com/
  2. Set breakpoints and press F5.
  3. Your breakpoint is hit, you can view/change the variables as easy as debugging C#/C++ programs.
  4. That's all :)

If you want to debug Django using PTVS, you need to do the following:

  1. In Project settings - General tab, set "Startup File" to "manage.py", the entry point of the Django program.
  2. In Project settings - Debug tab, set "Script Arguments" to "runserver --noreload". The key point is the "--noreload" here. If you don't set it, your breakpoints won't be hit.
  3. Enjoy it.

There are a few tools that cooperate well and can make your debugging task easier.

Most important is the Django debug toolbar.

Then you need good logging using the Python logging facility. You can send logging output to a log file, but an easier option is sending log output to firepython. To use this you need to use the Firefox browser with the firebug extension. Firepython includes a firebug plugin that will display any server-side logging in a Firebug tab.

Firebug itself is also critical for debugging the Javascript side of any app you develop. (Assuming you have some JS code of course).

I also liked django-viewtools for debugging views interactively using pdb, but I don't use it that much.

There are more useful tools like dozer for tracking down memory leaks (there are also other good suggestions given in answers here on SO for memory tracking).






debugging