Using Django Forms with Pylons

Solving the Validation Problem

You should use Django Forms with your Pylons application. In fact, I recommend it for every Pylons site you create.

Django developers rarely use third-party components like SQLAlchemy or Mako;¹ Pylons developers don’t often use Django components either. Both SQLAlchemy and Mako are great components: flexible, powerful, and fast. (If Django was created later, it might have used them instead.)

Form handling is another story, though. Pylons’ @validate decorator is a mess and has needed an overhaul for years, but no clear fix has emerged. A decorator is the wrong approach; it’s too inflexible. But manually working with FormEncode and htmlfill can be painfully cumbersome and verbose. While FormEncode is flexible, it isn’t elegant. Likewise, htmlfill can get in the way if you aren’t careful.

FormEncode is like playdough. It’s very flexible, but it’s a little messy. And you’d better not get any on the carpet.

I spent many hours working through solutions to this problem. I looked at alternatives like FormBuild, FormAlchemy, and ToscaWidgets; I even tried to build my own. Nothing stood out as a clean, elegant solution.

I turned to an unexpected place for salvation: I believe django.forms is the *best* option for working with forms in Pylons, hands-down.

Believe me, I was surprised too. Especially in light of that whole conceptual integrity debacle.

Why Django Forms? It’s widely considered to be a clean, flexible way to handle form validation and processing. It’s used by the largest Python web framework around with few complaints and much praise. And I know that it will be around for a long time, with lots of attention paid to backward compatibility.

I was concerned that it would be difficult to use just the forms part of Django. Fortunately, Django developers did a good job sandboxing the forms library from the rest of the Django suite. Surprisingly, django.forms works quite well out of the box; a few monkeypatches solidify the few areas that needed a little tidying.

This is good stuff, folks.

Simple Instructions

All of the tweaks needed for django.forms to work are included in a module named forms.py that you can obtain on GitHub. That module is intended to be a drop-in replacement for the django.forms module. (It does not replace your Django installation; you’ll just import that module instead. You must have Django installed first.)

Copy that downloaded forms.py module into your Pylons project (perhaps in your lib directory) and import that instead of django.forms. That’s it. The Django form documentation will take you the rest of the way.

Integration

The forms module I linked to above includes all of the following fixes; only read this section if you are interested in the technical details of what needed to be patched.

First, django.conf.settings.configure() was called (once and only once).
Django uses a function called mark_safe to escape HTML; that function was patched to mark strings with a WebHelpers literal as well.
forms.widgets.SelectMultiple was modified to support Paste’s MultiDict, as noted in this Django ticket.
Django’s FileField was patched to handle cgi.FieldStorage, which Pylons uses to upload forms.

That’s it. With those fixes, Django Forms work without a hitch.

You’re probably wondering about SQLAlchemy models. Django’s ModelForm obviously won’t work in Pylons without extensive modifications, but I’ve included a few helper utilities in forms.py that make model validation really easy. More on models below.

The Django Forms Workflow

Rather than a clunky @validate decorator, Django uses a simple idiom in which form processing and form display are included in the same action. This idiom keeps your controller code simple and clear, and that’s one of the reasons I liked Django’s form approach. Here’s how you would use the Django idiom in Pylons:

class CommentForm(forms.Form):
    name = forms.CharField()
    email = forms.EmailField()
    text = forms.CharField(widget=forms.Textarea())

# in your controller
def add_comment(self):

    if request.method == 'POST':
        c.form = CommentForm(request.POST)
        if c.form.is_valid():
            comment = Comment()
            comment.name = c.form.cleaned_data['name']
            comment.email = c.form.cleaned_data['email']
            comment.text = c.form.cleaned_data['text']
    else:
        c.form = CommentForm()

    return render('/add_comment.html')

(The /add_comment.html template would render the form itself, e.g. via ${c.form}.)

Simple, eh? No decorator nonsense. Just process the form in that action and be done with it. That code works perfectly fine in Pylons. The editing workflow is similarly easy:

# in your controller
def edit_comment(self, id):
    c.comment = Session.query(Comments).get(id)

    if request.method == 'POST':
        c.form = CommentForm(request.POST)
        if c.form.is_valid():
            # update the model's fields with the validated form data
            forms.update_model(c.comment, c.form.cleaned_data)
    else:
        # Fill the form with the existing comment's information.
        c.form = CommentForm(initial=forms.model_to_dict(c.comment))

    return render('/edit_comment.html')

Note the use of forms.update_model() and forms.model_to_dict(); I included these two utility functions in lieu of a complete ModelForm replacement.

forms.update_model(model, data, include=None, exclude=None): Lets you quickly assign validated form data to your model.
forms.model_to_dict(*items, include=None, exclude=None): Transforms a model into a dictionary that you could pass to Form‘s initial keyword argument; multiple items are merged together into the result so that you can override specific initial fields yourself:

>>> c.comment.name
'Steve Jobs'
>>> forms.model_to_dict(c.comment,
...                     {'name': 'John Doe', 'foo': 'bar'},
...                     exclude=['email'])
{'name': 'John Doe', 'foo': 'bar', 'text': '...'}

With forms.model_to_dict() and forms.update_model(), it’s trivial to work with SQLAlchemy models with forms. It’s a touch more verbose than Django’s ModelForm, but it’s explicit, simple, and easy to understand. View the source for more examples.

What about FormEncode?

As I mentioned above, a killer aspect of Django’s forms is the idiom they use to process requests. But Pylons isn’t going to add a dependency on Django just so that developers can use django.forms. What if you would rather use FormEncode?

You’re in luck. You can still use Django’s form processing idiom with FormEncode. I would argue that it should replace @validate as the recommended way to process forms.

Here’s the code, using the forms.FormEncodeForm class:

class CommentForm(forms.FormEncodeForm):
    html = render('/edit_comment_form.html') # <-- Note This

    name = formencode.validators.String(min=10)
    email = formencode.validators.Email()
    text = formencode.validators.String()

This idiom binds the form’s HTML template directly to the Form instance. You can either provide html as a class variable (as above), or as a keyword to FormEncodeForm.__init__. That way, the Form class can render itself just like Django’s forms do.

Here’s the controller code for a FormEncode-based form:

# in your controller
def edit_comment(self, id):
    c.comment = Session.query(Comments).get(id)

    if request.method == 'POST':
        c.form = CommentForm(request.POST)
        if c.form.is_valid():
            # update the model's fields with the validated form data
            forms.update_model(c.comment, c.form.cleaned_data)
    else:
        # Fill the form with the existing comment's information.
        c.form = CommentForm(initial=forms.model_to_dict(c.comment))

    return render('/edit_comment.html')

Look familiar? It should. It’s exactly the same as the code you saw earlier that used Django-style forms.Form.

Let me say that again: Form processing is **exactly the same*.*

Both forms.Form (Django-style) and forms.FormEncodeForm (FormEncode-based) use the same idiom for controller code. The FormEncodeForm class handles htmlfill behind the scenes. You provide the form HTML and the schema, and it just works. (With either of those classes, you’d render the form itself using code like ${c.form} within your page template, as explained in the Django documentation.)

Where does that leave us?

I don’t know about you, but I’ll tell you where it leaves me: I use Django’s Forms with Pylons, and I couldn’t be more pleased.

If I didn’t convince you to switch to Django Forms and you aren’t happy with the @validate decorator, consider looking at the FormEncodeForm class. It’s only a hundred lines or so, and that workflow works much more smoothly than @validate.

If you aren’t sure, or if my explanations were unclear, just go look at the source itself. It’s only around 300 lines long, and mostly docstrings; it should be easy to grasp if you understand a little about Django and Pylons.

Kudos to the Django team for decoupling most of the forms library from the rest of Django. Ideally it would be a separate library so as to not require Django in its entirety, but I’ll take what I can get.

This solution blindsided me, and I’m still shocked that it was that easy. A day or so of coding, and my form handling issues are done.

Django Forms. That was the ingredient I was missing for so long.

Pylons + SQLAlchemy + Mako + Django Forms.

That’s a winning combination.

[1]

Many hours have been spent trying to encourage Django to separate its codebase into separate components. Django does it all — models, templates, forms, routing — and its philosophy keeps the framework consistent. After Django was created, several third-party components evolved into arguably superior replacements for specific parts of the web stack: SQLAlchemy for databases, Mako for templates, and so on. Like Apple, Django controls the whole stack; as a result, its developers benefit from a large ecosystem of plugin applications. Django developers who want to use SQLAlchemy or Mako can certainly do so, but at the cost of Django’s iconic reusability.