I picked up Django last July when I discovered that it was more enjoyable to write web applications in Python than PHP. About a month ago, I revisited the Python web development scene and took a look at Pylons. Those two frameworks take very different approaches to web development in some ways; I’d like to comment on one of the ongoing discussions regarding conceptual integrity in Python web frameworks.
Adam Gomaa said this in a post about conceptual integrity:
Isn’t the idea of doing things just your way a great one?
It sure sounds great, but it’s too simplistic a view.
He’s referring to the fact that the Pylons architecture focuses on loosely coupled components: You can swap out a different DB engine, templating system, request dispatching system, etc., with ease, thanks to Pylons’ architecture.
But Mark Ramm provided a great example of an existing system — with completely pluggable components — which has proven to be very successful, stable, reusable, and compartmentalized. Whether that fits the definition of “conceptual integrity” doesn’t matter that much — the fact is, a pluggable architecture that works well does exist, and it’s more pervasive than you’d think.
It’s Unix.
Mark describes it well:
After all, unix based OS’s have a lot of conceptual integrity as a system, but that any idea of central control has left the unix world decades ago. Instead it’s been bult up from lots of independent implementations of lots of different components working together, along clearly defined pathways (pipes, etc). And OS X and Ubuntu have been able to leverage that existing set of diverse open components in a way that results in clean, powerful, user oriented operating system experiences.
So, there’s evidence that you don’t have to to retreat from the open component model in order to achieve conceptual integrity — even when you’re working at a much, much larger software project scale than a simple web framework ;)
Bingo. Unix proves that it’s possible to have a loosely coupled framework like Pylons which actually works, works well, and exhibits what I’d consider to be conceptual integrity.
On Pluggability & Compartmentalized Applications
James Bennett goes on to write about Django’s pluggable applications (django-tagging, django-registration, django-photologue, and the like). The existence of those applications can substantially decrease development time — they’re one of the great things about Django. Need a tagging system? Plug it in. Need a comments system? Plug it in.
Those plugins can be easily integrated thanks to the existence of a few applications that are bundled with Django by default, primarily django.contrib.auth. That contrib app provides a consistent interface for user accounts, which makes it easy to build things like django-tagging which take advantage of existing user accounts.
That pluggability is a fantastic quality to have, but it also has a potential downside: If you don’t want to use django.contrib.auth for user accounts, a lot of those third-party pluggable components become much less useful. Why? Because they can’t take advantage of a standard user account API any more.
Django and Pylons have different approaches to framework design, but I think they could both learn from each other. They’re both great frameworks. They each approach the framework design somewhat differently, but I think both could learn from each other.
What could Pylons learn from Django?
James rightly pointed out that one of Django’s greatest strengths is how one can fit together pluggable applications easily. I believe Pylons’ architecture does not prohibit the development of a collection of mini-applications (tagging/commenting, etc) either. In fact, I think it would be relatively simple to develop a reusable pluggable framework to take advantage of that just like Django has. That’s all pluggable components are: components designed to take advantage of well-defined interfaces.
The primary caveat with developing a system like that with Pylons is due to Pylons’ flexibility: If you develop a component which requires Mako and SQLAlchemy, your component won’t work if someone swaps Pylons’ default choices for something else (e.g. Genshi or SQLObject). But that’s no different than how Django relies on the existence of its own ORM in order to use the existing django.contrib addons.
Pylons can’t encompass everything-and-the-kitchen-sink — that’s not its architectural goal. But I propose that the concept of “pluggable components” — one of Django’s strengths — is indeed possible to achieve in Pylons through the use of well-defined interfaces. Pylons is not inherently incoherent. It simply is much more difficult to come up with a standard API for pluggable components when developers can swap out the database engine and templating system.
In fact, the primary hindrance to pluggability in Pylons is because most pluggable components would want to access a certain database and templating system. If those two components were guaranteed to be present, it’d be dead-simple to grow a suite of tagging/commenting applications which would be very pluggable and resilient.
As a side note: The primary glue behind many of Django’s mini-applications is django.contrib.auth, which provides the backbone for a lot of those applications. In the Pylons world, AuthKit could be considered a similar project. A pluggable authentication system — with User models and the like — would be an important API that many mini-applications would use.
I will continue to ponder this, because I think there’s potential for substantial growth in that area. If Unix can do it, so can Pylons. But it’s also important to keep in mind that Pylons doesn’t want to be a full-stack framework, so the word “pluggable” is an important notion here: What we’d really be aiming for would be a standard approach to integrating components with Pylons which could interface with a standard API for a user account engine. As mentioned before, AuthKit contains many of those ideas. I’m not suggesting that we’re far from any of these ideas; I’m just thinking out loud: For instance, if I wanted to create a Pylons-centric “comments” mini-application, to add comments to anything, where would I start and how would I make it pluggable and generic? Once we can answer those questions, we’re well on our way to being able to create a library of mini-applications which could be plugged into a Pylons app. The pieces exist: WSGI, AuthKit, etc.; I just haven’t seen it all pieced together yet.
What could Django learn from Pylons?
The Django developers and project leaders have their own vision for what scope Django should have. Django has been growing and maturing well, as old cruft is removed and the newforms-admin branch takes shape. It will be great to see Django’s admin system finally completely extracted with that branch.
I think that taking the best from existing projects and combining it under a cohesive system is a great way to approach a web framework: Django currently uses its own ORM and its own templating engine. While I think Django’s homegrown systems are reasonable and straightforward to use, I think it would have more potential if it would take advantage of existing engines like SQLAlchemy, Mako, and the like. Those projects focus on one specific area so that they can do their job very well; with Django, the Django developers are forced to do everything — if they wanted to support a different database backend, they’d need to develop that backend themselves. If Django used an existing database backend API, they’d be able to take advantage of the existing (focused) development. Even if Django kept Django’s syntax as an abstraction layer and just shifted out the backend, they’d be able to take advantage of existing development while allowing developers to dig into the lower-level framework if necessary.
But it’s not up to me, and I respect that the Django developers or users may not think that reusing something like SQLAlchemy is a practical, reasonable, or desirable goal. There’s nothing wrong with that — it’s just a different approach. Django, like Pylons, is a great project: While we could debate terms like “conceptual integrity” ad infinitum, it won’t do any good in the end. All we do is segment the python community. If there are to be two or more prominent web frameworks in python, great: I’d be perfectly happy to see both Django and Pylons succeed; they each have different approaches, and I’d choose one over the other depending on what type of project I’m working on. But both projects could improve, and would benefit from analyzing other approaches to gain a better perspective, and perhaps new ideas.
There’s a theme to all of this: code reuse.
Web Frameworks, Moving Forward
In essence, I’m emphasizing something that has been considered good software development practice for a long time: Code reuse is a good idea.
Whether that be reusing code from pluggable components (django.contrib and Django’s mini-apps) or reusing code from existing ORM/templating engines (Pylons), the theme is the same: When developing software, Don’t Repeat Yourself. If someone else has developed a great tagging application, it’d be great if you could plug it in. If a bunch of people have created a great database abstraction layer, consider using that rather than developing your own.
This is open-source development… we’re a community of Python developers working toward creating great web frameworks. Use what works, improve where you can, and see if you can take advantage of the development others have done on open-source projects.
Unix did it. Pluggability is great, and it’s possible to create a pluggable, reusable set of components even with a framework like Pylons. (It’s all about defining an API.) Django has the advantage in that field due to its architecture, but I think Pylons has great potential in that area too. They’re two different philosophies, but code can be reused and pluggable components can be created regardless of which framework you use.
You don’t have to be a full-stack framework to have conceptual integrity. Whether we’re there yet is up for debate; documentation always needs improvement. But I believe both Pylons and Django can have conceptual integrity: It’s not an either-or proposition.
