The Setup

After working for over a year as the only tech guy in our little start-up company, we brought another developer on to help with some of the reporting tasks. In an effort to mitigate the damage a contracted hit-man could do to our collective efforts (the hit-by-a-bus metaphor is so passé), I have been trying make sure my new teammate is familiar with the development tools I have been using.

We are from two very different worlds. I have been comfortable doing all of my development in Vim, though I occasional use Qt Designer for complex form layout and Eric IDE for visual debugging. My teammate found the Vim plus shell toolset to have too steep a learning curve, so I started a quest to find an IDE that we could both use comfortably.

Eric4 IDE

My first attempt was with Eric. It is a decent, open-source PyQt IDE. It is a snap to install on Ubuntu, but my teammate got frustrated installing the various dependencies for development on Windows. Though Eric is an adequate IDE for my purposes, it is very cluttered and has a confusing interface. Given how clumsy it can be to navigate, I didn’t feel good about pushing my team member to get it working. Eric does however have excellent integration with the Qt tools. I was a little nervous to move away from Eric until I realized that most of the magic is done by calling pyuic4 on the back-end.

Eclipse and PyDev

My next experiment was with Eclipse. I used Eclipse plus PyDev long ago, and found it adequate though I reverted back to Vim. When I gave it a fresh look, I was encouraged that PyDev was no longer half-proprietary and had lots of new features. Eclipse also has a lot of database plugins which might help my teammate visualize the structure of our embedded SQLite data store.

Installation on Linux was a single aptitude command, but it proved to be a complex IDE to set up. Learning how to install plugins wasn’t bad, but the selection was confusing. I’m still not sure which Qt plugins are necessary for PyQt development—I erred on the side of installing extra. Then I had to wrap my brain around perspectives, and configure them to be logical (Mylyn looks to address some of these problems, but it was yet another new paradigm that had to be grasped before I could get any work done). I also had to find a supported Mercurial plugin and figure out how to get my existing code into a project—a process which was not very intuitive. Riddle me this: Do you create a Mercurial Project or a PyDev project? Answer: Create a PyDev project and give it your existing Mercurial repo as the home directory, then right click on the folder in the project explorer, go to “Team”, and select “Share Project”. At that point you can tell it that you are using Mercurial. After all this work, I still felt like it was a clunky IDE with no good Vi keybindings (though the reasonably priced proprietary viPlugin might have solved that one). My other big gripe is that when stepping through my Python application, the integrated Python shell does not see the current debugging stack—a feature of Eric that I found to be extremely valuable. My teammate spent a little bit of time trying to install Eclipse and found it to have more dependencies than Eric.

At the end of the day, I think I like Eclipse but I don’t think it will work for my current project. It compares favorably with Visual Studio (which is also a bear to setup for an existing project, especially with a heterogeneous tool set), but it will take a lot of effort to get my team productive in it. Though I expect it would eventually prove adequate, I decided to explore alternatives before settling.

NetBeans Plus nbPython

At some point last month I came across a blog posting about the NetBeans Python plugin. It looked like it might meet our needs, so I decided to give it a shot. I think it is the most complete solution I have come across so far, though it currently has some annoying bugs.

My first impression is that NetBeans is ugly. I mean the kind of ugly that gets kids sent home from school. Yeah, it is Java Swing ugly. Swing’s default look and feel (Metal) payed more homage to Motif than to the competing toolkits of 1995. Metal is so ugly that it single-handedly prevented developers from adopting Java on the desktop (that and the fact that early expectations for Java far exceeded reality, oh and Microsoft’s abusive tactics). But enough ranting, because Sun finally sort of fixed it. To get NetBeans to use the not-so-ugly Nimbus look and feel, simply copy “/etc/netbeans.conf” into your local .netbeans folder, and add --laf Nimbus to the --netbeans_default_options line.

Impressions:

• NetBeans is pretty usable by default. It is much quicker to get started with than Eclipse.
• NetBeans appears customizable enough for my needs. It is much more configurable than it at first appears.
• The options dialog is very poorly organized and hard to use.
• My Dansguardian setup was blocking the automatic download and installation of updates and some plugins, so I had to add updates.netbeans.org to ”/etc/dansguardian/lists/exceptionsitelist”.
• I turned off the additional update check in the plugins dialog; it was annoying me.
• There are some decent dark themes, but it doesn’t look like it because the option window doesn’t change the background color.
• I installed the “Extra Color Schemes” package under the “Ruby” Category, and settled on “Dark Pastels”. I then changed the default to a fixed-width font (Lucida Sans Typewriter). It looks much better.
• Unfortunately, I don’t see how to change any window except for the editor window to be dark.
• The jVi plugin is awesome. It is everything you would expect an embedded Vi plugin to be. Manually installing plugins was a little tricky: Under the “Tools” menu, select “Plugins” then select the “Downloaded” tab and click “Add Plugins”
• The Mercurial plugin was already installed in the Ubuntu package. It works okay. It is straight-forward to use.
• It reduced confusion for me to disable the plugins that I am not using (like CVS). The menus are less cluttered now.
• The Python plugin works good. The Python shell is aware of the symbols in the debugger. There doesn’t appear to be a command history in the python shell, which is annoying.
• There is a nice SqlBrowser plugin. It’s pretty basic, but it is useful. To hook it up to SQLite, you need to install the JDBC Sqlite Driver plugin, which is downloadable from http://plugins.netbeans.org. Once it is installed, go to the “Window” menu and select “Services”. Then in the Services explorer, you can configure an SQLite Driver data source. Leave the username and password blank. Specify the jdbc connection like so: jdbc:sqlite://home/<dir>/<dbfilename>.db The SqlBrowser does not provide any visual view of tables with foreign key relationships mapped out. But there is a tree view of the database schema and you can execute SQL line-by-line and browse the results.
• There are some cool tools. It says that I don’t have a very consistent coding style. That is true, and useful. I wish I had started the project with a good coding style and then followed it’s warnings—for new I just have to turn it off.
• PyLint appears to be integrated already. I don’t see any pylint configuration, but it is giving me warnings that I usually expect from that tool (unused import, etc).
• I didn’t need to do anything special to work with a PyQt application.

Remaining Problems:

• It is not saving the Python Platform setting. I have to reselect Python 2.6 (instead of Jython 2.5) each time I open NetBeans. I am hitting the “Make Default” button, but it forgets every time it is closed.
• Debugging seems kind of buggy. It has problems with my signal handling (the error message says “ValueError: signal only works in main thread”). I commented out the signal assignment for now, and I’ll detect when I’m in debug mode in the future. Now the debugger runs the application alright (don’t be weirded out because it stops on the first line—that’s configurable), but once it hits the first breakpoint it doesn’t want to resume code stepping (though the application GUI is still responsive). I can’t get it to break at a second breakpoint. It sounds like this bug to me: https://netbeans.org/bugzilla/show_bug.cgi?id=169757
• The UI is sometimes laggy.

Summary: NetBeans is really close to being an excellent Python development environment. I think I’ll nudge my team in this direction and hope that the next release addresses the bugginess that I’m struggling with.

Banjo Six Months Later

Banjo is a good foundation for a blog engine, but it needs some work. It is certainly quicker to implement than writing your own blog engine, but it is not as finished as I had hoped. I fixed a number of bugs, and the banjo maintainer was nice enough to give me write access to contribute my changes back. However, my goal was to work with a community where I could learn from experienced Django developers, and there isn’t much of a community around Banjo right now.

Discovering Byteflow

When fireant came across my blog post on Banjo, he asked me why I wasn’t using Byteflow. My answer was that I had not heard of Byteflow. I started looking into it, and it looked like a good alternative to Banjo.

My first impression of Byteflow was very similar to Banjo: lots of features, lots of potential, but it looked like a dead project. The server wasn’t responding well, the homepage news was out of date, and the mailing lists were full of spam.

Then I looked at the IRC logs and found an active community that mostly speaks an Eastern European language (Russian or Ukrainian, based on the location of the authors). When I asked a couple of questions in English, I got very friendly and helpful responses from piranha, the main Byteflow author. Not only are the Byteflow developers active, but they are open to contributions. I helped clean up the newsgroup spam, and updated the homepage to reflect the release last spring. And as I adapted Byteflow to the specifics of my installation, they accepted my patches and got them into the code base in the same day.

So far Byteflow has exceeded my expectations. Not only has it been a reliable blog engine (for the last two days anyway), it also incorporates many of the features I was hoping to implement* in Banjo. Most of all, I’ve been learning a lot from working with the Byteflow developers (piranha already showed me a few cool Mercurial tricks). That is my biggest justification for spending time on a Django blog instead of using a mature product. I’ve been the only tech guy at my family startup for the past year, and I’ve needed to find a project where I can keep my skills fresh by working with others.

* I’m finally an OpenID provider! After almost ten years, I signed up for a Slashdot account just to test it out.

Setting up Byteflow

This week I had two free days between work projects, so I decided to migrate my blog to Byteflow. The installation went very smoothly—the best I have experienced with a Django app.

Dependencies

Byteflow has no dependencies that needed to be filled outside of a typical package management system.

Base (all from Aptitude):

• Debian Lenny
• Postgres
• Django 1.1 (1.0 should work)
• Python 2.5.2
• Apache
• mod_python*

* I’m convinced that I should switch to mod_wsgi, but Byteflow works fine with my existing mod_python install.

From Aptitude:

• python-psycopg2 2.0.7-4
• python-openid 2.2.1-2
• python-beautifulsoup 3.0.7-1
• python-imaging 1.1.6-3

Installation

In the past the Byteflow project has announced releases, but I don’t see them packaged for download anywhere. It would be preferable to have a known functional configuration. Since it has been a couple of weeks without a series of commits to the Mercurial repository, I decided to have faith that it’s in working condition and I cloned the repo.

hg clone http://hg.piranha.org.ua/byteflow/

It is a small repo and a clean codebase. Everything worked out of the box by copying settings_local.py.template to setting_local.py, editing it to point to a clean database, running syncdb, and pointing Apache to the Byteflow settings.py. The default template came up, and a test post through the admin interface appeared.

Very easy, very slick.

Configuring Byteflow as an App

I like to keep code maintained by other people strictly separate from code I maintain. It was as easy to do that with Byteflow as with most Django apps, but did require some tweaking. The basic idea is to keep Byteflow in my shared_django_libs directory (I’ll call this PROJECT_ROOT), and have the apps Apache sees in a completely different directory tree (I’ll call my personal site REPO_ROOT).

I’ll define a couple of variables to make this easier to follow:

• PROJECT_ROOT: my clone of the Byteflow repository.
• REPO_ROOT: the repository where I keep my site.
• WEB_ROOT: the Apache web root where I keep static files, .htaccess files, and other stuff serve-able by Apache. The location is REPO_ROOT/WEB_ROOT.
• STATIC_ROOT: the directory where Django serves static files. The location is REPO_ROOT/WEB_ROOT/static.
• STATIC_URL: the URL where Apache can see static files. It is SITE_NAME + ‘static’.
• MEDIA_ROOT: The Byteflow settings.py differentiates between media files and static files. I treat them the same, so my MEDIA_ROOT = STATIC_ROOT.
• MEDIA_URL: As described above, my MEDIA_URL = STATIC_URL.
• SITE_ROOT: the directory containing my settings.py and other local Django code. It’s at REPO_ROOT/SITE_ROOT.
• THEMES_DIR: The directory where my local themes live. It is SITE_ROOT/THEMES_DIR.

1. Copy the default settings.py into your app (not settings_local.py).
2. Edit your settings.py as follows:
   • At the top of the file, just before the lines where it adds ‘apps’ and ‘compat’ to the sys.path, set PROJECT_ROOT to point at your byteflow install. Then add this line:
     sys.path.insert(0, os.path.join(os.path.dirname(PROJECT_ROOT)))
   • Add variables for SITE_ROOT and WEB_ROOT. Then use those variables to define STATIC_ROOT, MEDIA_ROOT, and THEMES_DIR.
   • Remove the loading of settings_local.py.
3. Create a file containing your SECRET_KEY, and point your settings.py to it (I used the SITE_ROOT variable).
4. Add these lines at the bottom of your urls.py, where it can catch any URLs you don’t already deal with:
   import byteflow.urls
urlpatterns += byteflow.urls.urlpatterns
5. Copy from PROJECT_ROOT/static to STATIC_ROOT these directories: css, img, js.

That’s all I remember it taking. Now Byteflow plays happily with everything I’m already running.

Creating a Custom Theme

Of course the default theme doesn’t look like _my_ ugly blog. To get my distinctive look, I had to create a custom theme. It took a while to get everything looking right, but the process wasn’t really difficult. It mostly involved me merging the important parts of Byteflow’s default base.html into my existing template, and looking at the other Byteflow themes to figure out how to make changes to the Byteflow pages.

• In THEMES_DIR, create a directory with the name of the theme you specified in your settings.py
• It needs a base.html in that directory. My base.html includes the contents of PROJECT_ROOT/templates/base.html, header.html, and footer.html.
• You can over-ride any template in PROJECT_ROOT/templates by having a template with the same name and directory structure in your theme. For example, I changed the way a blog post looks by copying post_entry.html from PROJECT_ROOT/templates/blog into THEMES_DIR/my_theme/blog, and editing it. You can also override templatetags by copying them from PROJECT_ROOT/templates/templatetags into THEMES_DIR/my_theme/templatetags. I changed the way the dates looked by stealing the datelinks,html from one of the included Byteflow templates. (Thanks for the tip on that one, piranha.)
• Create a directory in STATIC_ROOT with your theme name, and then you can override the contents of css, img, and js. Took my existing CSS file, called it main.css, and then added a couple of entries to change the look of Byteflow generated elements.

Importing the Data from Banjo

Since I only had 30 or so posts to migrate, I figured I’d just do it by hand to gain experience with Byteflow’s interface and the various markup languages. It also gave me the chance to correct some typos and formatting problems introduced during the last two migrations and the use of Banjo’s unpredictable markup engine. Though I fixed a lot, unfortunately I suspect I added some new typos.

Before I installed Byteflow, I did a pg_dump on my blog database. Then I dropped and recreated the database for Byteflow. I then used that dump to enter each post through the Admin interface.

I couldn’t bear to lose the comments on my previous blog (all two of them), but the Admin UI doesn’t allow comment creation. So I submitted the comments through the blog interface, and edited the DB to have the original timestamps. Unbeknownst to me, Byteflow sent out an email for each comment alerting the original submitter that they had just created an account on my site. One of them contacted me, worried about being blamed for something they didn’t do. If you decide to do a similar migration, disable Django’s email sending ability until you complete the migration.

Problems with reStructered Text

I couldn’t use reStructured Text out of the box. I was getting weird errors that perfectly matched the bug reported here: http://code.djangoproject.com/ticket/6681

The workaround discussed there worked for me. All you need to do is comment out this line in django/contrib/admindocs/utils.py: docutils.parsers.rst.roles.DEFAULT_INTERPRETED_ROLE = 'cmsreference'

With that change, a was able to use reStructured Text on two posts, but the third had strange docutils errors showing up in my page. I didn’t have time to turn that in to a reproducible bug report, so I just switched to Markdown.

Impressions

Banjo features not in Byteflow

There are a few things I preferred in Banjo:

• Byteflow has no concept of a “Post Summary” and a “Post Excerpt”. The entire post shows up on the home page. I prefer this, but some people might not.
• Byteflow has a good tagging engine, but it has no concept of hierarchical Categories. Personally, I like Categories better than tags, but I’m willing to live without them for now (I’m emulating them with tags).
• Byteflow has no concept of a post update time. This also doesn’t bother me, but some might care.
• Byteflow can not configure the URL to a blog post in the same way as Banjo. I set Banjo up to by YYYY/slug, but byteflow insists on YYYY/MM/DD/slug. Given how rarely I post, I don’t need such a complex URL structure.

Best things about Byteflow

There is a lot to praise, but at the top of my list is the very helpful developer community (as represented by piranha). I appreciated their willingness to accept my patches and allow me to contribute.

The code is well organized and easy to modify.

I also like the way it handles comment spam. When a person posts a comment, a site account is automatically generated and an email is sent to the comment author. The comment is not viewable until it has been approved by either the site administrator or through the email sent to the comment author. In the future the author can log in to post without needing any moderation. This scheme should eliminate the majority of comment spam which is produced by people using non-legitimate email addresses. However, anonymity is sacrificed. I’m not sure if I prefer it to reCAPTCHA—that will take some thought.

Annoyances

There are a couple of design decisions that bother me a little bit, like the lack of admin control over comments and how subscriptions are handled, but these are pretty minor.

I worry about having problems with my data during upgrade, but the author is working on South integration so I’m hoping that is addressed soon.

The Byteflow.su project server responds to me only intermittently, but I’m starting to suspect it’s DNS related and might be on my end.

I’m not sure I understand the authentication system. The default template has some buttons related to authentication that mystify me (“Close”?). But it appears to be working okay.

On the whole, I have very few annoyances with Byteflow. The only big one I had (not being able to store themes outside of the byteflow root), was quickly addressed when they incorporated my patch.

Conclusion

As usual, the migration took longer than I expected, but it was less than two days of work (including research into Byteflow, documentation work, contributing changes back, playing with the CSS of my theme, and importing data). Manually importing and formatting posts took half of a long day. A clean setup without all the additional activities would take just a couple of hours.

I like how the Byteflow authors run their project. The pace of development has slowed recently because piranha is happy with the current feature set and has largely moved on to other projects. However Byteflow is not abandoned; piranha still uses it as his primary blogging platform. Since he is readily available on IRC and active on the mailing list, it is easy to discuss the architecture with him and make contributions. I am betting that I’ll be happy with Byteflow for a long time to come.

Summary: Herein the author finds that Django-cms is a pretty decent content management system.

I’ve finally gone live with the site for my new initiative: coopercate.org. As part of my effort to research the Django ecosystem, I decided to give a pre-packed CMS a try. There were two contenders: django-cms and Ella. Though Ella looked good, it appeared to be less mature than django-cms. So my efforts stayed focused on deploying django-cms and I never needed to fall back to Ella. It looks like Django-cms is going to meet my needs.

Which version?

There are two branches in the django-cms repository: trunk and cms2. It looks like cms2 is getting regularly merged in the trunk as features mature. At the time of my deployment there was a note in the wiki stating that django-cms is not compatible with the latest Django releases (which I needed for a different project), so I used the older r41 version that is downloadable from a link on the top-right corner of the home page.

It should be understood that r41 is fairly old—it is the original release from September of 2008. A lot of development has occurred since then which can be viewed on their demo site. I plan to install the newer version as soon as Django compatibility has been restored.

Installation

The installation instructions were sufficient to get django-cms up and running. Now that django-cms is successfully deployed, I want to document a couple of gotchas.

Prerequisites

Two prerequisites are listed: django-tinymce and django-filebrowser. Both prerequisites are optional, and in fact I only have them partially working as of now. They add some niceness, especially when building a site for a non-technical client, but they are not essential for the CMS to function.

django-tinymce depends on the TinyMCE javascript widget that is available as part of most Linux distributions. I didn’t like the layout of the Debian package however, so I downloaded version 3.2.1.1 from the project site. I then downloaded django-tinymce 1.5 and followed the instructions. I must not have put everything in the right places, however, because the WYSIWYG editor isn’t appearing for me. I’ll look into it more later.

I got django-filebrowser from SVN. Following the instructions resulted in what looks like a working link on the admin web site, but I haven’t tested the functionality much yet.

Configuration

I symlinked cms_global_settings.py and cms_settings.py into my project directory so that I could just override the variables I cared about in settings.py. I turned off LANGUAGE_REDIRECT and USE_I18N for my site. It is also important to make sure to have the DEFAULT_TEMPLATE set correctly. It appears to be relative from my project templates directory.

Base Template

The only step left was to copy cms/templates/base.html into my project’s templates folder and customize it. Now I have a functioning site.

Confusions

There are two areas where I find django-cms’s interface to be rather confusing.

• I don’t understand their distinction between a page and the content on the page. Each page can have multiple “contents”, but only the most recent one shows up on the live site. I am probably exposing my ignorance about content management systems in general.
• I don’t see an easy way of making django-cms display a list of all child pages on a specific page. Third-tier pages do not show up on the site navigation, so the only way I see to link to them is by manually maintaining a menu.

Impressions

Django-cms appears to have a decent user community and active development. Deployment wasn’t as easy as a PHP CMS, but it was easy by Django community standards. Though django-cms has seen a lot of activity recently, the older version I deployed is very functional and has had few quirks once I got it set up correctly. On the whole, I like it.

Summary: Herein the author discusses his selected blogging engine.

The Old Blog

I started blogging with a plain HTML file. I would simply append new entries to the top. The motivation was to have a place to record technical solutions that I repeatedly had to look up. The setup was functional, but didn’t look very sophisticated. It stopped being practical when I wanted to categorize entries on different topics.

I decided to upgrade a couple of years ago. I was learning python and figured that while I was at it I would brush up on my CSS+HTML skills. I chose PyBlosxom because I could actually understand the CGI script, and I created a layout with floats and divs. It worked pretty well, but due to a busy job schedule and switching hosting providers, I didn’t keep it up.

I really like PyBlosxom. It is ideal in its simplicity. It’s clear to deploy, the templates are easy to configure, and (best of all) the content is just ASCII text files in a directory. The categories come from the directory name, and the posting timestamp comes from the file name. This makes the blog very archive-able. Being able to access my data in twenty years is important if I’m going to invest any time and energy into creating it (thank goodness for the Internet Archive).

Unfortunately, PyBlosxom didn’t provide a lot of the niceties of a modern blogging platform. As my wife started using Blogger, I became envious of things like pingback support, feed subscriptions, and open-id authentication. I decided it was time to move on.

Discovering Django

I have played with a few web frameworks since learning Python. I’ve been searching for something as accessible as PHP, but better designed. I really like Spyce. It is well thought out, and I became a better engineer by tinkering with the framework. Unfortunately it doesn’t have much of a following. I missed the extensive documentation, libraries, and community I enjoyed as a young PHP coder. The enormous mindshare of PHP does bring some value.

Last year I discovered Django, and I’m hooked. It works the way I think. It is still pretty young, but it has a lot of momentum and I’m optimistic that it is maturing well. In my opinion, it is very successful at bringing the beauty of python to web application development.

Unfortunately, Django applications are very immature. Though initiatives have begun, there is not yet a central registry with ratings where good applications and plugins can gain mindshare and following. The Django community hasn’t yet developed habits like releasing code, preserving interfaces, and working off of stable versions. Though Django makes it very easy to build any application I might need, maintaining all of the nuts and bolts of my application makes it hard to compete with the PHP administrator who configures a couple of Drupal plugins and calls the site done. Most users won’t notice the difference in quality.

This realization led me to reexamine Drupal. I gave it a fair shot. Drupal does its job well, but it just isn’t elegant enough—it still feels like a PHP application. My python experience has made me finicky. Doggonit, I want a Django blogging application.

Bruce Kroeze says it well in his [blog post from 2007](http://coderseye.com/2007/banjo-blog-nearing-01-release.html]:

“Blogs look simple, and stripped of everything else, they really are simple. Basically they are just reverse chronological posting of text on a page. No biggie, at least at first or second glance.

“But the trick is in the phrase ‘stripped of everything else.’ Once you start considering the features commonly available in blog apps, it gets quite a bit more complex…”

The same can be said for a CMS, a shopping cart, or a photo gallery.

Getting Banjo

In the quoted blog post, Bruce Kroeze proceeds to describe Banjo: “the Django blogging application with bells and whistles”. Though immature when compared with some of the PHP blogging applications, Banjo has an impressive feature list. And being based on Django means that it should have a straightforward templating system and clean hooks for customization (this is where Drupal gave me a huge headache). It appears to be the only attempt at building a Django blog that has developed a proto–community. After mulling it over a number of months, I decided it was time to get my blog back online and give Banjo a test-drive.

Oh, The Agony

Deploying Banjo is not easy. The website is really a bunch of stub pages, and the mailing list is pretty quiet (I’ll have to save my rant about why Google Groups is awful for another post). What follows isn’t really a howto, but more of a map of the gotchas I dealt with in deploying Banjo.

I started by doing an hg clone http://hg.assembla.com/banjo and then followed the instructions here. Those instructions aren’t too bad—but the list of dependencies to install is daunting. High on the list is Satchmo—a Django shopping cart solution. It does seem odd for a blog to depend on a shopping cart, but Bruce Kroeze is a principle author of both applications and wanted to leverage some of the Satchmo libraries. Most of Satchmo is unnecessary—I’ll provide the details later. Unfortunately, Satchmo has its own daunting list of dependencies that must be installed. The dependencies are frustrating for a few reasons:

• Some of the projects depended on by Banjo and Satchmo appear to be abandoned.
• Some of the projects are disorganized, making it unclear which of the project source repositories is the most recent and compatible with Banjo.
• Some of the projects depended on by Banjo and Satchmo are under active development, but Banjo hasn’t seen any commits in a while, which makes me nervous that there will be breakage.
• I have to monitor all of these dependencies for security patches, instead of letting the Banjo community or my distribution security team do it for me.
• It’s simply a lot of installs (16 outside of my package management system), and when you include the source histories for all of these projects it becomes a chunk of disk space (129M for a blog!).
• Banjo usually depends on the unreleased source tips, instead of a specific released version. Making for a lot of moving developer targets.
• Not all of the dependencies are listed on the install instructions for Banjo or Satchmo.

The list of dependencies provided here is complete as of the day of this blog post. When it was necessary to check out source tips, I found that Banjo worked with the sources as checked out in early February 2009.

A note on installing python packages: As a general rule, I don’t use Python’s setup.py functionality to install packages, nor do I use methods that call setup.py such as easy_install and pip. Python’s distutils has two major problems:

• there is no clean uninstallation,
• it mixes manually maintained packages with ones that the package manager will track for me.

Instead, I copy the uncompressed files into a common directory and add that directory to my sys.path at the top of my Django settings.py. Sometimes it helps to check the Python Package Index (pypi.python.org) to find the latest released version. Occasionally it is necessary to run setup.py build and copy the files out of the build directory.

My prerequisites:

• Debian Lenny
• PostgreSQL
• Python 2.5
• python-psycopg2
• Apache
• mod_python
• memcached

I also had django-1.0 installed and running when I started setting up Banjo.

Dependencies:

• python-yaml from aptitude.
• python-simplejson from aptitude. I didn’t see this listed on the Banjo or Satchmo docs.
• python-imaging from aptitude.
• Django (svn tip just after version 1.1 alpha—Banjo doesn’t work with version 1.0). I wish Banjo stuck with the released version.
• django-registration (version 0.7 from pypi). This project is under active development.
• iso8601 (version 0.1.4 from pypi). This project appears mature but not active.
• typogrify (v 1.0). I didn’t see this requirement listed in the docs.
• sorl-thumbnail (svn tip). This project appears mature and maintained, though they don’t ever bundle their releases for download.
• django-template-utils (release 0.4p2 would probably work, but I used svn trunk because the one additional change looked safe). This project looks mature, but not very active. The Banjo installation instructions need to be updated to reflect this dependency.
• django-evolution (svn checkout). This is an active project.
• django-extensions (svn tip). This is an active project with a large community. It must be called “extensions” for Banjo to import it.
• django-atompub (svn checkout). The project doesn’t look active.
• django-xmlrpc (bzr trunk—version 0.1 doesn’t work). Don’t be deceived by the link to Google Code as the project’s home page (the Google Code svn source doesn’t work); Launchpad is the right place. The Banjo instructions need to be updated as well. This project doesn’t look actively maintained, especially since the source requires a trivial edit to atom.py (add an import re) to import. A patch was reported six months ago and hasn’t been applied.
• django-app-plugins (svn trunk). Project appears active, but has some trivial problems like missing a license notice (I just submitted the issue). The website says it’s released under the MIT license.
• django-threaded-multihost (version 1.3-0 downloaded from www.assembla.com/spaces/threaded_multihost/ documents). This project doesn’t look used outside of Banjo and Satchmo—not much of a community. But it appears to work well. One note: the released tar.gz does not contain the license and docs that are in the hg repo.
• django-site-skins (hg trunk). This was recently spun off of Banjo and Satchmo, and isn’t listed as a dependency in the docs.
• django-admin-plugins (hg clone of tip at http://hg.assembla.com/adminplugins). This is another Banjo/Satchmo library.
• Satchmo (release 0.8.1 might work because Banjo hasn’t changed much since then, but I downloaded the svn trunk). This is a Django shopping cart with a good community. However, Banjo only needs a couple of the Satchmo libraries.
• Banjo (hg checkout of http://hg.assembla.com/banjo. I didn’t try the 0.9-1 release downloadable from assembla).

I only had to make sure that a couple of Satchmo modules were in my python system path. I got them there by symlinking them into the common directory which I appended to sys.path. The source doesn’t appear to import anything else from Satchmo, so deleting the rest should be possible.

Satchmo modules:

• keyedcache
• livesettings
• satchmo_utils

After all this, I am finally able to start configuring Banjo!

Configuration

• I uncommented django_evolution from banjo/__init__.py
• When debugging, it is really helpful to comment out django_xmlrpc from banjo/__init__.py, and replace the line from banjo.blog.syndication import XMLRPC_METHODS in settings.py with XMLRPC_METHODS = None to eliminate a class of spurious error messages.
• The default skin relies on an out-of-date version of yui from Yahoo APIs. If you want to use it you should change the link so that you get some CSS.

A couple things I did differently with my Banjo configuration: * I merged the Banjo settings.py with the one in my application so that none of my changes would be in the Banjo source. It was pretty straightforward. * I added this rule at the end of my urls.py so that all URLs I hadn’t already dealt with go to Banjo:
(r’^’, include(‘banjo.urls’)), * I then created my own skin using the Banjo base_root template as a starting point.

Skins

The docs on creating a skin are non-existent. Here are the steps I posted to the mailing list:

1. I copied banjo/blog/templates/skins/default into myapp/templates/ skins/myskin
2. I edited myapp/ templates/skins/myskin/CONFIG.json to have a different name.
3. I added myapp/templates/skins to the SKIN_DIRS list in settings.py.
4. I went into the admin interface and selected my skin.
5. I then excised all the stuff that didn’t apply to my site and eliminated a couple of layers in the template inheritance hierarchy.

I found I couldn’t make all of the customizations I wanted in the skin, so I did have to edit a couple Banjo templates (_post_excerpt.html and view_post.html). I’m hoping to figure out a cleaner way in the future.

As you can see, I did eventually get everything working. It was less work than building from scratch such features as comment moderation, feed subscription, integrated search, scheduled post submission, caching, etc. And it’s higher quality as the Banjo developers included utilities I didn’t know about like Textile format and typogrify

Suggestions for Improvement

However, this process could be easier. Here are my suggestions for making Banjo more maintainable and more accessible to future users:

• There should be a bias towards using released versions of projects. This makes it easier to install a version that works, and it makes it easier to monitor the libraries for security updates.
• If a dependency project doesn’t have an active community, Banjo should consider folding that code into the Banjo core.
• Even if a dependency project does have an active community, if it isn’t likely to get distributed by an OS package manager, Banjo should consider including the project with the banjo source in a libs subdirectory.
• The maintainers should consider putting all the various Banjo and Satchmo libraries into a single package. In addition to making this easier to find and install, it also makes it easier to maintain. Currently they are scattered across various source repositories, issue trackers, and project home pages. This fragments an already small community.
• The necessary Satchmo libraries should be factored out of Satchmo and put into the shared libraries package as soon as possible—it will only get harder with time. Banjo shouldn’t require the installation of a full blown shopping cart, and the Satchmo users will benefit from the larger community using their libraries. Satchmo and Banjo can then select which version of the libraries to depend on for each release, which will simplify both projects.
• The project documentation can be improved with some of the information from this blog post.
• Though the multi–site capabilities are excellent, I think the defaults can be improved so that it doesn’t get in the way of a simple single-site deployment. It doesn’t currently prevent single site deployment, but it makes it harder than it needs to be.

Conclusion

I think Banjo is pretty slick, and I hope to be able to contribute to the project. It has the potential of becoming a full featured, easy to install and yet easy to modify, blogging platform. I’m also glad I discovered Banjo because it introduced me to Satchmo, which looks like a mature Django shopping cart application which will likely come in handy some day.

My next project is going to involve testing django-cms to see if I should add it to my my web developer toolbox.

Blog, shopping cart, CMS—Django is quickly getting to where it can fulfill most of my client’s needs.

Summary: Herein the author abandons one approach to spam filtering.

My family has demanded that I implement an anti-spam solution on our mail server. After doing some research, I decided to give the Anti-Spam-SMTP-Proxy a try. ASSP represent a lot of development work, and appears to be a mature product. It has everything I want: Baysian filtering, DNS RBL, Web customizable, a nifty email interface, and as an SMTP Proxy should require minimal configuration to use. Though ASSP reflects an important and valuable contribution to the open source community, it has some design and implementation deficiencies which make it the wrong tool for my environment. I want to document them here to help anyone else who might be investigating ASSP.

• First, it is a 9,000+ line Perl script. This lack of modularity makes it very painful to study and/or modify.
• In that huge Perl script, the web interface code is mixed with the processing code, and the default configuration file is thrown in there too. This makes it difficult to examine implementation methods because every search term will have a couple of non-implementation hits.
• Configuration files, data files, and source code files are all jumbled in the same directory. This makes it difficult to use lowest-level privileges, or to run ASSP on a read-only file system. Things like the current working directory, and the path to the configuration file, cannot be changed. This makes it hard to correct these organizational problems.
• The documentation is out of date. For example, it refers to the “Test mode” button. Unfortunately there is a whole section of test mode settings that are completely undocumented. Because the source code is obtuse, and the documentation lacking, some of the configuration settings have unexpected effects.
• ASSP does not speak TLS, so TLS connections have to bypass ASSP. This prevents users from using the email interface and complicates the auto-whitelisting.
• It also cannot relay mail directly to the Internet, so in TLS configurations two mail servers must be used: one which speaks TLS to receive mail from the user and hand it to ASSP, and one for ASSP to use to relay the mail to the Internet.

Though I can probably still get ASSP working to meet my needs, at this point the ease-of-use factor is negated. My difficulty is that there aren’t a lot of other really good anti-spam solutions. I’ve looked at both SpamAssassin and DSPAM. ASSP has some advantages over both of these solutions:

• The SMTP proxy design in theory should be easier to set up than other methods.
• I would predict that sender based auto-whitelisting and sent email automatically being added to the ham corpus would be more reliable for a small site than Baysean auto-whitelisting or receiver based auto-whitelisting.

I am considering using the knowledge and algorithms in the ASSP code base to start my own project. I could use an educational project, and there is much to learn in the ASSP scripts. Using the Twisted framework in Python would give a really big head start.

In the meantime, I’ll probably need to give one of these a try.