Content mirror and this site

Created: Apr, 03 and last updated: Apr, 03

How this site is configured and some more details on content mirror.

I've been asked for this a couple of times, so here's some details on how Content Mirror works on this site.

This site's main editing interface for editors (myself) is done in Plone. Plone is mirroring it's content into a Postgres database, which is being synced to a Django site, you are then reading it in something served by Django.

Server configuration

My Plone site is running in a Virtual Machine on my Mac. It is in fact an out of the box Plone 3.x, that's easy to install and set up. To that installation I added in Content Mirror and psycopg. For this I'm using Postgres, it would work with any database, but I much prefer Postgres.

Next I created a Postgres database locally. I did this locally because I wanted to develop the site and wanted to install and play with everything locally. Developing locally is fast and simple.

So I've got Plone and Postgres installed locally with Content Mirror pushing content to Postgres. With that up and running I tried it out by adding in a few documents, publishing them and checking that everything worked.

So here's how the "About page" of Djangozen, you'll notice it is vanilla out of the box Plone:

Building a front end

Once I've got everything going into the database, I can start to add in a front end. For this I chose Plango and then started modifying the heck out of it. Plango is here and is a front end in Django. It has a simple URL resolver that simply grabs the request, then looks that up in the database. So /about becomes a lookup in the database for a piece of content with no parent and an id of about.

There's certainly some opportunity for optimization in those URL look ups, but we'll about that later. The nice thing about having that Django front end is that it's a snap to add in comments, simple openid login and so on without doing any Plone development at all.

The plugin's section is added to and edited by end users, so that's a Django app (that's embarrasingly simple).

So after some fiddling around with the code, here's how the front end looks.

Deployment

Deployment of this site was actually very easy. I have a Webfaction account I use for various non-critical things. It's cheap and easy to set up - I'm running 4 sites on it at the moment for under $10 a month.

The Django code went into SVN. And then I made Django app on my server, copying down the code I'd developed locally. Then I added in Postgres database.

To move data up is pretty simple, I stopped the local postgres and made an ssh tunnel. Restart Plone and it's now writing to the remote postgres over the ssh tunnel. My script looks something like this:

sudo /etc/init.d/postgresql-8.3 stop
ssh -L 5432:server:5432 andymckay@server
sudo /etc/init.d/postgresql-8.3 start

The first time the connection is made, just run the bulk deployment script and the content is uploaded. After that all content will be incrementally deployed.

Here's a diagram of how the deployment looks:

Gotchas

• There's currently a bug in Content Mirror that means the ATEvent content type has none of the extra fields (such as startdate, enddate, location...). Only I seem to get this issue at the moment however.
• Changing the Postgres that the Plone site is talking to a lot can cause Plone to give errors, so try not to flip the Postgres database too often via the ssh tunnel.
• If you have a lot of content that is being added to the site on the front end, you loose a lot of the Plone advantage. For example I want people to be able to add events to Django zen and have a review step... there isn't a backwards step, I can't review content in Plone.
• Got to watch out for review states for things like navigation, RSS feeds, search and so on, if you want only published content to be viewed.
• Things will be written to the database without Django knowing about. So, for example, Plango has a full text search. A cron job runs every 5 minutes, looks for things that have been updated and adds or removes them from the full text search, as needed. A database trigger works well, until I found Webfaction doesn't allow them.
• You need to fight Kupu to get it not to resolve uid's in HTML so that if you embed an image in a document (such as this one) you can unresolve it.
• Not everything in Plone is going to be translated, but that's part of the cost.

Summary

For Djangozen it's probably overkill. Although writing this blog post in Kupu, adding images etc has been quite a pleasure compared to the Django admin interface. But now most of the above gotchas have been resolved, it's quite simple and quick to set this up and end up with a good site in no time.

Comments

There are 5 comment(s).