As I started moving forward with a plan to host Satchmo on my own, I began experimenting with hosting solutions. I already had a traditional hosting service and was able to get Satchmo running as a fast cgi. It worked reasonably well, but the control was less than optimal. For some reason, I never could figure out how to effectively start and stop my django processes. I also decided that I wanted to host my own Trac and Subversion instance so that I could have a similar level of integration as the Django site. For all these reasons, I decided I need more flexibility so started looking at VPS and VPS-like solutions.

I ended up going to http://www.grokthis.net for one of the advanced hosting accounts. After playing with it, I started running into some limitations and realized I really did want more control. So, I moved to one of their small Xen VPS accounts. These accounts have 96MB of RAM and 192MB of swap. I really liked the root access and quickly learned to like the Ubuntu distro.

As I configured the VPS, I tried to keep my memory footprint as low as possible. The first thing I did was get rid of Apache and move to lighttpd. If you want to see some good detailed instructions on getting lighttpd, Django and Satchmo running, take a look at Bruce's detailed tutorial here - http://coderseye.com/2007/setting-up-satchmo-on-a-debian-server.html

My exact setup is a little different and I will eventually post the examples of the scripts I use. For now, though, Bruce's write up is pretty close to mine. In addition to Satchmo, I wanted to run Trac and Subversion. Fortunately, subversion comes with it's own server which is pretty light-weight from a memory perspective. The only downside is that you have to access subversion via the svn:// protoccol instead of http://. This causes some problems with some brain-dead firewalls but it hasn't been a show stopper yet.

With lighttpd, the only option to run Trac is through fast-cgi. The process was pretty simple but I noticed that Trac started to use a bunch of memory. I would normally end up with 5 Trac processes all using 15M+ of memory. In order to conserve overall memory, I decided to run Django in the threaded fast-cgi instead of preforking. More on this choice later.

The other memory hog in this setup was MySQL. After looking around on the web, I found a few good suggestions. Here are some of the changes I made in /etc/mysql/my.cnf:

key_buffer = 8M
query_cache_size = 2M
skip-innodb
skip-ndbcluster

These changes drastically reduced MySQL's memory footprint. I honestly don't remember how much but it made a big difference. Obviously, I'm dealing with a very small database so the tuning I did may not work for a real production system with lots of users and data.

After the MySQL changes, everything seemed to run pretty well. Using top and ps, I could tell I was using a lot of my memory and was going to swap quite a bit but the site seemed to run ok. Usage was pretty low (less than 100 visitors a day) and it sometimes seemed slow to me but I was ok with it. I ran like this for a while, then we launched Satchmo 0.5.

Traffic picked up dramatically as the word spread and the vps couldn't keep up. As the small deluge subsided, I was able to see that my memory filled up and the constant swapping doomed the site. By no means was this a slashdot or Digg-level traffic but it was too

After doing some post mortem analysis, it was clear that I needed even more memory. I moved my account to a different grok-this program called vps-village. This plan gave me 256MB of RAM for only $20 so I figured I'd give it a shot. Because I was on the same hosting provider, I was able to get everything moved and did not have to reconfigure everything.

After the move, I noticed I had lots of free memory. I decided to move Django away from the threaded server and go to the prefork with the following options:

prefork minspare=2 maxspare=4 maxchildren=5

My trac config now looks like this:

## FastCGI programs have the same functionality as CGI programs,
## but are considerably faster through lower interpreter startup
## time and socketed communication
##
## Documentation: /usr/share/doc/lighttpd-doc/fastcgi.txt.gz
##                http://www.lighttpd.net/documentation/fastcgi.html

#server.modules   += ( "mod_fastcgi" )

fastcgi.debug = 1


fastcgi.server = ("/trac" =>
                  ("trac" =>
                    ("socket" => "/var/trac/run/trac-fastcgi.sock",
                     "bin-path" => "/usr/share/trac/cgi-bin/trac.fcgi",
                     "check-local" => "disable",
                     "min-procs" => 1,
                     "max-procs" => 3,
                     "bin-environment" =>
                       ("TRAC_ENV" => "/var/trac",
                        "PYTHON_EGG_CACHE" => "/var/trac/egg-cache")
                    )
                  ),
              "/main.fcgi" =>
                 ("shop" =>
                   ("socket" => "/var/django/satchmo.sock",
                    "check-local" => "disable",
                   )
                 )
                )

url.rewrite = (
"^(/admin.*)$" => "/main.fcgi$1",
"^(/shop.*)$" =>  "/main.fcgi$1",
"^(/account.*)$" => "/main.fcgi$1",
"^(/blog.*)$" => "/main.fcgi$1",
"^(/comments.*)$" => "/main.fcgi$1",
"^(/)$" => "/main.fcgi$1",
  )

This has server me well. Hopefully this is useful for others trying to figure out how to manage multiple apps on a small footprint vps server. In the future I'll probably move the trac and svn portions to another box but that is another post.