In parallel to the development of the GLSR (Gentoo Linux Script Repository a.k.a GLEP 15) myself, Scott (hadfield) and John (AllanonJL) have been working on developing a new Python based web framework. The decision to do this came about after it became apparent that we weren’t going to achieve any decent response times with CGI. We then spent a while working with FastCGI, but that turned out to suck also. With quite some code already laid down, we were reluctant to go down the mod_python roue and existing python frameworks either didn’t suit our style or looked rather sucky aswell.
So i’ll give you the blurb about Harmonious, if I may!
Harmonious consists of 5 distinct parts: A web server, a page dispatching agent, a templating engine, session handling routines and a database layer.
I’ll tell you about these in reverse order:
The database layer is still undergoing design and code changes so nothing is final yet. Basically, you’ll be able to easily use different database back ends much like phpBB can. Scott is the guy to talk to about this.
Session handling… pretty standard, not a lot of room for innovation in this area. Only thing worth mentioning is that it’ll be tightly integrated with the rest of the framework.
The templating engine has been around for some time now and is reaching the first stages of maturity. Its a fairly simple engine with nice, clean syntax:
{IF VAR == “fu”} fu {ELSE} bar {!IF}
{LOOP ITER_OBJ} ITER_OBJ.current {!LOOP}
{VARIABLE}
{IMPORT header.tpl}
It does caching also.
The page dispatching agent is where it gets a little more involved. To better explain the agent i’ll give you a quick lo down on the definition of a ‘domain’. A domain is simply a directory which contains sub domains, which in turn can contain sub domains of its own, for example: http://host/domain1/subdomain1/subsubdomain1/. The key feature is that domain modules can inherit ‘parent’ modules contained within the current domain or the domain above it, furthermore, parent modules can also inherit other parent modules from the domain above and so forth.
So, when the dispatching agent is told to load a specific domain module, it searches the hierachy for a module whos name matches the requested path and imports it. Hey presto! Only the code needed to handle the request has been loaded.
We also have ‘site’ modules which contain various classes and functions specific to domain modules (as little code as possible is contained within a domain module. As much as possible is abstracted into a site module), but i’ll not bore you any longer with those.
And finally the web server. This daddy does all the handing off to the page dispatcher and not a lot else in terms of servicing a quest, but wait, I haven’t told you about it’s threading model yet.
*deep breath*
As I mentioned earlier, one of the reasons for not using CGI was the speed factor, I couldn’t just implement a standard web server now could I (even though having the interpreter in memory is the main benefactor)?
The web server starts off with a base number of 4 worker threads ready and waiting to handle a request. If 4 proves to be a little too few, the server will create a new worker thread and add it to the queue. If 5 isn’t quite enough… etc, etc.
While all this is going on, another part of the web server is timing how long each request took to complete and is keeping a n point moving average of request times. If the past few averages got larger after adding more threads to the queue, the server will hold off and not allow any more worker threads to be created. As the load goes down it’ll gradualy kill any threads not doing anything.
I’m sure some of you are sitting there thinking ‘DoS’ to yourself, well no need to worry, as the server will not allow threads to spawn too quickly and no more than n in the last i minutes.
This threading module is all theoretical at the moment and if it actually improves performance under heavy loads will remain to be seen.
With the end of the exam period now in sight, i’ll be working on this as much as I can. A beta release shouldn’t be too far off, with a public release of GLSR following shortly afterward.
