programming

I'm considering changing the development environment we use to host our projects in work and I'm not alone. The reason why will become more clear further down. To be honest this article is partly to help me work out what my priorities for this are, and to appeal to the lazy web for any other suggestions.

So first of all, here are the things I'm looking for in this project.

These are essential:

1. must be free software;
2. must have flexible issue tracking with user defined fields;
3. must be able to handle multiple projects;
4. must be extremely transparent to non developer users;
5. must support svn and git and graphical front ends to them;
6. must support some kind of announcement system.

These are desireable:

1. should integrate in some way to mailing lists;
2. should support ad hoc tar bar downloads from scm;
3. should be easy to search issues;
4. should graphically depict progress on releases;
5. should have documentation (wiki like) integration;
6. should handles news and download areas;
7. should be able to exchange data (like issues) on projects;
8. should actively maintained;
9. should automate most sysadmin activity, account creation etc.;
10. should be packaged for Debian ideally.

The contenders thus far are:

• Savane, which we are currently using;
• Fusion Forge;
• Redmine;
• Trac.

Let's take these one at a time.

Savane

Savane currently supports all the essential features, with the exception of git support. Being a fork from the original SourceForge software it works in a similar way. A web front end (in PHP), with back end functionality (in Perl) that creates shell users and groups, interfaces with mailman and so on.

But there are some problems. First of all the official project seems to have ground to a complete halt. Fixes that were submitted by users are not applied, there seems to be no forward momentum whatsoever. This is a problem, the offical version needs patched (modestly) to run on PHP 5, and the code base is a mess, with a few pages still requiring register_globals to be on. Bad.

However, another fork was taken some time ago which addresses all these problems including git support, well actually, the code base is still probably in need or work, but most apps of a certain age have this problem. What I'd like to see in Savane is better graphical tools to monitor project progress, better documentation features and incidentally better Debian packaging, but I've signed up to the fork to work on the latter on the first instance (this looks like it will be quite a bit of work, it's non Debian policy compliant in many ways right now). The new fork is going to be forked again by the way, into a Python version. Could be interesting.

Fusion Forge

Previously known, or descended from GForge, Fusion Forge is another fork of the original SourceForge source code. It is well packaged for Debian (naturally, it runs Debian's own code hosting environment, Alioth) and well maintained. It has many of the features of Savane, which probably makes it puzzling why I didn't choose it in the first place. Well, there's one reason why, in my opinion the front pages of FusionForge are rather un-user friendly, I mean what the heck is "Code Snippets" doing there? They feel very aimed at developers, which is great... but I need a nice straightforward interface for less savvy users. Looking today, the navigation still feels it's just too developer centric, but as these sites double as developer sites and user sites (to acquire the software, report bugs and so on), that's not great. I really don't know yet whether Fusion Forge has better graphical tools than Savane.

In a conversation about this last week, Noodles suggested I should look at the ease with which the interface could be changed. A sensible suggestion I shall follow up.

Redmine

Written in Ruby on Rails and advocated by my work colleage Paul Vitty, Redmine is clean and elegant looking. It shows all the signs of benefiting from being a later generation project and has a plugin architecture that seems excellent too. It has easy clean wiki integration, graphical road maps and Gannt charts; and these things make a difference. I don't think it's as remotely scalable as the design of the first two systems, which isn't a huge problem for me, but I mention it in passing.

On the other hand, it seems to have little or no automated integration into the backend, user and group creation, mailing lists and so on. on the other hand it has a very rich set of plug ins, so it may already have such support or it may be possible to implement it.

Trac

Written in Python, Trac feels like Redmine lite, and that's in the wrong direction for me. It explicitly does not have multiple project support and that's a deal breaker for me.

I haven't come to a strong conclusion about this so far. I note John (whom I linked above) notes that Redmine is difficult to install and upgrade because of Ruby, this is a hassle I just don't need, but he also thinks it may be the best of the field. For me, it is probably more trouble free to migrate to the fork of Savane or to FusionForge. More thought needed, and your thoughts are most welcome!

Posted by Colin Turner

In PHP, sessions are by default stored in a file in a directory. Sessions can be specifically destroyed from within the code, for example when users logout explicitly, but frequently they do not. As a result session files tend to hang around, and cause the problem of how to clean them up. The standard way is to use PHP's own garbage collection which is normally enabled out of the box. In this, we define constants that specify the maximum lifetime for the session, and essentially the probability of clean up.

To make things more interesting, Debian, out of the box doesn't do garbage collection in this way. It has a cron job that regularly erases session files in the defined session directory. But, if like me, and many others, you put your session files in a different directory for each application to avoid clashes on namespaces for two applications running under the same browser from the same server, you have a problem. If you forget Debian's behaviour the session files will just grow indefinitely. I had forgotten this issue and found over a year's worth of session files in a directory recently.

Solving this problem is actually quite difficult to do optimally. I mean, I could create a cron job to mirror Debian's own, but then I'd have to put the maximum lifetime in a cron job somewhere, out of the way, and difficult for the average sys admin I'm working with to find and deal with. (That is, away from the main configuration of the project). Or I could parse this value out of the main configuration. But this leads to another problem. For some users, a 30 minute maximum idle time is acceptable (although in my case where actually a suite of applications are being used as a single gestalt entity that can even be a problem), but for many of my administrator users you need huge idle times, since they are used to logging in first thing, and periodically working at the application through the day.

In the end I settled on changing our framework to make it easy to pass through garbage collection values. This makes an interface to the configuration really easy, but it doesn't solve the problems of long session times that not all users need, and huge delays in garbage collection.

In my last article I talked about a Munin plugin for OPUS, but when you look at it you'll see these kind of cliff fall drops, which are caused by the garbage collection finally kicking in and removing sessions where users have not explicitly logged out. Currently, every ten minutes, OPUS runs through its user database and finds users who are allegedly online but have no active session file and then marks them offline. Then it updates the file with the online user count that Munin reads.

I suspect eventually, I will write a more sophisticated script that actually kills sessions depending upon idle time and the user class, which would make for a more accurate picture here. Any brighter ideas gratefully accepted.

Posted by Colin Turner

I've tried lots of programming editors and ides over the years, obviously in Unix and Linux this is a Holy War, particularly between the advocates of vi and emacs. It is common for both groups to suggest that the other editor is hopelessly over-complex or clumsy. I think there's some truth in that, because essentially, they both stink.

I tend to be an emacsen user myself, but I just think emacs is slightly less awful than vi. My first action on a new install is usually to use vi to edit my sources.list in Debian, to help me install emacs. Perhaps thats strange, because I really like sed. So what's the problem with them? They both share this kind of puritanically awkward interface that works well on a console, but sucks in a GUI. They both use ridiculously arcane sequences of key presses to do anything, and I mean even basic stuff like saving and quitting. Yes, yes, you don't have to lecture me about old terminals and their limitations, been there done that, got the t-shirt. I tend to do all my systems maintenance in emacs, but when I'm programming, I've started to love the softness of a decent editor that actually makes it plain and simple to edit multiple buffers of source code, even though its a pain to use different editors for console and gui work.

Continue reading "Geany and other Development Tools"

Posted by Colin Turner

A while ago, I discovered that my current main development project OPUS had an odd problem when printing out of a gecko based browser.

It would print the first page, whether in portrait or landscape, and if there was more content, it would be abruptly truncated and the second page would contain merely the footer off the page. I've been meaning to try and solve the problem with a print stylesheet for a while and finally did so today.

Continue reading "Fixing truncated printing with Firefox"

Posted by Colin Turner

So, in order to post that rant about PHP and SimpleXML I had to fix a problem that seems to have spontaneously arisen with Drupal (this content management system). For some reason it wasn't persisting login information, at least from firefox (sorry - iceweasel here on my Debian system). It's interesting to note, reading about the bug, that it has been around for literally months and doesn't seem to have been nailed. So, anyway, I've installed some beta of Drupal, and yes, it now seems to be fixed... If I could only solve the problem that I can't "uncollapse" parts of the content now. UPDATE: OK, this seems to be a problem with firefox version 2, or probably really the CSS file for it. It works with Galeon, or when I tell firefox to fake being IE .

Posted by ct

Defined tags for this entry: Gondolin, php, programming

Another night of coding in PHP, and I've officially decided that SimpleXML utterly irritates me. I'd already discovered, much to my irritation, that is virtually impossible to handle SimpleXML objects elegantly with the Smarty template engine - but now I discover I can't even shove them in a PHP session without trouble - when you next visit the site you get stuff like this: Warning: session_start() [function.session-start]: Node no longer exists and then more trouble. As part of a new Web Application Framework I'm working on I wanted to parse XML configuration files one time only, and then cache the results in the session. It looks like I now have to totally redesign my idea . You can see the work in progress at its home page.

Posted by ct