Proving the Obviously Untrue

I am not a fan of Microsoft Windows, most people who know me appreciate this to be a slight understatement. But I've used Microsoft's products. Like many people I've often had no choice. I actually use Microsoft trackballs. It's quite an irony that the best thing I can say about the world's largest software company that some of their hardware is good.

There's a tendency for people to believe I have no basis for this dislike, or that it is just kind of fashionable. But actually, I've used lots of operating systems over my time, and I've got good, solid reasons to dislike it. Here are just two.

1. Their software is ... crap;
2. Their conduct in the market place is just disgraceful.

So let's explore these a little more. The week before last, I was forced to use Windows (I know). So often, when one buys a consumer device (in this case a satnav system), it is will nigh impossible to update or maintain it without it. I had a huge download and install to do, and left it running overnight, safe in the knowledge that in the morning it would all be over and I could revert to Debian, my current OS of choice.

Of course not, I woke up in the middle of the night to find the box (laptop, I mean Vista came installed with no other choice, you don't think I wanted it did you?) had shut itself down due to inactivity, even when it was plainly on mains, and busy. Wearily I started it all over again, the download manager failed to resume and started from scratch. In the morning I found the laptop at a Debian prompt. Nice, but unexpected. Yes, Windows had helpfully, and without my consent restarted itself half way through the download to install updates. I note people wrestling with these stupid prompts doing presentations all the time. So more hours later I finally download the software, installing it is, quite simple, a nightmare. The task manager shows the box is waiting for consent, but there is no visible window. I have to wrestle with the whole thing for some hours. In a way, it's a good thing. It reminds me that I actually find this OS not just unethical, but just plain brain dead and ineffective.

But on the topic of ethics, I frequently lecture on these issues and list some of Microsoft's past misdemeanors. It often comes as a surprise to my students that these are often proven court findings, not just vague soapbox opinions. So for that reason, this report from ECIS, the European Committee for Interoperable Systems, is an excellent document. Well researched and referenced, it makes the long and consistent pattern of disgraceful behaviour in the market place very clear. It's a very readable report, but for the impatient, this Groklaw summary is very helpful.

On the one hand, these two reasons look very distinct, but on the other, they are actually inextricably linked. Microsoft's monopoly position, gained through some skill, and much luck, and bolstered by their persistent behaviour issues, is exactly why their products are so crap. Put simply, Microsoft is lazy; it is only in the presence of strong competition that they raise their game. Just look at the changes in Internet Explorer since Mozilla Firefox entered on the scene.

Yeah, for many of you, none of this is any surprise, but sometimes it's useful to remind myself, and everyone else, that the position I have on Microsoft is based on experience and fact.

Posted by Colin Turner

In PHP, sessions are by default stored in a file in a directory. Sessions can be specifically destroyed from within the code, for example when users logout explicitly, but frequently they do not. As a result session files tend to hang around, and cause the problem of how to clean them up. The standard way is to use PHP's own garbage collection which is normally enabled out of the box. In this, we define constants that specify the maximum lifetime for the session, and essentially the probability of clean up.

To make things more interesting, Debian, out of the box doesn't do garbage collection in this way. It has a cron job that regularly erases session files in the defined session directory. But, if like me, and many others, you put your session files in a different directory for each application to avoid clashes on namespaces for two applications running under the same browser from the same server, you have a problem. If you forget Debian's behaviour the session files will just grow indefinitely. I had forgotten this issue and found over a year's worth of session files in a directory recently.

Solving this problem is actually quite difficult to do optimally. I mean, I could create a cron job to mirror Debian's own, but then I'd have to put the maximum lifetime in a cron job somewhere, out of the way, and difficult for the average sys admin I'm working with to find and deal with. (That is, away from the main configuration of the project). Or I could parse this value out of the main configuration. But this leads to another problem. For some users, a 30 minute maximum idle time is acceptable (although in my case where actually a suite of applications are being used as a single gestalt entity that can even be a problem), but for many of my administrator users you need huge idle times, since they are used to logging in first thing, and periodically working at the application through the day.

In the end I settled on changing our framework to make it easy to pass through garbage collection values. This makes an interface to the configuration really easy, but it doesn't solve the problems of long session times that not all users need, and huge delays in garbage collection.

In my last article I talked about a Munin plugin for OPUS, but when you look at it you'll see these kind of cliff fall drops, which are caused by the garbage collection finally kicking in and removing sessions where users have not explicitly logged out. Currently, every ten minutes, OPUS runs through its user database and finds users who are allegedly online but have no active session file and then marks them offline. Then it updates the file with the online user count that Munin reads.

I suspect eventually, I will write a more sophisticated script that actually kills sessions depending upon idle time and the user class, which would make for a more accurate picture here. Any brighter ideas gratefully accepted.

Posted by Colin Turner

Munin is a great, really useful project for monitoring all sorts of things on servers over short and long term periods, and can help identify and even warn of undue server loads. It is also appropriately and poetically named for one of Odinn's crows (so I suppose I should have written this on a Wednesday).

We've been running Munin on one of our production servers at work for quite some time, and it gives us a lot of confidence that, to say the least, the server is running in its comfort zone around the clock. Among other bits and pieces, we run OPUS and the PDSystem on this box, two of our home grown projects that are available to the students. For some time now I've considered writing a plugin for OPUS to show logged in users, and I finally did this, albeit the counts are not nearly so reliable as I'd like for two reasons, but I'll probably discuss that in another post. Anyway, I arranged for OPUS to drop a simple text file which simply contains counts of online users with the syntax and so on, for each of the categories of users. Then I needed a plugin to deal with this. I decided to write it simple shell script, since its portable and I'm not much of a perl fan. The plugin has now been running for several days, and you can see its output here. There are problems with it, but that's more to do with PHP, Debian and user choice, and I'll comment on that another time. However, already it gives me a useful feel for a lot of user behaviour.

Writing Munin plugins is easy, and Munin does so much of the hard work of turning your creation into something useful.

Posted by Colin Turner

I read from Slashdot that Microsoft is taking legal action against TomTom over a number of alleged patent violations. Three of these are apparently targeted at the Linux kernel in use by TomTom.

Much of the focus of the discussion revolves around the use of FAT and FAT32 storage systems on media cards, and I have to confess this is an issue I hadn't previously considered very deeply. Microsoft has patents on these filing systems, and in fact it is important to note that Windows does not currently support any filing system that isn't one of Microsoft's own, patent entangled file systems.

There are very important, highly anti-competitive consequences to this.

• In order for media cards to work with Windows - still a practical monopoly offering in the operating system market - these cards have to support a MS entangled filing system.
• This likely means Microsoft is obtaining a revenue stream for all the licensing on many of these cards and their associated readers.
• Other operating systems, if they wish to be equally convenient to those using media cards, cameras, media players have no choice but to try and include support for these filing systems, even if they are not technically superior.

I would tend to agree with the observations of many that litigation here seems to be all that remains in a sad absence of innovation, but I would hope that if various legislatures have seen the browser issues as anti-competitive that they will consider issues like these too, especially if Microsoft starts throwing its weight around. It's not in any consumer interest for a single company to enjoy such dominance on such a wide range of products.

Update

Bruce Perens has written a good insightful analysis of this issue that also neatly encapsulates his well thought out view points on software patents in general.

Posted by Colin Turner

The FSFE recently interviewed me about my involvement with free software, and mainly, the education sector. You can read the interview here if you are interested.

Posted by Colin Turner

For some time now, I have been advocating that universities should create more free and open source solutions to fill the gaps needed within the education sector itself. Every year, thousands of computer science and engineering students in the UK alone rack their brains looking for undergraduate final year projects. Most of these will be isolated standalone demonstrations of skill that will never be seen again or used for any other purpose.

Too rarely do we encourage students to contribute to an existing major software project instead. Most often that will most practically be a FOSS project. There are several reasons why we do not, and probably the most significant is that there is still a large number of staff members in education that haven't the faintest clue about the FOSS world, and so many students are left unenlightened too. Another problem that is often cited is the difficulty in unentangling the contribution made by the student as an individual. However, with most source control systems it is trivial to extract this information and this could easily be made a requirement of the project.

So I was very interested when a friend referred me to a letter in the times expressing very similar opinions about major public sector projects. I agree completely, there is a major untapped resource out there in staff and students, which would gain real life experience in enterprise projects. As the letter's author suggests, the culture of pride in software (especially in free software) would help improve quality and education the public in the value of free software solutions.

Posted by Colin Turner

Yesterday I was given an advance preview of a document for comment. It finished with a paragraph warning the audience that since it was discussing an open source solution it would require more in-house expertise to cope with it. I won't mention more about the context since while this exasperates me, it's not the fault of the writer, who is simply under the same illusion as many others on this issue.

Suppose I write a product X, and I make it free software. Suddenly it requires lots of in-house knowledge to deal with it. If on the other hand, I take the exact same product, make it proprietary and slap a one thousand pound price tag on it, now it's a breeze to install and maintain. <sigh>. I hope when it's put this way the fallacy is obvious. Yes, some free software is good quality, some is poor quality which is just like every other type of software, but in general, I would argue free software is more likely to be of high quality since it is intrinsically open for peer review and the idea of the gift culture. There is much more reason to care about overall code quality in a free product than in a proprietary one, since the customer can see it! Even if they don't know much about the code, perceptions of quality can quickly arise from a cursory examination.

The fact that a product is free or open source, simply means you can use in-house expertise to deal with it, not that you have to. Working in a university, it's really informative to look at some of the really expensive "gold-standard" software provided at that level. It costs a small fortune, usually with annual fees. Is it slick, beautiful, well documented and easy to set-up? Often these products are appallingly difficult to install and maintain, the companies involved taking hefty consultancy for that as well as for the product. It's much easier to install many of the free products on the market. Anecdotes I know, but my experience none-the-less.

Writing this has reminded me of comments that Dirk Eddelbuettel alluded to in his blog which again, make it clear that people have completely the wrong end of the stick about this sort of software, thinking it's not appropriate to real life or mission critical engineering. On the contrary, it is more appropriate. By the way, the software being discussed in that article (the statistics software R) is slick, more powerful than any proprietary alternative I know of, and has superb documentation and supporting books.

Posted by Colin Turner

Debian is my favourite distribution of GNU/Linux. It's a well respected and well known brand, although I was almost stunned while on business in London last Friday to meet someone who knew about Suse, and Red Hat but wasn't even aware of the existence of Debian. I like Debian's astonishing infrastructure, and its community led approach. It has been on the verge of releasing its next stable version, Lenny, for some time. I've been waiting for that freeze to end to have my debian packages of OPUS uploaded, since there is currently no real rush and I wanted to do more "upstream" work on them. At that point I wanted to start the long path to DD status (via maintainer).

Noodles, who is a DD, has recently commented on his frustration that Debian procrastinates at the end of every release cycle. This seems to be true, and internecine war always seems to erupt over issues that should have been solved long before (or alternatively after) the release. I'm also sympathetic to his suggestion that Debian's voting procedures could be improved.

Reading through Planet Debian today, I am more depressed than ever. Debian's secretary has resigned over voting issues and the vitriol unleashed by them. I'm not making any comment on that. There is already a constitutional pathway for an interim secretary, but now, the project can begin to rip new wounds in itself over the resignation, rumours of expulsion procedures and what to do next. What to do next is damn important, but so is the release.

I was in Plymouth in September with Lenny installer beta 2 CDs with Lenny images that wouldn't even install on the new hardware in a problem frustrating similar to one I found months before (Ubuntu, derived from Debian, embarrassingly did so without problems). I dare say that issue has been fixed (I can no longer test it), but it's a horrible reminder of how a prolonged freeze can lead to an OS that can be born obsolete in the sense that it can't install on newer hardware.

Posted by Colin Turner