exim4 upgrades and configuration fragility

Last night I decided I’d catch up on sysadmin tasks. Some of that was trying to tighten up my spam filtering again. I had got in place a per-user Bayesian filter on spamassassin, which essentially should allow it to learn a much more individual pattern of what each user considers spam. I also had configuration for having my mail server (running exim) reject mail within the SMTP session for very egregious examples. That configuration hasn’t been working for a while – I had to revert lots of custom changes to my config for a significant exim4 upgrade a while back, and I haven’t had the time and patience to try and reinstate it all. So I thought I’d look at that.

I ran a standard apt update and upgrade before I started. I noted exim4 and its various binary packages were marked for upgrade which isn’t unusual and proceeded (the upgrade was between versions 4.95-RC2-1 and 4.95-1). Debian warned me of a change to exim4.conf.template and I examined the diff briefly, didn’t see anything extraordinary and retained my config. In any case, I am using a distributed config in conf.d so expected to see a more targetted diff on one of those. I didn’t. exim restarted without complaint.

I started using tail to follow my exim logs, and could immediately see that every single inbound message was being temporarily rejected, albeit, there was no information as to why. I spent probably the guts of an hour checking the changes between my configuration files and .dpkg-dist versions (the ones shipped as the new changes) and couldn’t see the problem. I tried copying over the exim4.conf.template and updating the configuration with update-exim4.conf and still I had the same problem. I checked the changelog and didn’t see anything profound that should really be a worry.

In the end I had to downgrade all the exim4 packages, and my mail started to be delivered again. Of course, this only buys me a little time to either find the problem, or hope it’s something upstream in the Debian package. I maybe should report this as a bug, but I feel I don’t yet have enough information.

However, it really got me thinking just how fragile exim4 configuration seems to be. I need to add a few tweaks to the shipped configuration if I want more effective handling of some things, such as my multiple domains, or being able to handle email addresses with extra bits like <name>-<website>@<domain> so I can filter email from various sites (and determine who sold my data). But in the main the challenges are in more effective spam filtering. All of those tweaks can be really easily disrupted by an update. I’ve been thinking of trying to generate a patch set against the default configuration in case that makes it easier to accept all new config files and applying the patch set, but that will have its own problems. greylist – a package that temporarily rejects initial attempts to deliver mail from unknown servers uses this patch approach.

In the meantime, if you upgrade exim4 and suddenly have all inbound messages being temporarily rejected you are not alone, but I can’t yet explain why. For me a temporary downgrade was the only solution.

aptitude install exim4=4.94.2-7 exim4-base=4.94.2-7 exim4-config=4.94.2-7 exim4-daemon-heavy=4.94.2-7

Installing Android Nougat on a Stock Galaxy Tab 10.1

My daughter uses an Android Samsung tablet (coded GT-P7510) which ended official support on Android 4.0.4. Unfortunately I didn’t pay any attention to this issue until the apps she most wanted to use, namely Netflix and YouTube stopped working on it as the Android version was too low.

I found a ROM to upgrade to Android 7.1 (Nougat) with some cost – for instance, the camera doesn’t work, but Aimee doesn’t care about that. So I decided to try upgrading it since the tablet was otherwise now utterly useless.

To make things more difficult, most of the information on upgrading this tablet on the Internet is outdated or wrong, or pre-supposes that the device was long since updated. I also don’t run Windows, and ran into some problems with the Heimdall alternative.

So this quick article is the result of a couple of evenings running into dead ends. It might help someone else. Certainly if I ever need to do it again it’ll help me.

But as usual, if you break something, you own all the parts. These instructions are completely specific to this particular tablet, and the wifi only version at that. Make sure your device is fully charged before you start.

A new recovery image had to be installed first, and some steps had to be undertaken just to get that far.


First of all there’s supposed to Windows software called Odin that is used to update the ROM, especially from a stock start. I can’t run that without emulation since I don’t run Windows, and in any case, I suspect it might behave badly in a virtual machine, and probably wouldn’t run correctly on modern Windows.

So I installed a Free and Open Source alternative known as Heimdall. For me, this was simple as it was Debian packaged. I couldn’t get the frontend to be useful, and I couldn’t get the Java version of the frontend to work online or offline. So I defaulted to the command line.

So, as root on Debian GNU/Linux:

aptitude install adb heimdall

This is also ensuring all the command line tools for android debugging are installed (I already had these).

Receiving TWRP

The device needs to be made ready for Odin / Heimdall upload. Turn the device off, and then hold Power and Volume Down till it appears with two icon choices. You want the one on the right. Use Volume Up to select, and use Volume Up again to bypass the dire warnings.

I had no success in using the Heimdall frontend, your mileage may vary. I got the correct archive for my purposes from here.

I downloaded the archive, and used tar xvf to extract the contents. You will find two .img files, recovery.img and hidden.img. You’ll need both.

Note that the partition target on the device for recovery is not called recovery but is called SOS at least on my device.

heimdall flash --SOS recovery.img --no-reboot

Because of the no-reboot option note that the tablet will continue to warn you not to restart it. You’ll need to watch the command line progress carefully to ensure that it is on. Now reboot the machine once again into the Odin / Heimdall mode again. I.e. power it off, and turn it on with Power and Volume Down.

Now flash

heimdall flash --HID hidden.img

For me this successfully got TWRP 3.0.3 loaded. It was a major odyssey of conflicting information to get this far. When you reboot make sure you hold down volume down to get to the recover menu, (and now choose the left hand option). If you don’t do this, the stock ROM overwrites the new one and you’ll need to start again.

Using TWRP

From here, things were relatively plain sailing. I got the ROM from here.
(EDIT: 2021-10-21: A reader brought it to my attention that the ROM images were no longer there, I think I found new versions here.) Incidentally, I’d tried other recovery ROMs I got onto the device before when I couldn’t get TWRP onto it, they did not allow the following steps to work.

I then used TWRP’s wipe option to wipe Cache, Data, and Dalvik Cache.

I used the Advanced button and put the device into sideload mode.

I then, from the Linux command prompt executed

adb sideload aosp-7.1-p4wifi-20170320.zip

I then did not reboot but went back in TWRP and selected sideload again, this time I was careful to uncheck the wipe data and cache items since I’m loading other items on top of the basic ROM image.

adb sideload P7500-open_gapps-arm-7.1-pico-20170119.zip

and I repeated the same for the last package

adb sideload superuser.zip

finally I selected to reboot the tablet. It took a pretty long time to boot. Don’t forget it’s a relatively underpowered device.

The device is up and running and now runs the apps my daughter wants again.

Boot problems with systemd? Check /etc/fstab

My (actually this) Debian server failed to boot after a power failure last week, it turns out the graphics card failed too, probably because of the cold and the thermal shock, but replacing the card did not allow the computer to boot.

With systemd, if something happens in the boot process, despite some obviously specific failure triggering the problem, it tends to fob you off with a message to run

journalctl -xb

This feels like the machine equivalent of “Hey, I just saw some needle go past that broke the machine. Let me hand you a haystack so you can find it.”

The command that is a whole lot more useful, is:

systemctl status systemd-modules-load.service

which allows much more rapid diagnosis of many problems. In my case the whole thing turns out to be an old line in /etc/fstab – an obselete line to mount /proc/bus/usb. It shouldn’t have been there anymore, but I am slightly amazed and irritated that the whole boot process was abandoned because of one line in /etc/fstab.

But hopefully someone else in this situation will find the suggestion to check that file first useful.

Migration from Savane to Redmine

I am admin for a server at work foss.ulster.ac.uk to host our open source development work. It used to run on GNU Savane, but despite several efforts, that project is clearly dead in the ditch.

So having to change the underlying system, I decided to move to Redmine (you can see some previous discussion here). I’m recording aspects of the migration here mostly for my own sake.

This install was on Debian Squeeze. I first of all installed the relevant package

aptitude install redmine redmine-pgsql

and followed the prompts for the configuration. The documentation for the Debian install is a little unhelpful about how to actually configure the web server, and while I have good experience with Apache, I have very little with Ruby on Rails.

I installed the Apache Passenger module.

aptitude install libapache2-mod-passenger

and copied the example config

cd /usr/share/doc/redmine/examples/
cp apache2-passenger-alias.conf /etc/apache2/sites-available/redmine

I then edited the newly created redmine file to look like this:

# These modules must be enabled : passenger
# Configuration for http://foss.ulster.ac.uk/redmine

ServerName foss.ulster.ac.uk
# this is the passenger config
RailsEnv production
SetEnv X_DEBIAN_SITEID "default"

# This is the example from the Debian package
# apache2 serves public files
#DocumentRoot /usr/share/redmine/public
#Alias "/redmine/plugin_assets/" /var/cache/redmine/default/plugin_assets/
#Alias "/redmine" /usr/share/redmine/public

# And my attempt (CT 20120816)
# apache2 serves public files
DocumentRoot /usr/share/redmine/public
Alias "/plugin_assets/" /var/cache/redmine/default/plugin_assets/
Alias "/" /usr/share/redmine/public

Directory "/usr/share/redmine/public"
Order allow,deny
Allow from all

In my case I wanted Redmine on the web root, so you can see the changes I made.

I then disabled the default config and enabled this:

a2ensite redmine
a2dissite default
a2dissite default-ssl

and restarted Apache

/etc/init.d/apache2 restart

Now you can login, with the default username and password (admin and admin) and change them and start some configuration.

Cinnamon; adding needed spice to Gnome 3

Ok, so I used Gnome Shell before it was officially released. I stopped using it because I thought it was intriguing, but awkward to use in its beta stages. Then Gnome 3 was released and gnome-shell was no longer an interesting option, it was the compulsory way to use the operating system; and I wrote at the time about some of the problems. Many were solved, but the underlying troubles in the design of Gnome Shell were a problem for me. And I’ve really tried to like it, I really have, and I don’t. I hate it. It makes almost every workflow I have tedious and exasperating. It has damaged my productivity. It looks pretty, it looks stylish, but it’s frankly slow and painful to get things done. I tried lots of other window managers and was frustrated at having the leave the good things of Gnome behind.

Until someone told me about Cinnamon. I looked at the website and thought it might be just the ticket. Unfortunately it’s not officially packaged for Debian (yet), and I currently lack the time to start building my own packages. Fortunately someone else has done it. I installed the packages on my laptop and breathed a sigh of relief (once I diagnosed a problem with the settings dialog). I installed it on my other boxes (that have GUIs), and now, well, the best thing is I am enjoying all the great things about Gnome 3 now. I’m even enjoying the great things about Gnome Shell, since Cinnamon is actually a fork, but all the stupid bits are gone.

There is a nice, elegant panel, so much cleaner than the Gnome Fallback mode. It looks like it belongs in Gnome 3, it does. Notifications are more subtle, coming up in out of the way bits of the screen. The screen effects are subtle but pleasant. In short it is what Gnome 3 should have been, or at least optionally. The “new” interface of Gnome Shell may suit many users, many devices, many workflows, but it most certainly does not suit all.

I have my nice comfortable desktop and workflow back after many months; kudos to the Cinnamon team, and kudos for them really showcasing all the excellence of Gnome 3 rather better.

If you want to try it out on Debian, follow the instructions here, and note the possible problem with the settings dialog.

Gnome 3, or Gnome Shell issues

I use the Debian operating system on several computers. My “main” computer (Imladris) runs Debian unstable (Sid) while the others mostly run on testing. I’ve been anticipating the Gnome 3 upgrade for some time, mainly because of the switch to Gnome Shell which is a completely new way of using the desktop. I had played with Gnome Shell a while ago, and was kind of impressed and worried by it in equal measure, I decided it wasn’t ready for prime time so stopped using it. Naturally I assumed it would be much more impressive upon release; especially since Debian is not (by far) the first GNU/Linux distribution to include Shell.

A while ago a big upgrade came through on imladris, and it was clear it was the Gnome 3 upgrade. I share this computer with three other users, two of which are children for whom I have implemented password less login (locally only). I can only say I think Gnome have significantly mishandled the upgrade. Here are some reasons why.

Login is seriously slow

The display manager can take up to a whole minute to display the list of users (and often doesn’t display the icons). There are some bug reports about a possible race condition that causes this, but seriously on a reasonable spec computer this is unacceptably slow. The same problems occurs when switching user.

I couldn’t login

My, admittedly old user account simply wouldn’t launch a working desktop. I had to (at a command prompt) delete configuration directories to get my account working again.

Absolutely zero support for the user in transition

So the average user does the upgrade and suddenly their entire desktop has changed. But when they first login there will be some guidance about where everything is gone… right? No. Having already used Shell, I knew, but I had to try and show everyone else how to use the machine again. It’s not that spectacularly intuitive.

Actually, a lot of functions have just gone

There’s a huge removal of existing functionality. All your carefully tweaked panels: gone. All your applets: gone. And bizarrely often with no working alternative.

Not friendly for children

It was possible to set up a Gnome 2 account to make it easy for kids. Low res graphics, and big panels with big select icons. The new paradigm completely ignores all that in favour of a sleek minimalist environment which is probably not that easy for young children to understand.

Dictatorial design choices

It’s been decided that we don’t need minimise buttons or maximise buttons. It’s been decided not to honour old desktop backgrounds. It’s been decided not to honour existing resolution settings. It’s been decided not to show anything on the Desktop (much to the confusion of many users). It’s been decided we can’t right click on the desktop.

Some of this kind of nonsense is exactly why I don’t like some other operating systems who believe they know what’s best for you with Messianic Zeal (I’m looking at you Apple).

All in all I find this transition very disappointing. There are lots of basic things no-one seems to have thought of, and years of desktop customisation have been swept away with an extraordinary arrogance. Don’t get me wrong, I support the idea of trying a new Desktop paradigm: but, for instance, if people used to have applets on their desktop for the weather, or for system monitoring, it’s because they needed it. Rolling out a new desktop that simply ignores these things in favour of how some people thing everyone should use their desktop is exasperating.

I’m seriously hoping that Gnome Shell improves significantly and fast. I won’t hold my breath.

Multi User Sound in GNU/Linux

For some years now, basically since Aimee became old enough to use a computer, I have had a need for decent multi-user sound. Specifically I would often have intricate work open in multiple work spaces on my desktop, and Aimee would want to do some artwork.

I guess Aimee was about two when she started using my computer a fair bit, and I immediately had a separate account for her with lower resolution graphics, easier menus, simple shortcuts and the like. I also had a graphics tablet for her, which she mastered very quickly. Another important reason to have a separate account was that if stuff got really badly messed up I could just nuke the account and start again, and she couldn’t really hurt my configuration.

As you would imagine, lots of the excellent free software for children, like GCompris, and Tuxpaint is very rich in its sound effects, and sometimes the sound is not just desirable, but essential for the activities. So a very annoying persistent problem has been that, once in a while, something in the sound stack in my login would stop Aimee’s sound from working.

Now I share my main PC with two other people, Tamsin and Aimee, and soon Matilda too, so this recurrent problem is more of an issue, it’s also very hard to nail down. Despite protests to the contrary the default ALSA setup still has this problem; ConsoleKit on its own doesn’t seem to get it quite right. I tried PulseAudio for a while, and generally it was an improvement, but the problem did still occur sometimes. I even made sure all the users were members of the right groups including pulse-rt.

Then I found this entry in the PulseAudio FAQ.

Sound doesn’t work when switching users

PulseAudio works with a single user, but when an additional user logs in (fast user switching), sound/audio does not work for the additional user.

Check that no users are part of the “audio” group.

In simple setups (e.g. singe user, without PulseAudio), users must be a member of the “audio” group to access the sound devices (/dev/snd/* (which have group “audio” write permissions)). Switching users will not automatically stop programs using those sound devices though, so those sound devices will not be accessible to a new (faster user switched) user’s programs.

By removing all users from the “audio” group (the PulseAudio server still runs in the “audio” group), PulseAudio is able manage access to sound devices (/dev/snd/*) amongst multiple users with the help of ConsoleKit.

It would never have occurred to me to remove the users from the audio group, but doing so seems to have solved the problem. I almost don’t want to say that, because every “solution” up to now has been partial, but so far no problems, so maybe this will be the fix.

Boot time sound problems with QuickCam Pro 9000 webcam

I’ve had a Logitech QuickCam Pro 9000 for some time now, and it works well, but I did have one bizarre problem with it when I used it with my main desktop machine (running Debian (Sid)). Namely, that if I had the camera plugged in (usb) at boot time the sound on the computer did not work, if you plugged it in after boot, everything was fine.

This was pretty irritating because if I forgot I would often have dozens of windows open and ready for work before I realised.

Normally this happens because the cards are loaded in the wrong sequence by udev. But, if you listed the sound cards with

cat /proc/asound/cards

the main card wasn’t just in the wrong order, it simply wasn’t there. I tried comparing modules loaded with and without the camera (at boot) and manually loading the differences, but this did not help. I tried forcing the index to be zero on the correct card, but this also did not help.

In the end, I made the following edit to /etc/modprobe.d/alsa-base.conf

# Keep USB (webcam from being loaded as first card)
options snd_usb_audio index=-2

and this did the trick. So if like me you were searching for the answer to this, I hope it helps.

Virtual folders with Dovecot and Debian

I use client side virtual folders a bit for my mail. Specifically, I tag messages with IMAP flags like todo and important, and then in Icedove / Thunderbird, I set up a special folder as a saved search which shows message that are either unseen, or marked todo in my inbox. It works rather well, and I use the same set-up on my laptop, and work and home desktop machines.

But it’s not very useful on my phone, which doesn’t allow such sophisticated client side behaviour. My phone mail applications shows the most recent 25 messages in a folder, but there are times when it would be really useful to look up messages that are labeled as important but rather old. It would be time consuming to look through the older messages, and difficult to find the one I want anyway.

As a result, I’ve been looking at the possibility of using virtual server side folders using dovecot on my Debian mail server. I was put off by the documentation which left a lot of questions unanswered.

Here’s how I did it on Debian. First of all edit the config file /etc/dovecot/dovecot.conf, back up this file first, so you can restore working behaviour if something goes wrong.

# You have to add the default namespace
# which is normally NOT added explicitly before
namespace private {
  prefix =
  separator = /
  # the next line is very specific to where you keep your mail
  location = mbox:~/Mail/:INBOX=/var/mail/%u
  list = yes
  inbox = yes
  subscriptions = yes
  hidden = no

# Then add the virtual namespace
namespace private {
    prefix = virtual/
    separator = /
    # pick where the virtual folders will be
    location = virtual:~/Mail/virtual
    list = yes
    inbox = no
    subscriptions = yes
    hidden = no

You must also add the virtual folder plugin.

## IMAP specific settings

protocol imap {

  # ... you need to enable the plugin
  mail_plugins = virtual

Now restart dovecot and check your normal folders are working.

/etc/init.d/dovecot restart

Note that I found dovecot will generally not serve physical folders correctly if the virtual mail folder (even if empty) does not exist. I consider this a bug, but one that needs to be worked around, at least for me.

If that’s all done and working you can begin to create virtual folders. I created two directories within my ~/Mail/virtual folders; which were inbox-todo and inbox-important respectively. Inside each I put the following files.

# ~/Mail/virtual/inbox-todo/dovecot-virtual

which shows all unseen and mail labelled todo in my inbox and


which shows only important mail in my inbox.

It seems to be working, my normal folders appear to be working perfectly correctly (but I’ll know better in a couple more hours/days); my phone has successfully subscribed to the two virtual folders, though the folder list shows a number of files which I’m certain it should not, again, this looks like a dovecot bug to be honest.