Implementing configurable work-flow patterns in Python Django

In my previous article, I discussed some of changes I’ve made to my WAM software to handle assessment and work-flow. I thought I’d have a look at this from the technical side for those interested in doing something similar, this is obviously extensible to general workflow management, where you might want to tweak the workflow later without diving into code.

My challenge was to consider how not to hard code a work-flow, but to have something that would be configurable, in my case in a SQL layer because I’m using Python and Django.

I had an idea about the work-flow I wanted, and it looked a bit like this (carefully sketched on my tablet). These nodes are particular states, so this isn’t really a flow chart, as decisions aren’t shown. What is shown is what states can progress to the next ones. But I wanted to be able to change the pattern of nodes in the future, or rather, I wanted users to be able to do this without altering the code. I also wanted to work out who could do what, and who should know about what.

Workflow Example
Workflow Example

Understanding States

The first thing I did was to create a State model class, and I guess in my head I was thinking of Markov Models.

As you can see, I created variables that told me the name of the state, and an opportunity for a more detailed description. I then wanted to be able to specify who could do certain things, and be notified. So, rather than a long series of Booleans, I want for a text field – the work-flow won’t be edited very often, and when it is, it should be by someone who knows what they are doing. So it’s just a Comma Separated text field. For instance.

will indicate that the Module Coordinator and Moderator should be involved (this is an HE example, but the principle is quite extensible).

So the actors field will specify which kinds of people can invoke this state, and the notify field those who should get to hear about it.

I want to draw your attention to this bit:

What on earth does this do? It allows a Django model to have a Many to Many relationship with itself. In other words, for me to associate a number of states with this one. Please also note that presence of

This is most easily explained by comparison to the Facebook and Twitter friendship model. Both of these essentially link a User model in a many to many relationship with itself.

Facebook friends are symmetrical, once the link is established, it is two way. Twitter followers are not symmetrical.

I wanted to establish which successor states could be invoked from any given one. And this should not be symmetrical by default. You can see in my example graph above, I want it to be possible to move from state A to either B or C, but this is not entirely symmetric, it is possible to move from B to A, but it should not be possible to go from C to A. Without symmetric=False, each link will create an implied link back (all arrows in my state diagram would be bi-directional) which would be problematic. By establishing the relationship as asymmetric we can allow a reciprocal link (as is possible in Twitter, and our A and B example), but we don’t enforce it, so that we prevent back tracking in work-flow where it should not be allowed (as in our A and C example).

Invoking States

I then created another model to keep track of which states were invoked.

This model allows me to work out who (the signed_by field) invoked a particular AssessmentState, when, and with any particular notes.

I also added a field to record when a notification (notified) had been sent. On creation, I leave that field as null. One of the many glorious things about Django is that it’s infrastructure for custom management commands allows you to easily build command line tools for doing cron tasks while your web front end runs without interruption. I found this rather awkward, but not impossible, in PHP, but in Django the whole thing is very organic, and you get access to all your models. If you have pushed plenty of your logic into the Model layer and not the View layer, this can really help.

In my new custom commend I can easily work out which signoffs have not been notified yet:

I can then act upon those, send notifications, and if that’s successful, set the notified field to the time at which I sent them.

Further Reading

In this article I have concentrated on the Model layer, with a few other observations, and in particular the relationship from a State model to itself.

All of the Forms and Views are available within my GitHub repository for the project. They aren’t a work of art, but if you have any questions feel free to look there, or get in touch.

I hope that might be helpful to someone facing the same challenge, and do feel free to suggest how I could have solved the problem more elegantly.

Assessment handling and Assessment Workflow in WAM

Sometime ago I began writing a Workload Allocation Modeller aimed at Higher Education, and I’ve written some previous blog articles about this.

As is often the way, the scope of the project broadened and I found myself writing in support for handling assessments and the QA processes around them. At some point this necessitates a new name for WAM to something more general (answers on a post card please) but for now, development continues.

Last year I added features to allow Exams, Coursework, and their Moderation and QA documents to be uploaded to WAM. This was generally reasonably successful, but a bit clunky. We gave several External Examiners access to the system and they were able to look in at the modules for which they were an examiner and the feedback was pretty good.

What Worked

One of the things that worked best about last year’s experiment was that we put in information about the Programmes (Courses) each Module was on. It’s not at all unusual for many Programmes to have the same Module within them.

This can cause a headache for External Examination since an External Examiner is normally assigned to a Programme. In short, the same Module can end up being looked at by several Examiners. While this is OK, it can be wasteful of work, and creates potential problems when two Examiners have a different perspective on the Module.

So within WAM, I put in code an assumption of what we should be doing in paper based systems – that every Module should have a “Lead Programme”. The examiner for that Programme should be the one that has primacy, and furthermore, where they are presented other Modules on the Programme for which they aren’t the “lead” Examiner, they should know that this is for information, and they may not be required to delve into it in so much detail – unless they choose to.

This aspect worked well, and the External Examiners have a landing screen that shows which Modules they are examining, and which they are the lead Examiner.

What Didn’t Work

I had written code that was intended to look at what assessment artefacts had been uploaded since a last user’s login, and email them the relevant stuff.

This turned out to be problematic, partly because one had to unpick who should get what, but mostly because I’m using remote authentication with Django (the Python framework in which WAM is written), and it seems that the last login time isn’t always updated properly when you aren’t using Django’s built in authentication.

But the biggest problem was a lack of any workflow. This was a bit deliberate since I didn’t want to hardcode my School or Faculty’s workflow.

You should never design your software product for HE around your own University too tightly. Because your own University will be a different University in two years’ time.

So, I wanted to ponder this a bit. It made visibility of what was going on a little difficult. It looked a bit like this (not exactly, as this is a screenshot from a newer version of an older module):

Old view of Assessment Items
Old view of Assessment Items

with items shown from oldest at the bottom to newest at the top. You can kind of infer the workflow state by the top item, and indeed, I used that in the module list.

But staff uploaded files they wanted to delete (and that was previously disallowed for audit reasons) and the workflow wasn’t too clear and that made notifications more difficult.

What’s New

So, in a beta version of 2.0 of the software I have implemented a workflow model. I did this by:

  • defining a model that represented the potential states a Module could be in, each state defines who can trigger it, and what can happen next, and who should be notified;
  • defining a model that shows a “sign off” event.

Once it became possible to issue a “sign off” of where we were in the workflow, a lot of things became easier. This screenshot shows how it looks now.

Example of new assessment workflow
Example of new assessment workflow

Ok, it’s a bit of a dumb example, since I’m the only user triggering states here (and I can only do that in some cases since I’m a Superuser, otherwise some states can only be triggered by the correct stakeholder – the moderator of examiner).

However, you can see that now we can still have all the assessment resources, but with sign offs at various stages. The sign off could (and likely would) have much more detailed notes in a real implementation.

This in turn has made notification emails much easier to create. Here is the email triggered by the final sign off above.

The detailed notes aren’t shown in the email, in case other eyes are on it and there are sensitive comments.

All of this code is available at GitHub. It’s working now, but I’m probably do a few more bits before an official 2.0 release.

I will be demoing the system at the Royal Academy of Engineering in London next Monday, although that will focus entirely on WAM’s workload features.