Implementing configurable work-flow patterns in Python Django

In my previous article, I discussed some of changes I've made to my WAM software to handle assessment and work-flow. I thought I'd have a look at this from the technical side for those interested in doing something similar, this is obviously extensible to general workflow management, where you might want to tweak the workflow later without diving into code.

My challenge was to consider how not to hard code a work-flow, but to have something that would be configurable, in my case in a SQL layer because I'm using Python and Django.

I had an idea about the work-flow I wanted, and it looked a bit like this (carefully sketched on my tablet). These nodes are particular states, so this isn't really a flow chart, as decisions aren't shown. What is shown is what states can progress to the next ones. But I wanted to be able to change the pattern of nodes in the future, or rather, I wanted users to be able to do this without altering the code. I also wanted to work out who could do what, and who should know about what.

Understanding States

The first thing I did was to create a State model class, and I guess in my head I was thinking of Markov Models.

class AssessmentState(models.Model):
    """Allows for configurable Assessment Resource Workflow

    name            The name of the state
    description     More details on the state
    actors          The user types who can create this state, CSV field
    notify          The user types to notify that the state has been created
    initial_state   Can this be an initial state?
    next_states     Permissible states to move to from this one
    priority        To allow the states to be sorted for presentation to the user

    actors and notify should be comma separated lists as in USER_TYPES below.
    """

    ANYONE = 'anyone'
    COORDINATOR = 'coordinator'
    MODERATOR = 'moderator'
    EXTERNAL = 'external'
    TEAM_MEMBER = 'team_member'
    ASSESSMENT_STAFF = 'assessment_staff'

    USER_TYPES = (
        (ANYONE, 'Any logged in user'),
        (COORDINATOR, 'Module coordinator'),
        (MODERATOR, 'Module moderator'),
        (TEAM_MEMBER, 'Module teaching team member'),
        (EXTERNAL, 'External Examiner'),
        (ASSESSMENT_STAFF, 'Members of AssessmentStaff'),
        ('exams_office', 'Exams Office')
    )

    name = models.CharField(max_length=200)
    description = models.TextField()
    actors = models.TextField()
    notify = models.TextField()
    initial_state = models.BooleanField(default = False)
    next_states = models.ManyToManyField('self', blank=True, symmetrical=False, related_name='children')
    priority = models.IntegerField()

As you can see, I created variables that told me the name of the state, and an opportunity for a more detailed description. I then wanted to be able to specify who could do certain things, and be notified. So, rather than a long series of Booleans, I want for a text field - the work-flow won't be edited very often, and when it is, it should be by someone who knows what they are doing. So it's just a Comma Separated text field. For instance.

coordinator,moderator

will indicate that the Module Coordinator and Moderator should be involved (this is an HE example, but the principle is quite extensible).

So the actors field will specify which kinds of people can invoke this state, and the notify field those who should get to hear about it.

I want to draw your attention to this bit:

next_states = models.ManyToManyField('self', blank=True, symmetrical=False, related_name='children')

What on earth does this do? It allows a Django model to have a Many to Many relationship with itself. In other words, for me to associate a number of states with this one. Please also note that presence of

symmetrical=False

This is most easily explained by comparison to the Facebook and Twitter friendship model. Both of these essentially link a User model in a many to many relationship with itself.

Facebook friends are symmetrical, once the link is established, it is two way. Twitter followers are not symmetrical.

I wanted to establish which successor states could be invoked from any given one. And this should not be symmetrical by default. You can see in my example graph above, I want it to be possible to move from state A to either B or C, but this is not entirely symmetric, it is possible to move from B to A, but it should not be possible to go from C to A. Without symmetric=False, each link will create an implied link back (all arrows in my state diagram would be bi-directional) which would be problematic. By establishing the relationship as asymmetric we can allow a reciprocal link (as is possible in Twitter, and our A and B example), but we don't enforce it, so that we prevent back tracking in work-flow where it should not be allowed (as in our A and C example).

Invoking States

I then created another model to keep track of which states were invoked.

class AssessmentStateSignOff(models.Model):
    """Marks a particular sign off of an AssessmentState for a module

    module              The module being signed off
    assessment_state    The AssessmentState being used
    signed_by           The user signing off (Could be Staff or Examiner)
    created             The datestamp for signoff
    notified            The datestamp for notificiations sent
    notes               Any comments
    """

    module = models.ForeignKey(Module, on_delete=models.CASCADE)
    assessment_state = models.ForeignKey(AssessmentState, on_delete=models.CASCADE)
    signed_by = models.ForeignKey(User, on_delete=models.CASCADE)
    created = models.DateTimeField(auto_now_add=True)
    notified = models.DateTimeField(null=True, blank=True)
    notes = models.TextField()

    def __str__(self):
        return str (self.assessment_state)

This model allows me to work out who (the signed_by field) invoked a particular AssessmentState, when, and with any particular notes.

I also added a field to record when a notification (notified) had been sent. On creation, I leave that field as null. One of the many glorious things about Django is that it's infrastructure for custom management commands allows you to easily build command line tools for doing cron tasks while your web front end runs without interruption. I found this rather awkward, but not impossible, in PHP, but in Django the whole thing is very organic, and you get access to all your models. If you have pushed plenty of your logic into the Model layer and not the View layer, this can really help.

In my new custom commend I can easily work out which signoffs have not been notified yet:

# Get all sign offs with no notification time
signoffs = AssessmentStateSignOff.objects.all().filter(notified=None).order_by("created")

I can then act upon those, send notifications, and if that's successful, set the notified field to the time at which I sent them.

Further Reading

In this article I have concentrated on the Model layer, with a few other observations, and in particular the relationship from a State model to itself.

All of the Forms and Views are available within my GitHub repository for the project. They aren't a work of art, but if you have any questions feel free to look there, or get in touch.

I hope that might be helpful to someone facing the same challenge, and do feel free to suggest how I could have solved the problem more elegantly.