Tag Archives: workload

Budgeting for Assessment

Workloads for Academics in Higher Education are often very complex, with teaching loads, research tasks and administration all juggling for our attention with lots of task switching adding to the complexity. For many academics, teaching loads are a significant part of their work, but explicitly looking at the time spent on assessment could bring better results for staff and students alike.

Contact Hours to Admin Hours

There are usually ad-hoc assumptions about the amount of administration time a module takes above and beyond the contact hours spent in front of a class. For our purposes contact hours could just as easily be delivering synchronous and asynchronous hours on-line as time spent in more traditional on-campus delivery.

A common such assumption is that it takes two hours of this administration time for each hour of contact, sometimes more. That administration time can be subdivided into various tasks such as preparation of teaching materials, delivery of assessment, and other correspondence with students etc..

Preparation time on teaching materials can obviously be markedly higher for the first presentation of a module, or after significant changes. Many academics are already undertaking substantial additional preparation time to re-factor materials for on-line delivery at the moment.

Assessment Hours

I want to focus on the time spent on assessment because I feel this is a serious time sink for most academics. This is partly because when it comes to reform of learning and teaching in a curriculum, assessment is often the last consideration because we are nervous about the serious consequences of getting things wrong. It is also partly because we can sometimes draw the conclusion that time spent on assessment equates directly to quality.

How often have you attempted to place a budget on your assessment time before delivering a module? I mean the time taken to design an assessment, deliver it to students, assess the submissions and deliver feedback. My guess is that very few of us have done this.

The outcomes of this can be serious. We often design assessments focusing on the first half of these tasks that take a tremendous and unquantified amount of labour to fully deliver, often much more than we really expected. This can result in a very stressed academic or team of academics, or the delivery of feedback is too late to be effectively useful to the students, or the quality and depth of the feedback suffers. Any combinations of these outcomes is also potentially likely.

Budget influences Design, poor Design blows the Budget

Agreeing a budget with a line manager, or even with yourself, can be informative. If you think the budget is too low you are faced with the choice of making the argument that additional resource is really required, or that you need to re-design the assessment to fit within your budget.

Of course, there are often times when additional resource really is required, but my argument here is that this should be a conscious choice, planned for and if possible agreed with your line manager who may be able to bring practical assistance, or at least balance out the rest of your workload.

Even if you agree a high budget, a good plan, and a good design minimises the risk of blowing that budget.

Design Choices

So what choices can we make to reduce the time burden? Some choices not only have no adverse effect on quality, but can actually deepen the quality of feedback or reflection opportunities for students. Here’s a very non-exhaustive list of thoughts in this direction.

  • Do you really need all those questions to confirm your learning outcomes? Do you have some questions that are just repeating the assessment of the same aspects? Trim them if so. Extra material can be used for tutorials instead.
  • Have you considered the use of a good rubric if you aren’t already using one? This can improve transparency of outcomes to students both before and after assessments and provide some generic feedback, leaving you with more time to give more focused feedback and can hugely improve the speed of marking.
  • Can you partially automate some of the assessment? If assessments are being delivered on-line many Virtual Learning Environments allow you to set assessments with questions with set or calculated answers so some of the marking and feedback can be automated. You can combine these with deeper more free response questions.
  • Can peer assessment accomplish some of your goals? If you are nervous about using peer assessment (and it does need care) what about using it in a formative way as part of your assessment diet. This can also greatly deepen students’ understanding of how their work is marked and assessed.
  • Can self assessment accomplish some of your goals? This can encourage highly reflective learning and allow you to guide the feedback based on the students’ initial assumptions.

What are your ideas to reduce your assessment budget will keeping, or even deepening the quality?

Even if you don’t undertake this formally with your line manager, try setting yourself as assessment budget, and consider how to work within it so that you can deliver authentic assessments, quality feedback in a way that leaves you time, focus and attention for the other parts of your job.

Workload Allocation Monitoring (WAM) Prototype

I decided to start writing a workload allocation monitoring system for Higher Education. I found one written as part of a JISC project at Cambridge, but despite my experience with PHP I found it difficult to set-up, a bit crude (sorry) and hard to maintain. It was clearly very flexible, and I wanted something flexible, simple and clean.

So I decided I’d try writing something quickly using the Python django framework. This is my first web-app written in Python and so I dare say I would do some things differently with more experience, but I have now reached the point where I have a workable prototype that I can start to use myself. I’ve got to say, I found django to be pretty neat.

At its heart is a list of the loads against Academic Staff in a department or school. The idea is to try and increase transparency. There are problems with this approach: some known irregularities of loading can be for confidential reasons; small numbers of staff with key skills can cause issues as well, but it is intended to provide a basis.

Overall loads for staff.
Overall loads for staff.

 

 

 

 

While classically the word semester implies that there are two of them, most Universities operate a three semester system with the third covering the Summer. Unevenness in loading over the Summer is another cause of potential trouble, so the system tries to show loading as spread across semesters. A scaled column accounts for staff who do not have a 100% FTE contribution but their hours are up-scaled for comparison.

Naturally staff will want to see some granularity of these loads and they are broken into individual activities that are allocated to given members of staff.

Breakdown of activities for a staff member.
Breakdown of activities for a staff member.

An individual activity can be specified as occupying a number of hours, or alternatively a percentage of a staff member’s time. It can occupy one or more semesters (in which case it is spread evenly across them). Types can be allocated for activities to help track contributions of different types. It might be that an activity is related to a module or study, or not.

Activities are long term parts of work allocated hours or a percentage of time.
Activities are long term parts of work allocated hours or a percentage of time.

Speaking of modules basic information is stored for these, and another issue I think will help, tracking the submission of exams and coursework through various QA processes.

At a glance the most recent information about the exam and coursework status can be seen.
At a glance the most recent information about the exam and coursework status can be seen.

While activities are considered to be events with long engagements, another issue for staff are tasks that are allocated to them, usually of comparatively short duration. It can be hard to staff to remember all of these tasks, and hard for manager to follow up their completion, especially without annoying staff who have completed them already.

Tasks can be allocated against individual members of staff or groups or both.
Tasks can be allocated against individual members of staff or groups or both.

The web-app will allow tasks to be defined against one person, many people, categories of people and so on.

A list of tasks and their deadlines.
A list of tasks and their deadlines.

 

 

 

 

 

It is possible to easily see which tasks are still open and whether their deadline has come and gone.

The staff required to complete a task are shown, and those that have indicated completion. The system politely nags those still outstanding.
The staff required to complete a task are shown, and those that have indicated completion. The system politely nags those still outstanding.

A look at a given task will show who has completed it and who still needs to.

A given staff member can sign off their own task.
A given staff member can sign off their own task.

 

It is often the case that admin and clerical staff check off colleagues who have responded to a given call, so the system allows for staff with given permissions to indicate someone has having completed the task. Alternatively the member of staff can do this for themselves.

So while it is still a bit rough and ready I’ve reached the point where the system is stable enough for use. Of course the challenge comes when we consider the assumptions to come up with the hours and percentage loading in the first place. So I hope to pick the brains of some colleagues about this and start testing the system.

I’ve yet to make a formal release, but the code is Affero GPL (you can use the code free of restrictions (and charge) but cannot deprive others of the same freedom on derivative works) so feel free to have a look at it.

My roadmap for an initial release can be found on foss.ulster.ac.uk, where I will eventually host the code as well, but at the moment it can be found at GitHub. My previous post detailed how to get the app to work with a central authentication system your University likely has, or something similar.

Yeah… design and CSS is not my strongest skill, more work to be done on that.

Share and enjoy.