Tag Archives: OPUS

OPUS and Assessment 3 – Regime Change

This is the third and final article in a short series on how OPUS, a system for managing placement on-line, handles assessment. You probably want to read the first and second article before getting into this.

Regime Change

It’s not just in geo-political diplomacy that regime change is a risky proposition. In general you should not change a regime once it has been established and students entered on to it. If you do, there is a risk that marks and feedback will become unavailable for existing assessments, or that marks are calculated incorrectly and so on. Obviously it is also non-ideal practice for the transparency of assessment.

Instead you should create a new regime in advance of a new academic year, change the assessment settings in the relevant programmes of study to indicate that regime will come into force in the new year, and brief all parties appropriately. All of this is done by the techniques covered in the first two articles. If you have done all that, well done, and you can stop reading now.

This article is about what to do if students are on a given assessment regime in OPUS, and somebody decides to change that regime midstream, when marks are already recorded for early items.

TL;DR DON’T DO THIS, TURN BACK NOW!

This shouldn’t ever happen, as noted really you need to ensure your regime changes are correctly configured and enabled before any students start collecting marks.

And yet, it does happen, or at least it has happened to me twice that I have been asked to make tweaks to a regime where student marks already exist. Indeed it is happened to me this week, hence this article.

Even changing small details like titles will effect the displayed data for students from previous years. Tweaking weightings could cause similar or more serious problems.

So what happens if we create a new regime and move our students onto it midstream? Well, the existing marks and feedback are recorded against the old regime, so they will “disappear” unless and until the students are placed back on that regime.

If you want to do this, and copy over the marks from the old regime into the new regime, there is a potential way to do this. It is only been used a handful of times and should be considered dangerous. It also probably won’t work if your original marks use a regime where the same assessment appears more than once in the regime for any given student.

But, if you’re here and want to proceed, it will probably be possible using what was deliberately undocumented functionality.

You will need command line, root access (deliberately – this is not a bug), in order to do this. If you haven’t got root access you need to get someone who does so you can… Read all the instructions before starting.

0. BACK UP ALL YOUR DATA NOW

Before contemplating this insanity, ensure your OPUS database is backed up appropriately. I’d also extract a broadsheet of all existing collected assessment for good measure from the Information, Reports section of the Admin interface.

That said, this functionality deliberately copies data, it doesn’t delete it – but still.

0. NO REALLY, BACK UP ALL YOUR DATA NOW, I REALLY MEAN IT.

 

Ok, you’re still here.

First of all this approach only makes sense (obviously) if the marks you have already captured are valid. I.e. the assessment(s) you want to change are in the future for the students and haven’t been recorded. If not, then obviously OPUS can’t help you do anything meaningful with the marks you have already collected.

1. Make your New Assessment(s)

Maybe you plan to just change from one stock assessment to another, or perhaps you want to adjust a weighting on an existing assessment that hasn’t been undertaken by students in this year. In this case, you can skip this step.

But if needed, create and test any new assessments following the approach laid out in the second article in this series. Do make sure you spend some time testing the form.

2. Add and Configure a New Assessment Regime

Create your new assessment regime, as detailed in the first article, but don’t link it to any programmes yet.

Your new regime should be configured as you wish it to be. Remember, for there to be any point in this exercise, the early assessments already undertaken by the students need to be the same (though not necessarily in the same order) – otherwise OPUS can’t help and you need to sort out all the marks in transition entirely manually.

3. Note the IDs of the Old and New Regimes

Things start to get clunky at this point. Remember, we are heading off road. You will need the database ID of both the old regime and the new one.

You can obtain these by, for instance, going to Assessment Groups in the Configuration menu and editing the regimes in turn. The URL will show something like this:

URL

At the very end, you will “id=2” so 2 is the id we want. Write these down for both regimes, noting carefully the old and new one. It’s almost certain the new id will be larger than the old one.

4. Choose your timing well

You want to complete the steps from here on in, smoothly, in a relatively short time period. It is advisable that you switch OPUS into maintenance mode in a scheduled way with prior warning. This can be done from the Superuser, Services menu in the admin interface, if you are a superuser level admin – if you aren’t you shouldn’t be doing this without the help of such a user. You can also enter maintenance mode with the command line tool.

5. Use the Command Line Tool with root access

OPUS ships with a command line utility. With luck, typing “opus” from a root command prompt will reveal it. It’s usually installed in /usr/sbin/ and may not require root access in general, but it most certainly will insist on it for this use.

OPUS Command Line Tool

 

 

 

 

 

 

 

 

If that didn’t work, go find it in the cron directory of your OPUS install and run it with

If you needed this to work, you’ll need to use instead of just using “opus” in the next command. We need a command called copy_assessment_results and you’ll note it’s not on the list. It’s not on the dev_help list either, because … did I mention this is a stupid thing to do? You need to enter in the command as follows changing the id for old and new regimes to be those you wrote down in step 3. All on one line.

Don’t run this more than once, the code isn’t smart enough not to copy over an additional set of data with possibly “exciting” results.

This copies assessment results and feedback, and marks from one regime to another. It’s potentially wasteful but it can’t identify the correct students and doesn’t delete data as an obvious precaution.

6. Enable the New Regime for Students

Even in maintenance mode, Superuser admins can log in and act. You can switch over your regime now. Maybe do this for one programme and test the results before using the bulk change facility discussed in the previous article.

With luck you will see your shiny new assessment regime with the old marks and feedback for the existing work in the old regime copied over. Older students on the old regime should still show their results and feedback correctly.

If not – well, this is what that backup in step 0 was for, right? And you’ll have to do it manually from the broadsheet you exported as well.

7. Re-enable Normal Access

Either from the command line tool with

or from the Superuser, Services menu, re-open OPUS for formal access.

8. Corrective Action

Explain to relevant colleagues the pain and stress of having to do this and that in future all assessment regime changes should be done appropriately, before students begin completing assessments.

OPUS and Assessment 2 – Adding Custom Assessments

This is a follow on to the previous article on setting up assessment in OPUS, an on-line system for placement learning. You probably want to read that first. This is much more advanced and requires some technical knowledge (or someone that has that).

Making New Assessments

Suppose that OPUS doesn’t have the assessment you want, then you will have to build your own, from scratch, or by modifying an existing one. This takes some minor HTML skill and access to your OPUS code to add a new file. So if you can’t do this yourself, ensure you get appropriate support.

Look at an existing assessment closely first. Go back to Advanced on the OPUS admin menu, and then Assessments.

For each assessment, clicking on Structure allows access to underlying variables that are captured. These can be numeric, text, or checkboxes, and some validation is possible too.

The Structure of an Assessment

you need to work out what things you will capture, and create a skin for the assessment, most usually from modifying one from another. This following snippet from a related Smarty template shows this is just HTML, but OPUS, through Smarty drops in an $assessment variable that gives access to any existing values, and any validation errors.

This is a representative snippet. You can see this full template here. Note the “special” code in between braces { }. The variables in the template pertain to the names in the structure.

Create and Save Your Template

Create your template, probably using one of the existing ones to help you understand the format. This provides the layout and skin for your pro-forma and allows you to do anything you can wish with HTML/CSS. Be mindful of security considerations, but you aren’t writing main code, just an included bit. OPUS will top and tail the file for you when it runs.

Save it under the templates/assessments directory in your OPUS install. I recommend you make a subdirectory for your institution.

Avoid using the “uu” directory. This is used for pre-shipped assessments and those used at Ulster University. There is a chance your changes will get clobbered by a new OPUS version if you put your template in there.

Adding the Assessment variables into OPUS

Then you need to create your new Assessment item itself as at the top of the article. Once you have created it, click on structure and add each variable you will capture in turn, whether it is text, a number, or a checkbox, and any simple validation rules – such as minimum or maximum values.

The final detail of one variable
The final detail of one variable

The description appears in feedback and validation, so make sure it is meaningful to the end user. The name is the variable name as it appears in your template. The weighting field is used to determine if numeric values contribute to the score. Usually use 1 if you want the score to be counted, and 0 if you want the score to be ignored. Finally you can choose whether each field is compulsory or not. Optional fields will be ignored in a total when OPUS creates a percentage.

Once complete, add your new assessment into a test regime as detailed in the first article and do some careful testing before adding the regime to live students.

OPUS and Assessment 1 – The Basics

OPUS is a FOSS (Free and Open Source Software) web application I wrote at Ulster University to manage work based learning. It has been, and is used by some other universities too.

Among its features is a way to understand the assessment structure for different groups and how it can change over years in such a way that legacy data is still correct for audit.

You don’t have to use the in-built assessment functionality in OPUS, but the features were written to promote transparency of assessment, and ensure all stakeholders could easily access assessment information for a student.

So here’s how to do it, it takes a bit of set-up but then should run smoothly until you ever decide to change how you assess. This is one of a short series on the matter.

Assessment Regimes

OPUS uses a “bank” of individual assessments that can be built from different weightings into as assessment regime. To be precise OPUS provides a means of capturing the rubric for each assessment and the feedback to students. Each assessment has a Smarty template which “skins” the assessment form. These can be found in the Assessment section of the Advanced tab of the admin interface.

A list of OPUS assessments
A list of OPUS assessments

For most people using OPUS, you build an assessment regime from these components in a pick and mix fashion. Head to the Configuration tab, and select Assessment Groups. This may well be empty, in an out-of-the-box install, in which case create a group with an appropriate name and some commentary on what it is for.

A list of Assessment Groups
A list of Assessment Groups

Once you have a group, you will see an option to edit the regime that is associated with it.

A typical assessment regime.
A typical assessment regime.

When we add an item, a dialog appears to enter some information.

A regime item.
A regime item.

In this we pick which of the assessments from the very start we want to use, you might decide, for instance, to use the same assessment twice in a given regime, at different stages. Give the student a description of what the assessment name should be for them, a weighting (which could be zero for formative only assessments).

You can also specify who should assess this – it could be the academic tutor assigned to the student, the workplace supervisor, the student themselves or labelled as “other”.

The year is specified in relation to the year of placement, and should usually therefore be zero. Finally start and end are the month and day (MMDD) for when work should begin on such assessments, and the deadline. These are used to help prompt staff and order assessments for students.

Adding Regimes to Programmes

Once an assessment regime has been created, you need to tell OPUS you want to use it with students in a given programme.

Go to Configuration and then Organisation Details and get to the school of study that’s relevant and pull up their list of programmes. For each programme you can click on assessment, from here you can select which regime is appropriate for the programme, and the year in which the regime started and ended being valid. You can leave out an end year to let the decision roll on.

More often than not you wish to apply these changes to at least a School. Clicking on Bulk Change Assessment will allow you to select all the programmes within a School, the new assessment regime you want and the start year, and it will do the rest.

Once you have done this the functionality in OPUS to show the assessments, their structures and marks, and to enable marking will appear for all relevant students and the staff working with them.

Sample Assessment Information
Sample Assessment Information

A table like that above will appear under each related student (this one is dummy information) and students can click view to see the pro-forma whether complete or not to understand how they will be assessed, or what the results were as appropriate.

An assessment pro-forma
An assessment pro-forma

Naturally staff who have no business with a student cannot see the marks or information pertaining to them.

When completing an assessment on a student a member of staff has 24 hours to edit their findings before the results “lock” and can only be removed by an administrator – this allows most minor errors to be corrected.

OPUS and ASET, ten years on

Ten years ago today, I and a few colleagues from Ulster University presented some of our work on on-line Placement Management at the ASET conference in York. At that time our system was simply called the Placement Management System or PMS, and yes of course this led to more than a few comments.

At that stage we had been working on the project for some 5 years, so it’s a useful reminder just how much time I ended up spending on that project.

Now called OPUS that system still exists, was released as Open Source and was and is used by a number of Universities. Though Ulster is developing an alternative system it hasn’t yet subsumed all the functionality in OPUS and I’m back to maintaining the system in a low key way.

I recently fixed some bugs introduced by a well meaning volunteer over two years ago, which felt quite good – while they were low on impact they were irritating in some aspects of usage. In the process I found that our custom framework, written by myself and Gordon Crawford for version 4 of both OPUS and the PDSystem to work with the Smarty Template Engine, is broken with Smarty version 3.

I intend to fix that problem, and do what may be a last release of OPUS, which will bring some improvements in speed, and localisation and internationalisation. Of course the source is still available directly from the version control on the site, so nobody has to wait on me – but I’ve had some recent queries from HEIs in India, so there is still interest in the system and its Debian packaging.

For those wanting the walk down memory lane, and for my own archival purposes, those slides from ten years ago are here: aset-york-pms-2005-09-05.

Importing data from lots of systems with PHP and Regular Expressions

Preamble: I found this article sitting as a draft for several years (2009), so the software release it mentions was years ago! But I figured I’d finally proof read and release the article since the problem it addresses is still a real one.

On Friday I released version 4.1.0 of OPUS which has been in the pipeline for a while. Although the previous version had been working remarkably well, I hadn’t realised just how many bugfixes and new features had accumulated in the last months, and as several Universities had just last week installed the older version, I really needed to get the new version out.

One of the features I introduced in the 4.x family, and which seems to have settled and matured nicely in 4.1 was functionality to import student data, but really, it’s useful for importing diverse CSV formats into a standard base, where the CSV conversion can have more complex problems than just shuffling columns. Our software really uses a thin layer of web services to interface to our student records system, but for the first few versions we imported data from the CSV files that our system could easily export. This is still the solution other universities will have to use unless and until they want to write their own web services layer. Even back when I needed this for internal use, this functionality was a pain, since the University was liable to change the format without notice, and even had slightly different formats for different lists. This made data validation and interpretation a real pain.

So I hit upon the idea of using Regular Expressions (regexps) to validate the data, naturally enough, but then had an idea that would go further. I thought I would define a “standard” CSV format that OPUS would expect, a very simple one. Then I would, for each input system define three regexps.

  1. A match regular expression that would define what each line of data should look like, and also group the bits we wanted to capture in parentheses.
  2. A replacement regular expression that would be used to remap the captured data into a “standard” line.
  3. An exclude regular expression that would define lines to be explicitly exempt from parsing, particularly to deal with header rows, for example.

Since University’s frequently change the content and order of their CSV files and they vary from institution to institution this provides a mechanism to not just validate the input data, but to re-map data from a specified format from any given institution to this “common” format for use within OPUS.

Here is the basic bones of the algorithm.

You can now analyse the arrays of bad lines to explore problems. Rejected lines indicate the fundamental regexp might be wrong. Excluded lines should be expected, but not if they contain valid data obviously. Mismapped lines indicate that the conversion (replacement) regexp is likely to be at fault. This functionality has so far, allowed me to easily adapt OPUS to import data from lots of different university systems without fuss. (Although setting up the initial regexps can be a real pain.)

To give an example, I use the following regexp’s for the University of Ulster’s current programme listing.

This excludes the top, key row of labels, and reorders some of the columns appropriately. This little trick may be of value to you if you have the same problem.