Keeping track of your question papers - Alison Brennan

Each year we mark around 3 million question papers, known as scripts, and last year we investigated approximately 9,100 cases of ‘at risk’ scripts. These are usually scripts which, for various reasons, end up in the wrong place during the initial transit after an exam has been taken e.g. those packaged in an incorrect script packet. It also includes investigating scripts which don’t exist in the first place e.g. where candidates are incorrectly marked as present on an attendance register our system identifies we should have received a script for them and we begin to investigate. 

Our checking process identifies all these cases and we carry out rigorous investigations to locate the script/check whether it exists, and ensure all candidates receive a grade. Every single script is important to us and we track all scripts to ensure each candidate's hard work is recognised. Last year, my team was able to track down all but 70 of the 9100 ‘at risk’ scripts they investigated. I thought it would be useful to outline what steps we take when we investigate each case and answer some of the most common questions we get asked. 

How are we alerted to an issue? 

There are three main ways in which we know a script may be at risk:

• If you’ve read Helen O’Leary’s latest blog on ‘Sending scripts for marking’, then you’ll know that, when scripts arrive with us, the majority are scanned. At this stage my team is notified if there is no script for a candidate who, according to the attendance register, was present for the exam.
  
• The second method is via our examiners – they may discover a script is missing when marking.
• The third is via our specialist team who investigate all cases of candidates who have entries, but who do not have associated marks. 

Why do we ask for attendance registers and seating plans? 

It’s always helpful to have as much information as possible when we’re trying to establish whether a candidate was present for an exam and where their script might be. Please remember to complete your attendance registers clearly and don’t leave any candidates as ‘blank’. An accurate attendance register tells us whether a candidate was present for an exam and can help us to work out if we should have received an exam script for them. 

Your seating plans can also provide us with clues, e.g. if there was more than one exam being completed in the exam room at the same time, scripts from different exams sometimes get put into the same script packet and may then be matched to the wrong unit or component – seating plans can help us to work out whether this may have happened. When we receive scripts, we scan the bar or unit code on the yellow Parcelforce label which you attach to the outside of the script packet. All the scripts in the packet are then scanned and digitally ‘attached’ to that code. 

Why don’t we tell you we're investigating the whereabouts of scripts straightaway? 

In the majority of cases we don’t contact you straightaway because we carry out a number of standard checks which usually allow us to solve queries we may have about scripts. We don’t want to cause you unnecessary worry so it’s only if we need more detailed information that we get in touch. 

Why do we tell you a script packet is missing and then later ask about some individual candidates from that packet? 

Individual scripts and script packets, both of which sometimes need investigating, are actually two separate issues. If there’s been a delay in your script packet being delivered to us then we’ll often contact you to ask for proof of postage or a copy of your despatch log. Once the script packets are opened and the scripts are scanned we may then find a candidate’s script isn't present and we’ll contact you again about this. 

The most common issue with speaking tests is that the CD hasn’t been ‘finalised’ so it can be played on other devices

Similarly, when speaking tests are sent to us, our examiners will notify us of any packets which haven’t arrived straightaway but they may not identify whether a candidate’s recording is missing until they begin to mark the test itself. This may be a while after the packet itself has been received, particularly if the examiner has a large number of tests to mark. 

The most common issue with speaking tests is that the CD hasn’t been ‘finalised’ so it can be played on other devices – this means it will only play on the device on which the recordings were made. It’s really important to check your CDs will play on a different device before sending them to us.

What if we can’t find a script? 

In a very small number of cases, despite our best efforts, we are unable to find the scripts. In the majority of these cases, we can put the candidate forward for an ‘assessment’ where we apply consistent criteria to produce an estimated mark. This mark is then combined with the candidate’s other unit/component results to produce an overall grade. We do this to ensure that no candidate misses out on receiving a qualification grade. 

How do we calculate an estimated mark and overall grade? 

So that all candidates are treated fairly and consistently, for unitised qualifications we use a JCQ-agreed statistical method to produce an estimated uniform mark for the missing unit (sometimes called the z score method). This method takes into account the results the candidate achieved in all other units for the qualification at the same level, and compares their performance in those ‘supporting’ units with the performance of all other candidates who also achieved the same units. Further details about this calculation process and some examples are provided on the JCQ website. 

The method used for new linear AS Levels is similar. However, as uniform marks do not exist for linear qualifications, calculations are completed using component raw marks. An estimated raw mark is then calculated for the missing component. 

What happens if a script is found after a grade is issued? 

If a script is eventually delivered to us, even after the publication of results, we will make arrangements for it to be marked. We will then apply the actual mark to the candidate, whether this is higher or lower than the previously estimated mark. This may affect the candidate’s overall grade. If this happens, we will contact you. 

Can an enquiry about results be submitted for an estimated mark? 

An enquiry about results can be submitted for any unit or component for which we have a script. Without a script we are unable to carry out a review of marking and therefore we can’t process an enquiry about results for a unit or component where we have produced an estimated mark.

About the author

Alison Brennan - Manager, Post-Assessment Service

Alison Brennan manages our Post-Assessment Services team and has worked at OCR for over 30 years in various post assessment roles. Her team investigate instances where coursework marks and coursework are not present as expected. The team also process changes to coursework marks from centres and moderators, and track the completion of moderation.