Training and testing
OCR trains all reviewers on a yearly basis, before the summer review of marking period. Assessors complete a remote training package covering both the process of completing reviews of marking or moderation and the rules and approach about when to change marks or outcomes of moderation. They then have to pass a test to show that they have understood and can apply these rules appropriately. Assessors are only allowed to undertake reviews after they have passed this test.
Monitoring of reviews of moderation
OCR monitors reviewers throughout the review of marking and moderation period. For reviews of moderation, we check all moderation outcomes against the review of moderation reports in order to ensure that the rationale for any changes is appropriate and consistent with the outcome. If this is found not to be the case, the review is returned to the reviewer with feedback so that they can correct it before the outcome is processed and the centre (school or college) notified. If a similar issue is then found on subsequent reviews for the same reviewer, they can be stopped from completing any further reviews and will not be used again to complete reviews for the same component in future series.
In addition, at the end of the review period when all reviews of moderation are complete, OCR analyses all outcomes to check if there are any patterns at reviewer or component level that suggest that the approach to reviewing and rules about when to change or confirm outcomes are not being followed consistently. If this is found to be the case, the reviewer(s) in question will be given additional feedback and training before being allowed to complete reviews in future series.
Monitoring of reviews of marking
OCR monitors reviewers throughout the review of marking and moderation period to ensure that they are making appropriate and consistent decisions and only changing marks to correct marking error. For reviews of marking, OCR monitors all reviewers daily throughout the review period by checking a sample of each reviewer's work. If we find a problem, the reviewer will be given feedback, re-trained or stopped from reviewing as necessary, and we will carry out the review in question again. Checks are timed to take place before review outcomes are released to centres, so you can be confident in the review outcome, whether a student’s mark has changed or not. In the rare event we are unable to complete all the additional checks for the priority reviews by 31 August due to late requests, we will issue ‘provisional’ results of the review. We believe this is still in the best interests of the student.
We also monitor all reviews of marking outcomes at weekly intervals throughout the review period in order to check that there are no patterns in changes to marks that suggest a potential problem with the original marking during the series. If this monitoring identifies any potential errors in marking, these are investigated and if necessary reviews of marking are extended to cover all candidates that further investigation has identified as being potentially impacted. Marks and where appropriate results will then be updated to correct the effect of any error, after the impact of this on candidates and the potential for adverse effect has been considered on a case by case basis.
In addition to monitoring at weekly intervals, OCR will run a final report at the end of the reviewing period, and this will be analysed to identify any trends or patterns between panels, reviewers and across time. Where the final monitoring report suggests that this has not been fully effective in achieving consistency, OCR will address this by amending training, testing and monitoring processes as required.
Review of training and monitoring arrangements
OCR will undertake a yearly review of training and monitoring arrangements for reviews of marking and moderation after the summer series review of marking and moderation period. Any changes to arrangements resulting from this review will be made before the next summer review series.
Where to find more information
For more information about our post-results services, please see the post-results services section. This covers how to request a review of marking or moderation, key dates and deadlines, our charging policy and links to the fees that are payable.
Further information can also be found in the JCQ post-results services booklet.