When you finish your comparative judgement session you have a set of scaled scores, which you may wish to relate to a set of standards represented by grades or levels (NB - in our national projects, we will do this for you).
Examples of standards may be mark bands, GCSE grades 9-1, Pass or Fail, or working at an Age Related Expectation.
If you have examples of work at specific marks or grades you can include them as scripts to be judged. Ideally the examples would be clear of any marks to minimise bias. A selection of two or three pieces can help, but be clear about whether the scripts represent the lowest possible standard (the threshold, the grade boundary) or the mid point of the standard. At the end of the session you can use the position of the scripts to guide you towards positioning the levels.
If you don't have examples of standards, then you can use a statistical rule of thumb. For example, you may know that, on average, year on year, 75 per cent of your pupils achieve Age Related Standards. In that case you look down 75 per cent of the way through the rank order and examine the script. You can then make a decision on whether you think it could be described by your standard. If not, then look above or below it until you are happy you have found the threshold.
Ideally you would use a combination of both methods to make your decision.
Setting the levels
To set the levels in your task, first of all, to update your scores, go to the particular task and click the 'Refresh Scores' button. Then click on Check results, and then arrange the candidates in order of their scores, by clicking on the Scaled Score column heading to order the results:
Now decide on your 'threshold' scripts which are at the bottom of each level that you want to apply. You can view each script by clicking on the link in the Code column for each candidate. Make a note of the Scaled Score for each threshold script and decide on the lower threshold Scaled Score for each level.
Now go to the LEVELS tab and click on the Add Level ('+') button to add a level:
Check the tiny box next to this new level, and then press the Edit Level button (pencil icon):
Give the level a name (e.g. Below Pass, Pass, Distinction) and assign a Level Value which can be used if you want to calculate the average level of a cohort.
Finally, in Scaled Score Boundary, enter the lower threshold of the Scaled Score for the level that you recorded previously, then press the 'Update' button: