CodeRunner Documentation (V3.2.1)
8.2 Combinator-template grading
The ultimate in grading flexibility is achieved by use of a "Combinator
template grader", i.e. a TemplateGrader with the
Is combinator checkbox checked.
In this mode, the JSON string output by the template grader
should again contain a 'fraction' field, this time for the total mark,
and may contain zero or more of 'prologuehtml', 'testresults', 'epiloguehtml'
The 'prologuehtml' and 'epiloguehtml' fields are html
that is displayed respectively before and after the (optional) result table. The
'testresults' field, if given, is a list of lists used to display some sort
of result table. The first row is the column-header row and all other rows
define the table body. Two special column header values exist: 'iscorrect'
and 'ishidden'. The \'iscorrect\' column(s) are used to display ticks or
crosses for 1 or 0 row values respectively. The 'ishidden' column isn't
actually displayed but 0 or 1 values in the column can be used to turn on and
off row visibility. Students do not see hidden rows but markers and other
The 'showdifferences' attribute can be added to the JSON outcome to render the standard 'Show differences' button after the result table; it is displayed only if there is actually a result table present and if full marks were not awarded to the question.
Combinator-template grading gives the user complete control of the feedback to the student as well as of the grading process. The ability to include HTML in the feedback allows for complex output containing SVG or images.
The combinator-template grader has available to it the full list of all test cases and their attributes (testcode, stdin, expected, mark, display etc) for use in any way the question author sees fit. It is highly likely that many of them will be disregarded or alternatively have some meaning completely unlike their normal meaning in a programming environment. It is also possible that a question using a combinator template grader will not make use of test cases at all. For example it might test the students code with thousands of random tests and display just a few of the failing cases using the result table.