Question Authors' Forum

Creating my own result columns

Picture of Richard Lobb
Re: Creating my own result columns
by Richard Lobb - Thursday, 30 June 2016, 9:33 PM

Thanks for posting a great question, Heli.

Wow - you just recently starting using CodeRunner and you're trying template graders?! That's called "Jumping in at the deep end"!

Certainly you can achieve what you want, but you do have to use a template grader. Template graders are simple enough in principle - you just have to construct a program the output of which is a JSON record describing a line of the result table - but are a bit tricky to implement when students supply the code.

The template grader example in the documentation isn't a very good one, particularly not for your needs, which are actually much simpler. [The documented version is not made any easier by the fact that  '<' and '>' characters in the input HTML are not being displayed in the version on :-(]

Here's an example of a template grader, closer to your needs. I'm assuming any simulation parameters are given to the student, so there's only a single testcase, in which the 'expected' field is the expected output. For simplicity I'm assuming that's always exactly 5 lines but it's trivial to generalise. I'm giving one mark for each correct line.

The 'Result columns' field of the question (in the 'customisation' part of the question authoring form) is

[["Expected", "expected"], ["Got", "got"], ["Comment", "comment"], ["Mark", "fraction"]]

and the template is (**edit** fixed wrong indentation on last statement):

import json
got = """{{ STUDENT_ANSWER | e('py') }}"""
expected = """{{ TEST.expected | e('py') }}"""
got_lines = got.split('\n')
expected_lines = expected.split('\n')
mark = 0
if len(got_lines) != 5:
    comment = "Expected 5 lines, got {}".format(len(got_lines))
    comment = ''
    for i in range(5):
        if got_lines[i] == expected_lines[i]:
            mark += 1
            comment += "Line {} right\n".format(i)
            comment += "Line {} wrong\n".format(i)

print(json.dumps({'expected': expected, 'got': got,
      'comment': comment, 'fraction': mark / 5}))

The output if the student's answer is correct is then, say:

Simple template grader example: right output

and if they make some mistakes:

Simple template grader example: partial marks

Obviously you can extend this to handle more elaborate marking rubrics.

Hope that helps.