Thanks for posting a great question, Heli.
Wow - you just recently starting using CodeRunner and you're trying template graders?! That's called "Jumping in at the deep end"!
Certainly you can achieve what you want, but you do have to use a template grader. Template graders are simple enough in principle - you just have to construct a program the output of which is a JSON record describing a line of the result table - but are a bit tricky to implement when students supply the code.
The template grader example in the documentation isn't a very good one, particularly not for your needs, which are actually much simpler. [The documented version is not made any easier by the fact that '<' and '>' characters in the input HTML are not being displayed in the version on coderunner.org.nz :-(]
Here's an example of a template grader, closer to your needs. I'm assuming any simulation parameters are given to the student, so there's only a single testcase, in which the 'expected' field is the expected output. For simplicity I'm assuming that's always exactly 5 lines but it's trivial to generalise. I'm giving one mark for each correct line.
The 'Result columns' field of the question (in the 'customisation' part of the question authoring form) is
[["Expected", "expected"], ["Got", "got"], ["Comment", "comment"], ["Mark", "fraction"]]
and the template is (**edit** fixed wrong indentation on last statement):
import json
got = """{{ STUDENT_ANSWER | e('py') }}"""
expected = """{{ TEST.expected | e('py') }}"""
got_lines = got.split('\n')
expected_lines = expected.split('\n')
mark = 0
if len(got_lines) != 5:
comment = "Expected 5 lines, got {}".format(len(got_lines))
else:
comment = ''
for i in range(5):
if got_lines[i] == expected_lines[i]:
mark += 1
comment += "Line {} right\n".format(i)
else:
comment += "Line {} wrong\n".format(i)
print(json.dumps({'expected': expected, 'got': got,
'comment': comment, 'fraction': mark / 5}))
The output if the student's answer is correct is then, say:
and if they make some mistakes:
Obviously you can extend this to handle more elaborate marking rubrics.
Hope that helps.
Enjoy
Richard