A user posted the following to the Moodle plugins repository. I'm answering here too for more general consumption.
First of all - THANKS!!! This tool is great and I'm glad this module is added to the LMS for schools in Hamburg.
I tried to write an own template grader and in general it is working as I print "print(json.dumps({"fraction":points, "got": output, "abort": False, "comment": comment}))".
If i try to avoid error messages in the pre_check I don't know what to print? I do think, I've overseen some important hints in the documentation and I'd be pleased to get some help.
I've added the following section to the documentation.
Use of the Precheck button
The question authoring form allows the author to provide students with a Precheck button in addition to the normal Check button. This is intended to allow students to do a penalty-free preliminary sanity check on their submission. The form of the sanity check can be controlled by the question author, but in the simplest case it is just a syntax check on their code.
When the Precheck button is clicked, the answer is submitted for grading in the normal way except:
- A Twig variable IS_PRECHECK is set to True. This is typically used by template graders to control the feedback that is given to the student when prechecking versus when doing a full check.
- The set of testcases to be run is restricted according to the setting of the Precheck dropdown menu in the authoring form. Options are:
- Empty: The list of testcases is a single empty testcase, which is a hidden test with the empty string for testcode, stdin, expected and extra.
- Examples. The set of testcases is all the ones with the Use as example checkbox checked.
- Selected. If this option has been chosen, each test case has a dropdown menu labelled Precheck test type, with options Check only, Precheck only and Both. The set of cases on a Precheck is then all those marked either Precheck only or Both, while the set of cases on a full Check is all those marked either Check only or Both.
- All. All testcases are included, as with a normal Check. This is meaningful only if the question author has made use of the IS_PRECHECK Twig variable to provide different feedback from the normal.
The feedback presented to the student contains the output from the run, as with a full check, but no marking takes place, so there are no penalties. Additionally the correctness of the Precheck is indicated to the student by a striped background shading that is blue for an OK submission and red for a failed one. There may also be a "Precheck failed" warning message.
The correctness of the precheck is determined as follows.
- If the Empty precheck option has been set, and a combinator template grader has not been used, any output (which is usually compile errors) is taken to denote a failed precheck. In this case, there is also an extra "Precheck failed" or "Precheck passed" message.
- If a combinator template grader is being used, the Precheck is deemed correct if the returned Fraction is 1.
- In all other cases, i.e. when a subset of test cases is being run, the Precheck is correct only if all tests pass.