First off, thanks for CodeRunner, it is a great tool
I am currently trying to create a question type for control theory questions (see the python-control project). I am using a custom combinator template, that does the following:
- It grabs the variables a student leaves in the workspace in a Python dict
- I wrote check classes for the different types of answer variable. Each test case consists of an instantiation of a check class object, which tests a single variable, either a control theory object, like a transfer function, or a matrix, or a single floating point variable. Not all valid ways of solving the problem result in numerically the exact same answer, and some procedures, like root-locus or tuning with a Bode plot, require students to read off values from a plot, resulting in even larger margins; the check classes find the respective variable from the workspace, and validate it with the given tolerances.
- When in precheck mode, the template produces a table (in epiloguehtml), that indicates whether the answer was found in the workspace. This helps ensure that students don't forget to provide one of the variables asked for a question.
- When in check mode, for a practice mode I can immediately give feedback on the question, for an exam I just want to give a confirmation that the answer variables have been found, i.e., the same information given in the precheck mode.
The feedback epiloguehtml is however only shown when feedback "show" is selected, so I selected "force show" for the feedback, and I have a template variable that tells my code whether to show the individual score and answers in the table, or suppress those and only show whether the required variable was found.
My problem is that the score that is calculated, and returned in the "fraction" property of the JSON object, influences the style of the div that contains the feedback table. So rather than only confirming to students whether the required variables have been supplied, it also informs an astute student of whether the answer was correct or partially correct.
I see two potential solutions.
- Finding a way to show the presence of the required variables (basically a sanity check), irrespective of the immediate feedback mode, and in a way that does not provide clues about the fraction value returned in the JSON object
- Continue misusing the feedback "force show" mode, but suppressing the style set on the div that contains the epilogue and feedback table, maybe by modifying the template?
Are there any suggestions? Is there a different way for showing the sanity check table?
An impressive question type. It's gratifying to see people using the most advanced features of CodeRunner.
I can't think of a nice solution to your problem, but an ugly one that might work for you is to make use of the 'partially correct' background colouring. This is used whenever the all_or_nothing checkbox is unchecked and the fraction is in the range 0.00002 and 0.99998 (roughly). So when in exam mode you could return marks in that range without any information leakage and with negligible actual impact on marks, which are always displayed rounded in Moodle, AFAIK.
Alternatively the way I dealt with a somewhat similar problem myself was to have the Check button simply print a message saying that Check functionality was disabled during the exam. The Precheck button still worked as usual, so they could still sanity check their answers. Then after the exam was over I switched grading on again and regraded all submissions. The regrading can take a long time, though, as all prechecks are regraded too.