Hello!
We are using CodeRunner for an exam where students may check if their code compiles through prechecks, but they do not see if it is functionally correct (check is hidden). All questions are combinator template graders.
We have decided that we would like to implement additional logic based on the student's precheck history (in case their final answer contains compilation errors). Naturally the first place I turned to was the graderstate of the QUESTION.stepinfo field. However, it seems that the state isn't maintained in some cases. I observe the following behavior:
- stepinfo is maintained if the student submits after a check
- stepinfo is maintained after regrading, even if the question code changes
- stepinfo is maintained throughout changes during the attempt
- stepinfo is not maintained if the student submits after a precheck
- stepinfo is not maintained if any changes are made between the last check and the final quiz submission
Is there any way to have the stepinfo from the precheck stick around such that it can then be used in the actual check, without students being allowed to run checks on their own?
I have considered as an alternative that we use incomplete question code during the exam and expose the check button directly, then swap out the question code for the real one and regrade. But this means the students see what they might think is an actual final grade for those questions. Also we have been badly burned by a bug that prevented regrading the quiz in the past, so I would like to avoid relying on that for the final grade.
Best,
Tom