Regrade seems to not rerun code for attempts after the first?

Regrade seems to not rerun code for attempts after the first?

by Pavel Krivitsky -
Number of replies: 0
Dear All,

I am using CodeRunner in a conventional assessment in a fairly large class. I've found the JavaScript trick to hide the Check button to be unreliable: it doesn't work on some browsers.

And so, I instead set the penalty to 0 and configured the test cases as follows:

  1. Precheck Test A
  2. Precheck Test B
  3. ...
  4. Check Test always set to fail, hide subsequent tests on fail, and inform the student that Check is disabled and to disregard error messages about hidden tests.
  5. Check Test A
  6. Check Test B
  7. ...
  8. Check Test always set to fail, to avoid revealing if the submission is all correct.
My intention was that after the quiz closed, I would delete Tests on lines 4 and 8 and regrade.

I had also enabled unlimited attempts, with each attempt building on prior attempts, and only last grade counting, so that students who wanted to update their submission could do so easily.

However, when I went ahead and deleted those questions and ran a full regrade, I discovered that for some students (and I haven't been able to figure out the pattern) attempts after the first still had their old Check results (with Test on line 4 results showing).

So far, I've tried:
  1. Selecting the target attempt and using Regrade Selected Attempts: No effect.
  2. Deleting all attempts but the last: No effect.
  3. Making other changes to the CodeRunner question (e.g., adding a dummy test): No effect.
  4. Changing some other setting (e.g., All or Nothing Grading): Student marks changed, but based on the old run.
How do I force CodeRunner to rerun all test cases? Has anyone else run into this? Is there some caching going on?

Thank you all in advance,
Pavel