CodeRunner question giving three different error messages for staff/student on live/dev servers

Re: CodeRunner question giving three different error messages for staff/student on live/dev servers

von Richard Lobb -
Anzahl Antworten: 1
I don't have any way to map from the regrading of a particular question to a JOBE log entry. There are two runs with a 765 byte payload, one with a 539 byte payload and two with a 1789 byte payload. I don't know if that's 3 questions, two with retries, or 5 questions with no retries.

The critical question here is whether the regrade of Q5 does or does not reach JOBE. That needs to be resolved definitively.

There are two possibilities:
  1. The request never reaches Jobe.
  2. The response never gets back to Moodle.

The OOM-killer theory is ruled out if it is guaranteed that one of those JOBE log entries corresponds to Q5, since all returned with a 200 OK response. So what might stop a request getting to JOBE or a response getting back? Is there a firewall/WAF/reverse proxy/load balancer in front of production Jobe that isn't in front of your test one? If so, worth checking its logs (not Jobe's) for both inbound and outbound traffic during a Q5 regrade— particularly anything about a reset, timeout, or oversized/blocked response.

Some things to possibly try:

  1. Customise Q5 and in Advanced customisation > Sandbox parameters set {"debug": 1} . Regrade Q5 for that one student (and maybe another student for comparison). Then on the Jobe server look in /home/jobe/runs (which you should also check before starting). The complete record of the run - code, compile output, run output etc - will be there if the job made it to Jobe. If there's nothing there, your theory that the job never makes it to Jobe is confirmed (though please double-check with a regrade of a question that works).
  2. Are you able to pull the current Development branch from GitHub ? If so, you'd get the exact cURL error message rather than the unhelpful 0:null response. But I'm guessing that's not possible?
  3. Look in /var/www/html/jobe/writable/logs for any error reports during regrades. 
  4. Try to carry out the test I suggested earlier of switching your test server to use the same Jobe as the production server and see if you then get the 0: null response.
Als Antwort auf Richard Lobb

Re: CodeRunner question giving three different error messages for staff/student on live/dev servers

von Chris Nelson -
Hi Richard,

We tried adding the debug in #1, but cannot move the student to the latest question version through any method, and we're not allowed to regrade the entire exam on our live system. And as stated, we're unable to replicate this issue in our testing servers. Curiously, all three CodeRunner questions in the exam don't move to the latest version (usually methods are to go to Quiz Settings and press 'Save', or regrade the students' attempt). I had a brief chat with Tim Hunt about this, and he's not sure why that has happened either.

For #2 No, sorry. Next time we upgrade (for our December release in late November), I can try running the regrade and see if anything changes.

#3 I don't have sight of those, but my developers have said they can't see anything obvious.

#4 Unfortunately our environments are heavily locked down and we can't mix and match the live and test servers/sub-servers :-(

I think we've probably spent as much time troubleshooting this as we can our side, but at least this problem is documented for the future in case anyone else has a student response get stuck in the same sort of state. Hopefully #2 will shed some more light on it after our next upgrade hits the live environment.