Developers' Forum

This question is closely related to this thread. But time-limit exceeded is perhaps a bit more problematic. Do you really want to display all test results, even if every one gets stuck in a loop? If there's, say, a 5 second timeout and 10 tests, that's 50 seconds of wasted time for both the Jobe server and the student waiting for their response. I think the current system, which throws the job out if a test case is looping, is the right approach if you're using all-or-nothing grading. If you're giving marks per test case I suppose there's a case for letting the tests grind on, though my gut feeling is still that it's better to let the student try again, with a penalty, rather than trying to give marks to a flawed submission.

Anyway, to answer your original question, if you want testing to proceed after a run-time error of any sort you'll need to catch the error yourself. We use a complex combinator grader in which we have our own watchdog timer that kills the run well before the Jobe timeout limit is reached. You can see our question type on the github repository here if you like, but it's very complex, having evolved over years, and it probably not what you want. A simpler solution would be to follow the general approach in that other thread (here). Where that thread recommends catching runtime errors, you could instead use a watchdog timer.