CodeRunner: Testing was aborted due to error.

Hi,

We use Moodle 3.9 version and for the code runner 4.0.2.In python3_w_input question type some of our instructors faced with the error "Testing was aborted due to error." We assumed that problem caused that reason : In the solution shared by the instructor, the code ends with the given inputs, while in the student's solution, the code is not finished yet and more inputs are waiting. However , we couldn't figure out it .I am sharing a sample question that we have same errors. Could you please help us for solving this issue?

Thanks a lot..

our question ,

inp = input("Enter an input: ").lower()

while inp != "exit":

  inp = input("Enter an input: ").lower()

student answer

inp = input("Enter an input: ")

while inp != "exit":

  inp = input("Enter an input: ")

Test case1:

InancDuyguExit

Test case2 (Hidden):

InancDuyguexit

Anexo 1635921797_Screen Shot 2021-11-03 at 09.41.16.png

Re: Testing was aborted due to error.

por Richard Lobb - martes, 9 de noviembre de 2021, 10:12

To run a question that uses standard input, CodeRunner writes the input that the question author supplies to a file. The program is then run with standard input redirected to read from that file. Since the student's code is checking for the line 'exit' (without the call to the 'lower()' method), it doesn't recognise the line "Exit". So the program attempts to read another line from the file, but there isn't one there. It then gets an EOF (End Of File) exception.

I'm not sure what you mean by "solving the issue". This is really a program error - the student has written bad code, so it fails. I think you just have to tell the students that the EOF error message means they failed to recognise the terminator line in the file and have tried to read more input than has been supplied by the question author.

Re: Testing was aborted due to error.

por Sueda Denizer - martes, 9 de noviembre de 2021, 19:15

Hi,
I aimed to mention with "solving issue" in this sample we have two test case and when the first case failed should it be skipped other test case the hidden one ?

Re: Testing was aborted due to error.

por Richard Lobb - martes, 9 de noviembre de 2021, 21:24

In all the standard question types anything that generates standard error output is taken to be a runtime error and testing is aborted. It doesn't usually make sense to continue testing after a runtime error, which might for example be an undefined variable in Python, so all tests will fail. In the usual case of "All-or-nothing" grading, if one of the test cases has failed the student already has zero marks so, again, why continue testing? The student should fix their code and try again.

However, if you wish to award marks separately to each test in your example, continuing after an EOFError, you can customise the template as follows.

Click the customise checkbox.
Turn off all-or-nothing grading.
Turn off "Is combinator" in the Template controls.
Replace the template with the following:

__saved_input__ = input
def input(prompt=''):
    s = __saved_input__(prompt)
    print(s)
    return s

student_answer = """{{ STUDENT_ANSWER | e('py')}}"""

try:
    exec(student_answer)
except EOFError:
    print("Oops, you seem to have failed to recognise the terminator")

Re: Testing was aborted due to error.

por Anton Dil - martes, 9 de noviembre de 2021, 22:38

I guess this is a matter of philosophy, but I have edited all my templates for Java to look like that - catching Throwable, allowing testing to continue. Ideally we'd provide a useful error message for different classes of error, but I haven't tried to do that (yet).