Hidden test cases appearing in code dump

Hidden test cases appearing in code dump

de Padmanabhan Rajan -
Número de respuestas: 4

Hi all,

I recently did a quiz with a c_function question. One of the students managed to get full scores in programming question, but his code was just checking for the test cases (including hidden ones) and printing the output. I thought he had somehow hacked into the Moodle server. When I confronted him, he admitted that the his submission crashed in the CodeRunner window and it threw up a code-dump, including the main program with all the test cases. His was the only such submission in a class of 90. Is this a known vulnerability in CodeRunner?

I have asked him to try and reproduce the error.

A screenshot of the students code is attached. The quiz included a single CodeRunner programming question along with other regular Moodle-type questions (non programming.) The quiz was sequential and the questions were shuffled.

Thanks

Padman

IIT Mandi, India


En respuesta a Padmanabhan Rajan

Re: Hidden test cases appearing in code dump

de Richard Lobb -
I can't think of any sort of C-program crash that would result in the source program being dumped to stdout or stderr, which would be required if it were to show up in the CodeRunner result table. However, if it did happen, you should be able to see it in the student's submission history for that question. Have you checked all the student's submissions?

More generally, yes there is a vulnerability there. A student could enter code to open the working directory and list its contents. Then, once they'd figured out how it all worked, they could dump the main program to stdout, and find all the tests, including the hidden ones (subject to a few conditions - see below). BUT, everything a student does in this way has to be done through CodeRunner, and all their submissions are recorded and available for you to inspect. So it's a dangerous strategy for a student to try. Also, they still have to be able to figure out what the expected output is from each test, since that's not present in the program (except with template graders).

In our in-house Python custom question type we've got some extra protection against these sorts of things, e.g. by replacing the open function with one that disallows access to the CodeRunner system files. That's much harder to do in C, though you could if you were keen enough add your own C function library, to be searched by the linker before the system one, with a restricted implementation of open

Another way to prevent students cheating in that way would be to add some dummy text to one of the test's stdin fields. If CodeRunner discovers any of the tests have stdin, it switches to doing a separate Jobe run for each test case, rather than wrapping them all into a single run. [It has to do that, at least with the built-in question types, because a new stdin needs to be set up for each test.] Any attempt by a student to dump files to stdout would then not expose the tests for hidden test cases.

Other strategies are to inspect the student's code before compiling and running it, looking for certain regular expressions that you might disallow, such as "open *\(". This is most easily done by a Python preprocess, e.g. by adapting the "c_via_python" question type. This is inelegant and might give false positives, but it's easier than rewriting library modules. 

Yet another ploy (again using c_via_python or equivalent) would be to delete the source file after compiling and before running. That might break some types of questions, such as questions using valgrind to check students' submissions for memory leaks.
En respuesta a Richard Lobb

Re: Hidden test cases appearing in code dump

de Padmanabhan Rajan -
Thanks Richard, for all the details. I am not sure if I want to write a custom C library to handle this issue (seems too daunting at this stage!)
But I am unable to figure out how to see the submission history for a student. Is it available readily or requires running administrator scripts?

Thank you.
En respuesta a Padmanabhan Rajan

Re: Hidden test cases appearing in code dump

de Richard Lobb -
When you review a student's attempt on a quiz, there's a Response history section at the end of each question. For example

Screen shot of response history
If you click the number in the Step column, the get to see the answer the student submitted and the response they got.
En respuesta a Richard Lobb

Re: Hidden test cases appearing in code dump

de Padmanabhan Rajan -
Thank you. But I am unable to see the response history option at all when I review the attempt of a particular student. Maybe this feature is installation dependent, and we don't have it. I will check with our system admin. I am not seeing the option seen in this link: https://chtl-bu.hkbu.edu.hk/elearning/self-help/selfhelp_staff_23/