CodeRunner: Run Code without changes?

Hi!

I want to setup a question, where there is already some solution given that contains an error. So for example, I give them three example test-cases where only one of them works. The student should fix then the error, but without seeing the output, or the actual error message, it may be a bit hard.
However, I noticed, that running the tests seems to be not supported, when there was no change to the code and even trivial changes like adding a blank line have no effect and prevent the tests from being executed.

I looked at all the options I could find in the test settings, but there seems to be no setting to allow running unmodified code. Is that true, or did I miss some setting somewhere?

Best,
Sebastian

Re: Run Code without changes?

de către Richard Lobb- luni, 4 august 2025, 09:31

The inability to submit just the preloaded answer was introduced for performance reasons. Without that constraint, at the end of a test or exam, all questions with an answer preload would be sent to the jobe server for marking. With large classes this could result in thousands of jobs being thrown at the jobe server within a few seconds, which could overload the server. It's really an extension of the idea that Moodle doesn't grade empty answers - the student has to have done something in order to get graded.

However, I agree that it's a nuisance in your particular use case and I don't have an immediate solution. My only suggestions are that you can tell the students to make a trivial edit, such as adding a space character on the end of a line, or you could paste the results of the initial (disallowed) submit into the question specification with wording like "If the preloaded code were tested, it would fail as follows: ....".

You said that "even trivial changes like adding a blank line have no effect and prevent the tests from being executed." I think you'll find that's true only if you add whitespace at the end of the answer. Whitespace at the end is trimmed off before comparing the student answer with the preload. But whitespace in the middle of the program should be sufficient to allow grading.

I'm reluctant to further complicate the question authoring UI with yet another checkbox for this particular edge case. But I can look into whether we can enforce the constraint only on quiz submission, and not when Check or Precheck is clicked. That would involve changes to both the CodeRunner plugin and its associated Behaviour plugin.

Re: Run Code without changes?

de către Sebastian Bachmann- luni, 4 august 2025, 17:44

Thanks for the explanation, that makes sense to me!

> But whitespace in the middle of the program should be sufficient to allow grading.

Yes indeed, I tested it again and any number of blank lines at the end of the file has no effect. However, adding blank lines somewhere inside the program makes it work.

It is easy enough to work around this and, as you say, give the error message in the question, or ask them to remove a specific comment.

Re: Run Code without changes?

de către Paul McKeown- luni, 4 august 2025, 19:18

Hi,

Another option would be to use the scratchpad_UI (eg, as in unsupportedquestiontypes/prototypes/PROTOTYPE_python3_scratchpad.xml in the Coderunner repo). You would preload the scratchpad with the test cases, make sure prefix with answer is set, and ask students to run the scratchpad. This would allow students to run the code straight-away, without modification.

A simple preload with comment on the last line (like "Remove me before running to see the errors!") is probably easier, and removes the need to make sure your scratchpad preload matches your actual test cases. But, your students might prefer to crash as fast as possible ;)

Cheers,
Paul

Re: Run Code without changes?

de către Sebastian Bachmann- luni, 4 august 2025, 20:20

Thanks for the idea - but I'm also very new to coderunner and thus feel a bit uncomfortable to play around with these options at the moment.
I'll implement it with the comment for now and see how it works with the students. It is actually the first term that we do it with coderunner, so it is new for everyone and explaining them once, that they have to change something seems to be acceptable.

Sebastian