CodeRunner: Resubmission function

Hello everyone,

we are evaluating the CodeRunner plugin in a programming lecture and are so far very satisfied. Thank you for the development of such a useful plugin!

However, we are missing a function that our previous tool was offering. It allowed students to resubmit their answers to coding questions after the grading. Depending on the amount of changed code, credits to compensate for the mistakes in their initial submissions were awarded. This allowed students to reconsider wrong solutions, learn from their mistakes and gain additional points.

There are some workarounds, for instance prechecking, but the original function was received very well by our students and was useful from a didactic perspective. I just wanted to ask whether you already thought about such a function (maybe we overlooked it), or whether there are plans to implement this function in CodeRunner if it is technically feasible.

Thank you and kind regards,
Frieder

Re: Resubmission function

de către Richard Lobb- joi, 23 februarie 2017, 14:28

Hi Frieder

I'm not quite sure I understand what functionality you're requesting. The latest version of CodeRunner always runs questions in Adaptive Mode, regardless of what mode the quiz is set to, so students have multiple grading attempts. When a student clicks Check (which is essentially "submit"), their code gets graded immediately and they can usually see what they got right and what they got wrong. They can then correct their code and resubmit. This cycle can be repeated as often as the student wishes, but they normally pay a penalty for each resubmission. The default penalty regime is 10%, 20%, 30%, ... although the question author can set this to anything they like.

I take it the above isn't sufficient for your needs. You say "Depending on the amount of changed code, credits to compensate for the mistakes in their initial submissions were awarded". I'm not clear how this might work. Do you mean that the penalty for a resubmission depends on (say) the size of a diff between the two submissions? Or maybe the ratio of the size of the diff to the original submission?

Setting a penalty regime that depends on the magnitude of the change is technically possible but it would be a major change to CodeRunner and I'm not (yet) convinced it would be worth the effort. Convince me! :-)

Richard

Re: Resubmission function

de către Frieder Loch- vineri, 24 februarie 2017, 21:21

Hi Richard,

first of all, thanks for your answer!

The main difference that I see is that we allow the resubmission within a certain timeframe after grading. For instance, resubmission opens 3 days after the exam (e.g., when the grades were published) and is open for 2 days. Within this timeframe, students can submit a corrected solution (correct = it passes all test cases) to the question based on their initial submission. This should allow to reconsider the solution without the pressure of a time boxed examination. Is there a way to implement this behavior, especially given that the examinations consist of programming, which can be resubmitted, and multiple-choice questions, which cannot be resubmitted? Is this even something that would require a change in Moodle?

Yes, the bonus for a resubmission depends on the number of changed characters. If a student changed, for instance, 20 characters to obtain a correct solution he would get 90% of the points.

Thanks and kind regards,
Frieder

Re: Resubmission function

de către Richard Lobb- sâmbătă, 25 februarie 2017, 21:49

Hi Frieder

That's an interesting approach.

Did your previous system also do instant grading of submissions, allowing students to immediately correct their code during the exam? My feeling is that giving students the option of resubmitting some time after the exam might not be as relevant when using CodeRunner. When students submit a CodeRunner answer that gets marked wrong, they normally try to fix it there and then. They're very reluctant (often too reluctant) to move on to the next question until they've got the marks for the one they're working on. I'm not sure how their behaviour would change if you were to say to them "You can come back after the exam closes and fix your program, paying a penalty that depends on how many changes you make". Would giving them that option really improve the quality of the final assessment?

Tim (below) has suggested a way in which you could give students the option of resubmitting after the exam has closed, but you would have to manually regrade the submission(s), applying your difference-dependent penalties. You could implement an off-line script to automate this using the exported responses both straight after the exam and again after the resubmission window. However, I'm not sure it would be worth the effort.

But it's certainly interesting to think about these things. Thanks.

Richard

Re: Resubmission function

de către Tim Hunt- sâmbătă, 25 februarie 2017, 01:51

You might be able to do this with Moodle's 'Each attempt builds on last' option (which is normally hidden behind one of the 'Show more...' links on the quiz settings form.

For the exam, create the quiz only allowing one attempt, and with that option off.

Later, after the exam, when you want to re-open the quiz, change the settings to allow 2 attempts, each attempt build on last, probably set it so the final grade comes from the first attempt. Then students should be able to have another go.

Re: Resubmission function

de către Frieder Loch- luni, 28 mai 2018, 21:06

Hello,

i just wanted to let you know that we finished our first examninations and got good results without the resubmission function that our old system offered. We just sensitised the students for practicing to write full and correct C-Programs and for the use of the pre-test function in advance.

Kind regards,
Frieder