A colleague has noticed that his students were getting a score of .96 despite several (>4) submissions.
It turned out that this was happening because a trailing comma in the score regime was missing. If there's no comma, then obviously it just applies the first two penalties. But it's really easy to miss! Would it be possible to highlight this issue in any way, or make progressive regimes be a bit more flexible?
Recognised as the penalties for the first two tries: '2%, 4%,' or '2% 4%'
Recognised as a progressive penalty: '2%, 4%...,', '2%, 4%...', or '2% 4% ...'
Thanks for reporting. Indeed that's a trap - I'm only surprised I haven't fallen into it myself. Or perhaps I have but I didn't notice and students unsurprisingly never complained.
I was surprised to see the '%' symbols in there, too. I would have expected those to cause an error but in fact the call to the function to convert the strings to floats just silently ignores all the stuff at the end so that never got picked up (like the trailing '...').
I've pushed an update to CodeRunner to github. As you suggested, it allows comma-separated or space-separated items, and allows a trailing '%' symbol after each number. However, it converts all the variant forms to the originally-expected generic form with a comma and a space as a separator and no percent symbols. That generic form is what you see if you then edit the question again.
This does mean the display of the penalty regime to your students will look slightly different. You would have had
(penalty regime: 0%, 10%, 20%, ... %)
but will now see
(penalty regime: 0, 10, 20, ... %)
which is what I see and always expected others would see. The percent on the end is added by my code when displaying the penalty regime - it's not valid input.
Non-progressive penalty regimes display as
(penalty regime: 0, 10%)
or, with only one penalty (the only common non-progressive case?)
(penalty regime: 0%)Again the final '%' is added by me.
I hope you don't mind the change. It will apply only to new or newly-edited questions, not to your existing ones.
Thanks for the very quick response, Richard!
That sounds great - thanks very much :-) We probably won't be able to grab the code for a couple of months for our main server, but I'll check it out on one of our test servers soon.