CodeRunner: scoring query

Hi Richard,

I'm trying to figure out how questions are scored based on exported output from a question that has 0 marks penalty per try, and 12 test cases. Tests 1-11 are worth 1 mark each, and case 12 is worth 2 marks. Tests 1 and 2 are marked 'both' Precheck and Check.

Looking at the scoring I see this for tests passed:

6/13 (1 mark each) scored 0.5, not .4615

7/13 (1 mark each) scored 0.58333, not .5384

8/13 (1 mark each) scored 0.666667 not .6154

So it seems clear that it is being marked out of 12.

What might I have missed?

Thanks!

Re: scoring query

de Richard Lobb - viernes, 17 de junio de 2022, 22:17

Hi Anton

AFAIK the partial grading system works fine. I just opened up a question with 5 test cases, 4 worth 1 mark and one worth 2 marks for a total mark of 6. I then submitted an answer that passed 2 of the 1 mark tests and it got graded as 0.33. I then changed the answer to pass another of the 1 mark tests as well and it got 0.5. All as expected.

However, I can't explain your results, unless the students attempted the quiz before you changed the mark for the last question to 2 marks, from 1? It might be worth regrading one of the submissions just to check?

If you can send me a question plus a set of steps to replicate the problem, I'll certainly be keen to have a look at it.

Re: scoring query

de Anton Dil - sábado, 18 de junio de 2022, 03:28

Hi Richard, Thank you for the reply - I wondered about this too. It's possible.
So if someone did the test before one test moved from 1->2 marks, they'd be marked out of 12? And those after out of 13? Maybe we'll get question bank versioning one of these days...

It's not important from a scoring point of view, just from an understanding point of view - the marks don't count for this quiz.

Not sure how to regrade though I looked at the date in the QBank and it is edited recently, unclear how, after most students did the quiz - 2 did it after the qb edit date, and they seem to be marked out of 13. So this might explain it!