Get full points if only one test is successful?

Get full points if only one test is successful?

by Steffen Trutz -
Number of replies: 3

Dear Richard, hello everybody,

I finally managed to try out that awesome plugin. Thank you Richard for all the work and great documentation.
There is now a point where I get stuck.
In one question (SQL) I have 3 test cases. Currently to get full points, all 3 tests must be successful. But I want that only one has to be successful, for full points. How/where can I set this?


Thanks a lot and stay healthy.
Greetings Steffen

In reply to Steffen Trutz

Re: Get full points if only one test is successful?

by Richard Lobb -
I'm a bit curious to know why you want to give a student full marks for a bit of code that fails 2 out of 3 tests?! That's the opposite of the normal CodeRunner approach, which requires that all tests must pass for the student to get any marks at all (but with the option for a small penalty to resubmit with corrected code).

Out of the box, CodeRunner lets you turn off All-or-nothing grading and allocate marks to each test case separately but there's no way in the standard question types to give full marks when only one test passes. To achieve that you need a custom combinator template grader, which is a big step up in complexity. If you're new to CodeRunner I suggest trying to get by with what's built in for a while.

If you really need this functionality urgently, though, post back and I can give further pointers.
In reply to Richard Lobb

Re: Get full points if only one test is successful?

by Steffen Trutz -

Dear Richard,
thanks for the answer. Maybe i am using the coderunner at the moment wrong. I give you an example.

Input
 Output
SELECT sum(pop) ...
 sum(pop)
-------
234567
SELECT SUM (pop) ... SUM (pop)
-------
234567
SELECT SUM(pop) ... SUM(pop)
-------
234567

At the moment i can only check one (right) solution, that why i ask for different right answers.
I have now solved it so that the output is always displayed as an example so that you can recognize the upper & lower case and spaces etc.

Maybe we will find in some months for the next classes a better solution :)
Greetings Steffen

In reply to Steffen Trutz

Re: Get full points if only one test is successful?

by Richard Lobb -
Thanks for clarifying the problem Steffen. I've never really taught SQL using CodeRunner and your example is interesting.

The general intent with CodeRunner in all languages is to assess whether code produces the right output, without regard to how it does it - objective rather than subjective grading. I also believe students should be given the opportunity to correct any errors, as you can in real-life programming - authentic assessment. In our courses we do have strict style guidelines, enforced by style checkers, but style checking is free and students must correct their code to conform to the style guidelines before it is is even accepted for grading.

Your example touches on all these issues:
  • The code has leaked through into the output in the form of column headers. A solution might be to specify the column headers specifically, e.g. as Total with a capital T. The student would then have to write SELECT SUM(pop) as Total and whether they chose to use capitals or not, with spaces or not, would be irrelevant.
  • Alternatively, you might have a style requirement, such as requiring all SQL keywords and function names to be in all-caps and without spaces following function names. Then the column header would have to be SUM(pop) and the other answers would be wrong.
  • I would hope that students can in any case resubmit their answer for a small penalty if their first attempt is wrong. For formative assessment we are currently using no penalties and even for summative assessment we allow at least on zero-penalty wrong submission, e.g. a penalty regime of 0, 10, 20, ...

Hope that helps and that you find CodeRunner useful.

Richard