Evaluate partial answer with a test case

Evaluate partial answer with a test case

by Uthpalani Kavisekara -
Number of replies: 1

When a student's answer is partially correct, I need to give a half mark for the answer (this is a MySQL question, and I'm using the sample "customers" table for this).

The expected answer is,

select name, city, grade
from customers
where city = 'London' or city = 'Paris';

and would like to give half marks for the following answer,

select name, city, grade
from customers
where city = 'London';

I tried the steps below to achieve this.

  1. Uncheck the "All-or-nothing grading" in "Marking" under question type section
  2. Create the following test case (test case 2)



But when I try to save it following error occurs...


In reply to Uthpalani Kavisekara

Re: Evaluate partial answer with a test case

by Richard Lobb -
The template for the question (which you can see if you click Customise) includes the following lines (in Python):

testcode = """{{ TEST.testcode | e('py') }}"""
extra = """{{ TEST.extra | e('py') }}"""
code_to_run = '\n'.join([prelude, extra, student_answer, testcode])
So the SQL test you're running consists of your hidden test code (in the extra field) followed by the student answer. That's two successive queries, and hence you get two successive query outputs, the first with just the two London rows and the second with the addition of the Paris row.

It's not actually all that easy to achieve what you want, as you're trying to give half marks for a wrong answer without penalising a student who has the right answer (which won't match that wrong answer).

One approach would be to Customise the question, then select use Regular expression grading rather than Exact match. The second test could for example try to match just one of the London lines or perhaps one of the Paris lines. To find more about regular expression grading, click the question mark beside the grading drop down. 

More elaborate regular expression graders could match either two London lines or one Paris line. 

However, regular expressions appear in the Expected column and will confuse the students unless you explain carefully in advance what's going on. They also tend to confuse question authors too :-(. Please read the documentation carefully.

Better grading strategies can be implemented by writing your own custom template graders but that's a big step up in complexity.

As a final comment: my preferred strategy is give marks only to exactly correct answers and do the grading via the penalty regime rather than trying to give partial marks to some arbitrary subset of the infinity of wrong answers.