Question Authors' Forum

cpp_class tests and customized test cases

cpp_class tests and customized test cases

by Alex Gunagwera -
Number of replies: 2

Hello, I am relatively new to this amazing tool. So far the inbuilt cpp_function and cpp_program have been sufficient. However, I need something slightly different. At the moment I wish to know two things:

  1. Defining the expect output as a range instead of an exact value. Lets say the program output is 87, I want to define the expected output in the test cases as anything between [84-90] and accept all values within this range i.e. accept if the program output is one of 84, 85, 86, 87, 88, 89 or 90. How can I achieve that? 
  2. Instead of setting up individual function tests, what If i want to test all methods in cpp_program that may include more than two functions. How most efficiently can I achieve this? Do I have to bite the bullet and set up tests for each function as a separate question or is there an easier way? Language in context is cpp. 

Thank you very much.

In reply to Alex Gunagwera

Re: cpp_class tests and customized test cases

by Richard Lobb -
1. Accepting variations in output

If you are using a cpp_function question type, then you can easily check for output variations with test code like
int result = student_function(...);
if (result >= 84 && result <= 90) {
    cout << "Correct!\n";
} else {
    cout << "Answer should be in the range 84 to 90 incl.\n";

where the expected output from the test is "Correct!".

However, with a cpp_program question, it's harder. One way is to click the Customise box and select Regular Expression instead of Exact match for the grading. Then you could set the Expected output to be, say,


The problem there, though, is that unless you've taught students regular expressions, they'll be baffled by the result table. When I use regular expression grading (which I do only rarely) I generally hide the Expected column of the result table and put a comment in the Test column (which is otherwise unused with this question type) to explain what is being tested for.

To do the job properly, you really need to capture the output from the program and analyse it with test code (usually in Python). This is far more flexible than regular expressions and allows you to generate helpful error messages when the output is incorrect. But it requires the use of a template grader which is a big step up in complexity. You need to define a Python template that compiles and runs the student supplied code in a subprocess (collecting the output rather than directly printing it), analyses the output and prints a JSON string that defines the mark to award and the feedback to the student. The section on Implementing new languages shows how to compile and run a C program from Python. You could tweak that to do C++ instead, switch it to use a template grader - see here - and then rather than simply printing the student output you could grade it, define the feedback, and print the required JSON string. It's probably too big a step if you're a relative newcomer to CodeRunner but I could (should?) document the process properly if you're sufficiently keen.

2. Marking both the program output and the behaviour of internal functions

No, there's really no way to grade the functions within a C++ program independently of the behaviour of the program as a whole. Having a series of questions that ask students to develop required support functions before assembling them into a final program is how we usually operate. At least in introductory programming courses this is a good pedagogical strategy ("scaffolding") that helps weaker students progress and at least get partial marks, which are otherwise hard to award.

For more advanced-level courses there are ploys you might use to investigate the internals of a program (e.g. using a profiler) but I doubt they're worth the effort.

In reply to Richard Lobb

Re: cpp_class tests and customized test cases

by Alex Gunagwera -
Thank you very much Richard, this answers my question.