Java method question: complex test case

Java method question: complex test case

by Antoni Oliver -
Number of replies: 4

Hello.

So far, CodeRunner seems to work perfectly for my needs.

Right now, I am writing some exercises in which students are asked to write a method to solve a task.

Let's use as an example a method isA that accepts a char as a parameter and returns if it was the letter a.

Similarly to the c_function example, I can write several test cases like these:

Test case 1: System.out.println(isA('a'));
Standard Input: nothing
Expected output: true
Extra: nothing

Test case 2: System.out.println(isA('A'));
Standard Input: nothing
Expected output: true
Extra: nothing

Test case 3: System.out.println(isA('b'));
Standard Input: nothing
Expected output: false
Extra: nothing

Is it advisable to do something more complex like this? (As far as I know, this might work)

Test case 4:

for (char c = 0; c < 256; c++) {
  boolean expected = (c == 'a' || c == 'A');
  boolean got = isA(c);
  if (expected != got) {
    System.err.printf("Expected isA(%c) = %b, but got %b.%n", c, expected, got);
    break;
  }
}

Standard Input: nothing
Expected output: nothing
Extra: nothing

If that is not a terrible idea, is there a way to place a correct method to call to get the expected value? It would not be necessary for this silly example, but more complex problems would benefit from that. I tried the fields Extra and Global extra and it could not find the method I wrote there.

Thanks!

In reply to Antoni Oliver

Re: Java method question: complex test case

by Matthew Toohey -

Hi Antoni

I certainly don't think this is a terrible idea. In fact, at the University of Canterbury we often have advanced test cases such as the one you describe.

You are right in thinking to use the extra and global extra fields. Unfortunately the java_method question type that comes with coderunner does not make use of these fields. To use them you will need to customize the question. I have attached a customized version of the question (in moodle xml format) you describe with two different ways of implementing the 'Test case 4' you describe.

The first way uses 'Global extra' to store a method for computing the correct answer.

The second way uses the 'extra' field of the test case to run the test code behind the scenes where the student cannot see it. This way the student is only told if they failed a particular test and cannot see any detail that might give away how to solve the question.

If you think this behavior is something you would want on other questions (which I suspect you might). You could consider making your own custom question prototype that uses the customization in my attached question.  


Kind regards,

Matthew

In reply to Matthew Toohey

Re: Java method question: complex test case

by Antoni Oliver -
Thank you!

That was awesome.

Just a quick question: in test 4, the name of the "extra method" would be shown to the student.

If they know that name, they can perfectly code their answer as this, right?

boolean isA(char c) {
  return correct_answer(c); // They would call the "extra method"
}

So... I guess it is better to stick with the test 5 method.

Is there a way to prevent this, though?

PS: I see that test 5 does not use correct_answer. I guess that this was a mistake and you mean to use it.

In reply to Antoni Oliver

Re: Java method question: complex test case

by Paul Powell -

You could randomly generate a function name...

In reply to Antoni Oliver

Re: Java method question: complex test case

by Matthew Toohey -

Hi Antoni 


Glad you found that helpful.


Yes, I believe you are right about test 4. I do not have much experience with righting java questions and did not think of that. Often our question types have extra parameters that allow us to ban the use of certain functions or strings in the students code. So with that sort of approach you could ban the use of the funciton 'correct_answer' by the student. Although I do not currently have a java question type that implements this behavior.


I intentionally didn't use correct_answer in test case 5 because I wanted to show it as an entirely separate approach. If you did not use test case 4 then there would not really be the need. I would only use global extra if there is code that needs to be shared across multiple test cases and as you point out there is a risk that the student could discover and use it (although unlikely if they can't see the function names).


Another thing that I'm sure what be possible would be to restructure the template code so that the tests are run in a separate class. That way the functions defined in global extra could be defined to be private and only callable by the test code and not the students code.


Cheers,

Matthew