Question Authors' Forum

Show (run) testcases until one of them fails?

 
Picture of Sergi García
Show (run) testcases until one of them fails?
by Sergi García - Tuesday, 5 December 2017, 7:10 AM
 

Hi, I want to ask a question.

Is it possible to show (run) testcases until one of them fails?

I want to imitate behaviour of USACO Training gateway http://train.usaco.org/usacogate


For example, we have test case A, B, C, D, E

If A and B are right and C is wrong, it stops and only show (run) test cases of A, B and C. D and D didn't (or at least hidden).

Is it possible?

Regards, Sergi

Picture of Richard Lobb
Re: Show (run) testcases until one of them fails?
by Richard Lobb - Tuesday, 5 December 2017, 11:44 AM
 

Sure. Just set each test case's Display value to Show but in addition check Hide rest if fail.

Richard

Picture of Sergi García
Re: Show (run) testcases until one of them fails?
by Sergi García - Thursday, 7 December 2017, 11:16 PM
 
Ohh I didnt watch that option, it is my fault.
Thanks Richard!!!
it is a way to use that in order to avoid running more test cases? (Instead of hide them)

Regards, Sergi.
Picture of Richard Lobb
Re: Show (run) testcases until one of them fails?
by Richard Lobb - Friday, 8 December 2017, 11:26 AM
 

No, the "Hide Rest If Fail" feature just suppresses the display of subsequent cases to the student. The rest still get run, and are displayed to staff (shaded darker).

If you want a "lazy evaluation" mode you have to implement it yourself with a template grader. For non-combinator graders you can set the abort attribute of the returned JSON record (see http://coderunner.org.nz/mod/book/view.php?id=193&chapterid=738). With a combinator grader you can return as many or as few test results as you like, so you simply stop testing on the first failing case.

Richard

Picture of Sergi García
Re: Show (run) testcases until one of them fails?
by Sergi García - Saturday, 9 December 2017, 12:22 AM
 

Thanks for your quick answer Richard!!

I was thinking in problems that not use combinator (for performance reasons). But I see the best way is desing problems that can use the power of combinator :)

Sergi.