Question Authors' Forum

Below is a combinator template for a C# "write-a-program" question type that attempts to compile and run all testcases in a single Jobe job. The "Allow multiple stdins" checkbox needs to be set for this to work. It should be straightforward to modify it for Java. It hasn't had much testing, but it should be enough to get you going.

Note that if anything fails - compile errors, run errors or timeouts - the CodeRunner framework will still fall back to compiling and running each test case separately. So this approach is only a winner if most submissions run OK and/or there are many test cases. It is possible to prevent that fallback only by using a combinator template grader, which is much harder.

I attach an export of the question type plus a single test question. Template debugging is turned on in the test question so you can see the actual runs.

Richard

""" The template for a question type that compiles and runs a student-submitted
    mono C# program using a combinator template that tries to run all
    test cases in a single compile-and-execute Jobe run.
    This assumes a "write a program" type of question, with no test code to
    be incorporated. The "Allow multiple stdins" checkbox needs to be set.
"""

import subprocess, sys

# Write the student code to a file prog.cs
student_answer = """{{ STUDENT_ANSWER | e('py') }}"""
with open("prog.cs", "w") as src:
    print(student_answer, file=src)

# Compile
return_code = subprocess.call(['mcs', 'prog.cs'])
if return_code != 0:
    print("** Compilation failed. Testing aborted **", file=sys.stderr)

{% if not IS_PRECHECK %}
else:
    # If compile succeeded and it's not a precheck run all the tests.
{% for TEST in TESTCASES %}
    testcode = """{{ TEST.testcode | e('py') }}\n"""
    standard_input = """{{ TEST.stdin | e('py') }}\n""".rstrip() + '\n'
    try:
        output = subprocess.check_output(
            ["mono", "./prog.exe"],
            input = standard_input,
            universal_newlines=True
        )
        print(output)
    except subprocess.CalledProcessError as e:
        if e.returncode > 0:
            # Ignore non-zero positive return codes
            if e.output:
                print(e.output)
        else:
            # But negative return codes are signals - abort
            if e.output:
                print(e.output, file=sys.stderr)
            if e.returncode < 0:
                print("Task failed with signal", -e.returncode, file=sys.stderr)
            print("** Further testing aborted **", file=sys.stderr)
            sys.exit(1);
{% if not loop.last %}
    print('#<ab@17943918#@>#');  # Testcase separator
{% endif %}
{% endfor %}
{% endif %}