Question Authors' Forum

How to make java program question run once for multiple test cases with standard inputs

 
Picture of Mohammad Oghli
How to make java program question run once for multiple test cases with standard inputs
by Mohammad Oghli - Thursday, 22 March 2018, 2:14 AM
 

I am trying to use template and is combinator to make my java program question run only one time for all test cases (standard inputs) so I can reduce the load on the server when many users submit there code for checking at the same time 

but I can't find any solution or references on how to do that with testCase.stdin

is there any way to change template code so I can do that.

Picture of Richard Lobb
Re: How to make java program question run once for multiple test cases with standard inputs
by Richard Lobb - Friday, 23 March 2018, 2:27 PM
 

Below is a combinator template for a C# "write-a-program" question type that attempts to compile and run all testcases in a single Jobe job. The "Allow multiple stdins" checkbox needs to be set for this to work. It should be straightforward to modify it for Java. It hasn't had much testing, but it should be enough to get you going.

Note that if anything fails - compile errors, run errors or timeouts - the CodeRunner framework will still fall back to compiling and running each test case separately. So this approach is only a winner if most submissions run OK and/or there are many test cases. It is possible to prevent that fallback only by using a combinator template grader, which is much harder.

I attach an export of the question type plus a single test question. Template debugging is turned on in the test question so you can see the actual runs.

Richard


""" The template for a question type that compiles and runs a student-submitted
    mono C# program using a combinator template that tries to run all
    test cases in a single compile-and-execute Jobe run.
    This assumes a "write a program" type of question, with no test code to
    be incorporated. The "Allow multiple stdins" checkbox needs to be set.
"""

import subprocess, sys

# Write the student code to a file prog.cs
student_answer = """{{ STUDENT_ANSWER | e('py') }}"""
with open("prog.cs", "w") as src:
    print(student_answer, file=src)

# Compile
return_code = subprocess.call(['mcs', 'prog.cs'])
if return_code != 0:
    print("** Compilation failed. Testing aborted **", file=sys.stderr)

{% if not IS_PRECHECK %}
else:
    # If compile succeeded and it's not a precheck run all the tests.
{% for TEST in TESTCASES %}
    testcode = """{{ TEST.testcode | e('py') }}\n"""
    standard_input = """{{ TEST.stdin | e('py') }}\n""".rstrip() + '\n'
    try:
        output = subprocess.check_output(
            ["mono", "./prog.exe"],
            input = standard_input,
            universal_newlines=True
        )
        print(output)
    except subprocess.CalledProcessError as e:
        if e.returncode > 0:
            # Ignore non-zero positive return codes
            if e.output:
                print(e.output)
        else:
            # But negative return codes are signals - abort
            if e.output:
                print(e.output, file=sys.stderr)
            if e.returncode < 0:
                print("Task failed with signal", -e.returncode, file=sys.stderr)
            print("** Further testing aborted **", file=sys.stderr)
            sys.exit(1);
{% if not loop.last %}
    print('#<ab@17943918#@>#');  # Testcase separator
{% endif %}
{% endfor %}
{% endif %}

Picture of Mohammad Oghli
Re: How to make java program question run once for multiple test cases with standard inputs
by Mohammad Oghli - Monday, 26 March 2018, 2:31 AM
 

I tried this template code for java program and it didn't work for me: 

import subprocess, sys
# Write the student code to a file prog.cs
student_answer = """{{ STUDENT_ANSWER | e('py') }}"""
with open("prog.java", "w") as src:
    print(student_answer, file=src)
# Compile
return_code = subprocess.call(['java', 'prog.java'])
if return_code != 0:
    print("** Compilation failed. Testing aborted **", file=sys.stderr)
{% if not IS_PRECHECK %}
{% for TEST in TESTCASES %}
else:
    testcode = """{{ TEST.testcode | e('py') }}\n"""
    standard_input = """{{ TEST.stdin | e('py') }}\n""".rstrip() + '\n'
    try:
        output = subprocess.check_output(
            ["java", "./prog.exe"],
            input = standard_input,
            universal_newlines=True
        )
        print(output)
    except subprocess.CalledProcessError as e:
        if e.returncode > 0:
            # Ignore non-zero positive return codes
            if e.output:
                print(e.output)
        else:
            # But negative return codes are signals - abort
            if e.output:
                print(e.output, file=sys.stderr)
            if e.returncode < 0:
                print("Task failed with signal", -e.returncode, file=sys.stderr)
            print("** Further testing aborted **", file=sys.stderr)
            sys.exit(1);
{% if not loop.last %}
    print('#<ab@17943918#@>#');  # Testcase separator
{% endif %}
{% endfor %}
{% endif %}


it gives this me error message when compiling it on code runner :

***Error***
Error: Could not find or load main class prog.java
** Compilation failed. Testing aborted **


Picture of Richard Lobb
Re: How to make java program question run once for multiple test cases with standard inputs
by Richard Lobb - Friday, 30 March 2018, 7:57 PM
 

The template code is written in Python so must be run with the sandbox language set to Python and the Ace language set to Java. But there are other problems in your template as well, and there are some additional complications that I didn't warn you about involved in running Java, such as naming the source file to match the main class name and setting the memory limit to 0 (leaving memory management to the JavaVM).

I attach an XML export of a working (I hope) prototype for running Java programs via Python in a single Jobe run, provided the code compiles and there are no run time errors or timeouts. It must be imported using the Moodle XML file format.

My usual warnings applies: this hasn't had much testing and certainly hasn't been used in a production environment, since I don't teach Java any more.

Note that this is only for write-a-program questions, without any test code.

Richard