Question Authors' Forum

Hi Richard,

Thank you for the speedy reply and useful example. I seem to have a per-test template working, although I am unsure about the best values for the various compiler and VM flags. I'm thinking of ~1000 students trying to run tests in an exam at once. So far, so good :-)

Regards,
Anton

I've been experimenting with the combinator and have it working up to a point, that is, when the answer is okay.

I'd like it to report just once if students' code doesn't compile, but I can't get this aspect working. I keep getting something similar to "Error in question Expected 3 test results, got 1. Perhaps excessive output or error in question?" when the answer doesn't compile.

I have been trying to compile the student's code separtely from the __tester__ class before compiling both together, but it doesn't seem to get that far in the template anyway.

Here's the template I am using (with some debugging output in it). When the student code doesn't compile I don't see that debugging output.

I could also compile the answer box code as a string, but I wasn't hoping to go that far, yet. Anyway t feels like something else is going on before the template is run.

import re
import subprocess
import sys

# 1. Get student answer and strip 'public' from class/interface/record
student_answer = """{{ STUDENT_ANSWER | e('py') }}"""

student_answer = re.sub(
r'\bpublic\s+(abstract\s+class|class|interface|record)\b',
r'\1',
student_answer
)

# 2. Extract classname from student code
match = re.search(
r'\b(?:abstract\s+)?(class|interface|record)\s+([_a-zA-Z][_a-zA-Z0-9]*)\b',
student_answer
)

if not match:
print("The answer didn't contain a class, interface or record.")
sys.exit(1)

classname = match.group(2)

# 3. Write student's Java code to file
with open(f"{classname}.java", "w") as f:
f.write(student_answer)

# 5. Compile student + tester files
cargs = ["javac21",
"-J-Xss512k",
"-J-Xmx64m",
"-J-XX:CompressedClassSpaceSize=32m"]

compile_cmd1 = cargs + [
f"{classname}.java"
]

comp = subprocess.run(compile_cmd1, capture_output=True, text=True)

if comp.returncode != 0: # first try compile by itself without template
print(comp.stderr)
sys.exit(2)
else:
print("Student code compiled")

# 4. Only now write __tester__.java using Twig loop syntax
with open("__tester__.java", "w") as f:

f.write("""
public class __tester__ {

private static final String SEPARATOR = "##";

public static void main(String[] args) {
__tester__ main = new __tester__();
main.runTests();
}

public void runTests() {
try {
{% for TEST in TESTCASES %}
{
{{ TEST.testcode }}
{% if not loop.last %}
System.out.print("##");
{% endif %}
}
{% endfor %}
}
catch (Throwable e) {
System.out.println("Exception: " + e);
}
}
}
""")

compile_cmd2 = cargs + [
f"{classname}.java",
"__tester__.java"
]

comp = subprocess.run(compile_cmd2, capture_output=True, text=True)

if comp.returncode != 0:
print(comp.stderr)
sys.exit(2)

# 6. Run compiled Java program
run_cmd = [
"java21",
"-Xms1m",
"-Xmx64m",
"-Xss512k",
"-XX:CompressedClassSpaceSize=32m",
"__tester__"
]

run = subprocess.run(run_cmd, capture_output=True, text=True)

print(run.stdout)
if run.returncode != 0:
print(run.stderr, file=sys.stderr)
sys.exit(3)