I'm relatively new to using coderunner, although I have had good success with a programming course in C++ this semester.
I have now come to the point where I wish to test for input and output. This is not a problem if I want to check the output to standard output using cout (I can use the support file feature for the input).
My question is: Is there a way to check the student output if I require them to write to a file?
If you're setting "write a function" type questions you can put code in the template to dump or inspect the output file after the student's code has been run. But it sounds like you're asking "write a program" questions, and that's a bit harder.
My normal approach in trickier cases like this is to use a Python question in which the Ace editor is set for C++ but the execution language is python3. The template is then a Python3 script that compiles and executes the student code and then (in your case) either dumps the student output file to stdout or runs a file comparator of some sort. It might even just run Linux diff. Something like the following:
import subprocess COMPILE = "g++ -Wall -Werror -o program program.cpp".split() RUN = "./program" f = open("program.cpp", "w") f.write(__student_answer__) f.close() try: subprocess.check_output(COMPILE, stderr=subprocess.STDOUT, universal_newlines=True) except subprocess.CalledProcessError as e: print("Error compiling program", file=sys.stderr) print(e.output, file=sys.stderr) else: try: subprocess.check_call(RUN, stderr=subprocess.STDOUT, universal_newlines=True) # Code to check the output file goes here. The simplest version might # be something like: print("After running program, output file blah.txt was:") print(open("blah.txt").read()) except subprocess.CalledProcessError as e: print("Error running program", file=sys.stderr) print(e.output, file=sys.stderr)
A variant on this might be to change the RUN command to something like "./program; diff expected.txt blah.txt" and run the subprocess in the shell (with the shell=True argument to check_call). [Caveat: I've not actually tried doing this myself.]
You could make this into a more generic question type by having a template parameter like POST_RUN, which defines what happens after the student's code has been run.
One slight disadvantage of this approach is that compile errors show up as Runtime Errors, because any output to stderr within a run is interpreted as a run error. But in practice this doesn't seem to cause much confusion as the compile errors are obvious (and, in the mode in which we run, shouldn't occur anyway, as students are expected to compile and test their code before submitting it).