Question Authors' Forum

If you have a write-a-function question that returns a string then you can easily write the test case to convert the returned string to lower case before printing it, and use a near-exact-match grader. However, if the function does the printing itself, or you're using a write-a-program question, it's much harder.

 

The only way to achieve what you want with an out-of-the-box question type is to use a regular expression grader, where the Expected field is a regular expression rather than literal text. But if you will allow upper or lower case everywhere you'll have a real mess, like "\A *[hH][eE][lL][[lL][oO] +[wW][oO][rR][lL][dD] *\Z" for matching Hello world case insensitively with arbitrary white space. Students won't understand the expected output, so you'd probably have to hide the Expected field in the result table.

 

Really, you need to use a custom grader, preferably defining your own question type. The grader has to run the student code in a subprocess, collect the output, convert to lower case, collapse multiple white space to a single space, perhaps trim leading and trailing white space (your call), then compare to the expected output, similarly transformed. This is probably rather a tall order for a newcomer. But it does make for a reasonably useful tutorial example. So ...

 

1. Create a new C++ write-a-program question
2. Set whatever test cases and answer you would normally use in a C++ write a program question
3. Click customise
4. Select Template grader from the Grading dropdown.
5. Open Advanced customisation and:
   • Set Sandbox language to python3 
   • Set Ace language to cpp
6. Set the template to the code below.
7. Click Save

""" The template grader for a question type that compiles and runs a student-submitted
    C++ program and checks the output matches the expected case-insensitively
    with arbitrary amounts of white space. Trailing white space is stripped.
    Extra white space at the start is an error (unless a non-zero amount is expected).
"""

import subprocess, json, re

def accepted(expected, got):
    """Return true if the output we got is deemed a match to the expected output"""
    expected_simplified = re.sub('\s+', ' ', expected.lower().rstrip())
    got_simplified = re.sub('\s+', ' ', got.lower().rstrip())
    return expected_simplified == got_simplified

# Write the student code to a file prog.cpp
student_answer = """{{ STUDENT_ANSWER | e('py') }}"""
with open("prog.cpp", "w") as src:
    print(student_answer, file=src)

# Assume the worst
fraction = 0
abort = True
    
# Compile
compile = subprocess.run("g++ -o prog prog.cpp".split(), capture_output=True, text=True)
if compile.returncode != 0:
    got = "** Compilation failed. Testing aborted **\n" + compile.stderr

# If compile succeeded, run the code. Since this is a per-test template,
# stdin is already set up for the stdin text specified in the test case,
# so we can run the compiled program directly.
else:
    run = subprocess.run(["./prog"], capture_output=True, text=True)
    if run.returncode < 0:
        got = f"Task failed with signal {-run.returncode}\n" + run.stderr + run.stdout
    else:
        got = run.stdout
        expected = """{{ TEST.expected | e('py') }}"""
        fraction = 1 if accepted(expected, got) else 0
        abort = False
        
outcome = {"got": got, "fraction": fraction, "abort": abort}
print(json.dumps(outcome))

 

That should do what you want. 

 

More generally you'd want to create a new question type for this. I attach a prototype for such a question which defines a type case_insens_cpp_prog plus an example question of that type, namely a Hello world program that will accept output like hello  \n World\n. You should be able to import these and be good to go.

 

But caveat: I just threw this together to answer your question. It has had next to zero testing and is not official coderunner code. Please post back with any corrections, issues, etc.