I'm working on a new question type using OpenSCAD. I'd like to ask students to create a 3D object using OpenSCAD and have them be able to submit any solution that successfully recreates the object. Here's an example:

Below are two possible and equally valid solutions.

Solution A:

translate([0,10,0]) cube([10,20,10]); translate([10,0,0]) cube([10,20,10]); translate([20,10,0]) cube([10,10,10]);

Solution B:

translate([10,0,0]) cube([10,10,10]); translate([0,10,0]) cube([30,10,10]); translate([0,20,0]) cube([10,10,10]);

I hope to do this by first verifying the student's solution is valid code (test 1), then subtracting the student solution from a correct solution and looking for a "Current top level object is empty." error (test 2), and then subtracting the correct solution from the student solution and again looking for a "Current top level object is empty." (test 3). If I receive those two errors, then I can conclude that the shapes are a perfect match.

I've done this by creating an OpenSCAD via python question prototype. Using

subprocess.call(["/home/pi/.local/bin/openscad","-o","prog.stl","prog.scad"])

I am able to perform test 1. If the output is 0, then the code is valid.

Using

student_answer = """difference() {\n {{ STUDENT_ANSWER }}\n{{ QUESTION.answer }} }""" with open("prog.scad", "w") as src: print(student_answer, file=src) test2 = subprocess.call(["/home/pi/.local/bin/openscad","-o","prog.stl","prog.scad"])

I am able to perform test 2. If the output is 1 then there was an error, which is what I want in this case. I use similar code to perform test 3, again looking for an error. I created a dummy question that shows the Expected and Got boxes are the same, except the Got box includes the error messages, which I think is causing the answer to be graded as incorrect.

I'm not sure how to solve this problem. Any help would be greatly appreciated. I've attached the question prototype and the dummy question for your reference.