Running python script to launch terminal command (MIPS code)

Running python script to launch terminal command (MIPS code)

by Asset Berdibek -
Number of replies: 6

Dear all, 

I've been trying to set up a MIPS code run through a Python3 code. So, students write a MIPS code, and they should be able to check it, using the CHECK button. The CHECK button must run a python script, which launches a java process using the Mars jar file to run and give the output of their MIPS code.

I tried it using the following settings:


Question type: Python3, and I also tried my own prototype

Customisation: on

Template:

import subprocess
import sys
#__student_answer__ = """{{ STUDENT_ANSWER | e('py') }}"""
with open("test.asm", "w") as f:
    f.write("""{{ STUDENT_ANSWER }}""")

command = ["java", "-jar", "Mars4_5.jar", "test.asm", "nc"]

try:
    result = subprocess.run(command, capture_output=True, text=True, shell=True)
    print("Standard Output:")
    print(result.stdout)
    # print(result.stdout.decode())
    if result.stderr:
        # print("ERROR:", result.stderr.decode(), file=sys.stderr)
        print("ERROR:", result.stderr)

except Exception as e:
    print("Runtime Error:", str(e), file=sys.stderr)
{% for TEST in TESTCASES %}
{{ TEST.testcode }}
{% if not loop.last %}
print(SEPARATOR)
{% endif %}
{% endfor %}

Grading: I tried "Exact match" and "Template grading"

Student answer: Ace; Template uses ace: on

The question is simple - create a hello world code in MIPS

Answer: 

.data
msg: .asciiz "Hello, World!"

.text
main:
    li $v0, 4
    la $a0, msg
    syscall

    li $v0, 10
    syscall

Now, whenever I try to save the question, it gives me "Incorrect syntax" error, which makes sense I guess, since it thinks students must write a python code. How can I tell it to ignore the language and just to run the Python script in the template and check the output of the terminal command run with java and including a Mars4_5.jar file?
In reply to Asset Berdibek

Re: Running python script to launch terminal command (MIPS code)

by Richard Lobb -

You're nearly there. There's one major issue, plus a few other potential issues.

  • The syntax error you're getting isn't because the question thinks students must write python code. It's because the program generated by that template isn't valid Python. To confirm that, I suggest you click the Template debugging checkbox, turn off validate on save, then save and run the question. 

You'll find that the line

#__student_answer__ = """{{ STUDENT_ANSWER | e('py') }}""" 

causes problems. This is a trap I've fallen into myself before. It looks like you've commented out a single line of Python, but because the student answer is a multiline string, you've actually only commented out the first line, and the rest of the student answer (which isn't Python) has then landed up embedded in the Python code. You should delete that line. That should fix your main problem.

  • In your f.write() statement you should escape the student answer using the syntax in the line you've commented out. You probably get away without escaping in most cases but if, say, the student answer ends with a double quote, you'd get a Python syntax error because it would merge with the terminating triple-double-quote and you'd be left with a hanging double quote. There may be other fail cases as well, though I can't think of any except if the student answer itself contains triple quotes.
  • In general you should write stderr output to the Python stderr stream, not to stdout. CodeRunner will take stderr output to indicate a run-time error and abort further testing. That's generally what you want if you have multiple test cases.
  • I'm not sure what your tests are, but if there's any testcode, it's probably not Python. So you don't want to expand that out into the Python template. Although if it's empty, and there's only one test with an expected field of "Hello, World!", it probably doesn't matter.
  • The grader should be Exact match in this case. Template graders need to output JSON. 
  • I assume the Combinator checkbox is checked? Otherwise the expansion of all the testcases into the template won't work.
  • You'll need to expand the Advanced customisation panel and set the student answer language to be some non-existent language (I'm assuming Ace doesn't have a syntax highlighter for MIPS assembly language). Otherwise the student answer will be syntax highlighted as Python.

See how that goes :-)

In reply to Richard Lobb

Re: Running python script to launch terminal command (MIPS code)

by Asset Berdibek -

1) I removed the commented out line and the main problem was gone. Thank you for noting that.

2) I fixed the f.write() line to f.write("""{{ STUDENT_ANSWER | e('py') }}""")

3) is Combinator check is ON. 

4) In this example, the grading is "Exact match", as we only need to compare output strings. 

5) In the advanced customisation menu I changed the Ace language to MIPS, so it doesn't highlight Python keywords

6) now, when I validate it, it gives me the output below. 

It's printing the message "Standard Output:", but after that, it's giving a segmentation fault error. I tried to remove the "shell=True" option from the subprocess.run and the segmentation fault is gone now. But still, it's not printing the "Hello, World!" message. Is it somehow related to stdout? I've read your sentence about stdout and stderr but I didn't get how it's related to this code.


import subprocess
import sys

with open("test.asm", "w") as f:
    f.write("""{{ STUDENT_ANSWER | e('py') }}""")
f.close()
command = ["java", "-jar", "Mars4_5.jar", "test.asm", "nc"]

try:
    result = subprocess.run(command, capture_output=True, text=True, shell=True)
    print("Standard Output:")
    print(result.stdout)
    # print(result.stdout.decode())
    if result.stderr:
        # print("ERROR:", result.stderr.decode(), file=sys.stderr)
        print("ERROR:", result.stderr)

except Exception as e:
    print("Runtime Error:", str(e), file=sys.stderr)
{% for TEST in TESTCASES %}
{{ TEST.testcode }}
{% if not loop.last %}
print(SEPARATOR)
{% endif %}
{% endfor %}

In reply to Asset Berdibek

Re: Running python script to launch terminal command (MIPS code)

by Richard Lobb -

Ah, yes. There's another problem I didn't think to warn you about.

Jobe sets tight limits on resource usage by individual tasks to prevent rogue tasks from overloading it. Java is very resource hungry and trying to run it from within the Python task will almost certainly run you out of memory and perhaps processes as well.

I think the best remedy is to turn off Jobe's memory limit and pass the problem of limiting memory to the Java VM. Which means setting arguments to the java command specifying resource limits. So you should first open the Advanced settings and set the Memory Limit to 0 (which disables it). Then add parameters to the Java command along the following lines.

I'm not Java user myself, but in this thread, Anton Dil has come up with the following command for running Java:

["java", "-Xss512k", "-Xmx64m", "-XX:CompressedClassSpaceSize=32m", "-Xrs", "-XX:ActiveProcessorCount=1"]

That gives a maximum stack size of 512k and a maximum heap size of 64m - the latter sounds far too tight for your needs. I'd suggesting starting with at least 500m. You'd probably want to increase the stack size by at least a factor of 10, too. I have no idea what CompressedClassSpaceSize would be appropriate for your needs - I suggest dropping that parameter. -Xrs is probably also not relevant to you unless you have large classes. You should keep the ActiveProcessorCount which I think should stop the Java VM starting a ridiculous number of worker threads. But if you do get an error message from the VM saying can't start threads, you can set a sandbox parameter to allow more processes to be started. Let's worry about that if it happens, though.

In reply to Richard Lobb

Re: Running python script to launch terminal command (MIPS code)

by Asset Berdibek -
With that, I got rid of the "Segmentation fault" mistake, but oddly, the server is giving me java usage error, saying I'm not using some of options correctly and returns a code 1.

command = ["java", "-Xss512k", "-Xmx64m", "-XX:CompressedClassSpaceSize=32m", "-Xrs", "-XX:ActiveProcessorCount=1", "-jar", "Mars4_5.jar","nc", "test.asm"]

Admin notified me that the server is using jdk 11, so I asked the admin to run this command on the server and it worked. So, I'm still foggy about the problem here. I attached the Mars4_5.jar file to the "Support files" section, I rechecked and rewrote the command, but still.
In reply to Asset Berdibek

Re: Running python script to launch terminal command (MIPS code)

by Richard Lobb -
Have you experimented to see which options are proving problematic?
In reply to Richard Lobb

Re: Running python script to launch terminal command (MIPS code)

by Asset Berdibek -
I removed the "shell=True" option from the subprocess.run method, and it worked. Even the input on mars code is working.
Thank you so much for your help!