CodeRunner: How to assess a Python program that accepts command line arguments

I'm trying to create a CodeRunner quiz to assess a write-a-python-program assignment in which the students are asked to pass a number to the program through the command line, compute the reciprocal of that number, and handle any possible exceptions (such as ZeroDivisionError). A simplified solution would be as follows:

import sys
try:
    number = float(sys.argv[1])
    inv = 1/number
    print(f'The reciprocal of {number} is {inv}')
except Exception as e:
    print(e)

And these are some of the tests I intend to perform:

$ python reciprocal.py
list index out of range
$ python reciprocal.py 0
division by zero
$ python reciprocal.py 5
The reciprocal of 5 is 0.2

I'm struggling to come up with a template that suits my needs. This is what I tried so far (inspired on this post):

#!/usr/bin/env python
import subprocess, sys

student_answer = """{{ STUDENT_ANSWER | e('py') }}"""
with open("reciprocal.py", "w") as src:
    print(student_answer, file=src)

SEPARATOR = "#<ab@17943918#@>#"

{% for TEST in TESTCASES %}
bash_command = """{{ TEST.testcode | e('py') }}"""
output = subprocess.check_output(bash_command.split())
print(output)
{% if not loop.last %}
print(SEPARATOR)
{% endif %}
{% endfor %}

But I'm getting this error:

***Error***
Traceback (most recent call last):
  File "__tester__.python3", line 15, in 
    output = subprocess.check_output(bash_command.split())
  File "/usr/lib/python3.6/subprocess.py", line 336, in check_output
    **kwargs).stdout
  File "/usr/lib/python3.6/subprocess.py", line 403, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/usr/lib/python3.6/subprocess.py", line 709, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.6/subprocess.py", line 1344, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'python': 'python'

Any help would be greatly appreciated.

Re: How to assess a Python program that accepts command line arguments

por Richard Lobb - martes, 14 de mayo de 2019, 09:37

Thanks for a very clear statement of the situation.

Your question more-or-less works on my machine, although I needed to add the parameter universal_newlines=False to the call to subprocess to avoid binary output. I also had to change the tests to use python3 (see below). But I certainly never got a 'python not found' exception.

That error means that there's no executable command 'python' to be found on your Jobe server, at least not within the set of directories specified by the current PATH environment variable. That's a bit surprising. On my Jobe servers, python is at /usr/bin/python, although that runs python2, not python3.

Clearly you do have a python on your system, since it's running the template. However, the template is running using python3 not python2. Is it possible you have a Jobe server with only python3 on it, and no python2? If that's the case your problem may be solved simply by changing your question's tests to use the command python3 rather than just python.

Otherwise, I think you'll have to explore a bit more. Perhaps the best way to figure out what's going on is to temporarily change your tests to run various bash commands instead of python commands, e.g.

bash echo $PATH
bash which python
bash which python3

Richard

Re: How to assess a Python program that accepts command line arguments

por Antonio Fernández Álvarez - martes, 11 de junio de 2019, 01:54

Thank you for your prompt reply.

You were right. I changed the command python to python3 in the template and everything worked like a charm.

Here is the workaround I came up with, just in case it would be useful to someone else:

import subprocess

student_answer = """{{ STUDENT_ANSWER | e('py') }}"""
with open("reciprocal.py", "w") as src:
    print(student_answer, file=src)

SEPARATOR = "##"

{% for TEST in TESTCASES %}
try:
    bash_command = """{{ TEST.testcode }}""".replace('python', 'python3')
    output = subprocess.check_output(bash_command, shell=True, universal_newlines=True)
    print(output)
except subprocess.CalledProcessError as e:
    if e.returncode > 0:
        # Ignore non-zero positive return codes
        if e.output:
            print(e.output)
    else:
        # But negative return codes are signals - abort
        if e.output:
            print(e.output, file=sys.stderr)
        if e.returncode < 0:
            print("Task failed with signal", -e.returncode, file=sys.stderr)
        print("** Further testing aborted **", file=sys.stderr)
{% if not loop.last %}
print(SEPARATOR)
{% endif %}
{% endfor %}

While trying to fix the issue I followed your advise and explored a bit. Concretely I embedded this snippet into the question's answer:

import socket
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.connect(("8.8.8.8", 80))
print(s.getsockname()[0])
s.close()
print(socket.gethostname())

And this is the output I got:

132.181.16.63
csse-jobe2

Researching a bit more, I found out that those data correspond to a server located at the University of Canterbury, New Zealand (the antipodes of Spain, the country I'm posting from).

At first I was quite surprised, but then I found this Stack Overflow thread and everything became crystall clear.

PS: CodeRunner is awesome! When you try CodeRunner for the first time you can not imagine how to teach programming without it. Thank you Richard for having developed such amazing software.

Re: How to assess a Python program that accepts command line arguments

por Richard Lobb - martes, 11 de junio de 2019, 11:40

Good to know it's working for you Antonio and thank you for the kind words.

I was amused at your detective work to discover you were using our jobe2 server. It sounds like whoever installed the CodeRunner plugin didn't realise they should install a local jobe server. jobe2 is just a lightweight server for helping people to get up and running. It has a limit on submissions per hour so if you try to use it in a classroom context you'll likely find it intermittently blocks you (and everyone else on the planet!) for an hour or so when its traffic limit is reached. More information on setting up your own Jobe, including links to a video and also to a Docker image, are available here.

Best of luck with your ongoing use of CodeRunner.

Richard