Pre-processing available?

Pre-processing available?

by Stefan Mueller -
Number of replies: 5

First off, thanks for making CodeRunner available! I am about to set it to use for an introductory Python course.

The problem is that the coding platform in use (TigerJython) extends regular Python by a repeat statement which can be used in either of the following forms: repeat n: or repeat:

The first is equivalent to for i in range(n): or an equivalent while-loop, the second to while True:

In order to convert the students answers into regular Python, I would some sort of pre-processing for literally replacing the repeat-statements.

Is any such thing available or does anybody see a simpler solution?

In reply to Stefan Mueller

Re: Pre-processing available?

by Richard Lobb -

The safest approach would be to install TigerJython on the Jobe server, then create a new question type to use it, along the lines described here. That guarantees you complete compatibility with TigerJython and any syntactic quirks. However, this assumes you can run a TigerJython job directly from the shell rather than through a GUI.

It is certainly possible (and much easier) to use a pre-process approach, as you describe. You could create a new question type (see here) with a template like the following:

import re
answer = """{{ STUDENT_ANSWER | e('py') }}"""
answer = answer.replace("repeat:", "while True:")
answer = re.sub("repeat (.+) *:", r"for i in range(\1): ", answer)

{% for TEST in TESTCASES %}
answer += "\n" + """{{ TEST.testcode | e('py')}}\n"""
{% if not loop.last %}
answer += 'print("#<ab@17943918#@>#")\n'
{% endif %}
{% endfor %}
exec(answer)

This is just a rough first approximation - it doesn't handle nested "repeat n" constructs, for example. And I'm guessing the loop variable i isn't meant to be exposed. Also the error messages aren't very user friendly. But it might be enough to get you going while you think about which approach you'd like to use longer term.

Here's a demo of it running.

Pseudo TigerJython

I attach the prototype for the above question type. Import it into a question category called, say PROTOTYPES and create a new CodeRunner question - you should see the new type tiger_jython in the CodeRunner question type dropdown.

In reply to Richard Lobb

Re: Pre-processing available?

by Stefan Mueller -
Wonderful, many thanks for your very elaborate answer! Your solution looks much simpler than I had expected. I clearly recognize the architecture of CodeRunner as explained in the Readme on Github which had not been very clear to me before.

My strategy now is to walk through 'answer' by means of 're.sub()' with 'count=1' and creating a new instance of a simple class*, containing a single integer variable, for each respective loop variable. For efficiency, I shall substitute a while-loop in both cases. -- Does that sound reasonable or do you see any obvious simpler way?

As you correctly guessed, TigerJython cannot (easily) be called from a shell as it is an integrated GUI workbench written in Java running Jython. It has become quite popular in the German speaking parts of Europe for beginners courses in programming. Therefore I expect this template being of use to others as well. I shall report back when I have it working.

*) Edit: probably a list containing all the integer loop variables will do.
In reply to Richard Lobb

PROTOTYPE_tigerjython_via_python3

by Stefan Mueller -
As this is my very first attempt to writing a prototype, I would like to ask for expert comments.

import re
answer = """{{ STUDENT_ANSWER | e('py') }}"""
# replace 'repeat :' by 'while True :'
answer = re.sub( r"(^[ \t]*)repeat[ \t]*:", r"\1while True:", answer, flags=re.M)
# replace 'repeat <expr> :' by 'for <var> in range( <expr> )'
loopCounter = 0
while True :
    saved = answer
    answer = re.sub( r"(^[ \t]*)repeat[ \t]+(.*):", r"\1for loopVariablesList4CodeRunner_TigerJython["+str( loopCounter) +r"] in range( \2 ) :", answer, count = 1, flags=re.M)
    if answer != saved :
        loopCounter += 1
    else :
        break
if loopCounter > 0 :
    answer = "loopVariablesList4CodeRunner_TigerJython = [0 for loopVariablesIterator4CodeRunner_TigerJython in range( " + str( loopCounter) + ") ]\n" + answer

{% for TEST in TESTCASES %}
answer += "\n" + """{{ TEST.testcode | e('py')}}\n"""
{% if not loop.last %}
answer += 'print("#<ab@17943918#@>#")\n'
{% endif %}
{% endfor %}
exec(answer)

My first tests with nested repeat-loops were successful. Yet I do not really know where the pitfalls might be lurking.

Many thanks in advance!

In reply to Stefan Mueller

Re: PROTOTYPE_tigerjython_via_python3

by Richard Lobb -

That looks fine to me. I don't see any real pitfalls myself, except that it won't match a multiline repeat, e.g. 

repeat (2 * 
len(data)):

I don't think that's a significant issue, though - just tell students that they're restricted to single line repeats (which are much nicer stylistically, anyway). Maybe they're already restricted in tigerjython?

Of course, the students will probably find other pitfalls - they usually do :)

I might be missing something, but I think you can simplify the template by just using dynamically generated identifiers of the form loop_control_variable_1, loop_control_variable_2 etc rather than picking them from a list. That way you won't need to insert extra code before the student code, which will ensure that Python error message line numbers match the students' own code. Also, if you use the version of re.sub that takes a function parameter you can avoid having an explicit loop. Here's my quick hack at it, which works on the one and only test I've tried. I'm not sure it's much of an improvement, though.

import re

loopCounter = 0
def replacer(match):
    """Return the replacement for a matched repeat n statement"""
    global loopCounter
    loopCounter += 1
    return "{}for __TigerJythonLoopControlVariable_{} in range ({}):".format(match.group(1), loopCounter, match.group(2))

answer = """{{ STUDENT_ANSWER | e('py') }}"""
# replace 'repeat :' by 'while True :'
answer = re.sub( r"(^[ \t]*)repeat[ \t]*:", r"\1while True:", answer, flags=re.M)
# replace 'repeat <expr> :' by 'for <var> in range( <expr> )'
answer = re.sub( r"(^[ \t]*)repeat[ \t]+(.*):", replacer, answer, flags=re.M)

{% for TEST in TESTCASES %}
answer += "\n" + """{{ TEST.testcode | e('py')}}\n"""
{% if not loop.last %}
answer += 'print("#<ab@17943918#@>#")\n'
{% endif %}
{% endfor %}
exec(answer)

Another possibility to consider: the error messages emerging from an exec can be a bit confusing. An alternative which gives simpler error messages is to replace exec(answer) with 

with open("student_prog.py", "w") as outfile:
    outfile.write(answer)
subprocess.run(["python3", "student_prog.py"], encoding="utf-8")
In reply to Richard Lobb

Re: PROTOTYPE_tigerjython_via_python3

by Stefan Mueller -

Many, many thanks for your elaborate answer! Not only is CodeRunner/Jobe a real masterpiece but you are a true master. Your sharing of your knowledge is really generous.

(I just spent a full weekend trying to install the Jobe Docker image within or parallel to the LXC-container that my Moodle instance is running in. It simply did not work either way I tried. Creating a separate LXC-machine for Jobe was done in less than 30 minutes and is working well now. Brilliant!)

Multi-line repeat-statements not working is not a real issue as the students are beginners and their programs are simple. Thanks for spotting and pointing it out, though!

You are right about the sequentially named loop variables. I did not clearly realize that the pre-processing and the following run of the program are disconnected. Your solution with the replacer function looks really elegant. I have never used this technique up to now. I shall test it extensively together with subprocess running the program file and report back here.