Question Authors' Forum

My Language

Picture of John Pap
My Language
by John Pap - Sunday, 14 January 2018, 9:38 PM

Hello guys, I'd like CodeRunner to accept as answer simple expressions like: Μ ← Ν * 2 ( with left arrow for assign and asterisk for multiplication ).
Is there a way to do that by changing the templates or otherwise ?
Please advise me, 
thanks a lot !

Picture of Richard Lobb
Re: My Language
by Richard Lobb - Sunday, 14 January 2018, 11:37 PM

Hi John.

I'm not quite sure what problem you're trying to solve. Certainly you can write a template (e.g. in Python) to accept text in some hypothetical language and then include your interpreter for that language within the template or as a support file.

However, the problem with the example you give is that the left arrow character isn't ASCII and currently Jobe (the sandbox server used by CodeRunner) runs only in ASCII. You could either ask students to enter a two-character equivalent, such as '<-', or you could wait until I've updated both Jobe and CodeRunner to handle UTF-8. As of yesterday, my development system is running UTF-8, so it shouldn't be too long before I publish the updates. 

If you want to live on the bleeding edge, you could try pulling the development branches of both Jobe and CodeRunner right now, but be warned the latest changes have had very little testing.


Picture of John Pap
Re: My Language
by John Pap - Monday, 15 January 2018, 1:59 AM

Thanks for your reply Richard,

my aim is to give simple instructions and get simple expressions as answers. E.g. write an expression which assigns to M the square of N, write an expression which assigns the integer remainder ( % ) of A with 3, back to variable A. 

I have a java .jar file as an application where I can test my code but I think there is no need for these simple commands, unless if you can give me some hints "how to".

I've found a temporary solution writing in C : 

puts("{{ STUDENT_ANSWER }}"); 

inside of main( ) but it accepts only the expression exactly as is ( M ← N * 2 ) while ( M←N*2 ) and ( M ← 2 * N ) are also correct but not accepted from CodeRunner as a solution. 

In short, I mind only to match the answers with the ability to ignore spaces and variable's order.

There I need a little help.

I can test Jobe and CodeRunner, yes, thank you very much again.

Picture of John Pap
Re: My Language
by John Pap - Monday, 15 January 2018, 10:10 AM

I'm not currently sure 100%, it needs testing, but I wrote some simple code in Python 3 and I have success. 

I tried many circumstances.

Thanks again !

Picture of Richard Lobb
Re: My Language
by Richard Lobb - Monday, 15 January 2018, 10:23 AM

Hi John

If you really just wish to match an answer ignoring white space and the variable's order, you could use a regular expression match. There are two ways of doing that. 

  1. Simply print the student answer, as with your C fragment, though you should escape the student's answer to handle embedded quotes, newlines etc, e.g. puts("{{STUDENT_ANSWER | e('c')}}". Customise the question, setting the Grading pull-down to Regular expression. Set the "Expected' field for each test case (only 1 in the simple case) to match whatever you're prepared to accept. You'll need to start and end the expression with \A and \Z respectively in order to match the entire string. See the in-line help. This is viable only if the students understand regular expressions; otherwise the 'Expected' column in the result table will confuse them. And CodeRunner isn't necessary for this approach anyway; there's a much simpler Regular Expression question type available.
  2. Write template code that tries to match the student answer with a regular expression and prints a more user-friendly message to indicate success or otherwise. For example, in Python3

import re
student_answer = """{{STUDENT_ANSWER | e('py')}}"""
pattern_to_match = .... # Whatever you require
if re.match(pattern, student_answer):
    print("Sorry, that answer doesn't match any of my expected answers"
And of course the "Expected" field for the test must be set to "Correct!".

There are of course better ways of checking the answer, including actually parsing the expression and interpreting it.

However, the above methods don't scale well and the output to the student is often not very helpful. To do the job properly you really need to add your language to CodeRunner as a new question type. If you already have an interpreter written for it, this isn't as hard as it sounds. The section on implementing or supporting new languages in the documentation indicates the general approach. If you have an interpreter rather than a compiler you wouldn't need the compilation phase. Rather than running the compiled program directly, as in the example, you would run your interpreter (which you would attach to the question as a support file), with the input program being, say, the {{TEST.testcode}} to set up the context for the student code followed by the student code followed by a bit of code to print the required variables. More generally again you could use the {{TEST.extra}} field to specify what checks to do after the student code has been run.

The above approach will be rather slow if your interpreter is written in Java and you have more than 2 or 3 test cases. To make it more efficient you'd need a combinator template, but don't worry about that until the per-test-case template is working.


Picture of John Pap
Re: My Language
by John Pap - Monday, 15 January 2018, 8:57 PM

I've found a solution, with different approach in Python 3. I used raw.split, join functions and I have convert every input to ASCII characters.

Then, I compare separately the first part ( before left arrow ) and the second part ( after arrow ) with the student's answer, respectively.

I was much easier than I thought.

One issue is that I get :

Error loading prototype: Error fetching prototype. qtype_coderunner/Prototype  is unavailable in this context, or does not exist.

This happening many times and I'm not sure how to fix it, ( except to restore Moodle )   :)

Picture of Richard Lobb
Re: My Language
by Richard Lobb - Monday, 15 January 2018, 10:18 PM

Restoring Moodle is certainly not required here!!

When you create a CodeRunner question you have to select the type of the question (C function, Python, Java class etc). Each question type is defined by a special coderunner question called a prototype. You can create your own question types by defining your own prototypes but you do have to be very careful not to then delete the prototype, or damage it by editing it and changing its type. If you do that you get the message you're reporting whenever you try to edit or open any question using the missing or damaged prototype. The same problems occur if, as a moodle admin, you edit or delete the built in prototypes.

Firstly: are you quoting exactly the error message you're getting? The message should contain the name of the missing prototype, e.g.

qtype_coderunner/Prototype python3_pylint is unavailable in this context, or does not exist.
for the case of a missing question type python3_pylint.

If the type name is indeed empty then you have somehow managed to get a question with an empty prototype. I'm not sure quite how that happens.

To figure out what questions are damaged and what prototypes are missing, point your browser at 


Click the link for the course in which you're getting those errors and you should get a report on what questions are using what prototypes and also what prototypes are missing.

The simplest way to eliminate the problem is to delete all the questions that have missing prototypes. However, if you need those questions you'll instead have to create new prototypes that define the required coderunner type name. See and proceed very carefully, reading all the on-line help in the question author form.


Picture of John Pap
Re: My Language
by John Pap - Tuesday, 16 January 2018, 12:16 AM

When my question type has Greek letters, I get the error.

I changed to Latin characters and it works fine.

How to try the UTF-8 'to be released' version ?


Picture of John Pap
Re: My Language
by John Pap - Tuesday, 16 January 2018, 5:55 AM

One other thing is that, Git version from December it doesn't work well for me.

With August's version works fine.

Also, just for the history, is there a way to disable the "check" button when someone attempts the question ?

Picture of Richard Lobb
Re: My Language
by Richard Lobb - Tuesday, 16 January 2018, 10:21 AM

OK, thanks for identifying the problem of question types with Greek letters. I've found the cause but I lean towards fixing the issue by preventing non-Latin question types in the question authoring form. I see them as akin to identifiers. For now, just stick to Latin.

If you really want to test the current development version you will need to do the following:

  1. On the Moodle server, pull the latest development branch of CodeRunner from github, login as a moodle administrator and accept the version upgrade request. Warning: this is irreversible.
  2. On the Jobe server, pull the latest development branch of Jobe. Run the install script as superuser (./install).
  3. On the Jobe server, follow the instructions in the Jobe documentation to set the Apache LANG environment variable to a UTF-8 aware value. Most conservative is C.UTF-8.
But are you quite sure you want to do this? It won't have any effect on your problem with using Greek letters in question types. Unless you can convince me otherwise, my fix to that will be to prevent their use when creating new prototypes.

Could you clarify exactly what goes wrong with the Git version of CodeRunner from December, please? No-one else is reporting issues. The only significant bug I'm aware of with that version relates to the setting of the is-combinator checkbox when creating new question prototypes - when the question is saved it can be set true when it should be false (or vice versa - I forget which). It's easily worked around by reloading the question and resetting the checkbox to the correct value.

I'm not sure what you're after when you ask " is there a way to disable the "check" button when someone attempts the question ?". Are you trying to allow students just a single attempt? There's no way to disable the check button to prevent a second attempt but you can set the penalty to 100%. Or perhaps set the review options to hide the result table and mark until the quiz attempt is submitted, depending on what you're trying to achieve.


Picture of John Pap
Re: My Language
by John Pap - Thursday, 18 January 2018, 2:17 AM

I should wait for the next stable release, maybe that way is better.

I ran the exact same code in the template ( sandbox language : Python 3 ).

One time with August's version and the second time with Git's.

The first worked fine while the second gave me incorrect for correct questions and vice versa.

For the check button I 'll set the penalty to 100 %

P.S. CodeRunner is my favourite plugin in Moodle, thanks for that.


Picture of Richard Lobb
Re: My Language
by Richard Lobb - Friday, 19 January 2018, 10:45 AM

Thanks John. Good to hear you're finding CodeRunner useful.

If the question that gave you problems with the latest version on Git was one for which you had written your own template, then it's possible you ran into the bug I alluded to earlier, where a particular sequence of actions in creating a question led to a combinator template running once on each test or a per-test template running as a combinator. As I recall, it was fixable just by re-editing the question and setting the correct value in the is-combinator checkbox.

I just re-loaded the December version into a Docker Moodle image and it seems to be doing all the basics OK. If you run into the problem again, would you mind posting a detailed description of what you did to create the problem, plus a screen shot, please?