Things one can mess up when working with CodeRunner
Here is my beginners' 101 recital. My first steps with CodeRunner were a mere stumbling straight into the pit and I am very, very grateful for Richard Lobb's generous help with getting out of it. May others find my account helpful for avoiding the pitfalls from the beginning!
Documentation
The CodeRunner documentation available is as complete as it is helpful. Read it and go there with any question as first thing. The question authors' forum on https://coderunner.org.nz is a valuable source of information as well as the central meeting point for users with a good chance that the main developper, Richard Lobb, may read cries for help.
Installation
There are two places to install some things. The Architecture of CodeRunner shows how the Moodle server interacts with the sandbox which should be a sepearate Jobe (i.e. job execution) server.
On the Moodle server
There are two kinds of question type plugins to install, namely moodle-qtype_coderunner and moodle-qbehaviour_adaptive_adapted_for_coderunner. They are available from moodle.org as well as from Github. The first one is responsible for the main handling of CodeRunner questions.
The Jobe server
Due to security considerations, the code should be executed on a separate server. The installation is straightforward and well documented.
Additional packages
Since the Jobe server is meant to run the code, the programming languages must of course be installed. This may sound fairly obvious but with Python2 being phased out from modern installations, there is a certain chance that when creating a prototype based on python2, one may run into the error message "qtype_coderunner/Language python2 is not available on this system".
Issues related to LXC
I was 100% unsuccessful when trying to install the Jobe Docker image within or parallel to the LXC-container that my Moodle instance is running in. It simply did not work either way I tried. Creating a separate LXC-machine for Jobe was done in less than 30 minutes and is working well.
A LXC container seems to be set up in such a way that /var/lock/ is a proper directory whereas on a physical machine it is a symbolic link to /run/lock/. This makes it necessary to add 'var/lock' to $config['clean_up_path'] in /var/www/html/jobe/application/config/config.php .
General trouble-shooting
In case something does not work with CodeRunner then the following steps may provide valuable information.
• Inspect the contents of /var/log/apache2/error.log on the Moodle server. It always has a number of messages in it about routine Moodle jobs, such as timed out web pages, but look for messages that might be related to your new problem.
• Take a look at /var/log/apache2/error.log on the Jobe server. Don't worry about messages stating that /usr/bin/find is not able to find /var/crash! Although these may look bad at first they are in fact harmless. At the end of each run, Jobe attempts to delete any stray files created in writable directories defined $config['clean_up_path']. Finding none of them is not a problem.
• Go to Site Administration > Development > Debugging and set Debug Messages to Developer and "Display debug messages" to On. Try creating some basic CodeRunner questions. See what errors you get coming up on screen.
• Clear the Moodle cache (Site administation > Development > Purge caches). Check the symptoms haven't changed.
• Reboot the server. Again check symptoms.
Messing with CodeRunner questions
There are two names involved in creating a new CodeRunner question and three when dealing with a question prototype. Further confusion may stem from the fact that a question prototype is itself a question. So let's get these things sorted out!
CodeRunner questions
Obviously, SQL queries behaves differently from Python code for example. Thus question must adhere to a question type whose name is chosen among the available prototypes as first thing. With the built-in question types there is not much to really mess up. The available examples give pretty clear directions on how to create one's own questions. The question is stored under its name in a (Moodle) question category.
Selectively customizing the question type
When creating a new question, one can see (and modify) the code provided by the prototype by checking "Customise". This makes the code supplied by the prototype available under "Template". Modifying it affects only the given question.
Question prototypes
Each question type is defined by a prototype which is itself a CodeRunner question. A bit of insight helps avoiding pitfalls when working with prototypes.
Names
The prototype is a CodeRunner question, too. As such it has to be created with a (Moodle) question name and based upon an existing question type. Two things to realize here: the choice of base question type does not really matter. Selecting one will make its code available in the same way as when selectively customizing the question type. There is, however, no inheritance among the prototypes.
Furthermore the (general) question name is only relevant to Moodle but not to CodeRunner! That is, the name under which the prototype will be available as a question type for new CodeRunner questions is not the question name. (I think of it as the "Moodle question name".)
For defining a question that should serve as a prototype, one has to specify that in "Prototyping" under "Advanced customization". The most confusing part for me was that the "Question type" field there is pre-filled with the name of the previously chosen question type and that the fields are identically named. To make it very clear: this is the "CodeRunner name" given to the question type. You must of course choose a unique name here, otherwise CodeRunner will be going bananas and your mind with it while trouble-shooting. In case of a name clash, a big red message "This question was defined to be of type 'something' but the prototype does not exist, or is non-unique, or is unavailable in this context" warns against the resulting mess.
Programming Languages
The whole code has to be processed by the Jobe server. As is well explained in the documentation under Supporting or implementing new languages, the language for interpreting the code may be different from the one executing the student answer. Most often one will use Python3 as interpretation language and call the execution language from there.
Messsing with categories
It is stated clearly in the documentation that question prototypes must be available to the questions that make use of them. As obvious as it may sound there still are ways to miss that point. If a prototype was missing, the same warning message as above would appear.
Furthermore I ran into problems when having question prototypes in a pseudo-category associated with the quiz itself, not positioned within the normal Moodle question category. I cannot recall what exactly was wrong and I do not want to find out again! The right way for handling question prototypes is to put them into a category as high up in the question bank tree structure as possible. In my case, I defined a question category named "CodeRunner Prototypes" in the top course category suitable. Multiple such question categories may exist at different locations in the tree so one can gather the CodeRunner prototypes in appropriate locations.
Additional thoughts
Do you have anything to add or correct? With your input this thread could start out some true beginners 101 section in the documentation.
I feel very much indebted to Richard Lobb both for his generous help and his continued efforts with CodeRunner!