Using moodle-ci-runner to test CodeRunner

Using moodle-ci-runner to test CodeRunner

by Tim Hunt -
Number of replies: 5
Sorry, this is probably unduly technical to be going in the Authoring forum, but I wanted to share what I worked out, and this was the easiest place. Feel free to ignore this; or, copy it to a more appropriate home for this sort of documentation.

Moodle HQ provides moodle-ci-runner as a way to run the Moodle automated tests, e.g. on a CI server. (If you want to do interactive development, then moodle-docker is more appropriate. These are worth knowing about if you suddenly find yourself needing to test your code in a sepcific environment that is a pain to set up (E.g test with PHP7.4 and Microsoft SQL server.) I don't really undersand how this works. Lets just waive our hands and say 'Docker magic' but I was able to understand enough to get the CodeRunner tests working.

The complication is, of course, that CodeRunner needs Jobe, so we need to add that to the setup. Here is, manually, what you need to do to get this running. (Becuase this is CI, I acutally have a bash script to do this automatically, but I think it is more readable to give the steps manually.

  1. To use this, you will have moodle-ci-runner checked out, and in there, the key files are in
    runner/main/
        jobtypes/
            behat/
                behat.sh
            phpunit/
                phpunit.sh
       modules/
            docker-... in various forms, and some other modules.

  2. The first thing to do is to add a new module for docker-jobe. So
        mkdir runner/main/modules/docker-jobe
    then save the attached file in there (assuming Moodle lets me attach something dangerous like a .sh file).

  3. Then we need to tell the jobs to use that, so edit each of runner/main/jobtypes/behat/behat.sh and runner/main/jobtypes/phpunit/phpunit.sh and in the list of required modules in function ..._modules() { add docker-jobe. (I added it just after docker-php).

  4. The other thing you will have to use this  is to have the Moodle code checked out somewhere (with the coderunner and adaptive_adapted_for_coderunner plugins added.) In there, you need to create the question/type/coderunner/tests/fixtures/test-sandbox-config.php file as usual to run the tests. The contents you need for that file in this setup is

    <?php
    echo "set_config('jobesandbox_enabled', 1, 'qtype_coderunner');
    echo "set_config('jobe_host', 'jobe', 'qtype_coderunner');
    That is, the 'hostname' of the jobe container of the virtual internal network the docker has set up is 'jobe'.
I think that is all. I hope that is useful to at least somebody.
In reply to Tim Hunt

Re: Using moodle-ci-runner to test CodeRunner

by Tim Hunt -

Acutally, I may as well share the key snippets of my script to do this:

git clone https://github.com/trampgeek/jobeinabox.git /var/jobeinabox
cd /var/jobeinabox
# Note: if https://github.com/trampgeek/jobeinabox/pull/19 is merged, we can remove this sed.
sed -i "s|--interval=5m|--interval=5s|" /var/jobeinabox/Dockerfile
docker build . -t my/jobeinabox --build-arg TZ="Europe/Amsterdam"

cp -r /var/moodle/local/ciextras/moodle-ci-runner-extra-modules/* /var/moodle-ci-runner/runner/main/modules

sed -i -e '/ docker-php/a\' -e ' docker-jobe' /var/moodle-ci-runner/runner/main/jobtypes/phpunit/phpunit.sh
sed -i -e '/ docker-php/a\' -e ' docker-jobe' /var/moodle-ci-runner/runner/main/jobtypes/behat/behat.sh

cd /var/ou/moodle
echo "<?php" > question/type/coderunner/tests/fixtures/test-sandbox-config.php
echo "set_config('jobesandbox_enabled', 1, 'qtype_coderunner');" >> question/type/coderunner/tests/fixtures/test-sandbox-config.php
echo "set_config('jobe_host', 'jobe', 'qtype_coderunner');" >> question/type/coderunner/tests/fixtures/test-sandbox-config.php

Note: we have to rebuild the JobeInABox docker container because we run our automated tests on arm64 servers (cheaper and better for the environment) and the pre-built JobeInABox does not come built for that architecture. The attached .sh file assumes you have done this (references my/jobeinabox rather than trampgeek/jobeinabox). If you use x64 architecture, you probably don't need to do that.


In reply to Tim Hunt

Re: Using moodle-ci-runner to test CodeRunner

by Richard Lobb -
Many thanks for posting that Tim. We've been using moodle-docker for developing ourselves; it's easy enough to switch PHP versions or to use (say) postgreSQL instead of mariadb, but switching Moodle versions is a bit clumsy. This is definitely worth a look.

[Edit] I followed your instructions and got it work, many thanks, Tim.

For anyone else who's interested, after following Tim's instructions I ran it with the Moodle 4.4 code I already had installed at /var/www/html/moodle, using the following commands:

export WORKSPACE=/var/www/html
export DBTYPE=mysqli
export PHP_VERSION=8.2
export JOBTYPE=behat
export BEHAT_TAGS=qtype_coderunner
./runner/main/run.sh

I find though that I have to stop and remove the docker jobe container before starting a new test run.

In reply to Richard Lobb

Re: Using moodle-ci-runner to test CodeRunner

by Richard Lobb -
A related question, Tim: the github ci runs have all been breaking lately. They break at the step Install Moodle (which is actually adding the plugin moodle-qbehaviour_adaptive_adapted_for_coderunner). The output for that step is:
Run moodle-plugin-ci add-plugin trampgeek/moodle-qbehaviour_adaptive_adapted_for_coderunner
 0/9 [>---------------------------]   0% < 1 sec [Starting install]
 1/9 [===>------------------------]  11% < 1 sec [Cloning Moodle]
 2/9 [======>---------------------]  22% 12 secs [Moodle assets]
 3/9 [=========>------------------]  33% 12 secs [Install plugins]
 4/9 [============>---------------]  44% 13 secs [Installing Node.js]
In VendorInstaller.php line 135:
                                
  Node.js installation failed.  
                                

install [--moodle MOODLE] [--data DATA] [--repo REPO] [--branch BRANCH] [--plugin PLUGIN] [--db-type DB-TYPE] [--db-user DB-USER] [--db-pass DB-PASS] [--db-name DB-NAME] [--db-host DB-HOST] [--db-port DB-PORT] [--not-paths NOT-PATHS] [--not-names NOT-NAMES] [--extra-plugins EXTRA-PLUGINS] [--no-init] [--node-version NODE-VERSION]

It seems to be a problem with Node.js but I don't think this is something I have any control over, is it?

Any suggestions on how to bring it back to life? It's been failing for at least 5 days so far.

In reply to Richard Lobb

Re: Using moodle-ci-runner to test CodeRunner

by Tim Hunt -
That is a known issue: https://github.com/moodlehq/moodle-plugin-ci/issues/309

Cause seems to be third-party, and may be fixed by now, or there are work-arounds. (Sorry for being vague. I am on holiday.)
In reply to Tim Hunt

Re: Using moodle-ci-runner to test CodeRunner

by Richard Lobb -
Many thanks Tim. Funny that my web search didn't bring that issue up - my search must have been too generic. Good to know it's not my fault, anyway. Enjoy your holiday.