CodeRunner V3.3.0 released

CodeRunner V3.3.0 released

por Richard Lobb -
Número de respuestas: 16

The latest version of CodeRunner, V3.3.0, has now been pushed to github and released on the Moodle Plugin Repository.

New features in V3.3:

  • A graph question type that allows you to set questions to which the answer is a graph that the student can draw directly into the answer box. This supports FSMs, directed graphs and undirected graphs. Thanks to Emily Price (UCan) for the initial prototype version of this.
  • A multilanguage question type that provides "Write a program" questions that the student can answer in C, C++, Java or Python3. Other languages could be added if required.
  • An SQL question type that allows you to set questions that require SQL answers (but restricted to the sqlite3 dialect). Thanks to Trent Lewis (Monash Uni).
  • Programs can now output non-ASCII UTF-8 characters. However, this only works with the latest version of Jobe (version 1.4.0) and if the Apache LANG environment variable is set to a UTF-8 locale. See the Jobe installation instructions for details.
  • A question can be customised to use a different Jobe sandbox from normal, allowing for load balancing and specialist Jobe servers e.g. for a language not supported by the production Jobe server.
  • The question authoring UI now allows one-click correction of test cases that fail during "Validate on save" (assuming the sample answer is correct - caveat emptor). Thanks Andrei Bautu (Trier Uni).
  • PHP and Python scripts are now available to download and analyse all submissions to a quiz, rather than just the final one by each student.
  • Various bug fixes.

As part of the refactoring required to support pluggable user interfaces the capability of explicitly setting the width of the answer box has been removed. The answer box now expands to fit the available space. Only the height (num_rows) can be set by the author, and the student can increase the height if they wish (except with Microsoft browsers).

One known issue: course backups created with this version of CodeRunner cannot be restored on sites running earlier versions of CodeRunner - database errors will occur. CodeRunner must first be upgraded to the latest version before the course restore is attempted.


Richard

En respuesta a Richard Lobb

Re: CodeRunner V3.3.0 released

por Martin Zwerschke -

Hi Richard,

very pleased, that the UTF-8 problem seems to be fixed!

I got the new version installed without problems and instantly tried a question in python3/C# with German characters inside the program and the test cases, wich failed before.

Now this works. (I had updated jobe server and LANG option in advance.)

That's really nice.

Will try some of the other new features as well.



En respuesta a Martin Zwerschke

Re: CodeRunner V3.3.0 released

por Richard Lobb -

Thanks Martin. Good to hear.

Richard

En respuesta a Richard Lobb

Re: CodeRunner V3.3.0 released

por Martin Zwerschke -
Announcement says:
  • A question can be customised to use a different Jobe sandbox from normal, allowing for load balancing and specialist Jobe servers e.g. for a language not supported by the production Jobe server.


I wonder if this is the right way for load balancing. If one fixes a question to one jobe server you can not switch over coderunner to another jobe like before.

I had some outings of or jobe server  recently because of unsuccessful dynDNS updates (my fault, but anyway).

I managed to switch back to my experimental home jobe server but I wonder if it woul be possible to enter more than one jobe server into the coderunner plugin settings ?

If one fails, another can do the job(e)s instead. Perhaps, if more than one of them is running, the load coud be balanced automatically.


In this context  I would like to feature-request a kind of "coderunner availabilty announcement "plugin for Moodle. 

I already had something like this realized with a small javascript inside a normal Moodle announcement.

So my students were able to see if Coderunner is clear and ready to check their answers at all.

But after I set up the firewall on jobe server this can not work any more, for traffic to and from jobe server is blocked save the traffic to the Moodle server.

So client's java script can not do it's test call to ("REST API get_languages()" ) like before.

So the jobe-status has to be checked by the Moodle server instead (inside coderunner php).



En respuesta a Martin Zwerschke

Re: CodeRunner V3.3.0 released

por Richard Lobb -

Hi Martin

Thanks for the feedback.

You're certainly right that setting a per-question or per-question-type jobe server is not the right way to go about load balancing. I shouldn't have even mentioned it, as it could only ever be useful as an emergency measure and even that's questionable. Being able to set a different Jobe server for a particular question was once requested by one of our staff members and, as it turned out to trivial, I added it.

I think the right way to do load balancing, or at any rate a much better way, is to use some form of DNS load balancing. That's scalable and requires no changes to CodeRunner. Open University in the UK does that. I understand that Round-robin DNS load balancing is trivial (just provide the DNS server with a list of Jobe IP numbers?) but that more elaborate load balancers, which poll the servers to determine their current load and availability, are possible.

Since I'm employed as a teacher not a software developer, I tend to focus my development efforts on things that are of relevance to me and my department. I can't recall an occasion when our production Jobe server has gone down unexpectedly, so providing a "coderunner availabilty announcement" feature doesn't seem like a priority to me. If the Jobe server does go down, students submitting code should get a message saying that the Jobe server appears to be down and to try again later. That's good enough for my purposes. That said, I appreciate that different sites have different needs so if there enough other users out there who would like such a feature, I'm prepared to reconsider. Squeak now, you other users!

Richard

En respuesta a Richard Lobb

Re: CodeRunner V3.3.0 released

por Tim Hunt -

Yes, at the Open University, our Jobe 'server' is actually two servers sitting behind a load-balancer.

However, I don't think round-robin DNS will work. The problem is files, where one evaluation of a student response may require several Jobe API calls. So, we set our load-balancer to use a bit of session persistence. I don't remember the exact config now, but I could dig it out.

En respuesta a Tim Hunt

Re: CodeRunner V3.3.0 released

por Richard Lobb -

Thanks Tim.

We've not actually used a round-robin DNS here and I'm interested to hear that you think it might not work. I just looked at my code and I would have expected it to work OK when different tests run on different Jobe servers. I don't currently allow students to submit files, only question authors. Files are cached on the Jobe server forever (even with several thousand CodeRunner questions, we only have around 700 files cached on Jobe), so only ever get uploaded once. If Jobe is asked to run a test case for which it doesn't have the required file(s), it returns an error code and Moodle must then upload the missing file(s) and try again.

So if a round robin DNS doesn't work I might have a bug somewhere - please let me know.

This discussion is helpful, because I now realise another downside of allowing students to upload files with their submissions - I'll need to implement a file cache expiry mechanism in Jobe. 

Richard

En respuesta a Richard Lobb

Re: CodeRunner V3.3.0 released

por Martin Zwerschke -

Thank you for this diskussion. Tim's approach sounds interesting but I don't think it will really work out for me.

The DNS names of my two jobe servers are dynamic because in Germany we usually get a different IP from our providers each time a system connects via DSL. Even if it holds it's connection, our providers usually interrupt and reconnect at midnight giving a new IP.

So we have to update DNS adresses via a service like dynDNS automatically with help of daemons in VDSL router or on the server directly exposed to VDSL connection. If for some reason the dynamic DNS update fails, I can not reach the jobe server any more. It is not the jobe service itself to get stuck. (Though I also run a MaximaPool docker on the same server, that could crash the server some time).

The DNS server is runnig on the service providers side - no way to let it do a round robin.

Our students connect from home as well as from our school's LAN. So twisting DNS on our LAN's DNS server would not solve the problem for home use.

I thought of two our three jobe entries in the coderunner plugin settings. If a connection to primary jobe server fails, coderunner connects to the secondary. This would realize failover instead of  load balancing in the first place. But it could also be used for load balancing, if both servers are up.

For the "Coderunner is working" announcement, I will try to proof in Javascript, if the SSH port of the jobe machine can be reached at all. If no it can show the outage, if yes the chance that coderunner is up is quite high. But of course doing a periodic test call to REST-API from inside Moodle would be a better proof of a working jobe machine.

But being a teacher myself I can understand, that Richard's time for development is limited...



En respuesta a Martin Zwerschke

Re: CodeRunner V3.3.0 released

por Richard Lobb -

I really don't think you should be trying to run Jobe on a server with a dynamic IP. It's fundamentally unreliable.

Have you considered running Jobe on something like an Amazon EC2 instance or a Digital Ocean server? I just brought up Jobe on a T2.micro Amazon EC2 instance (1 CPU, 1 GB RAM) and also on a Digital Ocean 1 CPU 1 GB RAM droplet. In both cases I used Ubuntu 16.04 base systems. I had to use jdk8 not jdk9 but otherwise all went very smoothly. Both run testsubmit.py and I also did a bulk test of 1200 Python questions on the Digital Ocean droplet.

The smallest DO droplet (1 CPU 1 GB RAM) costs only $5 per month, which is probably a lot less than the cost of the electricity to run your home server! Such a small server may or may not be sufficient, depending on what load you need to place on it, but surely your institution can afford to support you on a larger Amazon server, DO droplet or equivalent if necessary?

Richard



En respuesta a Richard Lobb

Re: CodeRunner V3.3.0 released

por Martin Zwerschke -

> surely your institution can afford to support you on a larger Amazon server, DO droplet or equivalent

It's not a matter of costs, I Think. But you don't know German communal burocracy  :-)  

But if i plug in a server more or less to the mains, no one cares. 

My home server is mainly for failover and experimenting. I have another one running in the cellar of our school next to the main electrical power supply connected to a otherwise not used and free VDSL line. This server is really isolated from any other network, so it would do no bigger harm if a hacker break into. This is why I risked to use it several weeks without firewall. Outages occured only the first  time I really switched on the firewall, because I forgot to let pass the dynamic DNS-Updater service.

Until then it was running like a charme. Same for my home server, that was protected by a firewall from the beginning. But it is behind a DSL-Router, that does the dynamic DNS-Update itself and when i copied the ufw settings to the school's server I forgot about the difference.

Of course it would be better to rent a dedicated server with fixed IP. But I think, I leave this as an option if bad realiability really becomes a problem.

Thanks for your hints anyway.



En respuesta a Richard Lobb

Re: CodeRunner V3.3.0 released

por oliver russell -
I am having difficulty running it on EC2. I was able to successfully install Moodle on AWS and set it up completely, but cant run the Jobe.
En respuesta a oliver russell

Re: CodeRunner V3.3.0 released

por Richard Lobb -

We'll need a lot more information if we're to help you. Such as: how is the Jobe server set up? Does it run the testsubmit test suite OK? Can you access the Jobe server from the Moodle server? What tests have you done from within Moodle? What error messages do you get?

En respuesta a Tim Hunt

Re: CodeRunner V3.3.0 released

por Rea Sutter -

Hi Tim,

I know this is an old thread, but:

You offered to dig out the config of your loadbalancer, and I would like to take you up on this offer. We have a Jobe Server that usually works OK, but we would be more comfortable with having two of them, to be more resilient.

I'd be grateful for a config, or at least more help with setting up the loadbalancer.

Thanks in advance!


En respuesta a Rea Sutter

Re: CodeRunner V3.3.0 released

por Tim Hunt -

Sorry, I forgot about that. I can't get the exact details right now, but the basic idea is to use some sort of session affinity, so that over a period of about 30 seconds or 5 minutes, all requests for a particualr source IP get routed to a particular JOBE server, so in the case where some files have to be transferred to go with the main execute job request, they all go to the same place.

En respuesta a Tim Hunt

Re: CodeRunner V3.3.0 released

por Rea Sutter -

Thank you for your quick reply! That does make sense.

We're planning to change to two servers the week after next, so there's still time if you manage to dig out those configs. :-)

Another question: what loadbalancer are you using?

En respuesta a Rea Sutter

Re: CodeRunner V3.3.0 released

por Tim Hunt -

We use a Citrix Netscaler, which is think these days is a virtual device.

More important then the specific configuration for a sepecific devive, a better question is "How do you know it is working after you have set it up?"

We tend to use three tests:

  1. Use testsubmit.py on each server, to make sure it is working.
  2. Then, point a development site at the load-balanced URL, and run the unit tests.
  3. Finally, if you already have questions in a Moodle site, use .../question/type/coderunner/bulktestindex.php to run all the question self-tests.
If all that works, you can be fairly sure that CodeRunner is working.