CodeRunner: Precheck on 3.1+

I am trying to get the usual thing of a quiz with a mixture of CodeRunner and non-CodeRunner questions working with a successful no-penalty pre-check. I'd like the feedback for the quiz to be deferred but the pre-check to still work. The release notes for CR 3.1 say:

"Adaptive behaviour is now forced for CodeRunner questions regardless of the question behaviour chosen for the quiz in which the question is running. This allows teachers to set tests in which standard questions like multichoice and short-answer run in, say, a deferred feedback mode while CodeRunner questions still allow students to correct wrong answers on the fly."

This gives the impression that for 3.1+, you should be able to get pre-check and / or check to work in a deferred feedback quiz; I have not yet managed to get this to work. During an attempt the buttons appear and if you click pre-check, the words "pre-check results" appear in the question number box at the top left, but no actual feedback appears. I realise most of the posts and docs still say you need immediate feedback mode, but most pre-date 3.1.

Can you please clarify?

Thanks

Re: Precheck on 3.1+ - still confused!

de Richard Lobb - domingo, 6 de enero de 2019, 13:20

Yes, I'm confused too! I thought what I had written was correct at the time but I discovered a couple of weeks ago that it isn't any more. I'm now not sure if it was ever correct.

To deal with this, I've added an extra pull-down menu option in the question editing form to control display of the result table. It has values "Normal" (i.e., the display is controlled by the quiz setting), "Always" and "Never". This should solve your problem, allowing you to select "Always" to force display of the result table (both Precheck and Check) regardless of the quiz mode and review settings. The code is in the Development branch, which I pushed to github a couple of days ago. However, it is barely tested as yet and unless you really like living on the bleeding edge I wouldn't recommend you install it. The new version updates the database and there's no going back, except via a Moodle server backup.

Merging the changes into the master branch is still a few weeks away. How much of a hurry are you in?

Sorry about the confusion.

Re: Precheck on 3.1+ - still confused!

de Stephen Parry - domingo, 13 de enero de 2019, 12:49

Sorry for delay, replied via email by mistake :-(

I can make do for a while - I'd rather have something stable and tested, although CR is going to become very important in our assessments and lessons. (Big thank you by the way; it's a truly awesome plugin to have at hand).

Re: Precheck on 3.1+ - still confused!

de Richard Lobb - martes, 15 de enero de 2019, 21:59

Yes, holding off until the latest version is officially released is always a good call unless you have an urgent problem.

The latest version (development branch) is now running on our production servers. I'll push it to the Moodle plugin repository in a couple of weeks.

Pleased you're finding it useful.

Re: Precheck on 3.1+ - still confused!

de Tim Hunt - martes, 15 de enero de 2019, 23:23

Just to say that I was planning to upgrade our version of CodeRunner some time in the next two weeks, so if you can release the new version before then, that would be lovely. (However, I am probably happy to take the latest version, even if it has not been tagged and pushed to the plugins DB.)

Re: Precheck on 3.1+ - still confused!

de Richard Lobb - jueves, 17 de enero de 2019, 21:29

Hi Tim

I'd like to give it at least a week on our production servers, both of which are only very lightly loaded at present, as it's our summer break. However, the latest version is on github in the Development branch if you want to give it a go. It should be pretty stable - the only things that might fail are the two new features: attachments (the biggie) and the Show feedback option.

Possibly of more interest/concern is the new Jobe version also in the development branch on github. I was worried about the implications of allowing attachments with large classes, as Jobe stores uploaded support files indefinitely and in a single flat file structure. So the new Jobe moves the file cache to /home/jobe/files and uses a two-level hierarchy like the moodledata file cache. I've also added a feature that clears all files last accessed more than 2 days ago from the file cache if the free space drops below 5%. Those changes should keep Jobe viable but Moodle server also stores all student attachments indefinitely, so it could still take a hit.

I've marked the Attachments capability experimental and warned question authors in two of the on-line help buttons about the possible impacts on file space and/or performance of allowing large attachments with large classes. The default upload size is only 10 kB but authors can increase it. So it's possible for a daft teacher to tell a class of 1000 students to each upload a 10 MB file, thereby blowing away 10 GB of disk space on Moodle. Should I worry about this?

Richard

Re: Precheck on 3.1+ - still confused!

de Tim Hunt - viernes, 18 de enero de 2019, 01:39

Oh drat! We just upgraded our Jobe servers yesterday and I was not paying attention. That was mainly do update the OS and PHP version, but we went to latest JOBE master branch then, and I don't want to have to do another deploy there. Therefore, I will probably just use v3.5.3, or more likely today's master branch, which seems to include some useful bug fixes.

So actually, there is not need for you to rush the release of the next version.

Re: Precheck on 3.1+ - still confused!

de Tim Hunt - martes, 29 de enero de 2019, 06:18

A further question about this:

With the new version of CodeRunner and Jobe, what are the rules about upgrading?

Jobe and CodeRunner must be upgraded at the same time or it will break.
It is fine to upgrade Jobe in first, then deploy the new CodeRunner a few days later.
It is fine to upgrade CodeRunner first, and then upgrade Jobe later.

Thanks.

Re: Precheck on 3.1+ - still confused!

de Richard Lobb - martes, 29 de enero de 2019, 10:11

You can do the upgrades in either order. All versions of CodeRunner (within the last few years, anyway) work with all versions of Jobe.

The caveat is that there might be a performance and/or disk-space hit with CodeRunner v3.6 (which I aim to release later this week) and a non-updated Jobe over time. It depends on how much use authors make of the new feature allowing students to attach files to their submission: what size files they allow, how many students, the amount of free disk space on Jobe and the file system it's using. If authors stay with the default file size limit of 10 kB, I wouldn't expect any problems with the old Jobe for months or even years.