Feedback: Using Attachments in CodeRunner for exams

Feedback: Using Attachments in CodeRunner for exams

de către Chris Nelson-
Număr de răspunsuri: 6

Hello,

I am passing on some feedback from one of our course teams who used CodeRunner in their June exams.

Short version: I don't think CodeRunner uses the same attachment interface design as Essay, and we'd humbly suggest that it would be better for CodeRunner to (re)use the Essay component for a more consistent student user interface.

Long version: the module team had CR questions that required students to attach files in their response. Settings were as follows:


However, when marking, the course team have found that several students were able to upload different filetypes, like the '.java.' restriction didn't appear to work. 

On testing, I think that's because of the exam set-up meaning that the student uploaded the file but didn't press the 'Check' button to validate the attachment (and therefore get an error) before the attempt time ran out and saved the attempt.

Additionally, when there is a CodeRunner attachment with a file restriction, it isn't stated:


If we compare this with Essay, any attachment area with file restrictions states the expected filetypes underneath the attachment area:


... and if you try to upload a filetype that's not on the 'accepted' list, it validates and tells you if there a problem immediately, e.g.:


Would it be possible to improve this area of CodeRunner so that file attachments are more consistent and validate on upload initiation, please?

Many thanks,

Chris Nelson,
On behalf of the M250 course team.

Ca răspuns la Chris Nelson

Re: Feedback: Using Attachments in CodeRunner for exams

de către Richard Lobb-
Thanks for the feedback, Chris.

As I recall, the reason I used a non-standard interface was because all that supported by Moodle at the time of implementaion was the ability to specify a set of basic Mime types, and that all that was available in that set was 'text', which covered all possible programming language source files.

I see that the essay question does now allow me to specify for example .c or .java files, but I don't see a way to specify a Python file, JavaScript, Pascal etc?

The existing code does also allow constraining the filename that students can upload which can be advantageous with data files.

You say "On testing, I think that's because of the exam set-up meaning that the student uploaded the file but didn't press the 'Check' button to validate the attachment (and therefore get an error) before the attempt time ran out and saved the attempt." This seems like very much an edge case. There are always students who are just in the process of answering a question when time runs out and they are of course upset. It's like arriving at the train station as the trail pulls out. It's tough, but that's life.

I totally accept that it would be better if the interface allowed students to select only files of the exact type required by the question author. But this has to include not only all programming languages supported out-of-the-box by CodeRunner but others that users might have chosen to support with their own question types. If this can be achieved using the standard Moodle UI, I'd be prepared to sacrifice the benefit of constraining the filename as well. However, we make very little use of student file attachments here at Canterbury and what we have at present seems to satisfy our needs. So while I'm happy to put it on our TODO list, it's not going to have a very high priority.

Would one of your team like to look into this?
Ca răspuns la Richard Lobb

Re: Feedback: Using Attachments in CodeRunner for exams

de către Chris Nelson-

Thanks for the quick reply, Richard,

We could certainly look into it :-)

I'd need to check with Core, but I'm wondering if we could marry your nifty little regex free text list with Essay, and then use that 'new' Core functionality in CodeRunner.

Ca răspuns la Richard Lobb

Re: Feedback: Using Attachments in CodeRunner for exams

de către Chris Nelson-
Hi Richard,

Having just chatted with Tim Hunt, Moodle allows you to add any filetype these days. The admin page is at https://[moodle]/admin/tool/filetypes/index.php.

I've just checked adding a Python filetype to one of our acceptance servers and its appears in the Essay attachments pop-up:

(I think having the 'CodeRunner' section would be the tidiest way to add the filetypes, but that needs to be thought through).

So if CodeRunner switched to using the Essay qtype's attachment component, perhaps the CodeRunner plugin could check for common programming filetypes and add any missing filetypes automatically on install? And for anything special, they could be manually added?
Ca răspuns la Chris Nelson

Re: Feedback: Using Attachments in CodeRunner for exams

de către Richard Lobb-
Thanks for checking that out Chris. The interface to add new filetypes looks nice.

Having a file type section called CodeRunner looks nice but there might be other plugins using some of those file types, eg. VPL lab, so I don't think CodeRunner can claim ownership of all progamming language source files :-) Also, there's already a motley collection of language files present in the out-of-the-box Moodle 4.3.3; these show up in the Essay question as Other files so it might be confusing if some language files were there and others in CodeRunner.

Managing the file types is slightly problematic. Adding all the file types for out-of-the-box languages supported by CodeRunner could (and should) be done at CodeRunner install time; I'm hoping your guys are going to do that? However, that doesn't cover custom question (language) types implemented by individual question authors. These can be implemented by any teacher, without Moodle admin capabilities, provided the support is present on the Jobe server. For example, one of our teachers has added a Clojure question type. Adding the corresponding file type to Moodle does require admin support, so the question author has an extra hoop to jump through.

But I don't think that's a show stopper. Extra language support had to be added to Jobe, so such question authors probably have sysadmins available to help. And if the required question type isn't present, then as a fallback the question author can allow All file types, and instruct the students what to attach - that blows away the whole point of this exercise, but would be a workable solution in most cases. With the caveat that I hardly ever use student-attached files, so I'm not confident I have a good handle on the needs.
Ca răspuns la Richard Lobb

Re: Feedback: Using Attachments in CodeRunner for exams

de către Chris Nelson-
Thanks Richard,

OK, we'll start to have a serious look at this and be in touch when we're worked up a proper spec for switching to the Essay component, with out-of-the-box filetypes added on plugin install/upgrade. Perhaps we can come up with a question-only setting for unusual allowable filetypes that makes adding them by non sysadmins possible.