Question Authors' Forum

File upload/submission question type

Picture of Richard Lobb
Re: File upload/submission question type
by Richard Lobb - Wednesday, 15 February 2017, 10:01 PM

Thanks for raising this Trent.

“Students are appear to more comfortable working in an IDE when developing code solutions and this is how one would develop code in the workplace. 
Totally agree. That has always been our model here, too. Although having said that, I think there are quite a few other users who don't operate that way. For example, the University of Auckland's Moodle site says "Because the system is on-line students do not need to have programming tools installed on their lab or private computer."

This might be overcome with more practice with CodeRunner quiz items out of test conditions
Yes, I think this is an important point. On-line tests and exams can be very stressful for students. In the past, when we had a special on-line exam environment that students were thrown into only for exams, a significant percentage of students crashed badly. But we now use CodeRunner for all labs, assignments, tests and exams. Students work with it both at home and in our labs from day 1 in the course. As a result, I think our on-line exams are no more stressful than written ones and possibly even significantly less stressful. It is still very important, however, to start every test or exam with some really trivial questions (as with written exams) to help students relax a bit.

Now, I’m not even sure that it is possible to have a “file upload” option for a quiz item in Moodle, so the request might not be technically possible anyway!
I certainly could have an option to upload data from a file rather than cutting and pasting. However, would this really  reduce the stress for your students? My feeling is that the stress arises because of the uncertainty and unfamiliarity with CodeRunner rather than the particular sequence of mouse and keyboard events required to transfer the code across.

If the quiz item had an upload file option this could allow the student to upload the package file that could be tested in the CodeRunner Quiz framework on the jobeserver still using the output text matching.
Yes, this is a reasonably compelling use case. However, presentation of the results is now less clear as students can no longer see on screen exactly what they submitted. They would have to be able to download again the jar file they submitted to see what the errors were. [I'm thinking of cases where they somehow submitted an incorrect jar or one containing the wrong version of a file.] 

It could also be interesting to entertain the idea of allowing code to be retrieved from a version control repository.
Uploading directly from a repository into Moodle is semi-doable even now. The "student answer" could be just the student's repository URL, and the template could begin with something equivalent to

git clone {{ STUDENT_ANSWER }}
The rest of the template could then cd into the directory and test the student's code. I've done something very similar to that in a web programming course, where students developed a website to certain specs and just pasted its url into the student answer.

However, this approach has a few problems, such as:

  • As with jar file upload, it's harder to understand the table of results, as the submitted code isn't clearly visible.
  • Grading is not automatically triggered by a commit to the repo; students have to independently request it.
  • The state of the repo changes over time so it would be difficult to go back through a student's submissions to see what happened unless the template were able (somehow) to record the state of the repo as the time it was being graded on each submission.
  • jobe needs to have git installed and the firewall configured to allow access to the repo(s).
I'd be really interested to hear from anyone else on the subject of file upload. Who else wants it? Why?


Picture of Trent Lewis
Re: File upload/submission question type
by Trent Lewis - Friday, 30 June 2017, 5:23 AM

Any further thoughts on a file upload.

I've investigated the version control (git) option, but my institution is not particularly okay with opening up the firewall on the jobeserver :(

I'd be happy to trial and test a file upload question type :)


Tim at Lone Pine Koala Sanctuary
Re: File upload/submission question type
by Tim Hunt - Friday, 30 June 2017, 11:28 PM

A file upload option could be implemented. The standard Moodle 'Essay' question type shows that the Moodle quiz system is capable of handling files as part of a question response.

However, one of the things that gives CodeRunner its flexibility is the use of a template to combine all the bits of code from the teacher and student. One would need to think through how that should be extended to cope with responses involving files.

For a response that is a single file, it is likely to be easier for the student to just copy and paste the content of their IDE into the question.

Picture of Richard Lobb
Re: File upload/submission question type
by Richard Lobb - Tuesday, 4 July 2017, 1:52 PM

OK, I'll look into copying appropriate bits of the essay question type into CodeRunner to allow file upload, but I'll do this only in the development branch for a start, as I'm still not sure where it's going.

Tim's question of how to deal with the files is a good one. My assumption is that any files attached to the student submission should just be copied into the working directory before the template-generated code is run. But then what?

The easiest approach from a developer's standpoint, as it doesn't require any extended functionality, is to use the template code as a control script, probably written in Python. It could just list the working directory to see what was uploaded and then take whatever action was necessary, e.g. unzipping a .zip attachment or compiling a Java program with a -jar <upload.jar> command line parameter. 

For extra flexibility I could add a template parameter FILE_NAMES, which contains the names of all uploaded files. One could then deal with uploaded files at the Twig level rather than at the template_scripting level. However you're limited in what you can do there, e.g., you can't add command line options for compiling. So I'm not sure that having the FILE_NAMES variable really helps much and it might just confuse inexperienced authors. Mostly you're going to have to fire the job off to Jobe for execution anyway, so dealing with the files there, in a much more powerful language, would usually be my preferred approach. Nowadays virtually all our advanced question types are written as Python scripts.

Thoughts, anyone? Particularly, Trent - what did you have in mind?

BTW: don't expect any lightning action on this - our second semester is about to start and I don't usually feel like playing with CodeRunner development while I'm teaching.