Question Authors' Forum

File upload/submission question type

 
Picture of Richard Lobb
Re: File upload/submission question type
by Richard Lobb - Tuesday, 4 July 2017, 1:52 PM
 

OK, I'll look into copying appropriate bits of the essay question type into CodeRunner to allow file upload, but I'll do this only in the development branch for a start, as I'm still not sure where it's going.

Tim's question of how to deal with the files is a good one. My assumption is that any files attached to the student submission should just be copied into the working directory before the template-generated code is run. But then what?

The easiest approach from a developer's standpoint, as it doesn't require any extended functionality, is to use the template code as a control script, probably written in Python. It could just list the working directory to see what was uploaded and then take whatever action was necessary, e.g. unzipping a .zip attachment or compiling a Java program with a -jar <upload.jar> command line parameter. 

For extra flexibility I could add a template parameter FILE_NAMES, which contains the names of all uploaded files. One could then deal with uploaded files at the Twig level rather than at the template_scripting level. However you're limited in what you can do there, e.g., you can't add command line options for compiling. So I'm not sure that having the FILE_NAMES variable really helps much and it might just confuse inexperienced authors. Mostly you're going to have to fire the job off to Jobe for execution anyway, so dealing with the files there, in a much more powerful language, would usually be my preferred approach. Nowadays virtually all our advanced question types are written as Python scripts.

Thoughts, anyone? Particularly, Trent - what did you have in mind?

BTW: don't expect any lightning action on this - our second semester is about to start and I don't usually feel like playing with CodeRunner development while I'm teaching.

Richard


Picture of Trent Lewis
Re: File upload/submission question type
by Trent Lewis - Thursday, 9 November 2017, 2:53 PM
 

Getting back into my CodeRunner head space (many months later)!

Did you progress any further with this?  I like your thinking above.  I agree that if a submission is a single file (that is text only) that it should be just a cut'n'paste into an answer box.  But larger, well-formed code might require multiple files (e.g. each class in Java should be in a separate file).

I do also see that this file upload for complex, multi-file code is probably not what CodeRunner is conceptually designed for as a quiz is more about testing parts of code, rather than large assignment submission.  However, CodeRunner works so well that it'd be a great extension ;)

I could see it as the student produces the smaller parts in the first few questions of the quiz and then needs to bring it all together with a multiple file submission (or a jar/binary) to show understanding of the bigger picture would be good.  This allows students to get marks for the "simpler" components and those who can conceptually bring it together gain extra marks at the end.

So, I am happy with having a file upload and then processing in python.  A FILE_NAMES would be a great idea.

Happy to test out anything.

Trent

PS. just about to starting investigating the CodeRunner and version control idea...

Picture of Richard Lobb
Re: File upload/submission question type
by Richard Lobb - Friday, 10 November 2017, 5:10 PM
 

This is very timely, Trent, as I'm just switching back into CodeRunner development mode, now that our teaching term has (almost) finished. I've got a list of things I want to do; file upload is on the list, near the top, so it should happen.

I think I'll just treat files uploaded by the students the same as support files included by the author. They just get copied into the working directory. Also, they're read only so they can't get changed when testing, which could introduce unwanted side-effects between tests.

I'll post back when it's done.

Richard


Picture of dirk grunwald
Re: File upload/submission question type
by dirk grunwald - Thursday, 7 December 2017, 5:42 AM
 

You may also want to look at http://www.inginious.org -- this is a autograding system that can interface with Moodle using LTI. It doesn't fit into a 'quiz context' but provides more capabilities on the backend grading (...because you have to cons it up yourself using a shell script..). It uses docker containers during evaluation so is slower / less responsive.

There are other similar options we've explored - we use coderunner extensively in our labs, but are looking for "assignment grading" solutions that need to integrate with version control but don't need to be as responsive.

For browser based development, there's the Moodle "VPL" ( http://vpl.dis.ulpgc.es/ ). We've also been using Cloud9 (c9.io) but that's now merging into Amazon and harder to use. Other web IDE's like codenvy are an option, but we haven't explored them yet.

We've considered inginious.org, autolabproject.org and okpy.org as replacements for an in-house solution. Our current thinking is to go with inginious in the short term and then adopt a solution using a university hosted gitlab and standard CI tools like jenkins. To handle the "file-up load", we were going to create an LTI application that uses a token to manipulate the gitlab instance and e.g. check in a student's solution for them. This would mean a simple interface for students who have yet to master git, but full git functionality for later classes.

Picture of Trent Lewis
Re: File upload/submission question type
by Trent Lewis - Thursday, 7 December 2017, 12:20 PM
 

Thanks for the hints and pointers.

I have just the other day successfully created a question type in CodeRunner that takes as an ‘answer’ an SVN revision number and then the jobeserver checkouts from a locally hosted subversion server, compiles and runs (using ant) and then compares against the tests given in the CodeRunner question setup.  The student just uses a standard IDE to develop the code and commits to the repository.  Allowing the student to specify the revision number as the ‘answer’ gives them control over what they submit and allows multiple submissions.  They could even, in theory, complete all the work, committing each stage/checkpoint  to the revision control system and then go to Moodle/CodeRunner and submit each of the revision numbers that satisfy the different questions.

It still needs some fine tuning but happy to share once I have made it a bit more generic and sort out some quirks.  One of the trickiest parts was getting the University IT to allow holes in the various firewalls!

I’m sure Richard will give me a slap over the wrist for this as I’m sure that is not the intention of CodeRunner :)

Trent

Picture of Richard Lobb
Re: File upload/submission question type
by Richard Lobb - Thursday, 7 December 2017, 11:00 PM
 

Certainly no slap over the wrist from me - it sounds great, Trent! Please share. 

I did something vaguely similar a few years ago when teaching a web programming course - the student's "answer" was an URL of a website they'd implemented, and the template code then explored and graded their site. And like you, I had to punch holes in the firewalls to make it happen.

I'm only a control freak with regard to CodeRunner itself - the more uses you can find for it the better, as far as I'm concerned :)

Richard

Picture of Trent Lewis
Re: File upload/submission question type
by Trent Lewis - Friday, 8 December 2017, 12:52 PM
 

I’ll keep you up to date and share it when it’s done. 


Trent