Question Authors' Forum

Matlab

 
Picture of Kristy Hansen
Matlab
by Kristy Hansen - Friday, 20 May 2016, 4:58 PM
 

Hi,

I'm new to CodeRunner and am having trouble finding documentation on setting up Matlab/Octave questions. Is it possible that you could send me some examples so that I can try to understand how it works? Is a template required? I am having trouble working out what I should put in the empty fields without knowing what the template is.

Regards,

Kristy

Picture of Richard Lobb
Re: Matlab
by Richard Lobb - Friday, 20 May 2016, 10:31 PM
 

Hi Kirsty

Out of the box, there's only one Octave/Matlab question type, called Octave function

You can find out a bit more about any selected CodeRunner question type by opening the Question Type Details section in the question authoring form and you can see the template that each question type is using by clicking the Customise checkbox. In the case of Octave function, you'll be told that the question type is experimental and has had very little use, but that's not true any longer. I'll remove that on the next push.

The template for the Octave function type is simply

{{ STUDENT_ANSWER }}

format free
        
{{ TEST.testcode }};

which means that the script passed to Octave for execution just consists of the student's code (whatever they pasted into the answer box), a command to use Octave's 'free' mode for output formatting (see the Octave documentation) and then whatever code you've chosen to paste into each of the Test case <n> boxes.

Here's what a trivial "Write an Octave function" would look like to a student:


I attach a Moodle XML export of this question; if you import this and open it you'll see how it's set up.

There are two other Octave question types provided if you import the file uoc_prototypes.xml from the db folder in the source into the Moodle Question bank. They are called Matlab function and Matlab script, though the names are slight misnomers as they actually use Octave. [We have several other Matlab question types that we use in-house, but they're way more complicated and not appropriate if you're just starting to use CodeRunner.] The Matlab function question type is more-or-less the same as the Octave function type except that the template starts with a re-declaration of the disp function that more-closely mimics Matlab's disp function so that the output the students see in a CodeRunner question is the same as they get (usually) when they run their code in Matlab. It doesn't use Octave's free output format mode.

Matlab script is a somewhat different type of question. The question type details for it say "Used for Matlab scripts (so-called). Runs the test code first (which probably sets up a context) and then runs the student's code, which may or may not generate output dependent on the context. Finally the extra code is run (if any)." I rarely use this. It allows you to ask questions of the form Given variables a and b, write code to compute or print some derived value c. The test code defines a and b, the student's code computes or prints c. If they just compute c, you can put code in the Extra field of the question authoring form to display c yourself, or check it in some other way. This question type is useful only for introductory questions when students are just learning to write simple assignment statements or expressions. Getting them to write functions early in their course provides a much safer context for feeding input parameters to them and getting printed output or computed return values back again.

If your students are developing in Matlab and submitting code to be graded with Octave, you should be warned that there are a few gotchas, particularly when students submit bad code - Matlab accepts all sorts of rubbish that Octave warns them about. There are also subtle differences in how some functions work and of course most of the toolboxes are missing from Octave. But if your students are using Octave too, none of these issues matter.

You should also be warned that regardless of whether you're using Octave or Matlab, you must be very careful with floating point tests, as the student's full-precision numeric output will only rarely be the same as yours. At very least you need to use fprintf to limit the precision to which numeric data is displayed and even then things can go wrong. One of my colleagues has developed an Octave question type that compares all floating point numbers to a specified tolerance, but this is a very complex question type that does lots of other clever stuff as well (e.g. lots of style checking) so ask us about it in a year or so :-)  In the meantime, if you run into this problem, you can always write more sophisticated tests that, rather than just printing the student's answer, compare it with the expected output and print a message like YES or NO (so the expected output is then YES).

Hope that's enough to get you going,

Enjoy,

Richard

Picture of Kristy Hansen
Re: Matlab
by Kristy Hansen - Wednesday, 25 May 2016, 11:57 AM
 

Dear Richard,

Thank you very much for the comprehensive reply :) I have been making great progress since I last wrote to you and have managed to code up several successful questions! I managed to import the .xml example that you attached to your message but I cannot find the db folder in the source for the other two examples. Should I be looking for a folder called "source" ?

Thanks again for your help.

Regards,

Kristy

Picture of Richard Lobb
Re: Matlab
by Richard Lobb - Wednesday, 25 May 2016, 8:41 PM
 

Hi Kirsty

Sorry for the confusion. I meant the db folder within the CodeRunner source. You'll find it in the github repository, here.

Richard

Picture of Kristy Hansen
Re: Matlab
by Kristy Hansen - Sunday, 29 May 2016, 4:34 PM
 

Hi Richard,

Thank you for the link and I'm sorry to bother you again but I am not sure how to save the .xml file from that location in order to be able to import it. I am not at all familiar with .xml format so that probably doesn't help :(. Can you please tell me how I can do this?

Regards,

Kristy

Picture of Richard Lobb
Re: Matlab
by Richard Lobb - Sunday, 29 May 2016, 7:33 PM
 

Hi Kristy

Steps are as follows:

  1. In your browser, navigate to https://github.com/trampgeek/moodle-qtype_coderunner/tree/master/db
  2. Left-click the link to the xml file you want e.g. uoc_prototypes.xml
  3. Left-click the "Raw" button at the top of the text window. That should get you a browser window containing just the xml file itself, without other stuff like line numbers.
  4. Save that file (probably by typing CTRL/S though it's browser/OS dependent) somewhere suitable, say as "questions.xml".
  5. Use Moodle's Course Administration menu to find Question Bank > Import. Click that link.
  6. You should now be looking at a Import questions from file form. Select Moodle XML format as the file type and use the Choose file dialogue box to select the saved questions.xml. Click Upload this file
  7. Click the blue Import button. You should get presented with a list of all the questions you just imported.
Hope it works this time. If not I'll just email you the files :)

Richard

Picture of Kristy Hansen
Re: Matlab
by Kristy Hansen - Monday, 6 June 2016, 1:34 PM
 

Hi Richard,

Sorry to make this seem so complicated! But I can't save the file as .xml. It just isn't a listed option for me. I can only save as .txt and then that format is not accepted using the import option. I would really appreciate it if you could email me the file :)

Thanks,

Kristy

Picture of Richard Lobb
Re: Matlab
by Richard Lobb - Thursday, 9 June 2016, 9:54 PM
 

Hi Kristy

OK, I attach an XML file containing the two prototypes: matlab_function and matlab_script. However, please please don't use these unless you understand question prototypes properly (see the CodeRunner manual, section 11). Normally, prototypes are installed by a system administrator at the system level, where they can't be altered by users. Experienced users manage their own question prototypes, but it is important that you know what you're doing. Several users have crashed CodeRunner by replicating prototypes, thinking they were sample questions for them to experiment with.

A pertinent question at this stage might be: what is the problem that you're trying to solve? If you're trying to ask a particular question that isn't supported by the existing question types, the first step is usually to edit the template on a one-off basis to do what you want. Only when you find yourself using the same template over and over again do you normally start using prototypes. If you want help designing a template for a particular question you have in mind, feel free to ask.

-- Richard

Picture of Kristy Hansen
Re: Matlab
by Kristy Hansen - Friday, 17 June 2016, 4:17 PM
 

Hi Richard,

Thank you very much for the attachment. I can appreciate what you mean when you say that you need to be careful with prototypes in order to avoid crashing the system.

I was actually hoping to create a question where students design a user defined function. Could you help me with that? I guess the prototypes that you sent are not really designed for this are they? Actually, I am wondering why the octave_function prototype is so much simpler than these latest versions that you sent? Are the latest ones more robust?

So far I have just been using the following template (slightly modified from octave_function) and it has been working well:

{{ TEST.testcode }};

format free

{{ STUDENT_ANSWER }}

Sorry for my delayed replies - it is a busy time of the semester for me!

Regards,

Kristy


Picture of Richard Lobb
Re: Matlab
by Richard Lobb - Wednesday, 22 June 2016, 10:02 AM
 

Hi Kristy

Good to hear that the modified octave_function template has been working well for you. I've used a similar template in the first week of my Matlab course to set questions in which I define a few variables and then ask the student to write some code using those variables to print a derived value. Presumably that's how you're using it too?

However, as soon as possible I switch to some variant of the (unmodified) octave_function template, i.e.

{{ STUDENT_ANSWER }}
format free
{{ TEST.testcode }};

With this template, I ask students to write a function to some spec and my test code then calls their function and prints the results. You can see an example of this type of question in my response to your first question in this thread - the sqr function. The exported xml for that question was attached to that posting if you want to look at it more closely. [If you're used to Matlab, you may be surprised that this template works, as in Matlab you can't define a function in a file then write global code that calls it. But Octave does not suffer from this restriction.]

Doing things this way round has several advantages:

  1. It gets students writing functions as soon as possible. While they don't initially really understand functions properly, they don't have any major problems writing ones if you specify exactly what they have to do. They just have to know the syntax for a function declaration. This sets them up much better for later in the course when they're expected to decompose larger scripts or functions into smaller ones by themselves.
  2. If you explicitly require that the student's function does not produce any output, your test code has complete control over the output from each test. This eliminates the frustration that occurs when students produce output that isn't exactly in the same format as the expected output. 
  3. Your tests can use fprintf to control numeric precision to reduce (but not eliminate) problems with rounding errors. Alternatively, your test code can compute the absolute difference between the expected and the actual results and print 'ok' or 'fail' accordingly.
  4. The environment in which the student's code runs is now completely explicit - the only variables they have available to them are the function parameters and the only result they have to produce is the function's return value. This improves the clarity of the question and also means you don't have to specify in great detail the expected output format. [You can still ask students to write functions that produce output if you like, but then of course you do need to specify the expected output carefully.]
You ask why the two Matlab prototypes are so much more complicated than the octave_function prototype. Mostly just because they include a custom disp function, written in Octave, that produces output in the same format as Matlab's disp function. My students are using Matlab for their coding but I am testing their code with Octave, so I try to minimise differences. I often test their functions by calling disp to display their answers and if they do the same tests in Matlab before submitting, and see noticeable differences between their output and the expected output, they get worried!

Richard