CodeRunner: Reuse 'answer' code from other question

I would like to reuse code I provided as answer in one question for testing code in another question.

As example in one question, I ask students to implement Newton's method so solve an algebraic equation. In another question (perhaps other quiz) I ask them to implement the Backward Euler method which requires a working Newton method. So to test it I need to use (in the 'question extra' field for example) my implementation of Newton's method. That leads to code duplication and if I decide to change the prototype for the Newton method for example then I would need to change that in many questions. Is it possible to somehow reduce this code duplication?

Re: Reuse 'answer' code from other question

por Richard Lobb - lunes, 4 de enero de 2021, 11:29

Support files in a question prototype get included in the working directory whenever a question of that type is run. So could you just put your newton code in a support file and import it as needed?

Re: Reuse 'answer' code from other question

por Andreas Dedner - martes, 5 de enero de 2021, 00:43

Thanks for the suggestion. I've tried out something that seems to work but want to check that that is what you meant.

I've copied the answer code for the Newton question into two files 'newton.m' and 'newton.py' and uploaded them into the 'support file' category of my prototype. I also uploaded '__init__.py'.

In the questions that use the Newton function, I've added to the 'answer' box (Python part):
from newton import newton
(nothing to be done for Octave).

That works well.

The only code duplication I'm left with is in the 'answer' box for the actual Newton question which contains the code. I could of course just write 'from newton import newton' but I would like to show the actual code to students after the quiz ends. Having that code duplication is no real problem - but I wanted to check if you have a suggestion for that - except using JavaScript :-)

Re: Reuse 'answer' code from other question

por Richard Lobb - martes, 5 de enero de 2021, 10:55

What you're doing is exactly what I was suggesting, but I hadn't considered that you might want students to see your own answer. The only solution to that that I can think of is to instead use template parameters in the prototype to contain the code, e.g.

{
"newtonpy": "def newton(...)\n...\n...",
"newtonoc": "[f, n] = function newton(...)\n ... \n..."
}

Then use TwigAll to expand out the code where you need it as, e.g. {{ newtonpy }}.

This is clumsy to maintain because you need an external program to convert the code to a JSON string, escaping double quotes and replacing newlines with '\n'.

The next version of CodeRunner will let you write template parameters using other languages as preprocessors so, for example, you'll be able to write the template parameters as a Python program, with triple-quoted strings. The output of that program will be the required JSON.

Re: Reuse 'answer' code from other question

por Andreas Dedner - martes, 5 de enero de 2021, 12:29

Hi Richard.

I think that suggestion went a bit over my head. I came up with a solution where I still have the code in the answer box but with a question template can provide the name of the script, the content of which is then compared with the answer. So I test 'newton.m + newton.py == answer'. This way I at least get automatically flagged if there is a difference between the uploaded scripts and the answer. I'm happy with that for now.

If I can make a suggestion:
I would quite like a 'test description' field which is put as a title (boldface perhaps) above each test in the example/result table where the author can give a brief statement to the students why this test was added. I've hacked my question prototype to include this optionally (without the boldface) which I want to use at least for the first few questions to get the students to consider why certain tests have been added.

Re: Reuse 'answer' code from other question

por Richard Lobb - martes, 5 de enero de 2021, 22:42

If you're happy with the support files approach then I'm happy too. That certainly gives you the most maintainable way of editing your solutions, at the cost of having to manually update sample answers when you change your solutions.

As to test descriptions: I normally just put comments in the test code to explain what's being tested in each case (when necessary). I'm not clear why that's not sufficient, except it doesn't appear in bold face.

Re: Reuse 'answer' code from other question

por Andreas Dedner - miércoles, 6 de enero de 2021, 01:52

It's not really an issue. I just thought that having the description highlighted might be nice for the students to focus on it. But of course I don't know how difficult it is to actually add something like that.

In my setting the issue came up that I would need to have the description twice (once for the matlab block and once for the python block). Or somehow make it possible for the grader to split off a description and add it to the output independent of the language. But that is of course specific to my setting.

I tried to use the 'Extra template data' field available for each test. From the documentation I wasn't quite sure if this is like 'global extra', i.e. a text field or like 'template params' i.e. a json structure where I can add multiple fields? I guess my question just I be able to write {{ Test.params.description }} in my grader and add { "description": "Super test" } into the extra template data field - I tried that but it didn't work that way.

Re: Reuse 'answer' code from other question

por Richard Lobb - lunes, 11 de enero de 2021, 12:30

I do need to clarify how the template parameters and twig expansion works, I admit. In the development version that I'm working on I've refactored the template parameter code to allow other preprocessors than Twig, so now would be a good time to rewrite the documentation.

For now I don't have a solution to your wish to display the test description highlighted. But I've just updated the code in the development version so that it allows the use of '%h' column format specifiers (i.e. raw html) in the "For example" table as well as the result table. The development version also allows you to define Twig macros in the prototype. The combination of those two features should allow you to format your tests in a better way. But you'll have to wait another month or so :)