Developers' Forum

question about section 8.1

question about section 8.1

by Jim Newton -
Number of replies: 4

I find the following text in section 8.1:  https://coderunner.org.nz/mod/book/view.php?id=198&chapterid=817

It is recommended that template graders be written in Python, regardless of the language in which the student answer is written, and that Python's subprocess module be used to execute the student code plus whatever test code is required. This ensures that errors in the syntax or runtime errors in the student code do not break the template program itself, allowing it to output a JSON answer regardless. Some examples of per-test template graders are given in the section Template grader examples.

I don't understand the logic of this recommendation.   It seems to me that someone who does know Python (such as myself) would rather write the checking code in his language of choice.  But for that he would need some simple harness code in Python which does the following things 1) calls a specified subprocess passing along a serialised versions of the QUESTION and STUDENT objects, 2) collects its stderr (which should be empty) and outputs a json string if there was stderr indicating error-in-subprocess, 3) outputs a json string indicating error-in-subprocess if the subprocess does not print a valid json string, 4) detects whether the subprocess failed to exit with status=0 (crashed etc) and output an appropriate json string indicating error-in-subprocess, 5) takes the json string produced by the subprocess and returns it to the system.

In reply to Jim Newton

Re: question about section 8.1

by Richard Lobb -

Thanks for the feedback on the documentation, Jim. I'm moving these posts to the Developers Forum, since they're mostly of relevance to developers. Please make further such postings to that forum.

In reply to Jim Newton

Re: question about section 8.1

by Jim Newton -

I hope it is obvious that there is a NOT missing in my claim above.  The sentence should be:

 don't understand the logic of this recommendation.   It seems to me that someone who does *NOT* know Python (such as myself) would rather write the checking code in his language of choice.

In reply to Jim Newton

Re: question about section 8.1

by Tim Hunt -

I think that is the point.

Some people, who happen to know python, have shown that if you get creative, then you can do all sorts of cool stuff by making the template into a script to do arbitrary processing.

I think the recommendation is "if you want to do cool stuff, you can. Here are our examples (in python)".

If you don't konw python, you can probably do something similar in another language of your choice - but no-one else has doen it yet, so you will have to work out the details.

In reply to Tim Hunt

Re: question about section 8.1

by Jim Newton -

In my opinion (others may disagree) there are two things to consider.  1) API: what does the system expect and what are the protocols it supports.   2) how to do cool creative things in python.

In my opinion these are two different things, but unfortunately the documentation conflates them.