CodeRunner: Propositions for CodeRunner plugin

I have had an interesting exchange with a CodeRunner user in Vietnam, and I thought it was worth sharing the exchange, with his permission of course.

Dear Mr. Richard Lobb,

My name is Khang N. Pham. I am a lecturer at Can Tho University, Vietnam. I am using the Coderunner plugin on Moodle for my lectures. Since 2015, I have used it for my programming courses such as: introduction to programming in C, algorithms, graph theory, automata, compilers, assembly. I found that it is a fantastic tool for teaching, especially for teaching programming.

Recently, I have begun to use other UI plugins besides ACE. The HTML ui plugin is great! We can combine other ui's and also javascript. I attached some screenshots (applying the Dijkstra algorithm for shortest path problems).

However, I have some troubles when dealing with UI plugins and TWIG. I hope you can help me.

1. Gapfiller UI: (lines 196 -197)

// The function to handle a 'textarea' tag.
function textarea(rows, cols) {
return '<textarea name="cr_gapfiller_field" class ="coderunner-ui-element" ' +
'rows="' + rows + '" ' + 'cols="' + cols + '" style="width:auto;" />';
}

On Chrome, a tag <textarea> without an end tag </textarea> does not work properly (even though we have />, i.e. <textarea />). So I think we should change these codes to ... "width:auto;" ></textarea>';

2. TWIG

For preprocessing questions, I found that TWIG is the best but its capacity is limited. For example, it supports json_encode but not json_decode. A JSON array from template parameters can be reused in other places but a JSON object (i.e. {"meber": value}) cannot be reused directly. For example, in the template parameters, we have an object A (i.e: a dictionary). In the Template section, we cannot use it directly but we can serialize it using json_encode and parse it using json.loads with Python! We cannot use A['some key'] directly. I found that this is very strange!

So, I wrote a json_decode filter to reconstruct the object from a string. This json_decode is also useful when we use C/C++ language questions. For example, Gapfiller returns a list of strings (in format of JSON string), and we have to parse it in order to extract different parts. I have written a TWIG extension file to support filters for my purpose (json_decode and PCRE filters). I work but I do not know if it is the right approach.

Another issue is that UI parameters cannot not be preprocessed using TWIG (even twigall is checked). This is useful when UI parameters depend on template parameters. For example, the number of rows of a Table Ui is set to the N variable.

I looked in the PHP code and found that in the file "edit_coderunner_form.php", lines 826 - 827, $this->validate_ui_parameters($data['uiparameters']); is called without TWIG rendering. By consequence, all TWIG code in UI parameters are considered as errors. I did some "dirty code hacks" to overcome this problem.

I think that it will be more logical if we validate ui parameters in the function "private function validate_twigables() " (lines 1069 - 1108).

I would appreciate it if you could kindly give me some advice for these issues.

Best regards,

Khang N. Pham

KhangPhang interesting graph question

Result table image

Re: Propositions for CodeRunner plugin

por Richard Lobb - lunes, 9 de agosto de 2021, 20:27

Dear Khang N. Pham

Many thanks for your interesting email. I'm pleased to hear you're finding CodeRunner useful. And I'm impressed at your Dijkstra's algorithm question. I forwarded your email to two other staff in our department in the hope it will inspire them to similar efforts.

With regard to the various issues you raise:

1. Gapfill UI bug with textareas. Thank you for drawing this to my attention. It's clearly wrong, and I've now fixed it in my development version. The fix will appear in the next release. I am very puzzled though; I've used textarea gaps several times in the past, most recently in an exam in June this year. The exam was checked by my co-lecturer and three tutors and was sat by 200 students. No one had any issues with it. Yet today the same exam question renders wrongly. I am absolutely baffled as to what changed, since the code for the gapfiller.js hasn't changed in two years!

2. Twig and UI_parameters. I did actually realise that you couldn't twig expand the UI_parameters, but I wasn't expecting it to be a problem. Now you, and another staff member here, have made me realise I was wrong. However, I'm afraid I'll have to put that problem on my to-do list, for attention when the teaching semester ends, as it's actually more difficult to fix than you might expect. But again thanks for drawing my attention to it.

I do vaguely recall once wanting a TWIG json_decode filter. Usually though I don't feel a need for it. Certainly you can have a template parameter like

{ 
    "animal": {"name": "Dog", "sound": "Woof"}
}

and then use it within the question template, text, tests etc in the form, say,

Write a program that prints {{animal.name}}s go '{{animal.sound}}'!

But it sounds like you have a much more difficult use case - could you send me an example of a question using your TWIG json_decode filter, please?

Related to this: do you have the latest version of CodeRunner, which came out early this year, that allows you to use languages other than Twig to process the template parameters? We started using this capability in the first semester this year and it's certainly a joy to use compared to Twig. You can do nearly all the work in the template parameters field, yielding just a list of Twig variables for substitution throughout the rest of the question. I am still a little nervous of what might happen if we used too many questions of this sort in a large exam but our experience to date suggests that performance is less of an issue than I feared. Indeed, it might well be possible that the bottleneck at the start of an exam is at the Moodle end rather than on the Jobe server, even with lots of such questions. Although that's assuming you're using a fast language like C or Python for processing the template parameters; Java would be of much more concern.

One of our tutors has used this capability to do some very impressive randomisation of questions. For example he has a numerical integration question that asks students to integrate a randomised polynomial between two given limits using a trapezoidal integrator with a given number of sample points. The template parameter code generates all the various test cases and their results and even generates an example image for inclusion within the question text. The image (generated as a data-URI) shows the curve to be integrated and, if the student mouses over it, superimposes an image of the shaded trapezoids whose areas should be computed and summed. The image and its overlay are of course different for every instance of the question.

One final thought: I think discussions of this sort might be of interest to other CodeRunner users. Would you mind if I posted your email and my response on the Developers Forum on coderunner.org.nz? I think that would also be a good place to continue such discussions or other ones of a similar nature.

Again, thanks for the interesting email and for raising the issues of the gapfiller UI and the non-expansion of UI_parameters by Twig.

Regards

Richard

Re: Propositions for CodeRunner plugin

por Richard Lobb - lunes, 9 de agosto de 2021, 20:47

Khang Pham's reply ...

Dear Richard,

Thank you very much for your prompt reply. I would be very happy if you posted my email and your response on the Developers Forum on coderunner.org.nz. It is a good chance for me to learn from other developers and experts.

I think that I have the latest version of coderunner plugin.

$plugin->version = 2021031400;

Actually, I am just a newbie in TWIG. I have just used it some time ago. You are right. TWIG objects work very well as you mentioned. However, what I really need is to use an object as a dictionary (i.e. an associative array). For example, in the Template parameters, I have:

===

{% set a = {"key": 123} %}
{{ a["key"] }}
{{ a.key }}

===

Both work perfectly! But when I put an object as an parameter like this:

{

"a" : {"key": 123}

}

and use it within the template

only the second works. The first one prints nothing! So, I found that very strange!

So, I did a trick like this:

Within the template,

{%set a_new = a|json_encode|json_decode %} {{ a_new['key'] }}

It works!

I guess that this relates to something called TWIG namespace.

Another use-case for json_decode is:

I created a C question which asks students to declare a struct and use it.

#include <stdio.h>
//Declare a struct of Point 
...
int main() { 
    struct Point a;
    //Write codes to read data from stdin to the variable a    
    ...
    printf("(%.3f, %.3f)", a.x, a.y);
    return 0;
}

Students have to fill two gaps. The Gapfiller UI plugin is very appropriate for this kind of question.

And in the template section, I set up:

===================

#include <stdio.h>

{% set STUDENT_ANSWER = STUDENT_ANSWER | json_decode %}

//struct definition
{{ STUDENT_ANSWER[0] }}

int main() {

//variable declaration
struct Point a;
//read data
{{ STUDENT_ANSWER[1] }}

printf("(%.3f, %.3f)", a.x, a.y);
{{ TEST.testcode }}
return 0;
}

================

In fact, Gapfiller returns a JSON string, and I have to decode it to a list. Without json_decode, I cannot complete my work. Although we can use a python template to handle what a Gapfiller (or other plugins) returns, I think that it is very complicated in this case. We have to fill student answers in gaps, write the source code to file, compile and run it, ...

The json_decode is something like this:

function twig_json_decode($str)

{

return json_decode($str, true);

}

Beside, I found that a random permutation of a list is also interested. It is very good for ramdomized question. To this end, I also wrote a "shuffle" filter.

function twig_shuffle_filter($array)

{

if ($array instanceof \Traversable) {

$array = iterator_to_array($array, false);

}

shuffle($array);

return $array;

}

When I design questions for my Java course, sometime I prefer a regex replace filter to a string replace. For ex,. a student can write "public class A" (with several whitespaces between public and class) instead of "public class A". So a preg_replace is really needed!

function twig_preg_replace_filter($str, $pattern, $replacement = '', $limit = -1)

{

if (!isset($str)) {

return null;

}

return preg_replace($pattern, $replacement, $str, $limit);

}

Related to preprocessors in template parameters, I am affraid of using other languages than TWIG because of your warning "overload the Jobe server". Another problem with twig preprocessor is that there is no error explanation except "we got Unknown error" :-(

Do you think a C preprocesser is better? If we use TWIG, preprocessing is in charge of Moodle, otherwise, it is run on Jobe server. I don't know if the Moodle is overload. I have tried to use C as preprocessor but I did not test on a large exam.

Each semester, we have about 800 students who use Moodle + Coderunner for their exams. We divided them in groups of 250 for every session. The system worked well with C questions without preprocessing. I would like it will works well with preprocessing (no overload occurs). I will test system capacities in this semester.

Because of COVID-19, online teaching will become an inevitable trend. I would like to do something for my students. They can learn themself without tutors. So, it is interested if we can generate as many as randomized questions. It is also a good solution for anti-plagiarism!

Again, thank you very much for your consideration on my email.

Best regards,

Khang

Re: Propositions for CodeRunner plugin

por Richard Lobb - lunes, 9 de agosto de 2021, 21:44

Dear Khang

Your observation that {{ a["key"] }} doesn't work but {{ a.key }} does is interesting. I don't fully understand this myself, but I can cast a little light on it.

In the first example you quote, the variable 'a' is defined within Twig itself. But when you define 'a' via a template parameter it is actually a JSON object, that is evaluated (using PHP's json_decode) when the question attempt begins to yield a PHP value. That value is then passed as a parameter to the Twig render method of the template. In my call to json_decode I use my default settings, which are to yield a PHP object. It seems that Twig respects the object nature of the parameter and rejects the first of your two usages. However, if I change the json_decode call to instead yield a PHP associative array, it seems that both your two usages work.

So it might well be that a change to the call to Twig render would allow your desired associative usage. However, I'm sorry, but I'm not prepared to make that change, as I don't know what other consequences there might be. You're the first person who has ever drawn my attention to this and I feel the risks are just too great. Happily, though, it seems you've found a solution yourself, via your clever json_encode filter. Alternatively - and this might be easier - I think you can use Twig's attribute function to access a particular attribute of an object, e.g.

attribute(a, "key")

seems to work (though I've never used this before myself).

You way of using the gapfiller UI for use is interesting. As you say, it does seem on the face of it much simpler than using a Python template to reconstruct the code then compile and run it. All the same, that's what we do. We have made this into a question type (or two actually, one for Python and one for C) so it only has to be done once. Thereafter, in all gap filler questions you simply fill in the globalextra with the code into which gaps are inserted. Once saved, the question answer field displays the code with gaps in it, and you can fill out the sample answer by just filling in the gaps, like the students do. The other big advantage of this approach is that you can now add other constraints, such as disallowing certain constructs, requiring others and so on, via template parameters. You can see an example of our python3_html_gapfiller on github here. You'll also find an exciting new style of gapfiller question implemented recently by Matthew Toohey, which implements on-line gapfiller questions within the Ace editor, which looks much nicer than the HTML gap filler. Unfortunately you won't be able to use this yet as it uses a new UI plugin and also requires a tweak to the PHP code, neither of which are yet available in the master branch.

You'll doubtless be horrified at the complexity of those question types but remember they only have to be written once. Compiling and running C or C++ from within such a template is a relatively trivial addition, and indeed we always run such languages from within a Python template nowadays in order to implement style constraints or perform transformation to the student's code or control compiler options, etc.