CodeRunner: Regex for analysing student code

Hello everyone,

First of all, thank you very much for this forum, which help me understanding this very good product.

We decided last week to choose, in our french company, to use CodeRunner to evaluate our atendees.

I'm very glad of that.

I never used such an automate, but i think that that marking someone only by the correctness of the printout is not sufficient to decide if he is able to create a good program. He can for example use a very badly constructed program, but which print in fact the good solution. He can take another easy way to do things, to avoid using lambda in Java for example, by replacing by a do loop.

I know that this question has already be set, but in another langage.

I would like to be sure, for example, that the student uses the Thread class.

I want him to write for example :

public class Programme{

public static void main(String args[]){

Test t = new Test();

t.start();

}

class Test extends Thread{

public void run() {

int i = 1;

System.out.println("From Thread" + i);

}

Then, i wrote a very short template using TWIG which is :

{% set cond1 = ('extends Thread' in STUDENT_ANSWER ) %}

{% if (cond1==true ) %}

{% else %}

public class Programme{

public static void main(String args[]){

System.err.println("Thread is not used in your program !");

System.exit(1);

}

{% endif %}

Is there an easier way to do what i want ?

and, what i would like to know about REGEX, is it possible to use REGEX so that patterns like

extends Thread

be accepted ?

I tried many possibilities like

{% if STUDENT_ANSWER matches '{^.+extends +Thread.+$}' %}
    
{% endif %}

But none works.

It it the right way to try it in TWIG ?

Or does the code analysis has to be done with another langage independant of the langage to to compiled ?

Many thanks for all,

Pierre from Paris

Re: Regex for analysing student code

by Richard Lobb - Tuesday, 10 December 2019, 10:56 PM

Taking the various issues/questions in turn:

It is most certainly true that "marking someone only by the correctness of the printout is not sufficient to decide if he is able to create a good program". This is a hard and ultimately subjective problem that can't be solved by a computer. But you certainly can (and should) subject student code to some sort of style check. In Python we use pylint plus lots of additional local style checks. I believe checkstyle is one of the standard Java checkers, but I don't teach Java. However, even code that passes style checks can be rubbish (dreadful choices of variables, meaningless comments etc). At some point, humans have to inspect code.
"Is there an easier way to do what i want ?" I'm not sure that there's an easier way for a one-off question like that. But if you're going to be doing many such checks, you'd want to write your own question type (see the documentation) which uses that template to ensure threads are being used or, better, that requires the code matches a (or a set of) regular expression(s) specified in a template parameter. Here at Canterbury nearly all question authors script their question types in Python regardless of target language, because it makes checking much easier. See the documentation.
Twig regular expressions use PERL syntax: the expression must be delimited by a single character (optionally followed by modifiers). In your case, the line you want is
```
{% if STUDENT_ANSWER matches '/^.+extends +Thread.+$/' %}
```
or replace the '/' with any other character of your choice. Also, since matches matches anywhere within the string you can dispense with the '^.+' and '.+$'.

A useful site when investigating Twig is twigfiddle.com - you just enter your twig code, evaluate it, and see what you get.

Re: Regex for analysing student code

by Pierre FACON - Wednesday, 11 December 2019, 12:42 AM

Thanks a lot Richard for your very valuable replies and work you do for that project.

Pierre from Paris