Question Authors' Forum

How to implement Regex Grader?

How to implement Regex Grader?

by Khen Brian -
Number of replies: 12

Hello,

I have been exploring the possibilities of cr in implementing an HTML-question-type which has been successful and is exciting.

However, I need some direction on how to implement the regular expression grader for the new question-type.

Here's a scenario: I am expecting the opening and closing tag with some data in the answer But I want to provide the flexibility of expecting any data in between the opening and ending tag. i.e. <h1>{any data}</h1>. I am using the Python HTML Parser. Hence when any code is provided as the answer to a question, it detects the tags sequentially. So I want to write a regular expression that would accept any word character.

I am not sure where exactly to write the regular expression whether in the Expected column of the test case when defining the question-type or in the test case for each question that uses the newly defined question-type. 

I have already selected the Regular Expression as the Grading option when I was creating the Question-type I'm using for this question. I am currently trying to write the regex in the Expected column of a test case of this question but it isn't working.

Regex in question test case


This is the results:

ResultsBut I want to accept any word character for the Encountered some data row using regex, how do I go about it?

Thanks in advance.

In reply to Khen Brian

Re: How to implement Regex Grader?

by Richard Lobb -

The Expected field is a single regular expression, without PERL-type delimiters. Thus you should drop the '/' characters and you might as well drop the parentheses as well, as there's no way you can get at the captured text. So your Expected field should be just

Encountered a start tag: h1
Encountered some data : .*
Encountered an end tag : h1

I'll update the in-line documentation in the next release to make this a bit clearer.

Alternatively, since you're apparently using a Python question type, you could do the regular expression matching within the template instead and, using an Equality Grader, print a very simple output like "OK" if everything matches and a much more comprehensive error output otherwise.


In reply to Richard Lobb

Re: How to implement Regex Grader?

by Khen Brian -

I will implement the alternative solution as well but thank you. That was what I needed

Brian

In reply to Richard Lobb

Re: How to implement Regex Grader?

by Khen Brian -
Hello Regard,

A follow-up on this.

The above solution works:

The Expected field is now just

Encountered a start tag: h1
Encountered some data : .*
Encountered an end tag : h1
REgex works
however, if any data is inserted before or/and after the expected result(s), the RegexGrader marks the answer to be correct which isn't what I want.
REgex worksas
 Why does the RegexGrader behave as such and how do I go around this: so any data introduced contrary to the expected results would fail the text case?

Thanks in advance.

In reply to Khen Brian

Re: How to implement Regex Grader?

by Richard Lobb -

As per the on-line documentation (click the question mark beside the Grader section in the author editing form):

"The 'regular expression' grader uses the 'expected' field of the test case as a regular expression and tests the output to see if a match to the expected result can be found anywhere within the output. To force matching of the entire output, start and end the regular expression with '\A' and '\Z' respectively. Regular expression matching uses MULTILINE and DOTALL options."


In reply to Richard Lobb

Re: How to implement Regex Grader?

by Khen Brian -
Thank you for the clarification, it works!

In an attempt to implement the alternative solution you suggested: "you could do the regular expression matching within the template instead and, using an Equality Grader". 

I tried that and I discovered that wouldn't allow me to grade a student's answer as correct when the student inputs any kind of data in between the HTML elements because the Equality Grader expects an exact match, therefore, it can't accomplish the solution to the problem I wanted to solve initially before turning to  RegexGrader. Perhaps I misunderstood you.



In reply to Khen Brian

Re: How to implement Regex Grader?

by Richard Lobb -

I may be misunderstanding what you're doing, but here's the sort of thing I had in mind.

Currently I assume you have a template of the form

class MyHtmlParser(HtmlParser):
    def handle_starttag(self, tag, attrs):
        print("Encountered a start tag:", tag)
    ...

s = """{{ STUDENT_ANSWER | e('py')}}"""
parser = MyHTMLParser()
parser.feed(s)

So what you can do instead is switch back to an equality grader, set the expected field of the test to just "OK :)" or some such and move the current expected regular expression to the test's Extra field where the student can't see it. Turn off iscombinator. Then change the template along the following lines:

import re
class MyHtmlParser(HtmlParser):
    output = ''

    def handle_starttag(self, tag, attrs):
        MyHtmlParser.output += f"Encountered a start tag: {tag}"
    ...

s = """{{ STUDENT_ANSWER | e('py')}}"""
parser = MyHTMLParser()
parser.feed(s)
expected = """{{ TEST.extra | e('py') }}"""
if re.fullmatch(expected, parser.output):
    print("OK :)")
else:
    print("Sorry, no banana")

That should work but the feedback to the student isn't particularly helpful (though arguably not much less helpful than a regular expression, unless your students actually understand regular expressions - mine don't). 

If you want to provide more useful feedback, one option might be to do a splitlines() on both the expected and the parser.output and loop, matching lines, until a mismatch. Display the failed match (and probably previous matches for context). Or you can do fancier things, like a diff between the got and expected (see the Python difflib library). It depends how much effort you want to put into this to improve the feedback quality.

An alternative to rewriting the MyHtmlParser as above is to capture print output by setting sys.stdout to a StringIO object before calling the parser, then setting it back to its original value afterwards. Less work but also less elegant.

Or am I misunderstanding what you're trying to achieve here?

In reply to Richard Lobb

Re: How to implement Regex Grader?

by Khen Brian -

Hello Richard,

Sorry for the delay, in between Exams now but yes, that's what I intended to implement so you understood me correctly. 

I will try implementing this solution and revert back to you.

I also wanted to find out if it is possible to perform and/or integrate automated testing in codeRunner?

Thank you.

In reply to Khen Brian

Re: How to implement Regex Grader?

by Michael Backus -

Khen,

I am interested in creating similar questions for my students. Do you have a Question Prototype and some sample questions you would be willing to share to help get me started?

In reply to Michael Backus

Re: How to implement Regex Grader?

by Michael Backus -
Figured out how to implement the Full Match method Richard described. Attached the prototype I created along with a demo question for Khen's benefit as well as anyone else who's interested in developing HTML questions.

Next steps:
  1. Incorporate a live editor
  2. Develop a way to provide really good feedback.
In reply to Michael Backus

Re: How to implement Regex Grader?

by Khen Brian -
Hi Michael,

Sorry for the late reply. 

Thank for sharing. I will incorporate yours and keep you informed on any new developments.

Kindly advice if you figure a way with your next steps especially automated testing.

Thanks again.
In reply to Khen Brian

Re: How to implement Regex Grader?

by Michael Backus -

I've been forging ahead with my question prototype and a couple sample questions. I've posted my latest versions in a response to the Grade HTML thread I started a few days ago. It's cool to see somebody else has a similar idea at about the same time. I'd like to collaborate if you're interested.

Here's my short term plan for the first few days worth of assignments:

  1. Tags Quiz
  2. Attributes Quiz
  3. Styles Quiz
Each quiz will be made up of 5 to 10 description/question pairs where a new tag/attribute/style will be introduced using a description and followed up with 1 or more questions.

What is your plan? Any ideas/feedback you have would be greatly appreciated.

Richard is willing to create a repository to help us share questions. Is that something you'd be interested in? Also, my prototype is still a work in progress. If we end up sharing, I'll need to figure out a way to assign version numbers so that we can ensure we're both using the latest and greatest version.

Might also be helpful to have a more direct method of communication. Use my Contact Me page and I'll reply via email.

In reply to Michael Backus

Re: How to implement Regex Grader?

by Khen Brian -

I'm interested in collaborating. However, I was working on this project on a limited contract and I have to handle it over to the next colleague because I will be working full-time else where. I will inform the project manager, so the whoever will take over from where I ended could contribute in their capacity.

The previous plan was to implement the tags questions to enable student get familiar with the HTML tags Hence, I was working solely on tag quizzes so I could also the upload prototype I implemented following my discusses is Richard and sample questions to the repository. 

I will keep you updated hopefully after my exams and discusses with the project manager on future plans because to the best of my knowledge the project is meant to be implemented to help students with the basics of web development courses such HTML, CSS & JS.