Question Authors' Forum

I may be misunderstanding what you're doing, but here's the sort of thing I had in mind.

Currently I assume you have a template of the form

class MyHtmlParser(HtmlParser):
    def handle_starttag(self, tag, attrs):
        print("Encountered a start tag:", tag)
    ...

s = """{{ STUDENT_ANSWER | e('py')}}"""
parser = MyHTMLParser()
parser.feed(s)

So what you can do instead is switch back to an equality grader, set the expected field of the test to just "OK :)" or some such and move the current expected regular expression to the test's Extra field where the student can't see it. Turn off iscombinator. Then change the template along the following lines:

import re
class MyHtmlParser(HtmlParser):
    output = ''

    def handle_starttag(self, tag, attrs):
        MyHtmlParser.output += f"Encountered a start tag: {tag}"
    ...

s = """{{ STUDENT_ANSWER | e('py')}}"""
parser = MyHTMLParser()
parser.feed(s)
expected = """{{ TEST.extra | e('py') }}"""
if re.fullmatch(expected, parser.output):
    print("OK :)")
else:
    print("Sorry, no banana")

That should work but the feedback to the student isn't particularly helpful (though arguably not much less helpful than a regular expression, unless your students actually understand regular expressions - mine don't). 

If you want to provide more useful feedback, one option might be to do a splitlines() on both the expected and the parser.output and loop, matching lines, until a mismatch. Display the failed match (and probably previous matches for context). Or you can do fancier things, like a diff between the got and expected (see the Python difflib library). It depends how much effort you want to put into this to improve the feedback quality.

An alternative to rewriting the MyHtmlParser as above is to capture print output by setting sys.stdout to a StringIO object before calling the parser, then setting it back to its original value afterwards. Less work but also less elegant.

Or am I misunderstanding what you're trying to achieve here?