Passing information "over" a CHECK click

Passing information "over" a CHECK click

by Carroll Morgan -
Number of replies: 7

Is it possible for information from one run of a grader, within the same quiz, to be passed from one CHECK to the next? In particular...

I would like to allow CHECK to indicate that -some- answers are incorrect, but not necessarily to identify all incorrect answers. On the next CHECK (if there is one), I would like to do the same, but only within the answers already flagged (ie by the first CHECK). And so on.

That's intended to prevent the students' "accumulating" hints from one CHECK to the next.

More generally, there are other situations in which it would be useful for "this" CHECK to be able to remember what the "previous" CHECK saw.

I have experimented with writing a file, but (not unexpectedly) it seems to be removed in between CHECKS.

Maybe there could be a component of stepinfo (which my CR version does not yet have) that brings in to 'this' CHECK a json'ed string passed out in a 'forNextTime' entry in the results dict of the 'previous' CHECK?

Thanks!

In reply to Carroll Morgan

Re: Passing information "over" a CHECK click

by Richard Lobb -
Interesting idea. I don't think I've ever wanted to do this myself, but it does open possibilities.

If I understand correctly, you're proposing that with combinator template graders (the only context in which you could use such a facility?) you'd have an extra field available in addition to the usual prologuehtml, testresults etc. That would be a string-valued field (possibly JSON, though I think that decision could be left to the question author, couldn't it?). Whatever value the template grader placed in that field would be passed back to the next template run as a new field of stepinfo.

Looking at the code, this isn't too hard to implement. The biggest problem might be deciding on a name for the field. Since I'm using the Moodle PHP naming convention of pure lower-case alphabetic identifiers (yeah, I know - yuck) it would have to be fornexttime rather than forNextTime. But apart from looking bad, that's not a good name for it in the incoming stepinfo, only in the outgoing grader result. How about graderstate?
In reply to Richard Lobb

Re: Passing information "over" a CHECK click

by Carroll Morgan -
Hi Richard,

That sounds perfect.

FYI (and to make sure we're on the same wavelength), I am trying to give the students -some- feedback when they press CHECK or PRE-CHECK, but not too much and, in particular, not to tell them that any particular answer is correct. The problem there is (of course) that correct answers can propagate from one student to another during the test.

So I'm at the moment using (hoping to use) what's known as the "Random Response" protocol in the privacy literature. Adapted to this context it would mean: (1) the wrong answers in the current check are identified; (2) a randomly selected -proportion- of those are shown in the results-table as "WRONG"; (3) but correct answers, and "not selected" wrong answers are not flagged at all.

The idea is that un-flagged answers therefore cannot be assumed to be correct, and are thus of little value to other students. The issue is however is that repearted use of this "without memory" would allow a student to press CHECK repeatedly and rely on the randomness --activated each time-- to cover all the wrong answers eventually (and this identify the correct ones by elimination). To defeat that, each subsequent check will limit its flagging to the subset of questions randomly selected in the previous check --- and so the grader has to know what those previous ones were.

I expect there will be other uses for this, if it works nicely. Your suggestion "graderstate" sounds fine to me, and putting a json'd dict (for example) in there is a good suggestion.

BTW I'm assuming that writing to a file won't work, for the reasons I mentioned earlier, that grader-written files are removed between each invocation?

Carroll
In reply to Carroll Morgan

Re: Passing information "over" a CHECK click

by Richard Lobb -
Interesting ideas. Thanks for raising the issue. I'll put this in my todo list for the next push, which will hopefully be around the end of this month.

Yes, you can't write a file to remember grading info, as the Jobe server is stateless. It does have a file cache to save repeated uploads of the same support file or attachment but otherwise each run should leave the server in the same state as before the run.

That said, you can easily circumvent this if you have admin access to the Jobe server. Just create a directory somewhere that has write access for the various Jobe helpers - jobe0 through jobe. But in doing this you immediately open a security hole - any student job has access to the same directory, if it knows about it. A more secure way of providing state is to install and turn on mysql on Jobe, and set up an account with credentials only you know about. Then your template could record state info in that database, although keeping the credentials hidden from prying student eyes is a bit problematic still. Our in-house Python question types prevent student jobs opening 'our' files, but the standard question types all allow students to read the code in the working directory. That includes the file that is currently being run, i.e. the template code.

If you wanted to explore these options, you could set up a docker jobe server of your own on any server that the Moodle server can access, assuming it's not too locked down. There's a rarely used feature under advanced configuration that lets you set, in the Sandbox parameters field, the URI of the Jobe server to use for that question. Click the '?' help button for the Sandbox fields under Advanced customisation for details. It should still work, though I haven't used it recently and it's not tested by the standard test suite.
In reply to Richard Lobb

Re: Passing information "over" a CHECK click

by Carroll Morgan -
This is remains an interesting idea. If it's implemented, could you post the details? Thanks!
In reply to Carroll Morgan

Re: Passing information "over" a CHECK click

by Richard Lobb -

graderstate is now a field of the QUESTION.stepinfo template variable. See here. The use of it (really entirely over the question author) is discussed briefly in the section on combinator-template grading. 

Is that enough to go on with?

In reply to Richard Lobb

Re: Passing information "over" a CHECK click

by Carroll Morgan -
Hi Richard.

I've tried passing "graderstate" back to CR, but it appears not to be recognised. Maybe we here have not caught up with the latest version of CR? See attachment.

Carroll
Attachment graderstateNotRecognised.jpg