How to judge whether quiz's problem behavior is deferred feedback or immediate feedback in coderunner?
How to judge whether quiz's problem behavior is deferred feedback or immediate feedback in coderunner?
by Hongchuan Liu -
Number of replies: 9
In reply to Hongchuan Liu
Re: How to judge whether quiz's problem behavior is deferred feedback or immediate feedback in coderunner?
by Hongchuan Liu -
How to judge whether quiz's question behavior is deferred feedback or immediate feedback in coderunner?
In reply to Hongchuan Liu
Re: How to judge whether quiz's problem behavior is deferred feedback or immediate feedback in coderunner?
by Richard Lobb -
CodeRunner doesn't currently provide that capability. But I can see it might be useful. So I've experimentally added it to the development version, although it requires an upgrade to the adaptive_adapted_to_coderunner plugin as well. It is accessed from
within the template as an attribute of the new QUESTION.stepinfo variable, e.g.
print("{{ QUESTION.stepinfo.quizbehaviour}}")Edit: now changed to
print("{{ QUESTION.stepinfo.preferredbehaviour}}")
That prints a string such as adaptive, deferredfeedback etc depending on the quiz setting.
I've pushed the updated adaptive_adapted_for_coderunner plugin to github already and will push the updated CodeRunner soon, but only to the Development branch at this stage.
In reply to Richard Lobb
Re: How to judge whether quiz's problem behavior is deferred feedback or immediate feedback in coderunner?
by Tim Hunt -
Why might this be useful?
I think that thinking you need to know that is a sign you are asking like wrong question. (Like asking "How can I ensure that only teachers can use features X?" The answer is: you should not be hard-coding roles logic. You should be adding a capability "use feature X" then leaving it up to the admin which roles have that capability.)
So, I think adding this is a mis-feature.
In reply to Tim Hunt
Re: How to judge whether quiz's problem behavior is deferred feedback or immediate feedback in coderunner?
by Hongchuan Liu -
I can judge whether the precheck button is pressed by IS_PRECHECK. Can you realize a IS_CHECK var for check button？
In reply to Hongchuan Liu
Re: How to judge whether quiz's problem behavior is deferred feedback or immediate feedback in coderunner?
by Richard Lobb -
Sorry Hongchuan, but I don't wish to provide an IS_CHECK capability. That would imply that you wish to make a distinction between how Check and Submit events are handled. This would open the possibility of giving different marks for the two events which would in my mind be a breach of contract. Furthermore it would break some aspects of how I currently handle caching of grading results.
You might be able to achieve most of what you want by hiding the Check button and using the Precheck button to deliver different feedback according to preferred behaviour.
In reply to Tim Hunt
Re: How to judge whether quiz's problem behavior is deferred feedback or immediate feedback in coderunner?
by Richard Lobb -
From my understanding, Hongchuan wants his questions to behave differently when used for formative assessment (quiz behaviour = adaptive) and for summative assessment (quiz behaviour = deferred feedback). For example, he might wish to present a full result table when in formative mode but present only the "For example" cases when in summative mode.
I could of course have separate plugins like "deferred_feedback_adapted_for_coderunner", "interactive_adapted_for_coderunner" etc but this would go only halfway towards meeting the goals (since the actual feedback is defined by the question type not the behaviour) while at the same time generating lots of extra plugins.
So I don't think it's a particularly inappropriate feature to provide. And it's very low cost, being buried down in a QUESTION.stepinfo variable that most users will never encounter.
I myself have sometimes wanted to have my questions behave differently
under exam conditions. The "Hide check" button option now in the
development branch arose after several teachers both inside and outside our
department asked for it. I never wanted it myself when writing coding questions but last year when I started making use of the HTML UI in exams, I wished to limit the feedback provided to students. In those questions a Precheck was useful in that it allowed students to do a sanity check on their answer (e.g. a dynamic programming table) but I didn't want them running a full grading check.
Hongchuan has asked for the Hide check functionality to include a "Set by quiz" option, but at this point I admit I do start to feel uncomfortable as my behaviour plugin would then be required to "know" about how other behaviours present themselves. That for me feels like a step too far. Letting question authors decide to use workarounds like that is one thing, but I don't wish to build it into the architecture of CodeRunner.
We could discuss this in a Zoom session if you still have grave misgivings.
[Edit: a better approach to all this might be to have user-specified quiz info, such as Tags or something like my template parameters, which could be seen by CodeRunner questions and used to customise their behaviour. For example there might be an Exam tag, or a quiz-specific default penalty regime usable by CodeRunner - a more general solution than having Adaptive and Adaptive No Penalties behaviours). But at present the only information I seem to be able to access about the quiz from within CodeRunner is the behaviour being used. Unless I'm missing something? Would you be interested in adding some such feature to the quiz engine?]
In reply to Richard Lobb
Re: How to judge whether quiz's problem behavior is deferred feedback or immediate feedback in coderunner?
by Tim Hunt -
Thanks for explaining.
"Letting question authors decide to use workarounds like that is one thing, but I don't wish to build it into the architecture of CodeRunner." - This I can agree with.
Something I cannot really agree with is "formative assessment (quiz behaviour = adaptive) and for summative assessment (quiz behaviour = deferred feedback)" - that is your usage, but not a given.
The feature that exists, which is supposed to give this sort of control, is the 'question_display_options' class. That is what lets the teacher decide what is visible 'during the attempt' or 'later'. However, I suppose that is not fine-grained enough. We just have 'specific feedback' as a boolean option.
And, I was forgetting that CodeRunner aleady subverts the intended design of the system, by always using qbehaviour_adaptive_adapted_for_coderunner behaviour, whatever the quiz setting is. Hmm.
OK, I guess this is a useful hack. However, can I suggest you keep the standard variable name 'preferredbehaviour'. Questions can be used in many places, not just quizzes (e.g. https://moodle.org/plugins/filter_embedquestion).
In reply to Tim Hunt
Re: How to judge whether quiz's problem behavior is deferred feedback or immediate feedback in coderunner?
by Richard Lobb -
Thanks Tim.
I've changed the name to preferredbehaviour and pushed the updated adaptive_adapted_for_coderunner and qtype_coderunner plugins to github (but the latter to the development branch only).
Sorry about the subversion of your intended design :/ You may not recall, but some years ago we did very briefly discuss this. As per my rant in the coderunner appendix I felt very strongly (and still feel) that fair assessment of programming skills demands that the student/programmer should always be presented with immediate feedback on the correctness of their code via something like a Check button. They should get such feedback regardless of the behaviour of other questions in a quiz, e.g. a quiz running with deferred feedback. I didn't wish to write behaviour subclasses for all existing behaviours so you actually drew my attention to the fact that I could force adaptive behaviour regardless of the preferred behaviour if that was what I wanted, though I will admit that it wasn't something you expressed any special enthusiasm for :)
In reply to Richard Lobb
Re: How to judge whether quiz's problem behavior is deferred feedback or immediate feedback in coderunner?
by Hongchuan Liu -
That's exactly what I want, and I can't wait to use it.