Question Authors' Forum

Proposed new feature: "Stop and read feedback" button

Proposed new feature: "Stop and read feedback" button

by Tim Hunt -
Number of replies: 8

This is something that has come up several times before:

The 'problem' we are trying to solve is that for CodeRunner question, the 'General feedback' can only be shown after the whole quiz attempt is finished - and if you don't have a quiz, perhaps because you are using our wonderful Embedded question filter, it never shows.

So, I just had a go at implementing this on the editing form there is a new option:

And, then during the attempt:

Clicking that button takes the qusetion into exactly the state it it in after the quiz attempt has been submitted - but only that question. So, Check/Precheck/Stop buttons no longer appear, and all feedback (including General feedback) is shown - well, providing quiz settings allow it.

I am struggling a bit with the wording. What's in the screen grabs is my current best effort. Suggestions welcome.

This was suprisingly easy to do. I will create a pull request with the necessary changes shortly. It requires changse to both the behaviour and the question type.

In reply to Tim Hunt

Re: Proposed new feature: "Stop and read feedback" button

by Matthew Toohey -
Interesting, I like that it takes the question into the same state as it would be after the quiz has been submitted. Makes it very clear that you are 'giving up' on the question for this particular attempt of the quiz and can no longer get marks from it.

As for wording, the only thing I could come up with was calling it the 'concede' button. But I'm not sure that this is informative enough.


Matthew
In reply to Tim Hunt

Re: Proposed new feature: "Stop and read feedback" button

by Tim Hunt -
In reply to Tim Hunt

Re: Proposed new feature: "Stop and read feedback" button

by Richard Lobb -
Many thanks Tim. I agree that this is something several people have asked for ,so while I'm not sure it's something I want to use myself, I'm more than happy to have it as an option. I like that its availability can be set to Always, Never or When mark cannot be improved.

The two things that do worry me a bit are the wording and the button location and style. One poster has suggested that the wording might be author-customisable and I rather like that idea. I can imagine different users wanting to provide very different forms of feedback (particularly in the case of non-programming questions) and being able to specify more exactly what happens when the button is clicked would be nice. For example "Abandon and see answer", "Give up and move on". Also, the wording "Stop and read final feedback" might be open to misinterpretation if people see it as similar to "Stop and think about the problem", expecting to continue after having read the feedback. Granted, they'd soon learn the error of their ways, so this certainly isn't a show stopper, but might happen occasionally.

So, I'm wondering: how much effort would it be to have an option for the author to provide alternative wording, if they chose to?

The other concern relates to the position and style of the button. We use the Preview button on most but not all of our questions. This means that the Check button might be alone, or the rightmost of 2 buttons or the leftmost of 2 buttons, or the centre of 3 buttons. I feel that that's not good UI design. So ... would it be possible to float the new button to the right-hand side, and preferably colour it differently? Red might be nice :)

Lastly some musing (longish) ...

From a pedagogical standpoint I'm not all that keen on providing students with any sort of "Give up" button, particularly if what it presents is the answer. Students like to believe that they can learn to program by reading answers to questions - it's so much quicker and easier than actually writing them, right? But in my experience they're deluding themselves and I'm very reluctant to give them that easy out. But it's hard to see what else you could provide, other than the answer. If it's a hint for solving the question, then it's too late as the question is now closed. And anyway, providing hints for programming questions is extremely problematic because different students get stuck in a myriad of different ways. The only idea we've come in our teaching team discussions that would keep the learning process going is to provide students with a Parsons Problem version of the question. While it's probably much easier than the raw question, and should be worth less marks, it at least keeps the active learning going. We do have the option in our current question type to provide a scrambled answer (like a Parsons problem) to students after a given number of failed attempts, but it's not very intuitive. What I'd much prefer is a button that students could click to change the mode of a question to a Parsons problem, forfeiting some but not all of the marks in the process.

So can you see where this is taking me ... ?

I'm wondering if we could can think of the new button as a Custom button, where the author chooses the text (possibly from a drop down with 'Customise' as one of the options) and where the action can be either to abort the question (as with your implementation) or merely to set another question state variable, like the Preview button, that the question author can then use within the template for whatever purposes they want?

I accept that this involves more work than the current one, and complicates the UI but I think the complication could be minimised if the Customise behaviour were available only via a checkbox or at the bottom of a dropdown.

What do you think? Is it worth pondering further on this? We're at the end of semester here and things will be pretty busy until the end of exams (11 November) but after that I hope to have some time to put into CodeRunner development.
In reply to Richard Lobb

Re: Proposed new feature: "Stop and read feedback" button

by Tim Hunt -
Thanks for your very full reply Richard.

Customisable button text (with one of the wordings as a default) can be done. As can the styling changes.

And, what stopping the question really reveals is the General feedback for the question, which is whatever the question author likes, which might partially address some of your fears. And, the new button is optional. You don't have to use it.

One way this can work is in combination with random questions, so if you start again, you get a similar but different problem to solve. With either our embedded question filter, or one of the advanced quiz settings, once the question is finished, there is an option to 'Try another question like this one'.

Also, the context we are in, students studying at a distance, in their own time, is different from students working in a classroom, where when they get stuck, there is someone available to ask. We really don't want to leave students completely stuck.

I don't think a completely custom button is something I want to try to implement at this time. It is a lot more work that just providing a way to trigger the 'Finish' action, and this is were I must admit that I am on a deadline here (and I have left things to the last minute for a variety or reasons). Ideally I need to get this feature finished this coming week, if possible. Can we get my change in, even if you want to push the concept further in future?
In reply to Tim Hunt

Re: Proposed new feature: "Stop and read feedback" button

by Richard Lobb -
Thanks Tim. The "Try another question like this one" is interesting. I admit I've been thinking almost entirely about questions in our learning modules, which are worth marks. And as you say, we have physical labs where students can get one-on-one tutor help if needed. However, we do have drill quizzes, where students can practice, for no marks, and the "try another question like this one" in conjunction with your new button does make a lot of sense in that context. That sounds like a great topic for our teaching team to discuss. Also, we really should explore the use of your embedded question filter, which has great potential.

If you're tight on time and are happy to implement just the styling change, I'd be happy to merge your changes straight away. The customisable button text is a nice-to-have, but could be added later by either of us as the need arises. The idea of a more general custom button was an idea I wanted to float to see what people thought. I can investigate that during my summer break, which is when I do my coding. I think (hope) it could be added on top of what you have without disrupting the UI too much. Or it might prove totally unworkable - I'm happy to explore the space.

Thanks again for your work on this. I'm at last starting to see its potential :)
In reply to Richard Lobb

Re: Proposed new feature: "Stop and read feedback" button

by Tim Hunt -

OK, I will do just the stylnig change. Moodle styling is based on Bootstrap these days: https://getbootstrap.com/docs/4.0/components/buttons/

So, I think

  • Stop button gets btn btn-danger
  • While I am add it, do you want Check to be btn btn-primary?
In reply to Tim Hunt

Re: Proposed new feature: "Stop and read feedback" button

by Tim Hunt -

Currently looking like this:


or


In reply to Tim Hunt

Re: Proposed new feature: "Stop and read feedback" button

by Richard Lobb -
Ah, many thanks for picking up on mis-styling of Check. I had been thinking for some time I should look into why it was so muddy, but hadn't got around to it. That looks much nicer. And I think floating the Stop button to the right looks great too. The red is a trifle fiercer than I had in mind but certainly gets across the message that this is not a button to be trifled with!