CodeRunner: Prototype problem

I created a SQL question using a customized template, and it worked fine. Somebody else copied the question into a quiz. The template seems to exist in the same category in the question bank, and I cannot find a duplicate. However, it does not work. We get an error:

This question was defined to be of type 'sql_woodmont' but the prototype does not exist, or is non-unique, or is unavailable in this context. You should Cancel and try to (re)install the prototype. Proceed to edit only if you know what you are doing!

What is really happening here? How can I fix this?

Note that 2 students are currently in the middle of the quiz and have completed the other questions, so I do not want to have to delete the question unless it is necessary.

Re: Prototype problem

de către Richard Lobb- duminică, 12 martie 2023, 22:28

You say you used a customized template, but the error message says the question is of type sql_woodmont. That's not an out-of-the-box question type, so it seems you have created a new question type, rather than just customizing a template. In other words, you have defined a new question prototype at some stage, via the Advanced Customisation section. Was this deliberate? Note that prototypes cannot be run as actual questions - this used to be supported, but not any longer.

The error message says that the question you are trying to run is defined to be of type sql_woodmont, which means there needs to be exactly one instance of a prototype for that question type anywhere in the question bank for the course (except the quiz-specific bit, which is isolated) or any outer question banks (e.g. the system). The error message unfortunately doesn't tell you whether you have zero, or more than one, such prototypes. [This is fixed in the current development branch of CodeRunner.]

If you try to create a new CodeRunner question in the same category of the question bank as the question giving an error, do you see sql_woodmont as an available type? If so, you probably have multiple prototypes, if not you probably have zero prototypes. [This is not totally guaranteed, but I think that's the case for all recent CodeRunner versions.]

Since you say "Somebody else copied the question into a quiz", I'm guessing someone managed to copy the prototype as well. [There are checks in the code to catch attempts to create multiple copies of a prototype but some ways of copying questions bypass the CodeRunner codebase altogether.]

If you have multiple prototypes, then simply deleting one of them should solve the problem.

Another tool that might be useful to you is a script at <moodlehome>/question/type/coderunner/prototypeusageindex.php which lists all the questions in a course according to their prototype. I'm not sure, though, what it does when there are multiple copies of a prototype. But it might pick up an error - I don't remember, having written it over 10 years ago and rarely used it.

Re: Prototype problem

de către Mark Stern- duminică, 12 martie 2023, 23:22

Yes, I have created a new question type. Yes, it was deliberate. Sorry, I thought that was clear.

Oh good, you have fixed the ambiguous error message. Is the fix considered production ready yet?

If I try to create a new question in the same category, I do see sql_woodmont as an available type.
I see a question with name PROTOTYPE_sql_woodmont. If I try to edit that, I see the big red box at the top. Under advanced customization, I see It is marked as Is Prototype? Yes (user defined) and question type sql_woodmont. There may be a duplicate, but I cannot find it. Is is possible that the duplicate is in another category?

Re: Prototype problem

de către Richard Lobb- luni, 13 martie 2023, 08:31

Certainly the prototype can be in another category. In fact, the documentation (here) recommends you keep all prototypes in a category PROTOTYPES, and also that you name each prototype question with a name like PROTOTYPE_blah so you can readily find the one you want.

I just checked the prototype usage script I mentioned earlier and it doesn't handle duplicate prototypes. It reports on only one occurrence of a particular prototype. However, you could use that to find the extra prototype by:

Running the script by using the URL I gave you above in your browser
Checking if the location of the prototype that if reports for sql_woodmont is the one your know about
If not, you've found the stray prototype else ...
Edit the prototype you know about to temporarily change the question type name to something else
Re-run the script

Note that step 4 will probably fix the immediate problem of your child questions not working, as it will remove the duplicate. But it still leaves you with a stray prototype floating around, which is a bad idea.

To answer your question about the development branch - yes, it's stable and we're running it on our production server, used by well over 1000 students. But it's for Moodle 4 only, not Moodle 3. And really there's no reason to panic here - the problem shouldn't be hard to solve.

Re: Prototype problem

de către Mark Stern- luni, 13 martie 2023, 10:39

OK, I think I have found it. There is a copy of the prototype in category A and a copy in category B (not the actual names). What I do not get is that it seems that if I have in question in category A it works, but if I have a question in category B it does not work.

Re: Prototype problem

de către Richard Lobb- luni, 13 martie 2023, 14:20

Good to hear you've found the rogue prototype.

I'm not sure if your question refers to the position of the prototype in the question bank or the position of the question. But a possible source of confusion here is that each quiz has its own private category (containing Top for quiz x and Default for quiz z) that is visible only to that quiz and can't be seen in the general question bank. I never put questions in these private categories as they disappear when you delete the associated quiz. Are you using those private categories?

Except for the quiz-specific categories, if a prototype is anywhere in the course question bank, it should work anywhere. So could you please clarify the situation that is causing confusion. Where are the various questions and prototypes, and exactly what error messages do you get (and when)?

Re: Prototype problem

de către Mark Stern- luni, 13 martie 2023, 21:38

There is a prototype in category A and in category B.
Question in category A works.
Question in category B does not work. When I click Check, I get:
Broken question (missing or duplicate prototype 'sql_woodmont'). Cannot be run.

If it helps, it seems that category A is a global category that can be accessed by all courses. Category B is specific to the course.

Anyway, I modified the prototype in category B to have a different question type (sql_woodmont_dummy). Now most questions seem to work, but not the one that is already in the quiz does not. If I click on edit question and run a preview based the question in the quiz, I now get a different error:

Error from Jobe sandbox server: runs_post: invalid run specification

My guess is that if I delete the quiz and recreate it, all will be fine apart from the inconvenience to the (now 3) students who are in
the middle of the quiz. Is there a better solution?

Re: Prototype problem

de către Richard Lobb- luni, 13 martie 2023, 22:56

A full explanation of CodeRunner's prototype search strategy is complicated. But for a simple analogy, consider a program in which there is a global variable x and a local variable x inside a function. In the global context, y = x is unambiguous, but if the statement y = x is within the function, is x the local variable or the global variable? CodeRunner refuses to allow any such ambiguity and reports an error.

To put it another way, if you have a question in the global context, only the global prototypes are available to it. If it's in a course context, both the course prototypes and the global prototypes are available. And you have identically-named prototypes in both contexts, hence the category A question fails. But the question in A is global so does not see the prototypes within the course context; it works fine.

You should have been able to fix this problem simply by deleting the prototype in the course context. Both questions would then use the prototype in the global context.

The "Error from Jobe sandbox server: runs_post: invalid run specification" sounds like a broken prototype. At any rate it doesn't result from the prototype ambiguity that you have described.

You do need to be very careful with prototypes. As the documentation says: "NOTE: User-defined question types are very powerful but are not for the faint of heart. There are some known pitfalls, so please read the following very carefully."

Re: Prototype problem

de către Mark Stern- luni, 13 martie 2023, 23:59

I figured it was something like that. I think the prototype ambiguity is now fixed. The problem seems to be just with the question inside the quiz. I am guessing that something is cached from the previous attempts. It looks like we are going to have to recreate the quiz (and annoy the students).