I am linking the new(-ish) bulk-test output for testing upgrades, but I have a couple of suggestions:
- Where it re-prints a summary of all the failures at the end, it woudl be useful if that list gave some sort of contextual information about where the failing questions are from.
- I personally would like green/red styling of passes and fails.
- We had some weird ones, where a question failed in the bulk run, but then passed when run separately. However, I now cannot reproduce that.
- For most failures, it is OK to follow the link to questiontestrun.php, and from there to "See this question in the question bank". However, when the failure is something like "Failed to load question Unknown question: qtype_coderunner/Prototype python3_cosc121 is unavailable in this context, or does not exist. ****", then that path does not work, because questiontestrun.php page give a fatal error.