First, thanks a lot for this wonderful tool.
I want to test student code on some wrong input, and they should raise an exception, or in SQL return an error (that will be the simplest example) :
If they forgot to put a primary key,
one can put 2 lines with the same value for the key. When testing the system says :
expected : X got Y
X and Y are the same message (...
UNIQUE constraint failed: JURY.numéro ...
checked with diff) ,
but I imagine because of the error code returned,I searched in the manual, and forum, may be I missed how to test negative examples.
they are considered as a wrong answer
Thanks to let me know the best way to test this kind of configuration.
I don't fully understand the situation. It shouldn't be possible to have a test case fail with the Expected and Got columns being exactly the same. Yet you say you've checked them with diff? I assume there's no "Show differences" button at the end of the result table? Internal errors that are detected and handled by the test code or the student code shouldn't be relevant. If CodeRunner itself sees an error it will add text like "*** Runtime Error *** to the output.
I think we need more detailed info. Could you reply with a screenshot of the result table and attach an XML export of the question question, please? To get the export file, Preview the question and click the "Download this question in Moodle XML format" link at the bottom.
I test with a real SQL table the output is attached.
Then I made a simpler version attached as XML moodle file. It does the same.
To sum up :
I expect the student not to forget to put a PRIMARY KEY. If there is one, using twice the same value should be impossible.
It is, but I don't know how to make codeRunner consider an error as the good answer.
Thanks in advance
an attachment with semi-colon and exact same number of line ...
When we want to test an error, that could be another issue. may be just testing :
would be enough without the exact same traceback
OK, thanks. Now I understand. In most languages we don't set tests where if the student's code crashes, that's the right answer! But I can see that SQL poses some challenges here (I don't actually teach SQL myself).
To deal with this you need to customise the question's Python3 template (or, better, define a new question type) so that it doesn't raise an exception if the sqlite3 command fails. You also need to catch the standard error output and display it as "normal" output, because CodeRunner interprets standard error output as a failed execution.
I attach a new SQL question prototype type sql_allow_errors that does all the above and in addition deletes the "near line nnn" part of the error message so that the Expected part of the test case can match the error regardless of what line it occurs on. I also attach your Create TSimple question modified to use that new question type.
Note that CodeRunner hides any completely empty columns in the result table so I would suggest trying to have a set of tests such that there is always some output in every column.