Dealing with exception and error code

Dealing with exception and error code

de către gabriel illouz-
Număr de răspunsuri: 9

Hello,

First, thanks a lot for this wonderful tool.

I want to test student code on some wrong input, and they should raise an exception, or in SQL return an error (that will be the simplest example) : 

If they forgot to put a primary key,

one can put 2 lines with the same value for the key. When testing the system says :

expected : X got Y

X and Y are the same message (... 

UNIQUE constraint failed: JURY.numéro ...
checked with diff) , 

but I imagine because of the error code returned, 
they are considered as a wrong answer  

I searched in the manual, and forum, may be I missed how to test negative examples.

Thanks to let me know the best way to test this kind of configuration.

Best

Gabriel

Ca răspuns la gabriel illouz

Re: Dealing with exception and error code

de către Richard Lobb-

I don't fully understand the situation. It shouldn't be possible to have a test case fail with the Expected and Got columns being exactly the same. Yet you say you've checked them with diff? I assume there's no "Show differences" button at the end of the result table? Internal errors that are detected and handled by the test code or the student code shouldn't be relevant. If CodeRunner itself sees an error it will add text like "*** Runtime Error *** to the output.

I think we need more detailed info. Could you reply with a screenshot of the result table and attach an XML export of the question question, please? To get the export file, Preview the question and click the "Download this question in Moodle XML format" link at the bottom.

Ca răspuns la Richard Lobb

Re: Dealing with exception and error code

de către gabriel illouz-

Hello,

I test with a real SQL table the output is attached.

Then I made a simpler version attached as XML moodle file. It does the same.

To sum up :
I expect the student not to forget to put a PRIMARY KEY. If there is one, using twice the same value should be impossible.
It is, but I don't know how to make codeRunner  consider an error as the good answer.

Thanks in advance
Gabriel




Atașament CodeRunnerSQL.png
Ca răspuns la gabriel illouz

Re: Dealing with exception and error code

de către gabriel illouz-

an attachment with semi-colon and exact same number of line ...

When we want to test an error, that could be another issue. may be just testing :

***error***

would be enough without the exact same traceback

Ca răspuns la gabriel illouz

Re: Dealing with exception and error code

de către Richard Lobb-

OK, thanks. Now I understand. In most languages we don't set tests where if the student's code crashes, that's the right answer! But I can see that SQL poses some challenges here (I don't actually teach SQL myself).

To deal with this you need to customise the question's Python3 template (or, better, define a new question type) so that it doesn't raise an exception if the sqlite3 command fails. You also need to catch the standard error output and display it as "normal" output, because CodeRunner interprets standard error output as a failed execution.

I attach a new SQL question prototype type sql_allow_errors that does all the above and in addition deletes the "near line nnn" part of the error message so that the Expected part of the test case can match the error regardless of what line it occurs on.  I also attach your Create TSimple question modified to use that new question type.

Note that CodeRunner hides any completely empty columns in the result table so I would suggest trying to have a set of tests such that there is always some output in every column.

Ca răspuns la Richard Lobb

Re: Dealing with exception and error code

de către gabriel illouz-
thanks a lot,
I'll test it and let you know...
Best
Ca răspuns la gabriel illouz

Re: Dealing with exception and error code

de către gabriel illouz-

thanks a lot Richard,
it works perfectly,
the adding to the python code is really clear to understand.
Best
Gabriel
 

Ca răspuns la Richard Lobb

Re: Dealing with exception and error code

de către Christian Kruggel-

Hello Richard,

thank you so much for questions-moodle-PROTOTYPE_sql_allow_errors.xml. Unfortunately my Python is too poor to understand


To me both expected and got do match but somehow Python does not agree.

Can you spot at the first glance what is wrong here?

Many thanks

Christian

Ca răspuns la Christian Kruggel

Re: Dealing with exception and error code

de către Richard Lobb-

It looks like you've pasted the expected output into the Input field instead of the Expected field by mistake!

Empty result table columns are not displayed, so you're seeing only the Input column where you expected to see Expected.