How to calculate within a given tolerance?

How to calculate within a given tolerance?

by Alexander D -
Number of replies: 4

Hi,

if I have an octave question type and I want the students to calculate something like this:

Write a function calculating the Simpson rule on the interval [a,b], with a<b!

Which would be something like this

function S = simpson(f,a,b)
    m=(a+b)/2;
    S=((b-a)/6)*(f(a)+4*f(m)+f(b));
end


So, now the question is how to evaluate the students answer within a given tolerance, e.g. relative tolerance should be equal to 0.01.

Is it possible to check the students answer without modification of the coderunner template?

Thnak you very much in advance!
Alex
In reply to Alexander D

Re: How to calculate within a given tolerance?

by Alexander D -
Hi Richard
due to the great design of your question type I've managed to solve the problem: If anybody is interested I put a simple example here:
  • I have the customise checkbox checked
  • under "customisation" this is my template
    1; % Dummy statement to make a script
    {{ STUDENT_ANSWER }}

    format free

    {% for TEST in TESTCASES %}
    {{ TEST.testcode }};
    {{ TEST.extra }}; % This is the new line I added
    {% if not loop.last %}
    disp('#<ab@17943918#@>#');
    {% endif %}
    {% endfor %}

  • The answer is
    function I = simpson(f,a,b)
        m=(a+b)/2;
        I=((b-a)/6)*(f(a)+4*f(m)+f(b));
    end

  • This is a test case
    %cos(x) auf I=[1,2]
    f=@(x) (cos(x));
    %disp(simpson(f,1,2))

  • And this is the extra code of this test case
    %solution
    function S = mysimpson(f,a,b)
        m=(a+b)/2;
        S=((b-a)/6)*(f(a)+4*f(m)+f(b));
    end

    %function for testing
    f=@(x) (cos(x));
    expected=mysimpson(f,1,2);
    got=simpson(f,1,2);
    error=abs(got - expected);
    % chec answer against expected one
       if (error > 1.e-1)
          printf("Wrong answer! got: %.5f, expected: %.5f",  got, expected)
       endif

       if (error < 1.e-1)
         printf("Test passed!")
       endif


One last point:
In my case the answer is evaluated by checking the error and only a "Test passed!" is printed to the student. This way the student does not see the numbers his function calculates.
Is there a way to display the calculation of the student's function but to ignore this number in the evaluation?

Best,
Alex
In reply to Alexander D

Re: How to calculate within a given tolerance?

by Richard Lobb -
Looks good! Thanks for posting, Alex.

If you want to include the student's answer in the output but disregard it in the marking, the best option is to use a template grader. However, that's a lot more work. A more clumsy but cheaper option is to switch to a regular expression grader that checks, say, for the presence of the string "Test passed!" in the output.

A problem with regular expression graders is that students don't understand the output unless you go to some extra trouble to explain it. If the regular expression were just "Test passed!" with no metacharacters, you probably wouldn't confuse the students too much, but they could exploit it by themselves printing "Test passed!" in their function. Maybe that's not a significant risk to you? The regular expression "Test passed!\Z" would force a match only at the end, preventing the exploit but confusing the students. [Edit: probably you need to handle line terminator(s) in the expression, so something like "Test passed!\s*\Z"? You can experiment, if you want to try this.]

Our own in-house combinator grader has a template parameter float_tolerance which lets you set the tolerance on any floating point numbers in the output. But that requires matching all the floating point numbers in the output (splitting with a complex regular expression that matches floating point numbers) and then, when comparing the expected with the got outputs, checking if each float value is within tolerance. Non-floating-point tokens are required to match exactly. This is another level of complexity higher again. 
In reply to Richard Lobb

Re: How to calculate within a given tolerance?

by Alexander D -
Hi,
I had a look at the template grader and tried it myself. It works great. Thank you very much for this awesome documentation!
Just one thing:
I followed these instructions where I found a small mistake in the template which tests the student's student_sqrt function with 1000 random numbers in the range 0 to 1000:

In my case I had to replace the line
stud_answer = student_sqrt(n)
by
stud_answer = mysqrt(x)
where mysqrt() is the function created by the students. Otherwise I received an error.


In reply to Alexander D

Re: How to calculate within a given tolerance?

by Richard Lobb -
Good to hear that worked for you Alex. And thanks for pointing out the documentation error. I've fixed it in my development code and it will go out on the next push.