CodeRunner: nodejs Promise async issue with multiple tests

Promises work fine when there's just one test. With two tests however, the second test seems to get run before the async part of the first test has finished and the output intended for the first test ends up in the second. Both tests fail.

In more detail:

Question type: nodejs

Answer and Answer box preload:

const promising = () => {
    return new Promise(response => {
        setTimeout(() => {
            console.log('Timed out');
        }, 1000);
    });
}

Test 1:

(async () => {
    await promising();
})();

Expected output:

Timed out

Check gives:

Test passes

so everything runs as expected.

Now after adding a second test Check gives:

Tests fail

Unchecking Is combinator just gives no output for either test (and no output for a single test either).

The obvious workaround is to have just one test per question, but that sort of defeats the purpose of having multiple tests.

Re: nodejs Promise async issue with multiple tests

de către Richard Lobb- joi, 21 ianuarie 2021, 13:44

You either need to set up your question so that the tests are explicitly chained to execute in sequence, or simply turn off the Is combinator option. In the latter case, though, you need to replace the template with the much simpler version

{{ STUDENT_ANSWER }}
{{ TEST.testcode }}

Explanation: when using a combinator, CodeRunner supplies the template with a TESTCASES Twig variable containing all the various tests. But when not using a combinator, the template instead receives a TEST variable containing the current test. The reason nothing happened when you just turned off Is combinator is that the TESTCASES variable was undefined so you got no test code at all.

When not using a combinator, the question's tests are run sequentially, with a separate Jobe server run for each test. This increases the Jobe server load of course, but you probably won't notice unless you have very large classes sitting exams.

Re: nodejs Promise async issue with multiple tests

de către Peter Sander- vineri, 22 ianuarie 2021, 00:19

Nailed it!

And thanks for the useful explanation as well.