Running F# Questions using Dotnet Via Python.

Running F# Questions using Dotnet Via Python.

by Christopher Morley -
Number of replies: 3

Good Morning, 

We have been using Coderunner for the last year quite successfully with the built in languages. We recently decided we wanted to add the ability the run F# questions as well.  Following the documentation, I am able to get the jobe server to respond to the request from the Moodle server, but depending on how we structure the call, we either get a compilation error or a time exceeded error.  the example we are using to test is a simple printfn "Hello World"

For reason, I don't believe the code is executing past Line 19 ( print("**Compilation failed. testing Aborted **")). 

I based the creation of the question type on an older forum post describing how to compile C# using mono.  (for various reasons DotNet is to be used over mono).

I would greatly appreciate it if people could provide some insight into my problem.  Please find below a copy of the question we have created as well as images of the error messages. 

===============================================================

import subprocess, sys, os


# Write the student code to a file prog.cs

student_answer = """{{ STUDENT_ANSWER | e('py') }}"""

with open("test.fsx", "w") as src:

    print(student_answer, file=src)


e=os.environ.copy()

e["DOTNET_CLI_HOME"] = os.getcwd()

e["DOTNET_CLI_TELEMETRY_OPTOUT"] = "true"



# Compile

print("testing- Pre Call.")

return_code = subprocess.call(['dotnet','fsi','test.fsx'], env = e, shell=True) ## returns an compilation error see comp error

# return_code = subprocess.call("dotnet fsi test.fsx", env = e, shell=True) ## returns a Time Out Error. 

print(return_code)

if return_code != 0:

    print("** Compilation failed. Testing aborted **", file=sys.stderr)


# If compile succeeded, run the code. Since this is a per-test template,

# stdin is already set up for the stdin text specified in the test case,

# so we can run the compiled program directly.

if return_code == 0:

    try:

        output = subprocess.check_output("dotnet fsi test.fsx", universal_newlines=True)

        print(output)

    except subprocess.CalledProcessError as e:

        if e.returncode > 0:

            # Ignore non-zero positive return codes

            if e.output:

                print(e.output)

        else:

            # But negative return codes are signals - abort

            if e.output:

                print(e.output, file=sys.stderr)

            if e.returncode < 0:

                print("Task failed with signal", -e.returncode, file=sys.stderr)

            print("** Further testing aborted **", file=sys.stderr)



Attachment comperror.png
Attachment timeout.png
In reply to Christopher Morley

Re: Running F# Questions using Dotnet Via Python.

by Richard Lobb -

I don't use dotnet or F# so I can't help with those details. The first run seems to fail because 'fsi' can't be interpreted correctly, which looks like an environment variable problem. But I don't understand why it behaves so differently from the second method. What happens with the first method if you set shell to False?

The second method seems to be basically working. Could it not just be a genuine timeout error? What's the value of TimeLimit (secs) in the advanced customisation section? The default for Python is 3 seconds, I think, and with multilayered software like this, you might just be exceeding that? It's wallclock time, not CPU secs.

Another possibility is that you've hit the runguard memory limit - when that happens, malloc calls fail and software behaves in unpredictable ways. It might be telling you there's a timeout error because it's stuck in a loop trying to get memory that isn't available. I suggest you also set the memory limit to 0 (which turns off the limit altogether) under advanced customisation.

If all else fails you can set the sandbox parameters field (again under advanced customisation) to {"debugging": 1}. That will leave the entire run info on the Jobe server in a random directory in /home/jobe/runs. You can then log in to Jobe (I assume you have access) and poke around in there. One of the first things to do when things break badly is to find the command that's being executed (it's in one of the scripts there, I don't remember where from the top of my head) and run it without runguard. [runguard is the program that imposes lots of security restrictions on the program, and making sure everything works without those restrictions is the first step.] If it works without runguard, try removing all the restrictions from the runguard command line and then, if that works, adding them back in one-by-one until the command works.


In reply to Richard Lobb

Re: Running F# Questions using Dotnet Via Python.

by Christopher Morley -

Hi Richard, 

Thank you for the prompt response. You certainly provided a number of options for me to Look into and test. 

1) if I change the shell value associated with the first option it then produces the timeout exceeded error as generated by the second option. It does appear that if I use shell=True then it seems unable to generate the string too execute. 

2) I have increased the TimeLimit (secs) value under the advanced customization to be 30, and there appears to be no real change in behavior, it just takes longer to timeout. 

3) I saw in a previous thread that the memory usage could be a problem and you had suggested setting that value to 0, so there was no limit. so I have set the memlimit to 0, again this hasn't changed the behavior of the problem. 

4) I seem to have encountered a bit of a problem when applying {"debugging": 1} as a parameters under advanced customisation (removing both MemLimit and TimeLimit). While it is there and saved on moodle. It doesn't appear to be actually doing anything on the jobe server. I can see the files while it is being processed but it cleared out when the run ends.  Do you have any idea why this might be happening ?

In reply to Christopher Morley

Re: Running F# Questions using Dotnet Via Python.

by Richard Lobb -

Sorry, I mis-remembered. The correct debug option is

{"debug": 1}

I'll be interested to hear what the problem is - it sounds like the job is stuck in a loop somehow. My only other suggestion, short of the full on debugging within the jobe server itself, is to try increasing the number of processes.  So as well as turning on debugging, try

{"numprocs": 300, "debug": 1}