Running C# in CodeRunner

Running C# in CodeRunner

by Richard Lobb -
Number of replies: 13

I received an email asking:

We use C# with our first year students and I was wondering if anyone has adapted Code Runner for C#.  If not would it take much time and effort to extend it.

I thought my answer might be of interest to others, so I'm posting it here ...

"If Mono is sufficient for your needs, you can run C# questions with the existing CodeRunner provided:

"I experimented a bit and came up with a C# program question type (i.e., a question that requires the student to write an entire program, which is run for each test case using the supplied stdin) as follows:

""" The template for a question type that compiles and runs a student-submitted
    mono C# program. 
"""

import subprocess, sys

# Write the student code to a file prog.cs
student_answer = """{{ STUDENT_ANSWER | e('py') }}"""
with open("prog.cs", "w") as src:
    print(student_answer, file=src)

# Compile
return_code = subprocess.call(['mcs', 'prog.cs'])
if return_code != 0:
    print("** Compilation failed. Testing aborted **", file=sys.stderr)

# If compile succeeded, run the code. Since this is a per-test template,
# stdin is already set up for the stdin text specified in the test case,
# so we can run the compiled program directly.
if return_code == 0:
    try:
        output = subprocess.check_output(["mono", "./prog.exe"], universal_newlines=True)
        print(output)
    except subprocess.CalledProcessError as e:
        if e.returncode > 0:
            # Ignore non-zero positive return codes
            if e.output:
                print(e.output)
        else:
            # But negative return codes are signals - abort
            if e.output:
                print(e.output, file=sys.stderr)
            if e.returncode < 0:
                print("Task failed with signal", -e.returncode, file=sys.stderr)
            print("** Further testing aborted **", file=sys.stderr)

Use this instead of the template for C-via-Python described at http://coderunner.org.nz/mod/book/view.php?id=193&chapterid=749. Also you need to set the Ace language to cs to get C# syntax colouring in the student answerbox.

Here's what a run of that question type looks like:

A C# write-a-program question

For other question types, such as "write-a-C#-method", you will have to wrap the student submission in some extra code, including that supplied in the {{TEST}} variable. If you need help with that, post back here.

I should mention that this question type has had minimal testing - it's provided only as a starting point to get you going.

If Mono is not sufficient for your needs, say because you need Microsoft libraries, then you're pretty much out of luck, unless you want to write your own sandbox.

Richard

In reply to Richard Lobb

Re: Running C# in CodeRunner

by Martin Zwerschke -

Hi Richard,

this approach gets close to what I need for our school. But in fact not close enough.

I am teaching C# for beginners and could make use of CodeRunner, if there was a way to 

add C# compiles using an external sandbox with free access (like ideone or rextester).

I wonder, if it is possible to completely pass-by the jobe-sandbox?

Am I right, that even using the mono/C#-template CodeRunner always does a post to the jobe (configured in CodeRunner's admin settings) afterwards?

  • We do not have our own jobe Server and can not easily install one.
  • using Canterbury's outward jobe is not the solution for productive use, i guess as it will frequently reject requests for "worldwide overload". 
  • the selectable "ideone" could be a solution (Supports C#), but is not currently supported by CodeRunner. Selecting "ideone" in customization options always returns  an unsupported sandbox-language error, even if it is set as the only sandbox in CodeRunner's admin settings
    and username and Password for ideone Service are set.

I tried to modify a simple short-answer Quiz instead of CodeRunner before and managed to get a student's Code compiled and run doing an ajax/jquery code post to rextester" inside the moodle's question field directly editing the html content (<>)-Button in Editor. But I can not get the result submitted as the answer via jquery script.

$( window ).on( "load", function()
{ $("button").click(function()
 {
 var to_compile =
  {
    "LanguageChoice": "1",
    "Program": $("#code").val(),
    "Input": "",
    "CompilerArgs" : ""
  };  $.ajax (
          {
     url: "http://rextester.com/rundotnet/api",
     type: "POST",
     data: to_compile
   }
         ).done (
  function(data)
  { trying to submit part of data here as answer does not work - something like: $("[id$='answer']").val(... part of data ...); $("input[id$='submit']").click(); } ).fail( function(data, err) { alert("fail " + JSON.stringify(data) + " " + JSON.stringify(err)); } );


Do you see a chance of using rextester or ideone this way inside CodeRunner?

Cheers from Germany,

Martin


In reply to Martin Zwerschke

Re: Running C# in CodeRunner

by Richard Lobb -

Hi Martin

Certainly the Python/C# template I gave you requires Jobe, for the Python as well as the C#. All student code has to be run in a sandbox for security.

Sorry the Ideone sandbox didn't work for you. It was really just a proof-of-concept sandbox written during development. I haven't even tested it for a year or two and AFAIK it has never really been used in earnest by anyone. I just looked into the problem and I see that the language strings on the Ideone server have all changed - only python2 programs still ran OK. I've kicked the Ideone sandbox back into life on my development server and have even tested a C# program with it. However, I'm not sure I wish to continue to support the Ideone sandbox, so I'd prefer to look at alternatives, for now.

Running programs with rextester would require a PHP support module to interface to rextester. The CodeRunner architecture supports multiple sandboxes pretty well but each sandbox has its own complications such as how it handles compile errors, runtime errors, timeouts, memory limits etc. I personally have no interest in developing a sandbox module to interface with rextester, so any such module would have to be developed by the community.

You say: "We do not have our own jobe Server and can not easily install one." Is this something I can help you with, perhaps? Setting up a Jobe server is really pretty easy. All you need is an old PC with fairly modest specs (our outward facing Jobe server is just a 2-cpu 4GB VM). Installing Ubuntu 16.04 takes about 30 minutes, then you can install Jobe in a couple of minutes with just a few commands. Then the machine just needs to sit somewhere that the Moodle server can access it - in your own office, perhaps, or even somewhere off campus. I'm happy to help if something goes wrong.

I can provide you with an unrestricted-access API key to our outward facing Jobe server if you like, but this should be considered a temporary measure while you get your own Jobe server going.

Richard





In reply to Richard Lobb

Re: Running C# in CodeRunner

by Martin Zwerschke -

Hi Richard,

thanks a lot for your rapid answer and your efforts testing "ideone" once again.
I see, that ideone is not your way and of course your advice to use the JOBE server is reasonable enough.

Furthermore and after your explanation I understand that C# /mono would be executed on the JOBE server as well.

(Before this I assumed, only the python script would run on jobe but C#/mono on the moodle server - I had not read the postings thoroughly enough...)

I dare to program a PHP support module to interface to rextester, but it would take me weeks of work, which I can not afford at the moment.

I still hang on to the idea to let the student's program compile and execute by rextester via an ajax/jquery script inside the question's html code and
let it paste a part of rextester's output to the answer box. This already works. 
But somehow moodle erases the pasted content before my script presses the submit button. Perhaps I just do something wrong, because I am not too skilled in web scripting matters.


>You say: "We do not have our own jobe Server and can not easily install one." Is this something I can help you with, perhaps?

Thanks for your offer of help. But of course I could do the installation of Ubuntu and jobe myself. It is in fact easy but that's not the point.

Rather there are some other reasons, I would prefer to avoid this:

  • I am using the moodle system to enable my students to work from their homes. So jobe would also have to be accessible from outside, if I install it.
    A local jobe would not fit this needs.
  • In Germany even a bigger school does not have a fixed IP-adress, but gets varying IPs dynamically by the provider. So at least I would have to use DDNS to get a jobe accessable from outside. And DDNS has got some problems, as far as I know. 
  • The jobe server would be our first and only device that would be accessible from outside. Accessing Internet is done via a proxy server,
     our moodle and website run on a rented server and I  have not implemented any type of remote management. 
    I do not like the idea of "opening the doors" for hackers, just for a jobe, that is used only occasionally. We were quite safe by now.
    Opening ports to the outside world can be made quit safe, but this requires a good administration, keeping servers and router up to date and keeping  informed about actual risks.
    We do not have a full time administrator, but i do this together with one collegue parallel to our fulfilling jobs as teachers. I am not so sure I could garantee the safety of our LAN in case of a jobe server accessible from outside. 
  • Moreover a jobe server is more vulnerable than a moodle server or web server and should run on a separate machine and inside a "demilitarized zone" . This server would have to run around the clock, although it is only needed occasionally. This consumes energy, that could be saved using a service like rextester or ideone, than runs anyway. And in addition it is one machine more to take care of next to my job...
    Perhaps jobe could run on our proxy server, that is always running. But this does not remove my concerns about safety.

I really think for smaller an mid size schools a dedicated jobe server is some effort and safety risk, while for a university with a skilled IT-administration team it may be peanuts. This is why I would appreciate it, if you would keep ideone in CodeRunner alive.

But anyway I am thinking about finally doing or own jobe, if I do not manage to get my rextester method running with moodle.

Thank you as well for offering an unlimited key for your jobe on time - but this would not solve the problem permanently.

And finally thank you for publishing your plugin at all and for your efforts on improving modern learning methods.

Martin

In reply to Martin Zwerschke

Re: Running C# in CodeRunner

by Richard Lobb -

Hi Martin

OK, fair enough. I can appreciate that while getting a Jobe server up and running is straightforward enough, it is one more thing to maintain. And the fact that your Moodle server is on an outside rented server does mean that Jobe has to accessible from that server. That's the only machine it has to be accessible from, though. You should normally set the Linux firewall on the Jobe machine to prevent any other access, so it's really not vulnerable to attack in the way that the Moodle server or other web servers are.

I spent some more time testing the Ideone sandbox today, until I received this email:

Hi coderunner,

you have reached a limit of free submissions this month.

If you wish to submit new codes now,you can purchase a pool of submissions by visiting https://ideone.com/offer/users/.
Regards,
Ideone Team 

So as I rather suspected, Ideone is no longer a free service and I'm not prepared to buy a package of further submissions in order to do further testing.

You're welcome to the updated version of the PHP Ideone Sandbox code if you want it, but you'll have to fix any further bugs with it yourself. Do you have shell access to the Moodle server to edit the code?

Also, aside from the fact it's not a free service, you should be warned that Ideone is nowhere near as responsive as Jobe. Each run takes at least several seconds and so questions with non-combinator templates (e.g. programs that are run multiple times with different standard input) will take a rather unacceptably long time to grade. Lastly, you need to be aware that the Ideone sandbox has never, to my knowledge, been used in a production environment so there are bound to be "interesting" things happen in some situations. With all those caveats, are you still interested? :)

Richard



In reply to Richard Lobb

Re: Running C# in CodeRunner

by Martin Zwerschke -

Hi Richard,

that's quite interesting but also disappointing news. I had not expected ideone to be limited that much.
Considering the mail you got from ideone, it really is no free service any more. And of course no one would
expect you to buy submission packets for that and I won't do either if -as I do expect- ideone is too expensive for our needs.

Perhaps our occasional use of CodeRunner would be below the unknown free limit. But who would rely on that
for doing a graded test with his students?

>You're welcome to the updated version of the PHP Ideone Sandbox code if you want it.

Of course I would like that nevertheless. Thanks in advance.

Yesterday I managed to pass a rextester-generated output to the (made readonly) student's answer box by ajax/jquery scripting inside a simple moodle shortanswer-quiz. That is quite near to my demands, though there is no syntax highlighting and no multiple test cases and so on.

But at least there is a chance for a fallback option, if I neither get our own jobe online nor ideone's limits get it sidelined.

I have not detected limits in rextester by the way so perhaps i could consider to replace ideone with rextester in your refreshed ideonesandbox phps.

I do not have shell access to moodle server yet and I do not know if my collegue, who manages moodle on the rented server has got something like this himself. I have got admin rights in moodle, but I am not  "root".

Is it possible to edit php-scripts of the plugin any other way? If not I would have to ask him.

I never maintained moodle before, so I do not really know what's going on under the surface, though generally I know php well enough to do modifications if I have got some access to the files. 

Martin

 





In reply to Martin Zwerschke

Re: Running C# in CodeRunner

by Richard Lobb -

Hi Martin

The latest push to the Development branch of CodeRunner on github (https://github.com/trampgeek/moodle-qtype_coderunner) includes an updated ideonesandbox.php which I think should work with the language C# though I can't check because I've used up my free run ration. You can either copy that one file, to replace the existing one (which is in the classes folder) or, if you're feeling brave, pull the entire development branch. The latter option isn't recommended on a production server, though we're already running it on ours. If you use just the updated ideonsandbox.php with the question's language set to C# it won't be correctly syntax-coloured by the Ace editor, which instead uses the name 'cs' for C# (not recognised by Ideone). If you use the complete development branch, with a totally rewritten Ace interface, the syntax colouring should work with C#.

If you don't have sufficient access to your Moodle server to pull code in with git, then I'm afraid you'll have to wait for the changes to be merged back into Master and a new Moodle package released. That will probably happen around February next year.

Please be warned, though, that I've tagged use of the Ideone sandbox as "Deprecated". If there are any other users wanting the Ideone sandbox to stick around, now is the time for them to put up their hands!

Richard

In reply to Richard Lobb

Re: Running C# in CodeRunner

by Martin Zwerschke -

Hi Richard,

although I am still interested in keeping the ideone-sandbox support alive or even adding a rextester-support, I started to
experiment with  jobe server as well.

As I noticed, the canterbury ouside jobe-server does not have Mono installed, so I could not test the C#-template instantly.

For getting into things, I installed my own jobe-server on an outdate private PC.

I finally got everything to run, and your C# template works.

But I noticed some problems and questions on my way :

  • Why exactly do you suggest to implement C# by a python3 script instead of adding it as an additional language to the jobe server?
    Wouldn't that make things easier?
  • Concerning the documentation of jobe, securing the jobe server as suggested is not possible using the given ufw-commands in my scenario:
    ufw default reject outgoing
    ufw allow out proto tcp to <some_useful_ip> port 80,443
    ufw allow in 22/tcp
    ufw allow in proto tcp to any port 80 from <your_client_ip>
    ufw enable
    Have I understood  correctly, that the ufw firewall to configure is the ufw on the jobe server? Or do you mean ufw on a separate firewall PC or a PC that is run as an internet router? Perhaps documentation could be clearer on this. I suggest a draft of the scenario for that the securing commands are valid.
    I guess <some_useful_ip> means the IP of our single (rented) moodle-server  and <your_client_ip> stands for the IP of my private jobe-server in my scenario. Perhaps that could be made clearer as well.

    My jobe server has got a private IP type 192.168.x.x as a reserved DHCP-lease given by the DSL-Router (a AVM-Fritz-Box in my case), which also serves as a NAT for the clients inside my LAN.
    The DSL-Router gets a dynamic IP from my DSL-provider. To be reachable from the Moodle-server I had to use a (free) DDNS-Service, that is updated regularly by the router and the DDNS service refreshs the DNS-subdomain for that.
    Inside the router's configuration I had to open an incoming port 80 to my jobe's 192.168.x.x-IP
    Yet the ufw commands do not support DNS names, but only fixed IPs.
    The Moodle server is a rented server and seems not to have a fixed IP as well, as I noticed this morning.
    The Server-IP shown inside the Moodle's Server-administration "php-info" has changed since yesterday.
    The moodle is reachable by it's DNS-name, not by a fixed IP-address.
    Thus I can not fill in the <some_useful_ip> field as well, nor in the ufw of my jobe server nor in the DSL-Router,
    that can not open a port for a special IP anyway, just open it for ouside access in general.

    Is it right, that opening in 22 tcp is only necessary, if the jobe server is not accessible physically with a mouse/keyboard and screen?


  • The C#-template seems not to include arguments for the mcs-compiler like the c-via-python-template does.
  • Special chars like 'ä' are not possible inside a C#-program - but that's a known bug already
  • Is there a way to inject keypresses as parts of the test cases? I tried to create a question using the C#'s Console.ReadKey()-command,
    but that could not work out, because the "stdin" only contains a string of chars, no key events.
  • You wrote:

      >For other question types, such as "write-a-C#-method", you will have to wrap the student submission in some extra code,

      > including that supplied in the >>{{TEST}} variable. If you need help with that, post back here.
      Could you please add a template for that? 
      How would a combinatator-question-template be done by the way?
      (I am not so sure to have understood the concept of that mechanism any way...
      Perhaps  a screenshot of a combinator-question in question author's and student's view would help.
      What exactly is the purpose of the cryptic SEPARATOR-string and why does it have to by customizeable?)

In reply to Richard Lobb

Re: Running C# in CodeRunner

by James Bryan -

Hi Richard,

In regards to wrapping the student submission with a surrounding code template, would you be able to give a basic example for how you would do that using coderunner?

Thanks

Regards

James

In reply to James Bryan

Re: Running C# in CodeRunner

by Martin Zwerschke -

I did it this way (Prototype for question to create a c#-function):

""" The template for a question type that compiles and runs a student-submitted
    mono C# program. 
"""

import subprocess, sys

# Write the student code to a file prog.cs
student_answer = """using System;using System.IO;using System.Text;namespace myProg {    class program {"""
student_answer += """ {{ STUDENT_ANSWER | e('py') }} """
student_answer += """ public static void Main(string[] args) { """
student_answer += """ {{TEST.testcode | e('py') }}"""
student_answer += """ } } } """

with open("prog.cs", "w") as src:
    print(student_answer, file=src)

# Compile
return_code = subprocess.call(['mcs', 'prog.cs'])
if return_code != 0:
    print("** Compilation failed. Testing aborted **", file=sys.stderr)

# If compile succeeded, run the code. Since this is a per-test template,
# stdin is already set up for the stdin text specified in the test case,
# so we can run the compiled program directly.
if return_code == 0:
    try:
        output = subprocess.check_output(["mono", "./prog.exe"], universal_newlines=True)
        print(output)
    except subprocess.CalledProcessError as e:
        if e.returncode > 0:
            # Ignore non-zero positive return codes
            if e.output:
                print(e.output)
        else:
            # But negative return codes are signals - abort
            if e.output:
                print(e.output, file=sys.stderr)
            if e.returncode < 0:
                print("Task failed with signal", -e.returncode, file=sys.stderr)
            print("** Further testing aborted **", file=sys.stderr)

For example you ask your students to write a function that calculates the average of three values.

The Question for the students is:


Please write a C#-Function  (private static oder public static does not matter),

that calculates the average value of 3 Numbers (type int) and returns it as a float value.

Please only put this function into the answer box (and nothing else) !

Moodle calls your function and tests the result.


And they just have to fill this function into the answer-box.

The correct answer could be like this:

public static float Average(int a, int b, int c)
{
            int sum = a + b + c;

            return sum / 3.0f; 
}

Then use test cases like:

Console.Write(Average(3,4,2));

with expexted output of 3

or

Console.Write(Average(5,8,12));

with expected output of 8.333333


This workes for me, though I not really know, why the 

| e('py') 

is necessary. I just copied it from Robb's sample above.


Hope that will help you.


In reply to Martin Zwerschke

Re: Running C# in CodeRunner

by Richard Lobb -

Hi Martin

Thanks for posting. Just to answer your question about why the | e('py') is necessary....

When wrapping a raw string from the author form or student answer in quotes to make a literal string within the template code, you generally need to escape various special characters that might result in invalid string literals. As a trivial example, if a student's answer was "Hi" (including the double quotes) and you wrote, in JavaScript.

s = "{{STUDENT_ANSWER}}";

you'd get

s = ""Hi"";

which is broken. Hence Twig provides a range of escapers, so that for a JavaScript template you can write

s = "{{STUDENT_ANSWER | e('js')}}"

which would give you the valid

s = "\"Hi\"";

I've added to the Twig base set of escapers, e('py'), e('java') and e('matlab'). The Java escaper can be used for C and C++ too. 

The Python escaper is intended for use only within triple-quoted string literals. It escapes only double quotes (in case the raw string itself has triple quotes in it) and backslashes. Backslashes need to be escaped so that, for example, a 2-character \n in the student's code doesn't get converted to a single newline character in the triple-quoted literal.

Richard


In reply to Richard Lobb

Re: Running C# in CodeRunner

by Martin Zwerschke -

Hi Richard,

we are using C#  under coderunner using "mono" and the c#-via python prototype for a while now - works like a charme.

Now  I tried to use C# 7 features (like the "switch-case" pattern match with keyword "when")

mcs C# compiler is called in the prototype. But it does not understand "when" keyword.

I tried to add option -langversion:7.2  (which mcs can handle) but it still does not support "when"

So i tried to switch to  newer  "csc"  compiler , that can handle "when" and other C#7 features.

I used:

return_code = subprocess.call(['csc', '/unsafe+', '/nologo', '-langversion:latest', 'prog.cs'])

for the compilation. It seems to work.

But the code execution runs into runtime exceptions:

"Unhandled Exception:

[ERROR] FATAL UNHANDLED EXCEPTION: System.OutOfMemoryException: Out of memory
at (wrapper managed-to-native) System.Runtime.InteropServices.Marshal.AllocHGlobal(intptr)"

OutOfMemoryException


But if i use 

mono ./test.exe

to execute the csc-generated compilation on the jobe server's command line, the program executes without problems though.

Does jobe block the code?


In reply to Martin Zwerschke

Re: Running C# in CodeRunner

by Richard Lobb -

Jobe sets resource limits (processes, cpu time, memory, file output) on all tasks. It seems you have hit the default limit on memory, presumably because the newer C# version wants more memory than the older one. With this question type, which runs C# in a subprocess, you need enough memory for both Python and C# together. Depending on what version you're running, the default is somewhere between 300 MB and 500 MB as I recall.

Easily fixed. In your prototype, open the Advanced Customisation panel and experiment with the setting of the Memory Limit field. A value of 0 turns off the memory limit checking altogether so you may want to try this first, but that's not recommended as a permanent setting.

In reply to Richard Lobb

Re: Running C# in CodeRunner

by Martin Zwerschke -

Increasing the memory limit did the trick.


I was suprised, however, that the csc compiler needs so much memory.

I had to increase to at least 600MB to succeed and now use a limit of 650MB for my C#-questions prototype.

Of course it is not only csc that uses the memory but the whole jobe toolchain.

At least csc needs much more memory than mcs does.

(Perhaps because it comes from Microsoft...)

Thanks for your help.