If you create a python3 question and fill in the anwer box with 51 'pass' statement lines, the error
"An unexpected error occurred. The sandbox may be down. Try again shortly"
appears.
The same happens with fewer lines with longer statements.
What setting causes this??
I think you're closing in on the problem. It sounds like a misconfigured Moodle server that is limiting the size of the request to the Jobe server. For example, a php.ini value of post_max_size = 256 would be cause all the failures you're reporting. However, such a low setting would cripple Moodle in lots of other ways. I would have thought that the restriction would have to apply only within the CodeRunner context for the problem to go unnoticed elsewhere. I'm not sure how this would come about.
Is this a production Moodle server that's working OK in all other ways?
Can you do a phpinfo() on your server (Google it) and see if anything looks seriously wrong? Also, you could possibly try a PHP question that simply does phpinfo() from within CodeRunner.
Python questions with thousands of lines are common and normally give no problems. So I think there is something unusual about your Moodle server configuration.
Hi Richard,
nice to learn coderunner by debugging :-)
The phpinfo from our moode>admin>server>phpinfo() menu shows a post_max_size of 4096M to make sure students can upload large documents.
The phpinfo from inside a php question answer shows 8M, but I think that is the jobe2.cosc.canterbury.ac.nz server talking.
Total output from a phpinfo() coderunner answer is shown below, part of our moodle (production) server output in the attachment. We use redhat, php-fpm and redis.
\r
phpinfo()
PHP Version => 7.2.19-0ubuntu0.18.04.1
System => Linux csse-jobe2 4.15.0-33-generic #36-Ubuntu SMP Wed Aug 15 16:00:05 UTC 2018 x86_64
Build Date => Jun 4 2019 14:48:12
Server API => Command Line Interface
Virtual Directory Support => disabled
Configuration File (php.ini) Path => /etc/php/7.2/cli
Loaded Configuration File => (none)
Scan this dir for additional .ini files => (none)
Additional .ini files parsed => (none)
PHP API => 20170718
PHP Extension => 20170718
Zend Extension => 320170718
Zend Extension Build => API320170718,NTS
PHP Extension Build => API20170718,NTS
Debug Build => no
Thread Safety => disabled
Zend Signal Handling => enabled
Zend Memory Manager => enabled
Zend Multibyte Support => disabled
IPv6 Support => enabled
DTrace Support => available, disabled
Registered PHP Streams => https, ftps, compress.zlib, php, file, glob, data, http, ftp
Registered Stream Socket Transports => tcp, udp, unix, udg, ssl, tls, tlsv1.0, tlsv1.1, tlsv1.2
Registered Stream Filters => zlib.*, string.rot13, string.toupper, string.tolower, string.strip_tags, convert.*, consumed, dechunk
This program makes use of the Zend Scripting Language Engine:
Zend Engine v3.2.0, Copyright (c) 1998-2018 Zend Technologies
_______________________________________________________________________
Configuration
Core
PHP Version => 7.2.19-0ubuntu0.18.04.1
Directive => Local Value => Master Value
allow_url_fopen => On => On
allow_url_include => Off => Off
arg_separator.input => & => &
arg_separator.output => & => &
auto_append_file => no value => no value
auto_globals_jit => On => On
auto_prepend_file => no value => no value
browscap => no value => no value
default_charset => UTF-8 => UTF-8
default_mimetype => text/html => text/html
disable_classes => no value => no value
disable_functions => no value => no value
display_errors => STDOUT => STDOUT
display_startup_errors => Off => Off
doc_root => no value => no value
docref_ext => no value => no value
docref_root => no value => no value
enable_dl => On => On
enable_post_data_reading => On => On
error_append_string => no value => no value
error_log => no value => no value
error_prepend_string => no value => no value
error_reporting => no value => no value
expose_php => On => On
extension_dir => /usr/lib/php/20170718 => /usr/lib/php/20170718
file_uploads => On => On
hard_timeout => 2 => 2
highlight.comment => <font style="color: #FF8000">#FF8000</font> => <font style="color: #FF8000">#FF8000</font>
highlight.default => <font style="color: #0000BB">#0000BB</font> => <font style="color: #0000BB">#0000BB</font>
highlight.html => <font style="color: #000000">#000000</font> => <font style="color: #000000">#000000</font>
highlight.keyword => <font style="color: #007700">#007700</font> => <font style="color: #007700">#007700</font>
highlight.string => <font style="color: #DD0000">#DD0000</font> => <font style="color: #DD0000">#DD0000</font>
html_errors => Off => Off
ignore_repeated_errors => Off => Off
ignore_repeated_source => Off => Off
ignore_user_abort => Off => Off
implicit_flush => On => On
include_path => .:/usr/share/php => .:/usr/share/php
input_encoding => no value => no value
internal_encoding => no value => no value
log_errors => Off => Off
log_errors_max_len => 1024 => 1024
mail.add_x_header => Off => Off
mail.force_extra_parameters => no value => no value
mail.log => no value => no value
max_execution_time => 0 => 0
max_file_uploads => 20 => 20
max_input_nesting_level => 64 => 64
max_input_time => -1 => -1
max_input_vars => 1000 => 1000
memory_limit => 128M => 128M
open_basedir => no value => no value
output_buffering => 0 => 0
output_encoding => no value => no value
output_handler => no value => no value
post_max_size => 8M => 8M
precision => 14 => 14
realpath_cache_size => 4096K => 4096K
realpath_cache_ttl => 120 => 120
register_argc_argv => On => On
report_memleaks => On => On
report_zend_debug => Off => Off
request_order => no value => no value
sendmail_from => no val ...snip... s.freq => 1% => 1%
session.upload_progress.min_freq => 1 => 1
session.upload_progress.name => PHP_SESSION_UPLOAD_PROGRESS => PHP_SESSION_UPLOAD_PROGRESS
session.upload_progress.prefix => upload_progress_ => upload_progress_
session.use_cookies => 1 => 1
session.use_only_cookies => 1 => 1
session.use_strict_mode => 0 => 0
session.use_trans_sid => 0 => 0
sodium
sodium support => enabled
libsodium headers version => 1.0.16
libsodium library version => 1.0.16
SPL
SPL support => enabled
Interfaces => OuterIterator, RecursiveIterator, SeekableIterator, SplObserver, SplSubject
Classes => AppendIterator, ArrayIterator, ArrayObject, BadFunctionCallException, BadMethodCallException, CachingIterator, CallbackFilterIterator, DirectoryIterator, DomainException, EmptyIterator, FilesystemIterator, FilterIterator, GlobIterator, InfiniteIterator, InvalidArgumentException, IteratorIterator, LengthException, LimitIterator, LogicException, MultipleIterator, NoRewindIterator, OutOfBoundsException, OutOfRangeException, OverflowException, ParentIterator, RangeException, RecursiveArrayIterator, RecursiveCachingIterator, RecursiveCallbackFilterIterator, RecursiveDirectoryIterator, RecursiveFilterIterator, RecursiveIteratorIterator, RecursiveRegexIterator, RecursiveTreeIterator, RegexIterator, RuntimeException, SplDoublyLinkedList, SplFileInfo, SplFileObject, SplFixedArray, SplHeap, SplMinHeap, SplMaxHeap, SplObjectStorage, SplPriorityQueue, SplQueue, SplStack, SplTempFileObject, UnderflowException, UnexpectedValueException
standard
Dynamic Library Support => enabled
Path to sendmail => /usr/sbin/sendmail -t -i
Directive => Local Value => Master Value
assert.active => 1 => 1
assert.bail => 0 => 0
assert.callback => no value => no value
assert.exception => 0 => 0
assert.quiet_eval => 0 => 0
assert.warning => 1 => 1
auto_detect_line_endings => 0 => 0
default_socket_timeout => 60 => 60
from => no value => no value
session.trans_sid_hosts => no value => no value
session.trans_sid_tags => a=href,area=href,frame=src,form= => a=href,area=href,frame=src,form=
url_rewriter.hosts => no value => no value
url_rewriter.tags => form= => form=
user_agent => no value => no value
zlib
ZLib Support => enabled
Stream Wrapper => compress.zlib://
Stream Filter => zlib.inflate, zlib.deflate
Compiled Version => 1.2.11
Linked Version => 1.2.11
Directive => Local Value => Master Value
zlib.output_compression => Off => Off
zlib.output_compression_level => -1 => -1
zlib.output_handler => no value => no value
Additional Modules
Module Name
Environment
Variable => Value
PATH => /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin
LANG => C.UTF-8
PWD => /home/jobe/runs/jobe_Ry3nOh
PHP Variables
Variable => Value
$_SERVER['PATH'] => /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin
$_SERVER['LANG'] => C.UTF-8
$_SERVER['PWD'] => /home/jobe/runs/jobe_Ry3nOh
$_SERVER['PHP_SELF'] => __tester__.php
$_SERVER['SCRIPT_NAME'] => __tester__.php
$_SERVER['SCRIPT_FILENAME'] => __tester__.php
$_SERVER['PATH_TRANSLATED'] => __tester__.php
$_SERVER['DOCUMENT_ROOT'] =>
$_SERVER['REQUEST_TIME_FLOAT'] => 1563793262.5484
$_SERVER['REQUEST_TIME'] => 1563793262
$_SERVER['argv'] => Array
(
[0] => __tester__.php
)
$_SERVER['argc'] => 1
$_ENV['PATH'] => /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin
$_ENV['LANG'] => C.UTF-8
$_ENV['PWD'] => /home/jobe/runs/jobe_Ry3nOh
PHP License
This program is free software; you can redistribute it and/or modify
it under the terms of the PHP License as published by the PHP Group
and included in the distribution in the file: LICENSE
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
If you did not receive a copy of the PHP license, or have any
questions about PHP licensing, please contact license@php.net.
define('SEPARATOR', "#<ab@17943918#@>#\n");
;
From a 'standard' Ubuntu Moodle server all runs fine so maybe the trouble is with our "professional" Redhat setup with F8 loadbalancers and http(s) proxy servers that might limit outgoing data larger than x bytes?
That server cannot run python2 so what would be the simpelest submit test (python2 or php) to test this?
I am pleased you're making progress, because I couldn't think of what to suggest next.
I like your proxy server idea. I've had a couple of users hit a problem with proxy servers totally blocking outgoing POSTs but I've not seen any configured to allow only small ones to pass through. Seems a bit odd but it's certainly possible and would explain everything.
I don't understand the bit about "That server cannot run python2 ...". Perhaps you meant "That server cannot run python3..." but I'm still not sure which server you're referring to, nor what test you're trying to carry out. At any rate, a simple "Hello world" php question would have a single test case with no test code and expected output of "Hello world". The answer to the question would be
<?php
echo("Hello world");
Does that help? I look forward to hearing what the problem is/was.
Great detective work. Thanks.
However, the 4 MB figure doesn't sound right to me. You had a Python program that blew the limit with only 51 'pass' statements. I just ran a basic Python question (the "write a function sqr" example) and pasted in an answer with 51 pass statements in it. The total size of the POST payload was 1377 bytes. There's no way any standard question, including a graph question, could get anywhere near 4 MB unless the student answer was itself megabytes.
I could believe that a 4 kB limit would cause problems of the sort you've had, but 4 MB shouldn't have.
And there goes my hypothesis :-)
Well done, Sherlock! Good to know the problem is now fully explained. All that remains is to find a fix ...
I'm afraid https is not (yet) an option with jobe. I have had repeated requests for it (you'll find some in the forums) but have always resisted on the basis that the gains would be negligible from a security standpoint and the cost would be extra complexity, lower performance, and at least one more point of failure (expired certificates). However, I think I'll have to accept that there is actually one significant gain: compatibility with common IT service department policies, even when these are largely meaningless in the context of a Jobe server.
I'll look into it when I next have some development time (I'm currently teaching two courses). Until then I think you'll have to try to get the http proxy limit raised. It's hard to see how ITS could justify a 1k limit for HTTP but a 4M limit for HTTPS.