Issues with Loading Support Files on Jobe Server for SQLite Task

Issues with Loading Support Files on Jobe Server for SQLite Task

de către Leonardo Mogianesi-
Număr de răspunsuri: 6

Hello,

I'm experiencing a problem with the Jobe server when trying to run an SQLite task with a support file. I have created a custom task (SqliteTask.php) in the application/libraries directory of my Jobe server to enable SQLite functionality. Here is the structure of my task class:

namespace Jobe;
class SqliteTask extends LanguageTask {
    public string $databaseFileName;

    public function __construct($filename, $input, $params) {
        parent::__construct($filename, $input, $params);
        $this->databaseFileName = ':memory:';
    }

public function compile()
    {
        // SQLite non richiede compilazione
        $this->cmpinfo = '';  
    }
    
public static function getVersionCommand()
    {
    return array('sqlite3 --version', '/([0-9.]+)/');
    }

public function defaultFileName($sourcecode)
    {
        return 'query.sql';
    }

public function getExecutablePath()
    {
        return '/usr/bin/sqlite3';
    }

public function getTargetFile()
    {
        return $this->databaseFileName;
    }

public function execute()
    {
        $cmd = $this->getExecutablePath() . ' ' . $this->databaseFileName . ' < ' . escapeshellarg($this->sourceFileName);
        list($this->stdout, $this->stderr) = $this->runInSandbox($cmd, false, $this->input);

        if (!empty($this->stderr)) {
            $this->cmpinfo = $this->stderr;
        }
    }
}
}

Despite adding the SqliteTask.php correctly, I am encountering the following error when trying to load my database file (prova.db) as a support file:

ERROR - [Date/Time] --> runs_post: File missing/unavailable: prova.db (FileId: 22484f8f8a5fe85db13c2d063ec82293, Destination: /home/jobe/runs/jobe_ZcAl4U/prova.db)

I print this error in the LanguageTask.php, precisely in this method:

public function loadFiles($fileList)
{
    foreach ($fileList as $file) {
        $fileId = $file[0];
        $filename = $file[1];
        $destPath = $this->workdir . '/' . $filename;
        if (!FileCache::fileExists($fileId) ||
           ($contents = FileCache::fileGetContents($fileId)) === false ||
           (file_put_contents($destPath, $contents)) === false) {
            throw new JobException(
                "File missing/unavailable: " . $filename . " (FileId: " . $fileId . ", Destination: " . $destPath . ")", 404
            );
        }
    }
}



Obviously test's won't run because they doesn't find the db file.

Additional Information:

  • Jobe seems to recognize the SqliteTask, as sqlite shows up in the list of supported languages:
    curl http://coderun.myserver.it/jobe/index.php/restapi/languages:
    [["c","11.4.0"],["cpp","11.4.0"],["java","11.0.24"],["nodejs","16.20.2"],["octave","6.4.0"],["pascal","3.2.2"],["php","8.1.2"],["python3","3.10.12"],["sqlite","3.37.2"]]
  • The file prova.db is missing in the working directory during execution, even though it has been added as a support file in the CodeRunner question.

Question:

Has anyone tried manually adding SQLite3 support to Jobe? If so, could you provide any insights on how to correctly set it up or any configurations that need special attention?

Thank you!

Atașament aaaaa.JPG
Atașament bbbbbb.JPG
Atașament sql.JPG
Ca răspuns la Leonardo Mogianesi

Re: Issues with Loading Support Files on Jobe Server for SQLite Task

de către Richard Lobb-
Well done on getting an sqlite3 language running natively on Jobe. But I'm puzzled as to why you're doing that? The CodeRunner built-in sql question type runs sqlite3 via Python, which is how we run most non-native languages on Jobe. The overheads of using Python are neglible (roughly 50 msecs) and it saves having to modify the Jobe code.

File upload failures are most commonly due to one of two causes:
  1. Excessive file size. There are various settings in Moodle and the php.ini files on both the Moodle server and the Jobe server that limit the size of file that can be managed and transferred between the two servers. 
  2. Networking issues. CodeRunner first tries to run a task using an HTTP POST request without uploading the support files. If that fails, it then uses an HTTP PUT request to upload all support files and then tries again. Some network firewalls and proxies don't handle PUT requests in their default configurations.

In both cases, depending on the failure mode, you may not get any useful error message except a "File not found" when you run the job.

To check if (1) is the problem, try with a very small file. To check if (2) is relevant, try running a simple Python question with a (small) support file.

I'd also like to know if the very simple sample SQL programs from the CodeRunner website (here) work on your system.


Ca răspuns la Richard Lobb

Re: Issues with Loading Support Files on Jobe Server for SQLite Task

de către Leonardo Mogianesi-
Thank you for your prompt response, you are the best.

I hadn't considered using Python, but the issue remains the same. I checked the php.ini file, and post_max_size is set to 8MB, while upload_max_filesize is set to 20M, so the file size shouldn't be a problem. For reference, my prova.db file is only 12KB.

Regarding the security issue that might be blocking POST and PUT requests, I'd like to understand how I could properly debug this. The database file is placed in a different temporary folder each time, such as /home/jobe/runs/jobe_3FiKFt/prova.db. How can I verify if the PUT request is being blocked?

I tryed (1) and (2) but nothing relevant went out. :(

What else can i do?

Thank you very much for your time.
Ca răspuns la Leonardo Mogianesi

Re: Issues with Loading Support Files on Jobe Server for SQLite Task

de către Richard Lobb-
You say you hadn't considered using Python but the screen shot of your question looks like a standard built-in SQL question (which does use Python). Please can you open your question again for editing, click Customise, and post a screen shot of the Template please?

Also, still with Customise set, please expand the "Advanced Customisation" panel, and send a screen shot of that.

Lastly, I'd like confirmation that files are uploading OK to Jobe in other question types. So:

1. Create a new CodeRunner question.
2. Choose python3 as the question type.
3. Set the question name and question text to any junk values.
4. Set the expected output for test case 1 to "Hello"
5. Add a very small (e.g. 100 byte) support file containing just simple text.
6. Save the question.
7. Preview it.
8. Enter the following Python code into the answer box and click Check:

import os
print(os.listdir())

What's the output? It should be a list of filenames like ['__pycache__', '__tester__.python3', 'prog.err', 'prog.cmd', mysimplesupportfile.txt', 'prog.out'].
Ca răspuns la Richard Lobb

Re: Issues with Loading Support Files on Jobe Server for SQLite Task

de către Leonardo Mogianesi-
Thanks again for the answer. I tested this out:

1. Create a new CodeRunner question.
2. Choose python3 as the question type.
3. Set the question name and question text to any junk values.
4. Set the expected output for test case 1 to "Hello"
5. Add a very small (e.g. 100 byte) support file containing just simple text.
6. Save the question.
7. Preview it.
8. Enter the following Python code into the answer box and click Check:

import os
print(os.listdir())

What's the output? It should be a list of filenames like ['__pycache__', '__tester__.python3', 'prog.err', 'prog.cmd', mysimplesupportfile.txt', 'prog.out'].

And i receive this:
Failed to run tests
Unexpected error while executing your code. The sandbox server may be down or overloaded. Perhaps try again shortly?

I tryed WITHOUT the support file (5bytes) and it works fine. So the problem is when i import the support file, it "crashes" and it gives me the "The sandbox server may be down or overloaded."

It happens with python3, sql, java, ecc.
With every language when i import a support file, it doesn't works.

Thanks for your time.

Atașament lol.JPG
Ca răspuns la Leonardo Mogianesi

Re: Issues with Loading Support Files on Jobe Server for SQLite Task

de către Richard Lobb-
This sounds like the problem I mentioned, where PUT requests are being disallowed by the network infrastructure. I'm afraid I can't help you with finding where the problem lies as institutional networks can be arbitrarily complex. But as a first check, you might want to confirm that the same problem occurs if you set CodeRunner back to using the default University of Canterbury jobe server, jobe2.cosc.canterbury.ac.nz.

Also, you can download the testsubmit.py program from the Jobe repo on GitHub and use that when testing - it's easier than testing with the Moodle server. It includes tests with uploaded files.
Ca răspuns la Richard Lobb

Re: Issues with Loading Support Files on Jobe Server for SQLite Task

de către Leonardo Mogianesi-
With the Canterbury Jobe It works fine.
Thanks, i will try with the testsubmit.py to see what Is the result.