I've never tried it myself, but I like the idea.
The sequence of calls from CodeRunner to Jobe is roughly as you describe it. Note though that support files are cached on the Jobe server indefinitely, so running a test doesn't normally involve any file uploads except on the very first test case run of a particular question.
In a worst case scenario, multiple test cases need to be run for each question (no combinator) and multiple support files are needed. Each test case run is run independently as follows;
- Compute the md5 checksums of all support files - these are used as file ids.
- POST the test case to Jobe, containing the list of required file ids
- If you get a 200 OK response, you're done. Otherwise ...
- If you get a 404 File Not Found response, one or more of the required support files is missing, so:
- PUT each file
- POST the test case to Jobe (again with a list of required file ids).
- If you get anything other than 200 OK, you're dead.
Since all test cases are independent, you don't actually need to run them on the same Jobe server although you do need to ensure that the entire sequence above uses just the one server. Running different test cases on different servers wouldn't currently give you a performance benefit because the test case submissions are synchronous. However, if you're going to have multiple Jobe servers available, it might be interesting to think about making the test case runs asynchronous so they could be run in parallel then re-sequenced as results came back.
The steps you describe for load balancing should work fine, but it might be better to put the line $this->jobid = rand() into the jobesandbox::execute() method instead of the constructor. That keeps alive the possibility of exploiting parallel test case execution if any of us ever gets around to making the submissions asynchronous.
Of course, since I've never tried this, I guarantee nothing. But I look forward to hearing how it goes.
PS: I forgot to mention that the process of running all the test cases for a question actually starts with a single GET request to find what languages the Jobe server supports. That doesn't change the load balancing game, although all Jobe servers in your cluster would have to support the same set of languages.