Depending on your Jobe server configuration and possibly the type of question you're asking, increasing the number of Jobe users might help or might make things worse.
My rule of thumb is to set the number of jobe users to the number of CPU cores on the server or at most twice that. Most Jobe tasks are CPU bound (though I'm not quite so sure about Java as we don't use it much) and having one or at most two Jobe users per core is sufficient to achieve 100% CPU utilisation. At that point your server is saturated and unless it's a short-term peak, you're going to overload the Jobe server and students will get Jobe Server Overload failures.
If a request comes into Jobe and all Jobe users are busy, the job is held in a pseudo-queue for at most 10 seconds before the Overload response is returned. In a test or exam you certainly don't want students being held up any longer than that.
The easiest way to scale up Jobe server capacity is to add more servers: the Jobe server parameter in the CodeRunner Admin Settings can take a semicolon-separated list of server IPs or names. A server is chosen at random from the list.
Our in-house system has a single nginx server running as a reverse proxy, which distributes jobs to the upstream servers in a round-robin fashion. This is highly scalable but if you're currently getting by with a single Jobe server I recommend the simplicity of two or three semicolon-separated Jobe servers.
BUT before doing any of these things I strongly recommend some benchmarking. The program testsubmit.py at the root of the Jobe distribution includes performance testing. For example the command
python3 testsubmit.py --perf python3
tests performance in Python. It does two separate tests:
- If you throw 'n' jobs at the server all at once and then sit back and wait, what's the maximum value of n that the server can achieve without errors?
- What's the maximum rate at which the server can maintain a steady constant rate of submissions over a 30 second window?
WARNING: one such run of testsubmit.py will push the Jobe server into overload so don't do it on your production server when anything important is on.
Armed with those numbers and some back-of-the-envelope calculations of average and peak submission rates during your tests and exams you'll be in a much better position to assess your needs.