You're right to proceed with caution. Running CodeRunner exams is not to be taken lightly. I've heard many cases of institutions running large CodeRunner exams for the first time without adequate forethought and catastrophic consequences.
However, 50 - 70 students is quite a modest requirement. We've been running exams with 300 students doing Python for many years on a departmental 8-core 8 GB Moodle server. More recently we've upped the specs because we now have 500 students in a sitting, but the relatively light loading during exams suggests we would probably in fact have been fine with the older spec machine.
Possibly more important than the Moodle machine spec is the Jobe server spec. We've traditionally used a single 8-core 8GB server for Jobe (separate from the Moodle server) and have had plenty of capacity. In an excess of caution we've recently moved to multiple Jobe servers but again, probably unnecessarily.
Having a completely separate Moodle machine for your exams has the advantage of isolating you from the main institutional server so that its load doesn't affect you and vice versa. It also means that you don't have to worry about students communicating via chat (if enabled on your intitutional server, as it is on ours). But of course it does come at a cost in dollars and time.
CodeRunner questions shouldn't place much more CPU load on the main Moodle server than other question types, so if your main University server handles tests/exams with 300 students I wouldn't have expected problems with a class of 50 - 70 doing a CodeRunner quiz.
There is one difference, though, which can be significant. With most Moodle question submissions, the browser opens a connection to Moodle, sends the answer, has it graded within at most a few 10s of milliseconds, and the connection closes again. With CodeRunner questions, however, the connection is kept open while the job is sent to the Jobe server for grading. This can take up to a few seconds. That means that a CodeRunner quiz can mean a lot more in-flight requests holding webserver connections open. I've heard of at least one case where an institution refused to allow CodeRunner exams because of that. I think they must have been contemplating exams with way more than 50 - 70 students however.
I've just seen a figure from an AI (if you trust it) saying "As a general rule, 1GB of RAM can typically support 10–20 concurrent users, but this varies based on activity. "
There are a few caveats, though:
- You mention you're running Java questions. Java is a resource hog and runs take considerably longer than in Python or C. You need to do some careful throughput measurements on your Jobe server. There's a program testsubmit.py that comes with Jobe. You can check your throughput in your selected language(s) with that, keeping in mind that the tests use absolutely minimal programs.
- If you try to make use of CodeRunner's facility to hide the Check button, so grading takes place at the end of the exam, you need to be very careful that you don't flood the servers (most likely the Jobe server) at the end of the exam.
- Question run time can vary significantly. If you set exercises that require students to solve complex mathematical problems, run times might change from a few milliseconds to a few seconds with significant impact on the Jobe server. You need to measure typical run time, multiply that by the expected submission rate, and make sure your Jobe server has the capacity.
- Even if the questions are very quick to mark normally, some students will write code that times out due to endless loops.