Thanks, I sort of figured that this may not be the default use case.
The thread you linked to has some nice pointers - will follow up on those as well.
One other low overhead possibility I am looking at is to just allow students to submit code to an assignment, and then use the Jobe sandboxing to run the code and get results - of course with timeouts changed.
In any case, coderunner has opened up the possibility of having many "micro" assignments that I had not thought of before. Thanks!