I don't teach Java so have no experience to report on, but since no-one else has responded ...
I'm guessing you would want to use the Write a class question type (or perhaps a derivative of it, customised for your own needs). Then write a series of tests along the following lines:
- Try to instantiate the class (in a try ... catch construct) and expect to catch whatever exception Java throws when you instantiate an abstract class.
- Declare a subclass that does not implement the declared abstract methods. Try to instantiate it. Again expect to catch the appropriate exceptions.
- Declare a subclass with all the required extra functionality. Instantiate it. Test all the functionality that the abstract class is meant to provide.
- If you're very keen, you could try a series of variants of (3), each lacking an implementation of one of the abstract methods. Expect to get exceptions as in 2.