Error on CodeRunner for Python3 plugins TensorFlow, Keras, Pytorch

Error on CodeRunner for Python3 plugins TensorFlow, Keras, Pytorch

de sky dancer23 -
Número de respuestas: 9

Please note that this is poor English as I am putting it into Google Translate

I would like to implement TensorFlow, Keras, and Pytorch, which are Python3 plugins for machine learning, in a CodeRunner quiz.

We have already installed TensorFlow, Keras, and Pytorch on the Jobe server and have confirmed that they work on the CLI of the Jobe server.

However, I am having trouble creating a CodeRunner quiz on moodle, and when I run it, I get the error in the attached image, does anyone know anything about this?
They all fail to import the library and give an error, but we are not sure what the cause is, since we have confirmed that it works on the server.

↓TensorFlow Errors
TensorFlow Errors

↓Keras Errors


↓Pytorch Errors

En respuesta a sky dancer23

Re: Error on CodeRunner for Python3 plugins TensorFlow, Keras, Pytorch

de Richard Lobb -
You're almost certainly exceeding the default resource limits (memory and/or num processes). See the advice in this thread: https://coderunner.org.nz/mod/forum/discuss.php?d=651
En respuesta a Richard Lobb

Re: Error on CodeRunner for Python3 plugins TensorFlow, Keras, Pytorch

de sky dancer23 -
Thank you very much.
We are now able to run it successfully.
However, the output is incorrect, perhaps due to character encoding.
Please let me know if there is a workaround.


En respuesta a sky dancer23

Re: Error on CodeRunner for Python3 plugins TensorFlow, Keras, Pytorch

de Richard Lobb -
I'm not familiar with the software you're using, and I don't understand why the weird output you're showing me is in the Expected column rather than the Got column. However, \x08 is the backspace character so it looks to me like the software you're using is displaying its progress, and "thinking" it's writing to a terminal, it is trying to update a single line display by repeatedly backspacing over what it has just written to update the displayed state. This isn't appropriate in CodeRunner - you don't wish to see the progress over time but only the final result, right? Can you instruct the program not to display progress?
En respuesta a Richard Lobb

Re: Error on CodeRunner for Python3 plugins TensorFlow, Keras, Pytorch

de sky dancer23 -
Thank you very much. I'll give it a try.
En respuesta a sky dancer23

Re: Error on CodeRunner for Python3 plugins TensorFlow, Keras, Pytorch

de Michael Fairbank -
I am familiar with using Tensorflow/Keras in Coderunner.
You would set verbose=0 in the fit function, to avoid output like that.
However, I would not recommend auto-marking work in coderunner that involves the process of training the neural network on the Jobe server. You don't want time-consuming tasks like that going on on the Jobe server. Why not ask the students to train and save their neural network on their own computers, and then they upload their saved neural network to coderunner for auto-marking?

You can allow students to upload the saved neural network with the "attachments" feature. (I always make the students save their network in the .h5 format as this seems more stable across different versions of tensorflow.)

Then my marking script in coderunner just measures some properties of the neural network. E.g. you can print model.summary(), and that works quite well (to check there are the correct layers in the neural network). You can also print out the activation functions for each layer.  You can also calculate the validation accuracy on a given dataset (e.g. MNIST), and print "success" if the accuracy >95%. Then auto-mark on coderunner with a regular expression which searches for the word "success". Let me know if you want me to post my coderunner test scripts.
En respuesta a Michael Fairbank

Re: Error on CodeRunner for Python3 plugins TensorFlow, Keras, Pytorch

de sky dancer23 -
Thank you for all the details!
I would like to see the code runner test script.
En respuesta a sky dancer23

Re: Error on CodeRunner for Python3 plugins TensorFlow, Keras, Pytorch

de Michael Fairbank -
In this question I ask the students to attach a .h5 file of their saved neural network that they train on the Iris dataset.   This is what the students see:

Upload your code for the trained feed-forward neural network model on the Iris dataset, scoring >90% accuracy on the validation set.

Use the "Attachment" feature below.  Your file must be called "ModelIris.h5" and be under 1MB in size.


And here are the attachment options I offer the students:



Then there are 3 unit tests run by coderunner. (These unit tests look for exact string matches; but it's also possible to make it mark by regular expression, and just look for a specific keyword, e.g. "success".)

Test 1. Print the keras_model.summary():

import tensorflow as tf
print("TensorFlow version", tf.__version__) # This is a useful check to ensure the Jobe tensorflow package has not been upgraded since you wrote this test script.
keras_model=tf.keras.models.load_model('ModelIris.h5') # This is the filename of the saved neural network I ask them to attach
keras_model._name="my_neural_network"
c=0
for layer in keras_model.layers:
c+=1
layer._name = "layer"+str(c) # Ensures the layers are named consistently; so we can get an exact string match on the model summary
keras_model.summary()

Test 2.  Print out the activation functions from each layer.
import tensorflow as tf
import os, contextlib
with open(os.devnull, 'w') as devnull:
 with contextlib.redirect_stdout(devnull):
   keras_model=tf.keras.models.load_model('ModelIris.h5')
   keras_model._name="my_neural_network"
c=0
for layer in keras_model.layers:
   c+=1
   if isinstance(layer, tf.keras.layers.Conv2D) or isinstance(layer, tf.keras.layers.Dense):
     print("Layer",c,"Activation Function",layer.activation.__name__)




Test 3. Check their saved neural network has learned the validation set sufficiently well.
import pandas as pd  
import tensorflow as tf

inputs_val=pd.read_csv('iris_test.csv',usecols = [0,1,2,3],skiprows = None,header=None).values # Note here
I've added the file iris_test.csv as one of the "support files" in the coderunner question definition.
labels_val = pd.read_csv('iris_test.csv',usecols = [4],skiprows = None ,header=None).values.reshape(-1)
keras_model=tf.keras.models.load_model('ModelIris.h5')

predictions = tf.math.argmax(keras_model(inputs_val),axis=1)
accuracy=tf.reduce_mean(tf.cast(tf.equal(predictions,labels_val),tf.float32)).numpy()
print("Validation Accuracy Exceeds 90%?",accuracy>0.9)


Here is how I uploaded my support file:



To make tensorflow not display lots of warning messages, which would break the exact-string matching, we have these first 2 lines added to set TF_CPP_MIN_LOG_LEVEL...



I just have the  "answer" and "answer-box preload" fields empty.  

For the "expected outputs", I just run it once on my own answer (a single saved .h5 file) and capture the output.

This works well, but every year there is a new tensorflow version and I have to re-build the expected string outputs (as tensorflow seems to tweak the exact layout of model.summary() each new version!)
En respuesta a Michael Fairbank

Re: Error on CodeRunner for Python3 plugins TensorFlow, Keras, Pytorch

de sky dancer23 -
Thank you very much. I'll give it a try.
What are the files that students upload and what are the contents of the supporting files? Do you also have samples?