Neural network classification of data using Smile

Data classification is the central data-mining technique used for sorting data, understanding of data and for performing outcome predictions. In this small blog we will use a library Smilecthat includes many methods for supervising and non-supervising data classification methods. We will make a small Python-like code using Jython top build a complex Multilayer Perceptron Neural Network for data classification. It will have large number of inputs, several outputs, and can be easily extended for cases with many hidden layers. We will write a few lines of Jython code (most of our codding will deal with how to prepare an interface for reading data, rather than with Neural Network programming).

First of all, let us copy data samples. One sample will be for training, another file is for testing Copy these files to your local directory. We will import  import the necessary classes to be used in this example using the usual Python style:

from import AttributeDataset,NominalAttribute
from import DelimitedTextParser,IOUtils
from smile.classification import NeuralNetwork
from smile.math import Math
from jarray import zeros,array
from jhplot import *
import java

We import several classes from several packages: “smile”, “jhplot” and Java. An additional package “jarray” is used to work with Java arrays using Jython. The NeuralNetwork class is the most important for our example. It creates Multilayer perceptron neural network that consists of several layers of nodes, interconnected through weighted acyclic arcs from each preceding layer to the following. We will call it as:

nn=[Nr_input, Nr_hidden, Nr_output] # define structure of NN net=NeuralNetwork(NeuralNetwork.ErrorFunction.LEAST_MEAN_SQUARES, NeuralNetwork.ActivationFunction.LOGISTIC_SIGMOID,nn)
net.learn(x, y) # train neural net on input double array y, with the outcome given by x

Here we use logistic sigmoid function and the error function “LEAST_MEAN_SQUARES”. Before start using this NN, we will read data from the files, and create input arrays x,y. We will write a small function “getJavaArrays” that returns two arrays, double[][] and int[] (using Java style). After NN was trained, we will use the test sample and call the method “predict(x)” to verify that our prediction are close to the expected values from the test sample.

The full code looks as this:

from import AttributeDataset,NominalAttribute

from import DelimitedTextParser,IOUtils
from smile.classification import NeuralNetwork
from smile.math import Math
from jarray import zeros,array
from jhplot import *
import java

def getJavaArrays(dataset): # this function create x[][] and y[] array from datasets
    lst = [0.0]*rows
    twoDimArr = array([lst,[]], java.lang.Class.forName('[D'))
    x = dataset.toArray(twoDimArr)
    y = dataset.toArray(zeros(rows, "i"))
    return x,y

parser =DelimitedTextParser(); parser.setDelimiter("[t ]+")
parser.setResponseIndex(NominalAttribute("class"), 0)
x,y=getJavaArrays(train) # get input and output for training NN

print "Rescale data range .. "
p = len(x[0])
mu = Math.colMeans(x);
sd = Math.colSds(x);
for i in range(len(x)):
   for j in range(p):
      x[i][j] = (x[i][j] - mu[j]) / sd[j];

nin=len(x[0]); nout=Math.max(y)+1
print "Training: Nr for input layer=",nin," Nr for output layer=",nout

nn=[nin,50,nout] # 50 nodes in a hidden layer. Need another layer? Add integer after 50
print "NN layout=",nn
net = NeuralNetwork(NeuralNetwork.ErrorFunction.LEAST_MEAN_SQUARES, NeuralNetwork.ActivationFunction.LOGISTIC_SIGMOID,nn)

c1 = SPlot() # plot error vs epoch
c1.visible(); c1.setGTitle("Neural Network output error")
c1.setAutoRange(); c1.setMarksStyle('various')
c1.setConnected(1, 0); c1.setNameX('Epoch'); c1.setNameY('Error')

for j in range(70): # 70 is max value for epoch
   net.learn(x, y)
   for i in range(len(x)):
      if (net.predict(x[i]) != y[i]): error +=1
   error=error / len(x)
print "Epoch=",j," Error=",error

print "Testing using zip.test file.."

for i in range(len(testx)): # rescale data
   for j in range(p):
      testx[i][j] = (testx[i][j] - mu[j]) / sd[j];

for i in range(len(testx)):
   if (net.predict(testx[i]) != testy[i]): error +=1
print "Error rate =", 100.0 * error / len(testx),"%"

Note that we use one hidden layer with 50 hidden nodes. The number of input (256) and outputs (10) are given by the structure of the data (looks at the data file to see their structure).

Now you can run this example using the DMelt data program (dmelt). Install DMelt, open the above code in the editor, and save it in a file with the extension “.py” (for example,  as “” file). This extension is important since it tells DMelt how to run this code.  Then run this file: click the button with a green running man in the top menu, or press [F8]. You will see a image which shows how the learning error shrinks as a function of epoch.  The last line of the code gives the error rate for your prediction.

If you are interested in deep neural networks, increase the number of hidden layers using the input list. For example, this setup has 2 hidden layers, with 50 and 25 nodes:


If you will see that the number of epochs will be drastically reduced to obtain a similar (or better) rate for successful predictions (but, the CPU time, will increase since more time will be needed to adjust the weights).

Neural network classification of data using Smile