Tutorial: Introduction to neural networks

From stats++ wiki
Jump to: navigation, search

This tutorial introduces the use of neural networks with the executable version of stats++.

It will be demonstrated how machine learning can be used to learn the functional relationship between the input and the output of a labeled data set (supervised learning).

In particular, a neural network will be trained to learn the following simple function: \begin{equation*} f(x,y) = (x + y)^2 \end{equation*} [input: $x$ and $y$; output: $f(x,y)$].

You will learn:

Note that the data for this tutorial will not be generated, or preprocessed. See the tutorial on preprocessing data.

The files for this tutorial (input and expected output) can be obtained here: tutorial_mlp_1.tgz

Preparing the work directory

Create a work directory, work/. Switch to this directory.

In this directory, you should untar the file tutorial_mlp_1.tgz.

Note that all work in this tutorial will be carried out in this directory, unless otherwise specified.

Data

Typically, the first task in stats++ is to collect and preprocess the data.

As discussed above, data is provided for this tutorial. Note the following files:

  • train_in.dat
  • train_out.dat
  • test_in.dat
  • test_out.dat

The first two files will be used for training, and the latter two for testing. The *_in.dat files contain the input data ($x$ and $y$):

x_1 y_1
x_2 y_2
.
.
.
x_n y_n

(for $n$ data points; in this case, $n = 1000$ and $= 100$ for training and testing ,respectively) and the *_out.dat files contain the output data [$f(x,y)$]:

f(x,y)_1
f(x,y)_2
.
.
.
f(x,y)_n

Creating the neural network

The first task (given data) is to create a neural network.

Open a text editor, and enter:

mlp_file     = mlp.txt
 
prefix       = mlp
 
[create]
 
architecture = 2
architecture = 10
architecture = 1

Save this file as param_create.txt.

The first two lines of this file (mlp_file and prefix) are always required for the NeuralNet executable. mlp_file specifies the name of the file used to store the neural network. prefix specifies the prefix for output.

The [create] block sets that the following variables are to specify the creation of the network.

The architecture variables specify the architecture of the network. In this tutorial, two input parameters ($x$ and $y$), a single hidden layer with $10$ nodes, and one output [$f(x,y)$] are used.

Open a terminal, and type:

$ mlp param_create.txt

A neural network will be created and stored in the file specified by mlp_file (mlp.txt). Note that this step will be fast.

Training the neural network

Now that we have created a neural network (stored in mlp.txt), the next step is to train it.

Open a text editor, and enter:

mlp_file       = mlp.txt
 
prefix         = mlp
 
[train]
 
method         = 1
 
X_in_file      = train_in.dat 
X_out_file     = train_out.dat

Save this file as param_train.txt.

The [train] block sets that the following variables are to specify the training of the network.

The method variable specifies the method used for training. In this tutorial, 1 specifies QRprop.

The X_in_file and X_out_file variables specify the name of the files used for training. In this tutorial, train_in.dat and train_out.dat, respectively.

Open a terminal, and type:

$ mlp param_train.txt

The neural network (stored in mlp.txt) will be trained, and (re-)stored in the file mlp.txt. Note that this will take a couple of seconds.

As part of the training, the following additional files will be created:

  • training_errors.dat
  • fit_valid_err.dat

These files are discussed in detail (and additional details on training, in general) in the tutorial on neural network training.

Testing the neural network

Now that we have created and trained a neural network (stored in mlp.txt), it can be tested (or used) on unseen data.

Open a text editor, and enter:

mlp_file       = mlp.txt
 
prefix         = mlp
 
[test]
 
X_in_file      = test_in.dat 
X_out_file     = test_out.dat

Save this file as param_test.txt.

The [test] block sets that the following variables are to specify the testing of the network.

The X_in_file and X_out_file variables are analogous to those used for training.

Open a terminal, and type:

$ mlp param_test.txt

The neural network (stored in mlp.txt) will be tested. Note that this step will be relatively fast.

The following file will be created:

  • mlp.test.out.dat

The data in this file (predictions by the network) can be compared to test_out.dat. Note that the first column provides a default label for each data point.

Further investigation

Related tutorials

Tutorial: Preprocessing data

Tutorial: Neural network training (advanced)

Notes and references