Tutorial: Introduction to neural networks
This tutorial introduces the use of neural networks with the executable version of stats++.
It will be demonstrated how machine learning can be used to learn the functional relationship between the input and the output of a labeled data set (supervised learning).
In particular, a neural network will be trained to learn the following simple function: \begin{equation*} f(x,y) = (x + y)^2 \end{equation*} [input: $x$ and $y$; output: $f(x,y)$].
You will learn:
- How to create a neural network
- How to train the neural network
- How to test the neural network
Note that the data for this tutorial will not be generated, or preprocessed. See the tutorial on preprocessing data.
The files for this tutorial (input and expected output) can be obtained here: tutorial_mlp_1.tgz
Contents
Preparing the work directory
Create a work directory, work/. Switch to this directory.
In this directory, you should untar the file tutorial_mlp_1.tgz.
Note that all work in this tutorial will be carried out in this directory, unless otherwise specified.
Data
Typically, the first task in stats++ is to collect and preprocess the data.
As discussed above, data is provided for this tutorial. Note the following files:
-
train_in.dat -
train_out.dat -
test_in.dat -
test_out.dat
The first two files will be used for training, and the latter two for testing. The *_in.dat files contain the input data ($x$ and $y$):
x_1 y_1 x_2 y_2 . . . x_n y_n
(for $n$ data points; in this case, $n = 1000$ and $= 100$ for training and testing ,respectively) and the *_out.dat files contain the output data [$f(x,y)$]:
f(x,y)_1 f(x,y)_2 . . . f(x,y)_n
Creating the neural network
The first task (given data) is to create a neural network.
Open a text editor, and enter:
mlp_file = mlp.txt prefix = mlp [create] architecture = 2 architecture = 10 architecture = 1
Save this file as param_create.txt.
The first two lines of this file (mlp_file and prefix) are always required for the NeuralNet executable. mlp_file specifies the name of the file used to store the neural network. prefix specifies the prefix for output.
The [create] block sets that the following variables are to specify the creation of the network.
The architecture variables specify the architecture of the network. In this tutorial, two input parameters ($x$ and $y$), a single hidden layer with $10$ nodes, and one output [$f(x,y)$] are used.
Open a terminal, and type:
$ mlp param_create.txt
A neural network will be created and stored in the file specified by mlp_file (mlp.txt). Note that this step will be fast.
Training the neural network
Now that we have created a neural network (stored in mlp.txt), the next step is to train it.
Open a text editor, and enter:
mlp_file = mlp.txt prefix = mlp [train] method = 1 X_in_file = train_in.dat X_out_file = train_out.dat
Save this file as param_train.txt.
The [train] block sets that the following variables are to specify the training of the network.
The method variable specifies the method used for training. In this tutorial, 1 specifies QRprop.
The X_in_file and X_out_file variables specify the name of the files used for training. In this tutorial, train_in.dat and train_out.dat, respectively.
Open a terminal, and type:
$ mlp param_train.txt
The neural network (stored in mlp.txt) will be trained, and (re-)stored in the file mlp.txt. Note that this will take a couple of seconds.
As part of the training, the following additional files will be created:
-
training_errors.dat -
fit_valid_err.dat
These files are discussed in detail (and additional details on training, in general) in the tutorial on neural network training.
Testing the neural network
Now that we have created and trained a neural network (stored in mlp.txt), it can be tested (or used) on unseen data.
Open a text editor, and enter:
mlp_file = mlp.txt prefix = mlp [test] X_in_file = test_in.dat X_out_file = test_out.dat
Save this file as param_test.txt.
The [test] block sets that the following variables are to specify the testing of the network.
The X_in_file and X_out_file variables are analogous to those used for training.
Open a terminal, and type:
$ mlp param_test.txt
The neural network (stored in mlp.txt) will be tested. Note that this step will be relatively fast.
The following file will be created:
-
mlp.test.out.dat
The data in this file (predictions by the network) can be compared to test_out.dat. Note that the first column provides a default label for each data point.
Further investigation
Related tutorials
Tutorial: Neural network training (advanced)
