Neural network training

The following training methods are implemented in stats++:

• Backpropagation
• QRprop
• Conjugate gradient
• Scaled conjugate gradient

stats++ (header-only)

Training a neural network is handled in stats++ though the NeuralNet class, declared in statsxx/machine_learning/NeuralNet.hpp. The train() subroutine is used for this purpose:

double NeuralNet::train(
                        const int      method,
                        //------
                        const int      nepoch_min,
                        const int      nepoch_max,
                        // -----
                        const double   lr,
                        const double   lr_min,
                        const double   lr_max,
                        // -----
                        const double   momentum,
                        // -----
                        const double   weight_penalty,
                        // -----
                        const double   qrprop_u,
                        const double   qrprop_d,
                        // -----
                        const double   scg_lambda,
                        const double   scg_sigma,
                        const double   scg_convg_iterfrac,
                        const double   scg_rk_tol,
                        // -----
                        const DataSet &TS_tr,
                        const DataSet &TS_val,
                        const DataSet &TS_gen,
                        // -----
                        const int      npts_per_batch,
                        // -----
                        const bool     silent
                        );

Example code

// STL
#include <vector>                                 // std::vector<>
 
// stats++
#include "statsxx/machine_learning/NeuralNet.hpp" // NeuralNet
 
 
int main(int argc, char* argv[])
{
    NeuralNet nn;
 
    // create the neural network
 
 
 
    nn.creat_MLP(
                 ni
                 no,
                 nl,
                 nhn,
                 fully_connect,
                 recurrent,
                 af_type,
                 is_classif
                 );
 
    return 0;
}

stats++ (executable)

Training is handled by the [train] block in the input file to the NeuralNet executable:

[train]
 
method = 0
 
nepoch_max = 1000
 
lr     = 0.01
 
momentum = 0.1
 
weight_penalty = 0.0001
 
X_in_file      = train_X_in.dat 
X_out_file     = train_X_out.dat
 
npts_per_batch = -1