a set of C++ library classes
for neural networks development



class OneLayer

Simple perceptron - HTML documentation under construction.

Inheritance:


Public Fields

[more]static const double SIG_OFF
[more]static const double SIG_TRESHOLD
[more]static const double TANH_ON
[more]static const double TANH_OFF
[more]static const double TANH_TRESHOLD
[more]static const double ID_ON
[more]static const double ID_OFF
[more]static const double ID_TRESHOLD

Public Methods

[more] OneLayer ()
Default constructor
it does nothing.
[more] OneLayer (unsigned nout, unsigned nin, act_func fout=sigmoid)
Constructor
It builds a one layer perceptron with nout output units, nin input units and fout activation function.
[more] OneLayer (unsigned dummy)
Constructor
It builds only the codex matrix and initiailizes the queue of historical errors
[more]virtual ~OneLayer ()
Destructor
[more]virtual void init_weights_norm (int s = 0)
initializes all values according to a noraml distribution centerd in 0 with unitary standard deviation
[more]virtual void init_weights_const (double x)
initializes all values to x
[more]virtual void read_weights (char* namefile)
It reads a matrix of weights
The matrix is read from the file namefile.
[more]virtual void print_weights (void)
It prints on stdout the weight matrix
[more]virtual vect& compute (vect& vinput)
It computes the output vector of the net
[more]virtual unsigned predictclassWTA (vect& vinput)
It outputs the predicted class (WTA criterion)
Given an input vector, it predict a class using the winner-takes-all criterion.
[more]virtual unsigned predictclass (vect& vinput)
It outputs the predicted class (min.
[more]virtual void backprop (vect& t, vect& x)
It computes the error gradient of the net.
[more]virtual void print_info_grad (void)
It outputs the delta values, ie.
[more]void print_C (void)
It prints on stdout the decomposition matrix C
[more]virtual void save_weights (char* namefile)
It saves onto a file the weights and the paramenters of the net
It saves onto the file namefilenet the parameters of the net and its weights
File format:
"Version: " version number (unsigned) (actually always 1)
namefile (string)
"Number_layers: " Layers number (unsigned)
"Dim_output: " Output layer size (unsigned) "Actfunction: " Number of the activation function (unsigned)
"Dim_hidden1: " I hidden layer size (unsigned) "Actfunction: " Number of the activation function (unsigned) present if number layers > 1
"Dim_hidden2: " II hidden layer size (unsigned) "Actfunction: " Number of the activation function (unsigned) present if number layers > 2
"Dim_input: " Input size (unsigned)
"Weight matrix first layer :"
Weight matrix output layer (double)
"Bias vector :"
Bias vector output layer (double)
"Weight matrix first hidden layer :" present if number layers > 1
Matrix (matrix of double) present if number layers > 1
"Bias vector :" present if number layers > 1
Vector (vector of double) present if number layers > 1
"Weight matrix second hidden layer :" present if number layers > 2
Matrix (matrix of double) present if number layers > 2
"Bias vector :" present if number layers > 2
Vector (vector of double) present if number layers > 2
"END" File termination string
NOTE : the activation function can be: SIGMOID = 1, TANHYP = 2, IDENTITY = 3.
[more]inline queue<float> & read_qerror ()
It returns the queue of the historical errors along the epochs.
[more]void save_errors (char* file)
[more]virtual stop_cond Learn_by_pattern (TrainingSet& t_set, Learning& learn, unsigned& iter, double& error, double errtol = ERR_TOL, unsigned maxits = MAXITS, bool printout = true)
[more]virtual stop_cond Learn_by_epoch (TrainingSet& t_set, Learning& learn, unsigned& iter, double& error, double errtol = ERR_TOL, unsigned maxits = MAXITS, bool printout = true)
[more]matrix test (TrainingSet& tset)
Testing the neural net
It performs the testing of the net on the testing set tset.
[more]matrix test (TrainingSet& tset, deque<unsigned>& outputs)
Testing the neural net
It performs the testing of the net on the testing set tset.
[more]virtual void save_out (TrainingSet& tset, char* file)
[more]void save_out (TrainingSet& tset, ofstream& f)
It saves onto a stream the outputs of the net
File format:
y1 y2 .
[more]void save_predicted (deque<unsigned>& outputs, char* file)
Saves onto a file the classes predicted by the net
The labels are stored one for each line.
[more]void save_decomp_matrix (ofstream& f)
[more]int CrossValidate (char* name, unsigned nfold, unsigned num_attr, char* filerr, LearnType& learninfo, char* savecvfile = "", char* readcvfile = "", char* outfile = "", int saveout = 0, int seed=0, bool printout=true)
[more]virtual int CrossValidateTrialWeights (char* name, unsigned nfold, unsigned num_attr, char* filerr, LearnType& learninfo, char* outfile = "", int saveout = 0, int seed=0, int times=3, bool printout=true)
[more]void SaveCV (char* name, vector<unsigned>& iter, vect& abserr, vect& percerr, unsigned ntrain, matrix& conf, clock_t start, clock_t end)
[more]void print_resultCV (vector<unsigned>& iter, vect& abserr, vect& percerr, unsigned ntrain, clock_t start, clock_t end)
[more]void print_errors ()
[more]unsigned Code_act_fun (act_func g)
[more]act_func Decode_act_fun (unsigned fun)
[more]int Read_seed (void)
It returns the the random initialization seed for the weights
[more]double get_perc_errors (void)
Return the testing percentual of errors.
[more]unsigned Read_minHam (void)

Public Members

[more] Activation, deactivation and threshold values with respectively sigmoid, hyperbolic tangent

Protected Fields

[more]static const unsigned ERR_PRECISION
precision by which are saved error in the stream
[more]static const unsigned OUTPUT_PRECISION
precision of the net output in the stream
[more]static const unsigned MAXITS
default value of max.
[more]static const double ERR_TOL
default value of the tolerated RMS value
[more]static const unsigned INTEST_SIZE
width of the heading field of the file for saving weigths
[more]unsigned n_class
Cardinality of the classes
[more]matrix C
Codex matrix one-per-class of dimension n_outputs * n_class
Ith column correspond to the codeword of the class i+1; Matrix values are determined by the activation function used by the net.
[more]LayerNetTrain layer [MAXLAYERS]
Layers of the neurons.
[more]queue<float> Qerror
Queue for recording the errors sequence of the net along the epochs
[more]unsigned num_errors
Number of testing errors
[more]double perc_errors
Percentual of testing errors
[more]int seed
Random initialization seed for the weights
[more]unsigned minHam
Minimum Hamming distance among codewords of the classes

Protected Methods

[more]virtual void Init_C (void)
[more]void analyze (act_func g, double& on, double& off)
[more]double Calc_square_error (vect& t, vect& out)
[more]double calc_denom_RMS (vect& t, unsigned n_train)
[more]double RMS_error (double err, double denom)
[more]void read_weights (istream& f)
[more]virtual void save_weights_heading (char* namefile)
It saves onto a file the general paramenters of the net
It saves the number of layers, size of the layers and type of the activation function
[more]virtual void save_weights_heading (ofstream& f, char* namebuf)
It saves in a stream the general paramenters of the net
It saves the number of layers, size of the layers and type of the activation function
[more]virtual void save_weights_body (char* namefile)
It appends onto a file the weights of the net.
[more]virtual void save_weights_body (ofstream& f)
It appends to a stream the weights of the net.
[more]void save_weights_cv_heading (char* savecvfile, int nfold)
It saves onto a file the general paramenters of the nets of the cross validation
It saves the number of layers, size of the layers and type of the activation function
[more]void save_weights_cv_body (char* namefile, int fold_n)
It appends onto a file the weights of the net of the cross validation.
[more]void do_Sum_deltas (unsigned n_layers, matrix* Sum_Delta_W[], vect* Sum_delta_bias[])
[more]void Update_Sum_Delta (unsigned n_layers, matrix* Sum_Delta_W[], vect* Sum_delta_bias[])
[more]void Zero (unsigned n_layers)
[more]void Zero_Sum_Delta (unsigned n_layers, matrix* Sum_Delta_W[], vect* Sum_delta_bias[])
[more]void Build_Sum_Delta (matrix* Sum_Delta_W[], vect* Sum_delta_bias[])
[more]void Delete_Sum_Delta (matrix* Sum_Delta_W[], vect* Sum_delta_bias[])
[more]void Copy_Sum_Delta_To_Delta (unsigned n_layers, LayerNetTrain layer[], matrix* Sum_Delta_W[], vect* Sum_delta_bias[])
[more]virtual unsigned computeclassWTA (vect& out)
[more]virtual unsigned computeclass (vect& out)


Inherited from Network:

Public Fields

ostatic const char* const NET_EXT
ostatic const char* const NETCV_EXT
ostatic const char* const ERR_EXT
ostatic const char* const OUT_EXT
ostatic const unsigned OUTPUT
ostatic const unsigned HIDDEN1
ostatic const unsigned HIDDEN2
ostatic const char* const CV_SUFFIX

Public Methods

oinline unsigned Nlayers() const
oinline unsigned Ninput() const
oinline unsigned Noutput() const

Protected Fields

ounsigned n_layers
ounsigned n_inputs
ounsigned n_outputs


Documentation

Simple perceptron
It Implements a simple perceptron for classification with learning capabilities
ostatic const unsigned ERR_PRECISION
precision by which are saved error in the stream

ostatic const unsigned OUTPUT_PRECISION
precision of the net output in the stream

ostatic const unsigned MAXITS
default value of max. number of iterations

ostatic const double ERR_TOL
default value of the tolerated RMS value

ostatic const unsigned INTEST_SIZE
width of the heading field of the file for saving weigths

ounsigned n_class
Cardinality of the classes

omatrix C
Codex matrix one-per-class of dimension n_outputs * n_class
Ith column correspond to the codeword of the class i+1; Matrix values are determined by the activation function used by the net.

oLayerNetTrain layer[MAXLAYERS]
Layers of the neurons.
They are filled completely only by the children classes. OneLayer fills only layer[0].

oqueue<float> Qerror
Queue for recording the errors sequence of the net along the epochs

ounsigned num_errors
Number of testing errors

odouble perc_errors
Percentual of testing errors

oint seed
Random initialization seed for the weights

ounsigned minHam
Minimum Hamming distance among codewords of the classes

ovirtual void Init_C(void)

ovoid analyze(act_func g, double& on, double& off)

odouble Calc_square_error(vect& t, vect& out)

odouble calc_denom_RMS(vect& t, unsigned n_train)

odouble RMS_error(double err, double denom)

ovoid read_weights(istream& f)

ovirtual void save_weights_heading(char* namefile)
It saves onto a file the general paramenters of the net
It saves the number of layers, size of the layers and type of the activation function
Parameters:
- namefile name of the file for saving the net. The suffix net. is appended to namefile
See Also:
save_weights

ovirtual void save_weights_heading(ofstream& f, char* namebuf)
It saves in a stream the general paramenters of the net
It saves the number of layers, size of the layers and type of the activation function
Parameters:
f - streamfile for saving the net. Used by save_weights_heading(char* namefile).
- namefile name of the file for saving the net.
See Also:
save_weights

ovirtual void save_weights_body(char* namefile)
It appends onto a file the weights of the net.
Parameters:
- namefile name of the file for saving the net. The suffix net. is appended to namefile
See Also:
save_weights

ovirtual void save_weights_body(ofstream& f)
It appends to a stream the weights of the net.
Parameters:
f - name stream file for saving the net. Used by save_weights_body(char* namefile).
See Also:
save_weights

ovoid save_weights_cv_heading(char* savecvfile, int nfold)
It saves onto a file the general paramenters of the nets of the cross validation
It saves the number of layers, size of the layers and type of the activation function
Parameters:
savecvfile - name of the file for saving the nets. The suffix cvnet. is appended to namefile
nfold - total number of the folds

ovoid save_weights_cv_body(char* namefile, int fold_n)
It appends onto a file the weights of the net of the cross validation.
Parameters:
- namefile name of the file for saving the net. The suffix cvnet. is appended to namefile
fold_n - number of the fold

ovoid do_Sum_deltas(unsigned n_layers, matrix* Sum_Delta_W[], vect* Sum_delta_bias[])

ovoid Update_Sum_Delta(unsigned n_layers, matrix* Sum_Delta_W[], vect* Sum_delta_bias[])

ovoid Zero(unsigned n_layers)

ovoid Zero_Sum_Delta(unsigned n_layers, matrix* Sum_Delta_W[], vect* Sum_delta_bias[])

ovoid Build_Sum_Delta(matrix* Sum_Delta_W[], vect* Sum_delta_bias[])

ovoid Delete_Sum_Delta(matrix* Sum_Delta_W[], vect* Sum_delta_bias[])

ovoid Copy_Sum_Delta_To_Delta(unsigned n_layers, LayerNetTrain layer[], matrix* Sum_Delta_W[], vect* Sum_delta_bias[])

ovirtual unsigned computeclassWTA(vect& out)

ovirtual unsigned computeclass(vect& out)

o Activation, deactivation and threshold values with respectively sigmoid, hyperbolic tangent
Activation, deactivation and threshold values with respectively sigmoid, hyperbolic tangent

ostatic const double SIG_OFF

ostatic const double SIG_TRESHOLD

ostatic const double TANH_ON

ostatic const double TANH_OFF

ostatic const double TANH_TRESHOLD

ostatic const double ID_ON

ostatic const double ID_OFF

ostatic const double ID_TRESHOLD

o OneLayer()
Default constructor
it does nothing.

o OneLayer(unsigned nout, unsigned nin, act_func fout=sigmoid)
Constructor
It builds a one layer perceptron with nout output units, nin input units and fout activation function.
Parameters:
nout - number of output units
nin - number of input units
fout - type of activation function

o OneLayer(unsigned dummy)
Constructor
It builds only the codex matrix and initiailizes the queue of historical errors

ovirtual ~OneLayer()
Destructor

ovirtual void init_weights_norm(int s = 0)
initializes all values according to a noraml distribution centerd in 0 with unitary standard deviation

ovirtual void init_weights_const(double x)
initializes all values to x

ovirtual void read_weights(char* namefile)
It reads a matrix of weights
The matrix is read from the file namefile. The weights are read only if the dimension of matrix agrees with the dimension of the reading net.
Parameters:
- namefile file name of the recorded weights to be read.

ovirtual void print_weights(void)
It prints on stdout the weight matrix

ovirtual vect& compute(vect& vinput)
It computes the output vector of the net
Parameters:
vinput - input vector of the net
Returns:
output vector computed by the net

ovirtual unsigned predictclassWTA(vect& vinput)
It outputs the predicted class (WTA criterion)
Given an input vector, it predict a class using the winner-takes-all criterion.
Parameters:
vinput - input vector of the net
Returns:
predicted class

ovirtual unsigned predictclass(vect& vinput)
It outputs the predicted class (min. Hamming distance criterion).
Given an input vector, it predict a class using the minimum Hamming distance criterion.
Parameters:
vinput - input vector of the net
Returns:
predicted class

ovirtual void backprop(vect& t, vect& x)
It computes the error gradient of the net.
It records the error gradient respect to the weights.
Parameters:
- t target vector
x - input vector

ovirtual void print_info_grad(void)
It outputs the delta values, ie. the error gradient of the net.

ovoid print_C(void)
It prints on stdout the decomposition matrix C

ovirtual void save_weights(char* namefile)
It saves onto a file the weights and the paramenters of the net
It saves onto the file namefilenet the parameters of the net and its weights
File format:
"Version: " version number (unsigned) (actually always 1)
namefile (string)
"Number_layers: " Layers number (unsigned)
"Dim_output: " Output layer size (unsigned) "Actfunction: " Number of the activation function (unsigned)
"Dim_hidden1: " I hidden layer size (unsigned) "Actfunction: " Number of the activation function (unsigned) present if number layers > 1
"Dim_hidden2: " II hidden layer size (unsigned) "Actfunction: " Number of the activation function (unsigned) present if number layers > 2
"Dim_input: " Input size (unsigned)
"Weight matrix first layer :"
Weight matrix output layer (double)
"Bias vector :"
Bias vector output layer (double)
"Weight matrix first hidden layer :" present if number layers > 1
Matrix (matrix of double) present if number layers > 1
"Bias vector :" present if number layers > 1
Vector (vector of double) present if number layers > 1
"Weight matrix second hidden layer :" present if number layers > 2
Matrix (matrix of double) present if number layers > 2
"Bias vector :" present if number layers > 2
Vector (vector of double) present if number layers > 2
"END" File termination string
NOTE : the activation function can be: SIGMOID = 1, TANHYP = 2, IDENTITY = 3.
Parameters:
- namefile name of the file for saving the net. To namefile is appended the suffix .net.

oinline queue<float> & read_qerror()
It returns the queue of the historical errors along the epochs.

ovoid save_errors(char* file)

ovirtual stop_cond Learn_by_pattern(TrainingSet& t_set, Learning& learn, unsigned& iter, double& error, double errtol = ERR_TOL, unsigned maxits = MAXITS, bool printout = true)

ovirtual stop_cond Learn_by_epoch(TrainingSet& t_set, Learning& learn, unsigned& iter, double& error, double errtol = ERR_TOL, unsigned maxits = MAXITS, bool printout = true)

omatrix test(TrainingSet& tset)
Testing the neural net
It performs the testing of the net on the testing set tset. The number of errors is recorded in num_errors and the percentual of errors is recorded in perc_errors.
Parameters:
tset - training set
Returns:
confusion matrix

omatrix test(TrainingSet& tset, deque<unsigned>& outputs)
Testing the neural net
It performs the testing of the net on the testing set tset. The number of errors is recorded in num_errors and the percentual of errors is recorded in perc_errors. The computed classes are pushed back to the queue outputs.
Parameters:
tset - training set
outputs - vector of the output classes computed by the network.
Returns:
confusion matrix

ovirtual void save_out(TrainingSet& tset, char* file)

ovoid save_out(TrainingSet& tset, ofstream& f)
It saves onto a stream the outputs of the net
File format:
y1 y2 . yn computed_class right_class
Where yi (double) are the bits of the codeword coding the class, computed_class (unsigned) is the class compute by the net, right_class (unsigned) is the correct class
Parameters:
tset - training set
f - stream where the outputs are saved

ovoid save_predicted(deque<unsigned>& outputs, char* file)
Saves onto a file the classes predicted by the net
The labels are stored one for each line.
Parameters:
outputs - vector of the output classes computed by the network
file - file of the predicted classes

ovoid save_decomp_matrix(ofstream& f)

oint CrossValidate(char* name, unsigned nfold, unsigned num_attr, char* filerr, LearnType& learninfo, char* savecvfile = "", char* readcvfile = "", char* outfile = "", int saveout = 0, int seed=0, bool printout=true)

ovirtual int CrossValidateTrialWeights(char* name, unsigned nfold, unsigned num_attr, char* filerr, LearnType& learninfo, char* outfile = "", int saveout = 0, int seed=0, int times=3, bool printout=true)

ovoid SaveCV(char* name, vector<unsigned>& iter, vect& abserr, vect& percerr, unsigned ntrain, matrix& conf, clock_t start, clock_t end)

ovoid print_resultCV(vector<unsigned>& iter, vect& abserr, vect& percerr, unsigned ntrain, clock_t start, clock_t end)

ovoid print_errors()

ounsigned Code_act_fun(act_func g)

oact_func Decode_act_fun(unsigned fun)

oint Read_seed(void)
It returns the the random initialization seed for the weights

odouble get_perc_errors(void)
Return the testing percentual of errors.

ounsigned Read_minHam(void)


Direct child classes:
TwoLayer
OneLayerECOC
Dico1

Alphabetic index HTML hierarchy of classes or Java


Last Updated February 2001
For comments and suggestions mail to Giorgio Valentini