bagsvm |
Application for training and testing a bagged ensemble of SVMs
The application bagsvm implements a bagged ensemble of SVMs for classification problems. The classes can be coded according to the classical One-Per-Class (OPC) coding scheme or Error Correcting Output Coding (ECOC) scheme. Program option allow to build arbitrary MLP with one, two or no hidden layers, using a number of hidden units limited only by the available memory. The user can also select different flavours of back propagation algorithms and different learning algorithm parameters (e.g. learning rates, momentum). If an option is not set by the user, a default one is used.
Usage: bagsvm trainfile [options]
-test string test file name -type unsigned SVM base learner type: 0: linear (default) 1: polynomial 2: RBF (gaussian) 3: sigmoid (MLP-like) -nb unsigned number of base learners -c float regularization parameter of the base SVM (def. 1) -deg unsigned degree of the polynomial SVM (s a*b + r)^d (def. 2) -s float s parameter of the poly SVM or of the sigmoid SVM (def. 1) -r float s parameter of the poly SVM or of the sigmoid SVM (def. 1) -sigma float sigma parameter of the gaussian SVM: exp(-(a-b)^2/sigma) -sbmodel string save the learned base models onto sbmodel1..sbmodeln -sbtrain unsigned save training errors of the base learners: 0 (def) no, 1 yes -sbtest unsigned save testing errors of the base learners: 0 (def) no, 1 yes -setrain unsigned save ensemble train errors at each step: 0 (def) no, 1 yes -setest unsigned save ensemble test errors at each step: 0 (def) no, 1 yes -pred string save classes predicted by the ensemble with respect to the test set onto a file. -predtrain string save classes predicted by the ensemble with respect to the train set onto a file. -sit unsigned if 1 save save classes predicted by the ensemble onto a file at each iteration (def. 0) -train unsigned if 0 (def.) testing is not performed on train set, if 1 yes -p integer 1 = printing a point at each iteration; 0 = noprint (def.) Examples:
For all the examples we refer to a training set named foo.train and a testing set named foo.test with 5-dimensional examples and 7 classes. These are only few examples. See above Usage for a complete reference of the available parameters. Note that you can place the parameters in any order, and most of the parameters are not mandatory. More precisely, you can also supply no parameters, but remember that the number of inputs of the MLP must agree with the dimension of the input examples and you must also specify the correct number of classes.
- Training and testing a standard MLP with one hidden layer:
nn foo.train -test foo.test -d 5 -nc 7
Note that the order of the the supplied parameters is not important.
Note also that the number of inputs of the MLP must agree with the dimension of the input examples and you must also specify the correct number of classes. The selected MLP has one hidden layer (default) and uses a backpropagation learning algorithm with fixed learning rate.
If you want to select a learning rate equal, for instance, to 0.1:
nn foo.train -test foo.test -d 5 -nc 7 -rate 0.1
If you want to select 20 neurons for the hidden layer:
nn foo.train -test foo.test -d 5 -nc 7 -h2 20
- Training and testing a standard MLP with two hidden layer:
nn foo.train -test foo.test -d 5 -nc 7 -nl 3
If you want to select a learning rate equal, for instance, to 0.04:
nn foo.train -test foo.test -d 5 -nc 7 -rate 0.04 -nl 3
If you want to select 20 neurons for the fist hidden layer and 12 for the second hidden layer:
nn foo.train -test foo.test -d 5 -nc 7 -nl 3 -h1 20 -h2 12
- Training a standard MLP changing learning algorithms:
Using a backpropagation with a momentun term:
nn foo.train -test foo.test -d 5 -nc 7 -alg md
If you want to select a learning rate equal, for instance, to 0.1 and a momentum rate equal to 0.4:
nn foo.train -test foo.test -d 5 -nc 7 -alg md -rate 0.1 -mom 0.4
If you want to select a bold driver learning algorithm with increment rate equal to 1.04 and decremet rate equal to 0.5:
nn foo.train -test foo.test -d 5 -nc 7 -alg bold -incbold 1.04 -decbold 0.4
- Training and testing an ECOC MLP with one hidden layer:
nn foo.train -test foo.test -d 5 -nc 7 -type MLP-ECOC
- Training and testing a standard MLP and saving it onto a file:
nn foo.train -test foo.test -d 5 -nc 7 -s mymlp
The file mymlp.net stores the parameters and weights of the MLP. You can reload its weights and parameters and continue the training in a second, possibly different stage: nn foo.train -test foo.test -d 5 -nc 7 -alg boldmom -maxerr 0.02 -w mymlp
The MLP is initialized with the paramenters and weights stored in the file mymlp.net, and the the training starts using a bold driver with momentum algorithm and the learning algorithms ends when the normalized RMS errors drops down to 0.02 (or the maximum allowed number of iterations is reached).- Saving the outputs or the history error:
For saving the computed outputs of the net onto a file named output:
nn foo.train -test foo.test -d 5 -nc 7 -out output
For saving the history of the RMS errors during the learning epochs onto a file named history:
nn foo.train -test foo.test -d 5 -nc 7 -serr history
- A more complicated example:
We want to train and test an MLP with one hidden layer and 35 hidden neurons using a backpropagation algorithm with an exponential decrement of the learning rate during the epochs, with an initial rate of 0.2, initializing it with the weights stored in the file startweights. We want also that the learning ends when the normalized RMS errors goes below 0.02 or we have reached 5000 iterations; then we want that the resulting MLP is stored in the file finishweights and the outputs realtive to the test set is stored in the file output, while the history errors during the epochs stored in the file history.
In order to obtain this result you must type:
nn foo.train -test foo.test -d 5 -nc 7 -h2 35 -alg gd_de -rate 0.2 -maxerr 0.05 -maxit 5000 -w startweights -s finishweights -out output -serr history
Output:
Output of the application
Format of the file storing the net
File storing the outputs
Input data file
Alphabetic index Hierarchy of classes