org.strbio.util
Class NeuralNet

java.lang.Object
  extended by org.strbio.util.NeuralNet

public class NeuralNet
extends java.lang.Object

Class encapsulating a neural network.

Provides neural network support for SD (BP) and conjugate gradient (SCG) training methods. SCG is described in Moller, Neural Networks, Vol. 6, pp. 525-533.

The network object relies on an external object to show all the patterns to the network. The object needs to implement the NeuralNet.Trainable interface, which defines the method:

public void presentPatterns(NeuralNet n);

This method should not modify any weights itself; just present all the patterns and call prop() and backprop() for each.

All arrays presented to the user start with zero. Internally, lots of things start with one because of the extra bias unit in each layer. This representation should be invisible to the user to avoid confusion.

  Version 1.11, 5/20/98 - added getDimensions non-static method.
  Version 1.1, 5/18/98 - added saveBinary, more load() methods,
    getDimensions.  load(filename) now autodetects binary/ascii files.
  Version 1.0, 5/13/98 - adapted from network.cpp from 4/3/96.
    Took out all references to shared memory and OS-specific optimizations.
  

Version:
1.11, 5/20/98
Author:
JMC

Nested Class Summary
static interface NeuralNet.Trainable
          This is the interface you have to implement for objects that are to be used used in training neural nets.
 
Field Summary
 double alpha
          user-dependent variables for SD BP method: momentum
 double epsilon
          user-dependent variables for SD BP method: step size
 double low_error
          stopping parameter for scg and sd.
 double max_error
          stopping parameter for scg and sd.
 
Constructor Summary
NeuralNet()
          Make an 'empty' neural network.. must be loaded out of file to be useful.
NeuralNet(java.io.BufferedReader infile)
          Initialize a network by reading it out of an open BufferedReader.
NeuralNet(java.io.DataInputStream infile)
          Initialize a network by reading it out of an open DataInputStream.
NeuralNet(int[] layersize)
          Initialize a new network with a given topology and a new random seed
NeuralNet(int[] layersize, java.util.Random random)
          Initialize a new network with a given topology and a given random number generator.
NeuralNet(java.lang.String filename)
          Initialize a network by reading it out of a file with a given name.
 
Method Summary
 void back(double[] d)
          propagate desired outputs backward through the net, computing interesting variables.
 double bias(int layer, int i)
          return the bias of a given neuron.
 double error()
          get last error.
 int[] getDimensions()
          Get dimensions of a loaded network.
static int[] getDimensions(java.io.BufferedReader infile)
          Get dimensions of a network in a file, then reset to the point before anything (except comments) was read in.
static int[] getDimensions(java.io.DataInputStream infile)
          Get dimensions of a network in a binary file, then reset to the point before anything was read in.
static int[] getDimensions(java.lang.String filename)
          Get dimensions of a network in a file with a given name.
 double input(int i)
          return a network input
 int layers()
          return number of layers.
 int layerSize(int i)
          return size of a given layer.
 void load(java.io.BufferedReader infile)
          load from an open text file; check network dimensions.
 void load(java.io.DataInputStream infile)
          load from an open binary file; check network dimensions.
 void load(java.lang.String filename)
          load from a text file; check network dimensions.
 double output(int i)
          return a network output
 void prop()
          propagate inputs forward through the network.
 void save(PrintfStream outfile)
          save weights in an open text file, for portability (to C) and readability.
 void save(java.lang.String filename)
          save weights in text file, for portability (to C) and readability.
 void saveBinary(java.io.DataOutputStream outfile)
          save weights in an open binary file, for speed.
 void saveBinary(java.lang.String filename)
          save weights in binary file, for speed.
 int scg(int n, Printf outfile)
          Train using the scaled conjugate gradient algorithm for n cycles, from moller (1993).
 int sd(int n, Printf outfile)
          Train using steepest descent for n cycles, unless stopped by low_error or max_error.
 void setBias(int layer, int i, double x)
          set the bias of a given neuron.
 void setInput(int i, double x)
          set a network input
 void setOutput(int i, double x)
          set a network output
 void setPatternPresenter(NeuralNet.Trainable p)
          set the pattern presentation object.
 void setWeight(int layer, int n_from, int n_to, double x)
          Set a given weight.
 void sparseBack(double[] d, int[] mark, int n)
          fast version of back for sparse inputs
 void sparseBoolBack(double[] d, int[] mark, int n)
          fast version of back for sparse inputs.
 void sparseBoolProp(int[] mark, int n)
          prop optimized for lots of zero inputs.
 void sparseDeltaProp(int[] added, int na, int[] removed, int nr)
          prop, but with a vector of inputs that have been added since the last call, and a vector that have been removed.
 void sparseProp(int[] mark, int n)
          prop, optimized for lots of zero inputs.
 double weight(int layer, int n_from, int n_to)
          Return a given weight.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

alpha

public double alpha
user-dependent variables for SD BP method: momentum


epsilon

public double epsilon
user-dependent variables for SD BP method: step size


low_error

public double low_error
stopping parameter for scg and sd.


max_error

public double max_error
stopping parameter for scg and sd.

Constructor Detail

NeuralNet

public NeuralNet(int[] layersize,
                 java.util.Random random)
Initialize a new network with a given topology and a given random number generator.


NeuralNet

public NeuralNet(int[] layersize)
Initialize a new network with a given topology and a new random seed


NeuralNet

public NeuralNet(java.io.BufferedReader infile)
          throws java.io.IOException
Initialize a network by reading it out of an open BufferedReader.

Throws:
java.io.IOException

NeuralNet

public NeuralNet(java.io.DataInputStream infile)
          throws java.io.IOException
Initialize a network by reading it out of an open DataInputStream.

Throws:
java.io.IOException

NeuralNet

public NeuralNet(java.lang.String filename)
          throws java.io.IOException
Initialize a network by reading it out of a file with a given name.

Throws:
java.io.IOException

NeuralNet

public NeuralNet()
Make an 'empty' neural network.. must be loaded out of file to be useful.

Method Detail

layers

public final int layers()
return number of layers.


error

public final double error()
get last error.


layerSize

public final int layerSize(int i)
return size of a given layer.


weight

public final double weight(int layer,
                           int n_from,
                           int n_to)
Return a given weight.


setWeight

public final void setWeight(int layer,
                            int n_from,
                            int n_to,
                            double x)
Set a given weight.


bias

public final double bias(int layer,
                         int i)
return the bias of a given neuron.


setBias

public final void setBias(int layer,
                          int i,
                          double x)
set the bias of a given neuron.


input

public final double input(int i)
return a network input


setInput

public final void setInput(int i,
                           double x)
set a network input


output

public final double output(int i)
return a network output


setOutput

public final void setOutput(int i,
                            double x)
set a network output


prop

public final void prop()
propagate inputs forward through the network. function 1/(1+exp(-input)) is hardcoded for speed in back propagation.


sparseProp

public final void sparseProp(int[] mark,
                             int n)
prop, optimized for lots of zero inputs.


sparseDeltaProp

public final void sparseDeltaProp(int[] added,
                                  int na,
                                  int[] removed,
                                  int nr)
prop, but with a vector of inputs that have been added since the last call, and a vector that have been removed.


sparseBoolProp

public final void sparseBoolProp(int[] mark,
                                 int n)
prop optimized for lots of zero inputs. Assumes any inputs are binary.


back

public final void back(double[] d)
propagate desired outputs backward through the net, computing interesting variables. you have to have forward propagated previously, to set up inputs and outputs in the net again, the activation function is hard coded for speed.


sparseBack

public final void sparseBack(double[] d,
                             int[] mark,
                             int n)
fast version of back for sparse inputs


sparseBoolBack

public final void sparseBoolBack(double[] d,
                                 int[] mark,
                                 int n)
fast version of back for sparse inputs. Assumes any inputs are binary.


setPatternPresenter

public final void setPatternPresenter(NeuralNet.Trainable p)
set the pattern presentation object.


sd

public final int sd(int n,
                    Printf outfile)
Train using steepest descent for n cycles, unless stopped by low_error or max_error. If n==0, keep going until stopped. Returns the number of cycles trained. Reports progress to outfile, if present.


scg

public final int scg(int n,
                     Printf outfile)
Train using the scaled conjugate gradient algorithm for n cycles, from moller (1993). If n==0, keep going until minimum is found. Reports progress to outfile, if present.


save

public final void save(PrintfStream outfile)
                throws java.io.IOException
save weights in an open text file, for portability (to C) and readability.

Throws:
java.io.IOException

saveBinary

public final void saveBinary(java.io.DataOutputStream outfile)
                      throws java.io.IOException
save weights in an open binary file, for speed.

Throws:
java.io.IOException

save

public final void save(java.lang.String filename)
                throws java.io.IOException
save weights in text file, for portability (to C) and readability.

Throws:
java.io.IOException

saveBinary

public final void saveBinary(java.lang.String filename)
                      throws java.io.IOException
save weights in binary file, for speed.

Throws:
java.io.IOException

load

public final void load(java.io.BufferedReader infile)
                throws java.io.IOException
load from an open text file; check network dimensions. If network dimensions are zero, creates a new network. You can comment the file with lines beginning with a semicolon.

Throws:
java.io.IOException

load

public final void load(java.io.DataInputStream infile)
                throws java.io.IOException
load from an open binary file; check network dimensions. If network dimensions are zero, creates a new network.

Throws:
java.io.IOException

getDimensions

public final int[] getDimensions()
Get dimensions of a loaded network.


getDimensions

public static final int[] getDimensions(java.io.BufferedReader infile)
                                 throws java.io.IOException
Get dimensions of a network in a file, then reset to the point before anything (except comments) was read in.

Throws:
java.io.IOException

getDimensions

public static final int[] getDimensions(java.io.DataInputStream infile)
                                 throws java.io.IOException
Get dimensions of a network in a binary file, then reset to the point before anything was read in.

Throws:
java.io.IOException

getDimensions

public static final int[] getDimensions(java.lang.String filename)
                                 throws java.io.IOException
Get dimensions of a network in a file with a given name. File type (text/binary) is autodetected.

Throws:
java.io.IOException

load

public final void load(java.lang.String filename)
                throws java.io.IOException
load from a text file; check network dimensions. If network dimensions are zero, creates a new network. File type (text/binary) is autodetected.

Throws:
java.io.IOException