Multi-Layer Perceptron
and C language  |
Introduction
These pages present a very simple yet powerful C program: Bpsim. It illustrates
the behavior of a simple multi-layer perceptron on a very orignal example
based upon the tabula rasa Little Red Riding Hood example.
Bpsim provides an implementation of a neural network containing a single
hidden layer which uses the generalized backpropagation delta rule for
learning. A simple user interface is supplied for experimenting with a
neural network solution to the Little Red Riding Hood example. This program
might be a good starting point if you need to develop a multi-layer perceptron
in C.
Features
In addition, bpsim contains some useful building blocks for further experimentation
with single layer neural networks. The data structure which describes the
general processing unit allows one to easily investigate different activation
(output) and/or error functions. The utility function create_link
can be used to create links between any two units by supplying your own
create_in_out_links function. The flexibility of creating
units and links to your specifications allows one to modify the code to
tune the network architecture to problems of interest.
There are some parameters that perhaps need some explanation. You will
notice that the target values are either 0.1 or 0.9 (corresponding to the
binary values 0 or 1). With the sigmoidal function used in out_f
the weights become very large if 0 and 1 are used as targets. The ON_TOLERANCE
value is used as a criteria for an output value to be considered "on",
i.e., close enough to the target of 0.9 to be considered 1. The learning_rate
and momentum variables may be changed to vary the rate
of learning, however, in general they each should be less than 1.0.
Credits
The original applet was written by Josiah C. Hoskins (see source file for
more info).
Instructions
Download this simple C program: bpsim.c. Compile
it using your favorit C compiler, for example with gcc:
gcc -o bpsim bpsim.c -lm
Then run, bpsim from the shell.
Questions
-
Play with the program and try to make it able to recognize all characters.
-
What are the features of the Wolf, Grandma and Woodcutter ?
-
What are the behaviors associated with the Wolf, Grandma and Woodcutter
?
-
Is it a linearly separable problem ?
-
What is the structure of the network (number of inputs / ouputs / hidden
units / layers) ?
-
Try to improve the performance by changing either the learning_rate, momentum,
the number of hidden layer, the number of units in hidden layers. Can you
get better results ? Why ?