Multi-Layer Perceptron and C language 

Introduction

These pages present a very simple yet powerful C program: Bpsim. It illustrates the behavior of a simple multi-layer perceptron on a very orignal example based upon the tabula rasa Little Red Riding Hood example.

Bpsim provides an implementation of a neural network containing a single hidden layer which uses the generalized backpropagation delta rule for learning. A simple user interface is supplied for experimenting with a neural network solution to the Little Red Riding Hood example. This program might be a good starting point if you need to develop a multi-layer perceptron in C.

Features

In addition, bpsim contains some useful building blocks for further experimentation with single layer neural networks. The data structure which describes the general processing unit allows one to easily investigate different activation (output) and/or error functions. The utility function create_link can be used to create links between any two units by supplying your own create_in_out_links function. The flexibility of creating units and links to your specifications allows one to modify the code to tune the network architecture to problems of interest.

There are some parameters that perhaps need some explanation. You will notice that the target values are either 0.1 or 0.9 (corresponding to the binary values 0 or 1). With the sigmoidal function used in out_f the weights become very large if 0 and 1 are used as targets. The ON_TOLERANCE value is used as a criteria for an output value to be considered "on", i.e., close enough to the target of 0.9 to be considered 1. The learning_rate and momentum variables may be changed to vary the rate of learning, however, in general they each should be less than 1.0.

Credits

The original applet was written by Josiah C. Hoskins (see source file for more info).

Instructions

Download this simple C program: bpsim.c. Compile it using your favorit C compiler, for example with gcc:

gcc -o bpsim bpsim.c -lm

Then run, bpsim from the shell.

Questions

  1. Play with the program and try to make it able to recognize all characters.
  2. What are the features of the Wolf, Grandma and Woodcutter ?
  3. What are the behaviors associated with the Wolf, Grandma and Woodcutter ?
  4. Is it a linearly separable problem ?
  5. What is the structure of the network (number of inputs / ouputs / hidden units / layers) ?
  6. Try to improve the performance by changing either the learning_rate, momentum, the number of hidden layer, the number of units in hidden layers. Can you get better results ? Why ?