Pseudoinverse Network

Introduction

The pseudoinverse rule is a simple supervised learning algorithm in a Hopfield-like neural network. This associative memory system is based on learning rule inspired from Adaline:

where W_ik is the weight between neuron i and neuron k, n is the learning rate,  X_i^u is the target pattern, Sum_j(W_ij X_j^u) is the output of a linear neuron and X_k^u is the actual output.  The iterative learning rule converges to a connectivity matrix which is known as the pseudoinverse.

Credits

The applet was written by Olivier Michel (adapted from Matt Hill -- mlh1@cornell.edu).

Instructions

Use the mouse to enter a pattern by clicking squares inside the rectangle "on" or "off". Then, have the network store your pattern by pressing "Memorize". After storing some patterns (typically two), try entering a new pattern which you will use as a test pattern. Do not impose this new pattern, but use it as an initial state of the network. Press "Test" repeatedly to watch the network settle into a previously imposed state.

Applet: 10x10 nodes

1. What is the theoretical maximum number of random classes the network is able to memorize?
2. What is the experimental maximum number of random classes the network is able to memorize?
3. Do the experimental results agree with the theory?
4. Store a finite number of random patterns, e.g., 8. How many wrong pixels can the network tolerate in the initial state so that it still settles into correct pattern?
5. Try to store characters as the relevant patterns. How good is the retrieval? What is the reason?