Pseudoinverse Network


The pseudoinverse rule is a simple supervised learning algorithm in a Hopfield-like neural network. This associative memory system is based on learning rule inspired from Adaline:

dWik= n . [ Xiu - Sumj(Wij Xju) ] . Xku

where Wik is the weight between neuron i and neuron k, n is the learning rate,  Xiu is the target pattern, Sumj(Wij Xju) is the output of a linear neuron and Xku is the actual output.  The iterative learning rule converges to a connectivity matrix which is known as the pseudoinverse.


The applet was written by Olivier Michel (adapted from Matt Hill --


Use the mouse to enter a pattern by clicking squares inside the rectangle "on" or "off". Then, have the network store your pattern by pressing "Memorize". After storing some patterns (typically two), try entering a new pattern which you will use as a test pattern. Do not impose this new pattern, but use it as an initial state of the network. Press "Test" repeatedly to watch the network settle into a previously imposed state.

Applet: 10x10 nodes

  1. What is the theoretical maximum number of random classes the network is able to memorize?
  2. What is the experimental maximum number of random classes the network is able to memorize?
  3. Do the experimental results agree with the theory?
  4. Store a finite number of random patterns, e.g., 8. How many wrong pixels can the network tolerate in the initial state so that it still settles into correct pattern?
  5. Try to store characters as the relevant patterns. How good is the retrieval? What is the reason?