OCR with Multi-Layer Perceptrons

Introduction

This applet illustrates some pattern recognition capabilities of the multi-layer perceptrons with an Optical Character Recognition (OCR) application. A database is used to learn handwritten digits ranging from 0 to 9. Then, the user can draw digits himself, and try to make the perceptron recognize them.

Credits

The original applet was written by Olivier Michel.

Instructions

A database of bitmaps is used to learn to recognize handwritten figures between 0 and 9. Each bitmap is a 15x20 binary map representing an handwritten digit. 100 samples (10 for each digit) were drawn by hand, with a computer mouse. The input layer of the neural network fcontains 80 neurons representing a 8x10 matrix corresponding to a subsampling of the actual sample matrix. For each input unit, a preprocessing module computes a value equal to the average of the four corresponding input pixels of the input matrix. The output layer is made up of 10 units, each one indicating a symbolic value for the digit recognized.

Applet

Questions

1. Learning: How many iterations are necessary to make the error reach 0 on the training set ? Which parameters (learning rate, momentum, number of hidden units) provide an optimal compromise between performance and time of convergence ?
2. Learning: Try to find optimal parameters for a neural network witout hidden layer (set the H1 value to 0, or simply delete it). How good is the performance compared to that of larger  networks ? What is your conclusion regarding the separability of the classes in the learning set ?
3. Testing: Try to draw digits with the mouse and test the answer of the neural network. Do you think there are good generalization properties ? How can you explain this ?
4. Improving the learning: In order to get better results when testing, how can you change the learning so that generalization is better ? Does early stopping provides better results ?