Colorization Chapter

The portrait experiment

Training set (53 images)

These images are given as examples, in order to know what a color image looks like.

Test images (5 images)

The following images are converted into grey-scale values, and will be used later as test images.


Note that there can be issues concerning the true colors of a painting. For example compare :

Results

I modified the original training set, removing two images and down-sampling the biggest images, so that it fits the RAM during learning and prediction without requiring a program structure change; I updated the training set above accordingly.
The following result is based on all the training images above. Same parameters for the all test images. Pixels on a 3x3 grid only were colored.
For each test image are displayed:

• image to be colored
• result
• color chosen, without grey level
• edge prediction (black = possible strong color variation, white = no possible color variation)
• most probable color at the pixel level

• the edge (= color variation) prediction is quite good, much better than in previous experiments with very-small-sized training sets, and better than in the experiment with the 20-houses set. This indicates that the kernel parameters (number of nearest neighbors and standard deviation of the Gaussian) are correct.
• Locally, green is more often predicted than expected, even in the middle of faces.
• Blue artefacts disappear with the spatial regularization.
• One cannot reasonably expect parts other than the face to be colorized correctly, given the training set.
• Adding a new test image takes about half an hour (because of the search, for each pixel, for the 1000 nearest neighbors in the learning set); but a re-colorization with a different spatial coherency parameter takes only a few seconds.
• The spatial coherency parameter is the most important one, but, apparently, the one for smoothing the color probabilities in the color space, as well as the precise way color variation predictions are handle, are important too.