My solution is the following:
First read the input and count how many pieces are placed in the wrong boxes(i.e. there color is not the same as the color of the box) lets call that number cnt. You must move these pieces at least once.
Then I build a graph for which the boxes are the nodes and there is an edge between two boxes a and b iff there is a piece of color b in the box of color a. After building the graph one should count the number of connected components in it(lets call it count).
Now the answer to our problem is simply cnt + max(count-1,0)
as one should do at least max(count-1,0) moving of his hand to another box and it is always possible to solve the problem in exactly that many moves of the hand without moving a piece.

LSBG, thank you so much, it's the better solution I've ever seen)))