A Computer Vision Based Move Detection and Interpretation System for Chess Game

In this study, a real time computer vision based tracking system that enables to detect and interpret moves in a chess game is developed. With this system, it has planned to support the decision making processes of referees in chess matches. For the mechanical part of the system, a setup is established, which is positioned over the chess board and includes a RGB web camera and a laptop computer. For the software part, all the implementations are carried out with Matlab programming environment without using any third party chess engine. The proposed system performs move detection and interpretation processes individually. In move detection part, snapshots taken by the camera are processed one by one. For each snapshot, color and position of each chess piece are identified. This stage also enables the preprocessing steps such as geometric rectification of chess board and image cropping. In move interpretation part, move turns and the validation of the moves are identified. Illegal moves are notified by the system. Besides, the results are visualized in a 2D environment and logged. The system was tested with three game scenarios where each game is composed of 100 identical moves but played under different illumination conditions. The accuracies of move detection and interpretation are computed to be 96%, 98% and 95% for each scenario, respectively. There are some limitations for the system to work with high accuracy; the colors of the chess pieces are not very close to each other, the ambient light remains constant during the game and the camera angle and position are kept constant