In order to ensure the accuracy of checkerboard chess position detection, the location and recognition of the chessman in the chess visual system. In this paper, we first use the gradient method to obtain the board area, and the proportional method obtains the chess coordinate. Then use the Hough transformation to determine the existence of the chessmen and obtain chessman center coordinates in a subdomain at the chess position. ‘A’ component of Lab segmentation judges the color of the chessmen. Polar coordinates projection and fast Fourier transform are used to extract features. Finally, the Pearson correlation coefficient is selected for feature selection, and the Euclidean distance is classified for 12 types of fonts. 576 and 1024 chessman samples collected from the training set and the test set are tested respectively. The error rate of existence of chessmen, the number of chess position and the chess color judgment in the subdomain are all 0. After using the training set to get the font class center, the font recognition test is carried out in the training set and the test set. The error rates of the methods of the filtered fonts characteristics for identifying is 0.17% (wrong 1) and 0.29% (wrong 3). The error rates were reduced by 0.35% and 0.39%, respectively, compared with the non-feature-based method. The experimental results show the excellent performance of the chess position detection, the chessmen location and the recognition method. It lays the algorithm foundation for the construction of a chess robot vision system.