In order to study the deformation and failure characteristics of mining overlying rocks, similar material physical model test was used to simulate the remining process of coal mine face, and BOTDA distributed optical fiber and FBG were used to monitor the migration laws of mining overlying rocks. The three-dimensional model of 1.5 m × 0.6 m × 1.3 m was built experimentally. The geometric similarity ratio was 1:150. At the same time, two optical fibers were laid in the horizontal direction of the model, four optical fibers were laid in the vertical direction, and two fiber grating strain sensors were laid in the sub-critical layer. It is used to study the relationship between the migration state of overburden rock and the test results of BOTDA and FBG during model excavation. The experiment shows that the strain of FBG sensor is closely related to the migration state of rock formations, and can monitor the deformation breaking and delamination process of mining overburden rocks. The horizontal optical fiber test results showed a clear correspondence with the rock collapse movement. The peak distance of the test frequency shift was almost the same as the width of the upper layer of rock collapse. The average error was not more than 3 %. The change of frequency shift of vertical fiber can reflect the dynamic process of overburden collapse and delamination.