Due to the long-term external dynamic load, such as wind load, the tubular tower of an offshore wind turbine usually exhibits quite irregular stress responses, leading to fatigue failure. Considering the different probabilities of each wind speed condition and the different responses of the tubular tower under each condition, it is necessary to divide wind speed conditions into segments to compile the fatigue stress spectrum. In this paper, a three-dimensional refined finite element model of the tubular tower structure was established based on an offshore wind power project in the East China Sea. Combined with the Weibull distribution model of wind speed, the rain-flow counting method, and the linear fatigue cumulative damage law, the nominal stress method was used to calculate the fatigue life of the tubular tower. At the same time, the simplified tubular tower model was established and compared with the refined model, so as to provide ideas and references for the fatigue research of the wind turbine structure. The calculation results show that it is feasible to compile the fatigue stress spectrum by dividing wind speed conditions. The parts prone to fatigue damage are located at or near the junctions between the tower segments in the middle and lower part of the tubular tower. In the analysis of the fatigue characteristics of the tubular tower, the results of the simplified tubular tower model whose wall thickness changes uniformly with the height and the refined model are quite different, which will cause large errors. Therefore, it is recommended to establish a refined model to reasonably predict the fatigue life of the tubular tower.