Pulse thermography, as an emerging non-destructive testing technique, has been widely used in the infrared field. However, the low contrast and high noise in infrared images caused by uneven heating of the sample surface and low surface emissivity pose challenges for defect detection. In this study, non-destructive testing of composite materials, specifically CFRP (carbon fiber reinforced polymer) plates and 304 stainless steel, was conducted using long pulse thermography. A post-processing method based on temperature ratios was proposed. This method performs a division operation on each frame of the temperature sequence image and the previous frame image to obtain a sequence of temperature ratio images. The image with the highest signal-to-noise ratio is extracted from the temperature ratio sequence for defect identification and quantification. The results show that the method significantly improves the signal-to-noise ratio of the image compared with the previous method, and is more conducive to detecting deeper and smaller defects, compared with the original image, the signal-to-noise ratio is improved by nearly 98.46%. We combined the method with the half-height full-width method to quantify the size of the specimen. Through simulation and experimental verification, the method has good robustness and significantly reduces the error of defect size quantification. Therefore, this method can be considered as an effective infrared image post-processing and defect size quantification method.