Carbon fiber reinforced polymer (CFRP) composites are widely used because of their excellent properties such as high specific strength and high specific modulus, but their mechanical properties along the thickness are poor due to the laminar structure characteristics and the intrinsic brittleness of epoxy resin, and they are prone to delamination under out-of-plane impact and in-plane compression loads, which in turn reduce the strength of the composites. Therefore it is especially important to improve the interlaminar fracture toughness of the composites. In this paper, we attempt to improve the interlaminar fracture toughness of the composite by introducing highly oriented carbon nanotube fiber veils in the interlaminar region. To ensure that the fiber veils are well infiltrated by the resin, they are first immersed in an epoxy resin solution diluted with acetone. After the acetone evaporated, it is inserted into the interlayer region of the homemade carbon fiber prepreg and subsequently cured by a hot pressing process. The Mode I and Mode II interlaminar fracture toughness of the toughened samples are evaluated via ASTM testing standards. Combined with the optical microscopic observation of the cross-section and scanning electron microscopy analysis of the fracture surface, the crack propagation paths are clearly shown and the interlaminar toughening mechanisms of CNT fiber veils are revealed. The results show that the Mode I and Mode II interlaminar fracture toughness of CNT veil toughened samples are improved by 37.4% and 41.8%, respectively. The toughening mechanisms mainly include matrix toughening, strengthening carbon fiber bridging and crack deflection.