Abstract: In order to obtain the flame retardant epoxy resin (EP) composites with good comprehensive properties, the 9,10-dihydro-9-oxa-10-phosphaphenanthrene-10-sulfide (DOPS) derivative was applied to EP. First, maleic anhydride-9,10-dihydro-9-oxa-10-phosphaphenanthrene-10-sulfide (MAH-DOPS) was synthesized by the reaction of DOPS and maleic anhydride (MAH). The structure was determined by FTIR, ¹H NMR and ³¹P NMR. Secondly, DOPS and MAH-DOPS were added to EP respectively, to prepare composites DOPS/EP and MAH-DOPS/EP by blending. Thirdly, the thermal stability, flame retardancy and mechanical properties of DOPS/EP and MAH-DOPS/EP were compared. Finally, the effects of MAH-DOPS on the combustion performance and thermal degradation behavior of EP were discussed, and its flame retardant mechanism was analyzed in depth. The results show that the initial decomposition temperature of flame retardant DOPS (205.4℃) is lower than that of MAH-DOPS (235.2℃), and the thermal stability of DOPS/EP is lower than that of MAH-DOPS/EP when the same mass fraction of flame retardants is added, which is consistent with the thermal stability of flame retardants. Both flame retardants DOPS and MAH-DOPS can improve the flame retardancy of EP, and MAH-DOPS/EP has better flame retardant effect. When the amount of flame retardant is 15wt%, limit oxygen index (LOI) values of MAH-DOPS/EP and DOPS/EP are 28.6% and 29.1%, reaching UL-94 V-0 and V-1 rating, respectively. The mechanical test results show that compared with EP, the bending strength of MAH-DOPS/EP increases by 45.8%, while DOPS/EP decreases by 62.5%. The mechanical properties of DOPS/EP system decrease obviously, and almost lose the use value. Cone calorimeter tests show that the average heat release rate (av-HRR) and total heat release (THR) of the composite MAH-DOPS/EP decrease significantly. The results of TG-IR show that the H•, O• or HO• free radicals are captured by the phosphorus-containing free radicals generated by the pyrolysis of MAH-DOPS/EP, and the free radicals are quenched. SEM-EDS results show that MAH-DOPS/EP can form a more complete and compact char layer, and the content of P in the char layer is higher. The study shows that MAH-DOPS plays a flame retardant role in gas phase and condensed phase through flame suppression and char formation respectively, and the gas phase flame retardant mechanism is the main one.