Boron nitride (BN) exhibited poor compatibility with epoxy resin (EP) due to its inert surface, and its flame-retardant efficiency in EP was low. In this study, a coordination compound (FeD) was synthesized by a reaction between 9,10-dihydro-9-oxa-10-phosphaphenanthrene-10-oxide derivative (DMZ) and Fe(NO
3)
3. FeD was grown in situ on the surface of BN to prepare a flame-retardant functionalized boron nitride hybrid (FeD/BN). FeD/BN was incorporated into EP to prepare thermally conductive and flame-retardant EP composites. Limiting oxygen index (LOI) and vertical burning (UL-94) tests indicate that EP composite with 15 wt% FeD/BN achieves a V-0 rating in the UL-94 test with a LOI value of of 33.2%. Cone calorimetry test results show that the peak heat release rate, total heat release and total smoke release of the composite decrease by 28.2%, 18.9% and 30.1%, respectively compared to those of pure EP. Thermal conductivity of 15(Fe/B)/EP is 235% higher than that of pure EP. Tensile and impact strengths of 15(Fe-B)/EP are enhanced compared with those of pure EP.