Hybrid coating of epoxy acrylate(EA) and modified nano organic montmorillonite(OMMT) was prepared for optical fiber Bragg grating(FBG) sensors. Mechanical performance testing and heat performance testing were carried on the OMMT/EA hybrid coating. The results demonstrate that, the performance of OMMT/EA coating with 2% mass fraction of nano OMMT is the best. SEM photographs and XRD analysis show that thesingle-lamellar nano OMMT is even distribution in EA. The tensile property, adhesion force, pencil hardness and glass temperature of OMMT/EA coating can be improved by adding nano OMMT, while the linear thermal expansion coefficient is decreased. FBG sensors with OMMT/EA coating and EA were embedded in the composites which processed under certain temperature and pressure, respectively. Comparing two groups of data, the results show that the modified OMMT/EA coating could obviously decrease the hysteresis of FBG sensor signals.