The reliability-based optimization (RBO) of composite structures subject to buckling constraint was investigated. Considering the uncertainties of material properties and applied loads, the buckling reliability of the structure was evaluated through the combination of Response Surface Method (RSM) and Finite Element Method (FEM). The RBO of composite structures subject to buckling constraint was conducted in terms of the ply number and orientation angles, using the commercial code iSIGHT combined with the reliability analysis program. The numerical examples of the composite laminate and the cylinder validate the present RBO method, which is useful for the optimization of composite structures subject to buckling constraint in engineering.