By regulating spinning process of polyacrylonitrile (PAN) precursor, PAN based carbon fibers (CF) with rough surface and CF with smooth surface were produced, both of which share the same chemical structure as well as similar mechanical property to simulate the influence of surface morphology on the electrochemical oxidation behaviors. Under the same oxidized condition, CF with smooth surface shows a higher morphology resistance, and the O to C ratio is much more than that on CF with rough surface, indicating a higher oxidation degree of the former. XPS spectra reveals that the difference of O to C ratio is probably due to the changes of the relative amount of carbonyl groups on carbon fiber surface. The tensile strength and tensile modulus of CF with rough surface could be improved, however, the tensile strength could be obviously improved to a maximum of 17.3% at the initial stage of electrochemical oxidation. The interlaminar shear strength (ILSS) of carbon fiber reinforced epoxy composite was measured by a fragmentation test, and it is suggested that a better interfacial adhesion could be obtained from rough-surface carbon fiber reinforced epoxy composite, indicating the reinforced mechanism between carbon fiber and epoxy matrix is prone to be anchor force rather than chemical interaction.