Aiming at the problem of distinguishing split defects from mineral lines on the wood surface, a method based on local binary difference excitation pattern (LB_DEP) is proposed. Firstly, the potential defect regions are segmented with image preprocessing, then linear split and mineral line are screened using geometric parameters. Based on local binary pattern (LBP) and Weber′s law, an LB_DEP histogram reflecting the correlation relationship between image texture structure positions and difference excitation is established. Finally, the histogram features of LBP and LB_DEP are extracted, which are fused with feature data to form the feature vector that is used as the input of the SVM classifier to classify defects. Two feature extraction methods are proposed, namely ‘Hchisquare’ and ‘HPCA’, which are both evaluated on the selfbuilt data set. The experiment results show that for the two feature extraction methods the recall rates of 0937 and 0958, as well as the precision of 0950 and 0965 are obtained, respectively. Compared with other similar researches, the recall rate and precision are improved by at least 3% and 5%, respectively, and the time consumption is also at the level of milliseconds, which indicates the advantages and effectiveness of the proposed method.