Heavy metal pollution in soil has become a prominent problem affecting agricultural security and ecological health. Hyperspectral remote sensing is used as a rapid method to predict soil heavy metal concentrations. The processing of spectral data and the variables of the estimated model has an important impact on the predictive model of soil heavy metal elements. In this paper, smoothed and resampled spectral reflections are preprocessed by using three preprocessing methods, namely, standard normal variate (SNV), multiplication scatter correlation (MSC) and normalization (NOR). Then, first and second order differential (FD and SD, respectively) and absorbance transformation (AT) are performed. Based on the adsorption and retention of heavy metals by various soil components, the relevant spectral bands are extracted as modeling variables. An extreme learning machine algorithm (ELM) is used to establish the model, and the effects of different factors on the model are compared. Results show that the combination of the three preprocessing methods (SNV, MSC and NOR) with spectral transformation can enhance the stability and predictive ability of the resulting model. The combination of SNV and FD can predict the contents of Cr, Ni and Pb. The R2 of the model is 0.85, 0.87 and 0.80 respectively. The optimal model of Cu is derived from the combination of NOR and SD (R2 = 0.84), and the spectral responses of soil Cr, Ni, Cu and Pb, are closely related to clay mineral-related and organic matter-related bands. The model established by the clay-related bands enhances the stability of the prediction of Ni content, and the RPD value was increased from 2.46 to 2.72 compared with the full-band model. The combination of bands associated with organic matter and clay minerals can accurately predict the content of Cr and Cu in soil; indeed, the predict model R2 for these elements reaches 0.88. Accurate prediction of soil Pb by the full-band model indicates that the Pb concentration in the study area is related to a various of soil chemical components. The prediction effects of the four heavy metal elements show the order Cr > Cu > Ni > Pb. The results of the current study complement the theoretical basis for estimating the heavy metal content of soil by hyperspectral spectroscopy, and provide important insights into the application of hyperspectral remote sensing to monitor other heavy metals.