Twenty heavy metals in the ground surface dust and agricultural soil in three different areas, i.e., a former artisanal zinc smelting area, a medium-scale zinc smelting area, and a control area, in Guizhou province southwest China, were investigated to reveal the pollution status of these heavy metals. The results showed that the pollution was most serious in the artisanal zinc smelting area, followed by the medium-scale zinc smelting area, while the control area was relatively clean. Zinc smelting activities had caused environmental contamination for elements such as Ag, As, Bi, Cd, Cu, Hg, In, Pb, Sb, Sn, Zn. Whereas, Co, Cr, Ga, Mo, Ni, Sc, Th, Tl, V were in the natural levels, hinting they were irrelevant to the zinc smelting activities. Concentrations of most heavy metals that come from the zinc smelting were obviously higher in the ground surface dust than that in the agricultural soil. And Cd, Pb, Zn, Ag were the four elements that affected by the zinc smelting activities most.