The Heihe River basin is one of the important bases for commercial grain production in China. The climate change impacts on crop yield and crop water productivity (CWP) need to be examined for sustainable development of regional agriculture, especially under the pressure of population growth and regional water scarcity. A distributed hydrological model was constructed for the upper-middle reaches of the Heihe River basin on the basis of the Soil and Water Assessment Model (SWAT). The sensitivity ranks of hydrological parameters were evaluated for the runoff and evapotranspiration processes, and then independently calibrated with the observations of three gauging stations (Qilian, Zhamashike, and Yingluoxia stations) and the evapotranspiration data derived by remote-sensing. The study first simulated crop yield and CWP for four typical crop types, namely corn, spring wheat, spring barley, and spring canola-Polish, for the period of 1966-2014 using the SWAT model, calibrated with the crop yield data (during the period of 2005-2014) from Zhangye Statistics Yearbook. The spatial-temporal features of crop yield and CWP were analyzed. A fluctuating growth was found for the 49 year trend of corn, spring wheat, and spring canola-Polish. The spring barley shows a slightly decreasing trend. The high-yield area moved from the west part (Suzhou and Gaotai counties) to the east part (Linze, Ganzhou, and Shandan counties) of the basin for the past 49 years, and the area of high CWP concentrated in the Gaotai and Linze Counties. The responses of CWP to climatic forcing were further studied using wavelet coherence method. It is found that maximum temperature dominated the CWP evolutions, especially at the long-term scales. The corn showed the largest elastic responses to the climatic factors, which may be favorable to improve regional CWP by expanding planting area. In addition, the strategies of regional CWP improvement were discussed based on the results.