Building detection is a crucial task in the field of remote sensing, and which can facilitate urban construction planning, disaster survey, and emergency landing. However, for large-size remote sensing images, the great majority of existing works have ignored the image tilt problem. This problem can result in partitioning buildings into separately oblique parts when the large-size images are partitioned. This is not beneficial to preserve semantic completeness of the building objects. Motivated by the fact above, we firstly propose a framework that detecting objects in large-size image, particularly for the building detection. The framework mainly consists of two phases. In the first phase, we particularly propose a Tilt Correction (TC) algorithm which contains three steps: texture mapping, tilt angle assessment, and image rotation. In the second phase, the building detection is performed with object detectors, especially deep neural network based methods. Last but not least, the detection results will be inversely mapped to the original large-size image. Furthermore, a challenging dataset named Aerial Image Building Detection (AIBD) is contributed for the public research. To evaluate the TC method, we also define an evaluation metric to compute the Cost of Building Partition (CoBP). The experiment results demonstrate the effects of the proposed method for the building detection. CCBY