This paper presents a novel adaptive trajectory tracking control method, which can precisely control wheeled mobile robots only using an uncalibrated fish-eye camera fixed on the ceiling. Different from existing approaches, the inertial device, distorted image correction, and the trajectory expression are not required in the control system. The position and orientation of the mobile robot in the camera coordinate system are estimated by the extended POSIT (Pose from Orthography and Scaling with Iteration) algorithm in real-time. Based on estimation results, the controller considering both tracking errors and parameter estimated errors is designed by linear parameterization, where the camera intrinsic parameters are online updated. The asymptotic convergence of the tracking error and the estimated error to zero is proved by the Barbalat lemma. Circular trajectory and irregular trajectory tracking experiments have been conducted to verify the performance of our controller.