Vision-based grasping plays an important role in the robot providing better services. It is still challenging under disturbed scenes, where the target object cannot be directly grasped constrained by the interferences from other objects. In this article, a robotic grasping approach with firstly moving the interference objects is proposed based on elliptical cone-based potential fields. Single-shot multibox detector (SSD) is adopted to detect objects, and considering the scene complexity, Euclidean cluster is also employed to obtain the objects without being trained by SSD. And then, we acquire the vertical projection of the point cloud for each object. Considering that different objects have different shapes with respective orientation, the vertical projection is executed along its major axis acquired by the principal component analysis. On this basis, the minimum projected envelope rectangle of each object is obtained. To construct continuous potential field functions, an elliptical-based functional representation is introduced due to the better matching degree of the ellipse with the envelope rectangle among continuous closed convex curves. Guided by the design principles, including continuity, same-eccentricity equivalence, and monotonicity, the potential fields based on elliptical cone are designed. The current interference object to be grasped generates an attractive field, whereas other objects correspond to repulsive ones, and their resultant field is used to solve the best placement of the current interference object. The effectiveness of the proposed approach is verified by experiments.