基于重心调节与增强学习的新型仿生机器水母的研究

CORC > 自动化研究所 > 中国科学院自动化研究所 > 毕业生 > 硕士学位论文

题名	基于重心调节与增强学习的新型仿生机器水母的研究
作者	栗向滨
学位类别	工程硕士
答辩日期	2016-05-19
授予单位	中国科学院大学
授予地点	北京
导师	喻俊志
关键词	仿生机器人机器水母连杆机构重心调节增强学习仿真分析水下实验
中文摘要	经过亿万年的自然选择，海洋生物进化出了出色的水下运动能力。因此，基于仿生学原理对水下仿生机器人进行研究，模仿海洋生物非凡的机动性和隐蔽性，将具有重要的理论意义和实用价值。受生物水母自由灵活运动的特点以及独特的喷射推进模式的启发，研发了一种基于多连杆推进机构与重心调节机构的新型仿生机器水母，并对机器水母的机械结构与控制系统进行了详细的设计与研究，并得出了一套基于重心调节与增强学习控制的姿态控制方法。本文主要研究成果如下：第一，基于自然水母的喷射推进模式与灵活改变姿态的特性，设计了一款具有铰链多连杆推进机构以及重心调节机构的新型仿生机器水母。为了实现模仿真实水母收缩舒张的喷射推进过程，数字舵机带动肢部连杆进行周期往复运动；重心调节机构通过两个配重块水平方向与竖直方向的协调运动，完成对机器水母重心的调节，从而实现机器水母姿态的改变。防水密封设计包括静密封和动密封。第二，基于ARM Cortex M4主控芯片，构建了嵌入式的闭环控制系统。利用定时器的PWM模式产生四路舵机控制信号以及三路步进电机控制信号，通过调整PWM波的占空比实现对数字舵机转动频率与幅值的控制以及对步进电机转动速度的控制。机器水母通过串口收发指令与数据，通过射频模块实现指令与数据的无线传输，由此将上位机与下位机连成一体，构成闭环控制系统。第三，提出了基于机器学习算法的机器水母姿态控制方法，实现了机器水母通过自身学习而完成姿态的控制与镇定。由于机器水母通过重心调节而实现姿态控制是一个强耦合和非线性的问题，故利用增强学习算法，将机器水母姿态的控制模型转化为智能体与环境模型，通过对动作权重、状态权重与回报函数的计算，并建立回报函数值与下一步动作的关系，从而实现机器水母自主完成自身姿态的控制。第四，建立了机器水母喷射推进运动的运动学与动力学模型，以及机器水母重心调节过程的动力学模型，通过对三个模型进行仿真实验与分析可以得出，所设计的机器水母可以模仿真实水母收缩舒张运动过程，实现喷射式推进；机器水母重心运动轨迹完全包裹浮心，即可以实现对其三维姿态的完全控制，并且增强学习姿态控制系统是收敛的，即可以在有限动作数的情况下实现目标姿态。第五，验证了新型仿生机器水母的水下运动特性以及姿态控制能力。基于推进系统与重心调节系统的协调控制，对机器水母物理样机进行了水下运动控制实验，实现了机器水母任意方向的单向运动以及复合运动。通过对传感器数据的分析，机器水母可以通过Q学习控制算法快速、准确地到达目标姿态，验证了此新型仿生机器水母的机械结构和运动控制设计的可行性。
英文摘要	Through millions of years of natural selection, aquatic animals have evolved extraordinary underwater motion capacity. Therefore, based on the principle of bionics and imitating marine life excellent flexibility and concealment, researches on underwater bionic robots have important theoretical significance and practical value. Inspired by the characteristics of freedom and flexibility as well as unique jet propulsion mode of live jellyfish, a novel biomimetic robotic jellyfish based on multi-linkage propulsive mechanism and barycenter adjustment mechanism is developed. The mechanical structure and the control system of the robotic jellyfish are detailedly designed and studied. An attitude control method based on barycenter adjustment and reinforcement learning control is developed. The main research achievements of this thesis are as follows. First, based on the characteristics of the jet propulsion mode and the flexible change of attitude of natural jellyfish, a novel biomimetic robotic jellyfish equipped with the multi-linkage mechanism and barycenter adjustment mechanism is designed. In order to realize the simulation of the contraction-relaxation jet propulsion process of the real jellyfish, the limb linkage is driven by digital steering engine to achieve periodic reciprocating motion; barycenter adjustment mechanism completes regulating the center of gravity of the robotic jellyfish by coordinated movement of the two clump weights in horizontal and vertical directions, which make the posture of robotic jellyfish change. Waterproof sealing design includes static sealing and dynamic sealing. Second, based on an ARM Cotex M4 main controller chip, an embedded closed loop control system is built. Digital servo and step motor control signals are generated from the controller by PWM mode of the Timer. Adjusting the duty ratio of PWM waves realizes control of the position and the rotation frequency of the digital servo and control of stepper motor rotation speed. The robotic jellyfish receives and sends commands and data from and to the upper controller through serial ports. The robotic jellyfish achieves wireless signal communication through the radio frequency module, which make the host computer and the slave machine a whole, thus a closed-loop control system is established. Third, the attitude control method of robotic jellyfish based on the machine learning algorithm is put forward, which realizes the attitude control and stabilization of the robotic jellyfish through its own learning. Because the attitude control is a strong coupling and nonlinear problem with the regulation of the center of gravity, the reinforcement learning algorithm is used. The method transforms the robotic jellyfish attitude control model to an agent-environment model. Through the calculation of action weight, status weight and the reward function, as well as the relationship between the reward function value and the next action, robotic jellyfish attitude control on its own with reinforcement learning is achieved. Fourth, the kinematic and dynamic models of the jet propulsion motion of the robotic jellyfish, and the dynamic model of the regulation process of the center of gravity are established. The simulation experiments and analysis for above three models can be drawn that the designed robotic jellyfish can realize contraction and relaxation jet propulsion mode to imitate true jellyfish; the movement trajectory of the robotic jellyfish completely encases the buoyancy center, which means control of 3D pose can be achieved. Furthermore, the reinforcement learning control system is convergence, which means the robotic jellyfish can reach the target attitude in a finite number of actions. Fifth, the characteristics of underwater motion and attitude control ability of the novel robotic jellyfish are verified. Based on the coordination control of propulsion system and the barycenter adjustment system, a series of underwater motion control experiments for prototype of the robotic jellyfish are conducted. The robotic jellyfish can achieve unidirectional movement in any direction and composite motion. Through the analysis of the sensor data, the robotic jellyfish can reach the target attitude accurately and fast through Q learning control algorithm, which verify the feasibility of mechanical structure and motion control design of this novel bionic robotic jellyfish.
语种	中文
内容类型	学位论文
源URL	[http://ir.ia.ac.cn/handle/173211/11740]
专题	毕业生_硕士学位论文
作者单位	中国科学院自动化研究所
推荐引用方式 GB/T 7714	栗向滨. 基于重心调节与增强学习的新型仿生机器水母的研究[D]. 北京. 中国科学院大学. 2016.