The aim of this study is to investigate the effect of chitosan oligosaccharides (COS) on type 2 diabetes mellitus. Wild type C57BL/6J mice or diabetic db/db mice were treated with vehicle or COS for three months. COS treatment significantly decreased the blood glucose (P < 0.01) and reversed the insulin resistance (P < 0.05) in db/db mice, which was accompanied by suppressing the inflammation mediators (P < 0.05), down-regulating the lipogenesis (P < 0.01) and inhibiting the adipocyte differentiation (P < 0.05) in white adipose tissue. Additionally, COS treatment inhibited the reduction of occludin (P < 0.01) and relieved the gut dysbiosis in diabetic mice by promoting Akkermansia (P < 0.01) and suppressing Helicobacter (P < 0.05). Spearman's correlation analysis indicates that the COS-modulated bacteria are positively correlated with inflammation, hyperglycemia and dyslipidemia. The functional profiling based on the microbiota composition implicated that COS treatment may regulate the metabolic pathways of gut microbiota. In summary, COS treatment remarkably improved the glucose metabolism and reshaped the unbalanced gut microbiota of diabetic mice. Our study provided the evidence for application of COS to the treatment of diabetes mellitus.