Aiming at the problem of low recognition accuracy of underwater flaw detection due to the random and changeable shape of underwater structures and the difficulty of feature extraction, a lightweight small visual geometry group (LSVGG) model based on visual geometry group (VGG) network is proposed. The classical VGG network structure is adopted to reduce the convolution layer and improve the number of features, which can reduce the operation time and system overhead on the premise of ensuring the recognition accuracy. The experimental results show that the LSVGG model can be deployed on the small autonomous underwater vehicle (AUV) and has high flaw detection and identification accuracy for underwater structures. Compared with the traditional model, the recognition accuracy of underwater structure flaw detection is nearly doubled, and the recognition accuracy is as high as 99.7%.