Aiming at the problems of unsatisfactory defect detection effect and insufficient feature extraction of small and medium targets in steel surface defect detection, a YOLOv5s-ADW algorithm based on YOLOv5 algorithm was proposed. a mixed model of self-attention and convolution (ACmix) module is integrated into the backbone network layer to enhance the feature sensitivity of the model. The deformable large kernel attention (D-LKA) mechanism is added to the feature fusion layer to enhance the ability of the model to capture irregular defects in images. The original loss function was replaced by Wise-IoU loss function to reduce the influence of low-quality examples on the model detection effect and improve the detection ability of small target defects, and the experimental verification was carried out on NEU-DET. Experimental results show that the mean average precision (mAP) of the YOLOv5s-ADW algorithm reaches 88.3%, which is 14.4% higher than that of the original model. The average precision (AP) of small target defects and defects with high missed detection rate is also greatly improved, which can better solve the above problems compared with other mainstream algorithms.