Autonomous underwater vehicles (AUVs) equipped with visual detection systems are capable to detect underwater artifacts, which is of great significance to deep-sea archaeology. Underwater cultural artifacts are situated within a dynamic and intricate environment, where the target objects are often fragmented, layered and concealed under sediment. These conditions pose significant challenges to the effective extraction of discernible features, thereby impeding the capability of AUV for reliable and accurate detection of underwater artifacts. In order to solve these problems, this paper proposes an underwater cultural artifact detection algorithm based on the deformable deep aggregation network model. To fully extract the target feature information of underwater cultural artifact in complex environments, this paper designs a multi-scale deep aggregation network with deformable convolutional layers. Besides, the SimAM attention module is designed for the feature optimization, which enhances the potential feature information of cultural artifact target and weakens the background interference information simultaneously. Finally, the prediction of cultural artifact is achieved through fusing feature at different scales. The proposed algorithm has been extensively validated and analyzed on the collected underwater artifact datasets, and the precision, recall, and mAP of algorithm are 92. 7% , 90. 5% , and 92. 2% , respectively. Additionally, the proposed algorithm has been deployed to the AUV system, specifically the artifact detection frame rate of visual detection system reaches 19 fps in the actual deep-sea test scenario and this can satisfy the real-time detection task.