基于PPYOLOE+的水下生物目标检测_为什么要做水下显著目标检测

基于PP-YOLOE+的水下生物目标检测+部署

1 项目背景

水下目标检测旨在对水下场景中的物体进行定位和识别。这项研究由于在海洋学、水下导航等领域的广泛应用而引起了持续的关注。但是，由于复杂的水下环境和光照条件，这仍然是一项艰巨的任务。

基于深度学习的物体检测系统已在各种应用中表现出较好的性能，但在处理水下目标检测方面仍然感到不足，主要有原因是：可用的水下目标检测数据集稀少，实际应用中的水下场景的图像杂乱无章，并且水下环境中的目标物体通常很小，而当前基于深度学习的目标检测器通常无法有效地检测小物体，或者对小目标物体的检测性能较差。同时，在水下场景中，与波长有关的吸收和散射问题大大降低了水下图像的质量，从而导致了可见度损失，弱对比度和颜色变化等问题。

Al+水下勘探是一个新兴领域，目前专门用于水下研究工作的解决方案不多，高质量的数据集更是弥足珍贵。使用 PP-YOLOE+ 来推进水下目标检测的进步，从而使得水下机器人等设备能够更加智能化，提高海底资源勘探等方面的效率。而水下机器人又反哺出高质量的水下目标检测数据集，推动 Al+水下勘探的发展。

2 方案选择

2.1 问题与挑战

深度学习中，数据往往决定了性能的上限，算法只是不断地逼近上限。尽管基于深度学习的方法在标准的目标检测中取得了可喜的性能。水下目标检测仍具有以下几点挑战：

（1）水下场景的实际应用中目标通常很小，含有大量的小目标；

（2）水下数据集和实际应用中的图像通常是模糊的，图像中具有异构的噪声。

2.2 方案选择

因此，针对以上所述的背景和水中目标检测所遇到的挑战，本项目将选用 PP-YOLOE+ 这一基于飞桨云边一体高精度模型PP-YOLOE迭代优化升级的版本。针对性的解决以上问题。

2.3 模型特点介绍

PP-YOLOE+ 具有如下特点：

• 超强性能

• 训练收敛加速

• 下游任务泛化性显著提升

• 高性能部署能力

3 环境配置

3.1 环境准备

PaddlePaddle >= 2.3.2
Python == 3.7
1
2

3.2 环境安装

%cd /home/aistudio/work/
# gitee 国内下载比较快
!git clone https://gitee.com/paddlepaddle/PaddleDetection.git -b develop

# github
# !git clone https://github.com/PaddlePaddle/PaddleDetection.git -b develop

1
2
3
4
5
6
7

# 环境安装
%cd /home/aistudio/work/
# gitee 国内下载比较快
# !git clone https://gitee.com/paddlepaddle/PaddleDetection.git -b develop

%cd PaddleDetection/
!pip install -r requirements.txt > /dev/null
1
2
3
4
5
6
7

/home/aistudio/work
/home/aistudio/work/PaddleDetection
1
2

4 数据集预处理

4.1 数据集介绍

本项目选用数据集来源于水下目标检测算法赛（注：数据由鹏城实验室提供），训练集是5543张 jpg 格式的水下光学图像与对应标注结果构成，其中主要有海参、海胆、扇贝、海星四种目标。

4.1.1 标签类别

4.1.2 图像分辨率

4.2 数据集处理

4.2.1 数据集解压

# 1.数据集解压
!unzip data/data172711/fish.zip > /dev/null
!mv ./fish ./work/PaddleDetection/dataset
1
2
3

4.2.2 voc2coco

%cd work/PaddleDetection/
1

/home/aistudio/work/PaddleDetection
1

# 2.voc2coco

import argparse
import glob
import json
import os
import os.path as osp
import sys
import shutil

import numpy as np
import PIL.ImageDraw
import xml.dom.minidom as xmldom
import cv2
label_to_num = {}
categories_list = []
labels_list = []


class MyEncoder(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, np.integer):
            return int(obj)
        elif isinstance(obj, np.floating):
            return float(obj)
        elif isinstance(obj, np.ndarray):
            return obj.tolist()
        else:
            return super(MyEncoder, self).default(obj)


def getbbox(self, points):
    polygons = points
    mask = self.polygons_to_mask([self.height, self.width], polygons)
    return self.mask2box(mask)


def images_labelme(data, num):
    image = {}
    image['height'] = data['imageHeight']
    image['width'] = data['imageWidth']
    image['id'] = num + 1
    image['file_name'] = data['imagePath'].split('/')[-1]
    return image


def images_cityscape(num, img_file, w, h):
    image = {}
    image['height'] = h
    image['width'] = w
    image['id'] = num + 1
    image['file_name'] = img_file
    return image


def categories(label, labels_list):
    category = {}
    category['supercategory'] = 'component'
    category['id'] = len(labels_list) + 1
    category['name'] = label
    return category


def annotations_rectangle(points, label, image_num, object_num, label_to_num):
    annotation = {}
    seg_points = np.asarray(points).copy()
    annotation['segmentation'] = [list(seg_points.flatten())]
    annotation['iscrowd'] = 0
    annotation['image_id'] = image_num + 1
    annotation['bbox'] = list(
        map(float, [
            points[0][0], points[0][1], points[1][0] - points[0][0], points[1][
                1] - points[0][1]
        ]))
    annotation['area'] = annotation['bbox'][2] * annotation['bbox'][3]
    annotation['category_id'] = label_to_num[label]
    annotation['id'] = object_num + 1
    return annotation


def annotations_polygon(height, width, points, label, image_num, object_num,
                        label_to_num):
    annotation = {}
    annotation['segmentation'] = [list(np.asarray(points).flatten())]
    annotation['iscrowd'] = 0
    annotation['image_id'] = image_num + 1
    annotation['bbox'] = list(map(float, get_bbox(height, width, points)))
    annotation['area'] = annotation['bbox'][2] * annotation['bbox'][3]
    annotation['category_id'] = label_to_num[label]
    annotation['id'] = object_num + 1
    return annotation


def get_bbox(height, width, points):
    polygons = points
    mask = np.zeros([height, width], dtype=np.uint8)
    mask = PIL.Image.fromarray(mask)
    xy = list(map(tuple, polygons))
    PIL.ImageDraw.Draw(mask).polygon(xy=xy, outline=1, fill=1)
    mask = np.array(mask, dtype=bool)
    index = np.argwhere(mask == 1)
    rows = index[:, 0]
    clos = index[:, 1]
    left_top_r = np.min(rows)
    left_top_c = np.min(clos)
    right_bottom_r = np.max(rows)
    right_bottom_c = np.max(clos)
    return [
        left_top_c, left_top_r, right_bottom_c - left_top_c,
        right_bottom_r - left_top_r
    ]


def deal_json(output_dir, input_dir, img_list):
    data_coco = {}
    images_list = []
    annotations_list = []
    image_num = -1
    object_num = -1
    # labels_list =[]
    
    for img_file in img_list:
        img = cv2.imread(osp.join('dataset/fish/Images', img_file))
        w = img.shape[1]
        h = img.shape[0]
        img_label = img_file.split('.')[0]
        label_file = osp.join('dataset/fish/Annotations', img_label + '.xml')
        # print('Generating dataset from:', label_file)
        image_num = image_num + 1

        xml_file = xmldom.parse(label_file)
        eles = xml_file.documentElement
        images_list.append(images_cityscape(image_num, img_file, w, h))
            
        for i in range(len(eles.getElementsByTagName('name'))):
            label = eles.getElementsByTagName('name')[i].firstChild.data
            # if label == 'starfish':
            #     continue
            object_num = object_num + 1
            # print(label)
            if label not in labels_list:
                # print(label)
                categories_list.append(categories(label, labels_list))
                labels_list.append(label)
                label_to_num[label] = len(labels_list)
            points = []
            xmin = int(eles.getElementsByTagName('xmin')[i].firstChild.data)
            ymin = int(eles.getElementsByTagName('ymin')[i].firstChild.data)
            xmax = int(eles.getElementsByTagName('xmax')[i].firstChild.data)
            ymax = int(eles.getElementsByTagName('ymax')[i].firstChild.data)
            # print(xmin,ymin,xmax,ymax)
            # if xmin > 2000:
            #     print(label_file)
            points.append([xmin, ymin])
            points.append([xmax, ymax])
            annotations_list.append(
                annotations_rectangle(points, label, image_num,
                                    object_num, label_to_num))                
    
    data_coco['images'] = images_list
    data_coco['categories'] = categories_list
    data_coco['annotations'] = annotations_list
    # print(labels_list)
    return data_coco


import os.path as osp
import glob
import os
import shutil
train_img = 'dataset/fish/Images'
train_box = 'dataset/fish/Annotations'
    
# Allocate the dataset.
total_num = len(glob.glob(osp.join(train_img, '*.jpg')))
    
train_num = int(total_num * 0.8)
os.makedirs('data/cocome' + '/train')

val_num = total_num - train_num
os.makedirs('data/cocome' + '/val')

count = 1
train_list = []
val_list = []
for img_name in os.listdir(train_img):
    if count <= train_num:
        if osp.exists('data/cocome' + '/train/'):
            shutil.copyfile(
                osp.join(train_img, img_name),
                osp.join('data/cocome' + '/train/', img_name))
            train_list.append(img_name)
    else:
        if count <= train_num + val_num:
            if osp.exists('data/cocome' + '/val/'):
                shutil.copyfile(
                    osp.join(train_img, img_name),
                    osp.join('data/cocome' + '/val/', img_name))
            val_list.append(img_name)
    count = count + 1

if not os.path.exists('data/cocome' + '/annotations'):
    os.makedirs('data/cocome' + '/annotations')
train_data_coco = deal_json(
        'data/cocome' + '/train', train_img ,train_list)
train_json_path = osp.join('data/cocome' + '/annotations', 'instance_train.json')
json.dump(
        train_data_coco,
        open(train_json_path, 'w'),
        indent=4,
        cls=MyEncoder)
            
val_data_coco = deal_json('data/cocome' + '/val', train_img, val_list)
val_json_path = osp.join('data/cocome' + '/annotations', 'instance_val.json')
json.dump(val_data_coco, open(val_json_path, 'w'), indent=4, cls=MyEncoder)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215

4.2.3 将转换后的数据集移动至dataset/fish

# 3.将转换后的数据集移动至dataset/fish
!mv data/cocome/* dataset/fish/
1
2

4.2.4 将自己的数据集的路径进行修改和配置

metric: COCO
num_classes: 5

TrainDataset:
  !COCODataSet
    image_dir: train
    anno_path: annotations/instance_train.json
    dataset_dir: dataset/fish
    data_fields: ['image', 'gt_bbox', 'gt_class', 'is_crowd']

EvalDataset:
  !COCODataSet
    image_dir: val
    anno_path: annotations/instance_val.json
    dataset_dir: dataset/fish

TestDataset:
  !ImageFolder
    anno_path: label_list.txt # also support txt (like VOC's label_list.txt)
    dataset_dir: dataset/fish # if set, anno_path will be 'dataset_dir/anno_path'
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

5 模型训练

使用 PP-YOLOE+ 进行训练:

在./configs/ppyoloe/ppyoloe_plus_crn_s_80e_coco.yml中提供了基于PP-YOLOE+训练该场景的配置，训练脚本如下：

!python tools/train.py -c configs/ppyoloe/ppyoloe_plus_crn_s_80e_coco.yml -o LearningRate.base_lr=0.000125 -o snapshot_epoch=1 -o worker_num=1 --eval
1

6 模型评估

在训练模型以后，我们可以通过运行评估命令来得到模型的精度，以确认训练的效果。评估可以参考以下命令执行。

这里使用了我们已经训练好的模型。如希望使用自己训练的模型，请对应将weights=后的值更改为对应模型.pdparams文件的存储路径。

# 模型评估

!python tools/eval.py -c configs/ppyoloe/ppyoloe_plus_crn_s_80e_coco.yml -o weights=output/ppyoloe_plus_crn_s_80e_coco/best_model.pdparams
1
2
3

Warning: import ppdet from source directory without installing, run 'python setup.py install' to install ppdet firstly
W1016 22:54:41.591436 22455 gpu_resources.cc:61] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 11.2, Runtime API Version: 11.2
W1016 22:54:41.598824 22455 gpu_resources.cc:91] device: 0, cuDNN Version: 8.2.
loading annotations into memory...
Done (t=0.00s)
creating index...
index created!
[10/16 22:54:43] ppdet.utils.checkpoint INFO: Finish loading model weights: output/ppyoloe_plus_crn_s_80e_coco/best_model.pdparams
[10/16 22:54:44] ppdet.engine INFO: Eval iter: 0
[10/16 22:54:55] ppdet.metrics.metrics INFO: The bbox result is saved to bbox.json.
loading annotations into memory...
Done (t=0.01s)
creating index...
index created!
[10/16 22:54:55] ppdet.metrics.coco_utils INFO: Start evaluate...
Loading and preparing results...
DONE (t=0.50s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=3.09s).
Accumulating evaluation results...
DONE (t=0.43s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.278
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.541
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.267
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.237
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.275
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.304
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.160
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.436
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.543
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.514
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.509
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.572
[10/16 22:54:59] ppdet.engine INFO: Total sample number: 200, averge FPS: 19.422192298181745
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

7 模型预测

这里我们将训练好的模型对着整个数据集的验证集来一番批量预测，这些预测结果会记录在VisualDL中，可以很方便地与原图对比，观察预测效果。

!python tools/infer.py -c configs/ppyoloe/ppyoloe_plus_crn_s_80e_coco.yml -o weights=output/ppyoloe_plus_crn_s_80e_coco/best_model --infer_dir ./dataset/fish/val --use_vdl=True --vdl_log_dir=./output/image --output_dir ./output/results
1

8 模型导出

.pdparams只包括了模型的参数数据，实际部署还需要执行导出步骤。导出步骤可以参考下面列举的步骤：

注意，这里使用了我们已经训练好的模型。如希望使用自己训练的模型，请对应将weights=后的值更改为对应模型.pdparams文件的存储路径。如果没有指定–output_dir，那么导出的模型将默认存储在 output_inference/ 路径下。

!python tools/export_model.py -c configs/ppyoloe/ppyoloe_plus_crn_s_80e_coco.yml -o weights=output/ppyoloe_plus_crn_s_80e_coco/best_model.pdparams
1

至此，我们就完成了水底生物目标检测模型的从训练到导出的过程。接下来，看看该模型使用Paddle Inference部署时的具体性能表现。

9 模型速度测试

!python deploy/python/infer.py --model_dir=output_inference/ppyoloe_plus_crn_s_80e_coco --image_file=dataset/fish/val/c000418.jpg --run_mode=paddle --device=gpu
1

-----------  Running Arguments -----------
action_file: None
batch_size: 1
camera_id: -1
combine_method: nms
cpu_threads: 1
device: gpu
enable_mkldnn: False
enable_mkldnn_bfloat16: False
image_dir: None
image_file: dataset/fish/val/c000418.jpg
match_metric: ios
match_threshold: 0.6
model_dir: output_inference/ppyoloe_plus_crn_s_80e_coco
output_dir: output
overlap_ratio: [0.25, 0.25]
random_pad: False
reid_batch_size: 50
reid_model_dir: None
run_benchmark: False
run_mode: paddle
save_images: True
save_mot_txt_per_img: False
save_mot_txts: False
save_results: False
scaled: False
slice_infer: False
slice_size: [640, 640]
threshold: 0.5
tracker_config: None
trt_calib_mode: False
trt_max_shape: 1280
trt_min_shape: 1
trt_opt_shape: 640
use_coco_category: False
use_dark: True
use_gpu: False
video_file: None
window_size: 50
------------------------------------------
-----------  Model Configuration -----------
Model Arch: YOLO
Transform Order: 
--transform op: Resize
--transform op: NormalizeImage
--transform op: Permute
--------------------------------------------
class_id:0, confidence:0.7396, left_top:[299.31,0.93],right_bottom:[331.32,32.59]
class_id:0, confidence:0.7280, left_top:[293.39,100.21],right_bottom:[332.52,138.38]
class_id:0, confidence:0.6212, left_top:[528.09,-1.20],right_bottom:[566.73,20.30]
class_id:0, confidence:0.5986, left_top:[71.87,-0.63],right_bottom:[111.20,26.91]
class_id:0, confidence:0.5766, left_top:[120.02,101.65],right_bottom:[181.98,152.03]
class_id:0, confidence:0.5554, left_top:[405.12,215.59],right_bottom:[458.40,272.60]
class_id:0, confidence:0.5486, left_top:[476.20,49.54],right_bottom:[511.81,85.39]
class_id:1, confidence:0.6370, left_top:[446.48,281.86],right_bottom:[504.50,342.79]
class_id:1, confidence:0.5986, left_top:[450.84,89.30],right_bottom:[494.58,123.99]
class_id:1, confidence:0.5715, left_top:[510.54,89.15],right_bottom:[570.02,138.44]
class_id:1, confidence:0.5151, left_top:[512.79,94.07],right_bottom:[562.67,133.47]
save result to: output/c000418.jpg
Test iter 0
------------------ Inference Time Info ----------------------
total_time(ms): 1072.8, img_num: 1
average latency time(ms): 1072.80, QPS: 0.932140
preprocess_time(ms): 18.20, inference_time(ms): 1054.50, postprocess_time(ms): 0.10
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64

10 模型部署

本项目用EasyEdge端与边缘AI服务平台对水底生物目标检测模型进行部署。EasyEdge是基于百度飞桨轻量化推理框架Paddle Lite研发的端与边缘AI服务平台，能够帮助深度学习开发者将自建模型快速部署到设备端。只需上传模型，最快2分种即可获得适配终端硬件/芯片的模型。

10.1 上传本地模型

10.2 生成端到端模型

至此，本项目的一个完整流程就结束啦！如果对此感兴趣，不妨动起来呀！

请点击此处查看本环境基本用法.

Please click here for more detailed instructions.

此文章为搬运
原项目链接