香橙派5 RK3588 yolov5模型转换rknn及部署踩坑全记录 orangepi 5
零、写在前面
���于距离写这篇文章过去很久，有的部分，官方已更新，请多结合其他人的看，并多琢磨、讨论~
另外打个小广告：博客 https://blog.vrxiaojie.top/ 欢迎大家前来做客玩耍，提出问题~~
以后的文章都会在博客发布了，CSDN这边可能这是最后一篇文章。
一、烧录系统
1.准备工作：
(1) 使用官方提供的Ubuntu镜像：
Orange Pi 5系统下载（我使用Orangepi5_1.1.4_ubuntu_jammy_desktop_gnome_linux5.10.110）
(2) 下载 balenaEtcher 用于烧录系统
(3) 准备一个大于等于32GB的TF卡、一个读卡器。
2.打开软件，根据软件步骤烧录即可
完成后，插上TF卡、键盘、鼠标，上电开机。
二、在开发板上的准备工作
1. 在终端查看ip，并用ssh连接
香橙派接显示器，与电脑连接同一个网络，
使用 ip address查看ip
使用任意一个ssh软件，连接
默认的用户名有: root、orangepi，它们的默认密码都是orangepi
2. 安装ftp服务器，并连接
安装
sudo apt-get install vsftpd
修改配置文件
sudo vim /etc/vsftpd.conf
local_enable=YES
write_enable=YES
并确认它们前面没有井号(vim操作：按i键进入编辑模式，esc后输入:wq退出保存)
重启ftp服务
sudo /etc/init.d/vsftpd restart
filezilla设置
主机：香橙派的ip
用户名:orangepi
密码: orangepi
三、pc端的准备工作
1.准备一台装有Windows10及以上系统的电脑，并用WSL装Ubuntu22.04系统
WSl2安装过程参考微软官方教程
请自己手动配置好Ubuntu的运行环境以及conda，这里不多赘述。
2.下载rknn_toolkit2的所有源码 github页面 并解压
3.下载 yolov5官方源码，并训练模型
为节省时间，在这里先不自己训练了，就使用预训练权重yolov5n.pt
4.将pt转换成onnx格式
首先，根据官方文档的要求，在训练完成模型之后，再修改models/yolo.py Detect类下的forward函数
也就是修改为
def forward(self, x):
z = [] # inference output
for i in range(self.nl):
x[i] = self.m[i](x[i]) # conv
return x
然后，在yolov5目录下，打开终端输入命令
python export.py --weights yolov5n.pt --data data/coco128.yaml --include onnx --opset 12 --batch-size 1
注意注意！opset一定要为12，不然后面onnx转rknn会报错。weights自己选你训练完成的best.pt，data选你自己设置的，我这里只是做一个最简单情况的演示。
这里我遇到一个错误:
line 715, in run
shape = tuple((y[0] if isinstance(y, tuple) else y).shape) # model output shape
AttributeError: ‘list’ object has no attribute ‘shape’
解决方法
找到这一行，修改为
shape = tuple(y[0].shape) # model output shape
程序运行结束后，会在当前文件夹下生成一个yolov5n.onnx文件，待会要用。
5. onnx转化为rknn格式
进入你的Ubuntu系统，用conda创建一个新的环境
conda create -name rknn python=3.8
激活这个环境
conda activate rknn
将整个rknn_toolkit2源码解压后的文件夹复制进Ubuntu的 /home/用户名目录下，我将它重命名为rknn。
进入doc目录，使用 pip 安装依赖:
pip install -r requirements-cpu-ubuntu20.04_py38.txt -i https://mirror.baidu.com/pypi/simple
返回上一级目录，然后进入packages目录，安装rknn_toolkit2
pip install rknn_toolkit2-1.4.0_22dcfef4-cp38-cp38-linux_x86_64.whl
完成后，输入命令 python
from rknn.api import RKNN
如图所示，如果没有报错，说明安装成功，使用键盘Ctrl+Z退出此模式。
接下来，把(三、4)步所得到的onnx文件放入rknn/examples/onnx/yolov5文件夹下，终端里进入该文件夹。
用你喜欢的编辑器修改 test.py里面的一些内容，具体位置如图所示
-
onnx和要生成的rknn的文件名
-
classes改成你模型的类型，我这里是官方的所以默认
-
增加target_platform=‘rk3588’，根据你的板子型号自行调整
执行
python test.py
成功运行后的内容大概是这样的
此时这个目录下也会生成一个yolov5n.rknn文件，待会要用。
四、在香橙派上的部署
1.创建一个名为deploy.py的文件
内容如下
import numpy as np
import cv2
from rknnlite.api import RKNNLite
RKNN_MODEL = 'yolov5n.rknn'
QUANTIZE_ON = True
OBJ_THRESH = 0.25
NMS_THRESH = 0.45
IMG_SIZE = 640
CLASSES = ("person", "bicycle", "car", "motorbike ", "aeroplane ", "bus ", "train", "truck ", "boat", "traffic light",
"fire hydrant", "stop sign ", "parking meter", "bench", "bird", "cat", "dog ", "horse ", "sheep", "cow", "elephant",
"bear", "zebra ", "giraffe", "backpack", "umbrella", "handbag", "tie", "suitcase", "frisbee", "skis", "snowboard", "sports ball", "kite",
"baseball bat", "baseball glove", "skateboard", "surfboard", "tennis racket", "bottle", "wine glass", "cup", "fork", "knife ",
"spoon", "bowl", "banana", "apple", "sandwich", "orange", "broccoli", "carrot", "hot dog", "pizza ", "donut", "cake", "chair", "sofa",
"pottedplant", "bed", "diningtable", "toilet ", "tvmonitor", "laptop ", "mouse ", "remote ", "keyboard ", "cell phone", "microwave ",
"oven ", "toaster", "sink", "refrigerator ", "book", "clock", "vase", "scissors ", "teddy bear ", "hair drier", "toothbrush ")
def sigmoid(x):
return 1 / (1 + np.exp(-x))
def xywh2xyxy(x):
# Convert [x, y, w, h] to [x1, y1, x2, y2]
y = np.copy(x)
y[:, 0] = x[:, 0] - x[:, 2] / 2 # top left x
y[:, 1] = x[:, 1] - x[:, 3] / 2 # top left y
y[:, 2] = x[:, 0] + x[:, 2] / 2 # bottom right x
y[:, 3] = x[:, 1] + x[:, 3] / 2 # bottom right y
return y
def process(input, mask, anchors):
anchors = [anchors[i] for i in mask]
grid_h, grid_w = map(int, input.shape[0:2])
box_confidence = sigmoid(input[..., 4])
box_confidence = np.expand_dims(box_confidence, axis=-1)
box_class_probs = sigmoid(input[..., 5:])
box_xy = sigmoid(input[..., :2])*2 - 0.5
col = np.tile(np.arange(0, grid_w), grid_w).reshape(-1, grid_w)
row = np.tile(np.arange(0, grid_h).reshape(-1, 1), grid_h)
col = col.reshape(grid_h, grid_w, 1, 1).repeat(3, axis=-2)
row = row.reshape(grid_h, grid_w, 1, 1).repeat(3, axis=-2)
grid = np.concatenate((col, row), axis=-1)
box_xy += grid
box_xy *= int(IMG_SIZE/grid_h)
box_wh = pow(sigmoid(input[..., 2:4])*2, 2)
box_wh = box_wh * anchors
box = np.concatenate((box_xy, box_wh), axis=-1)
return box, box_confidence, box_class_probs
def filter_boxes(boxes, box_confidences, box_class_probs):
"""Filter boxes with box threshold. It's a bit different with origin yolov5 post process!
# Arguments
boxes: ndarray, boxes of objects.
box_confidences: ndarray, confidences of objects.
box_class_probs: ndarray, class_probs of objects.
# Returns
boxes: ndarray, filtered boxes.
classes: ndarray, classes for boxes.
scores: ndarray, scores for boxes.
"""
boxes = boxes.reshape(-1, 4)
box_confidences = box_confidences.reshape(-1)
box_class_probs = box_class_probs.reshape(-1, box_class_probs.shape[-1])
_box_pos = np.where(box_confidences >= OBJ_THRESH)
boxes = boxes[_box_pos]
box_confidences = box_confidences[_box_pos]
box_class_probs = box_class_probs[_box_pos]
class_max_score = np.max(box_class_probs, axis=-1)
classes = np.argmax(box_class_probs, axis=-1)
_class_pos = np.where(class_max_score >= OBJ_THRESH)
boxes = boxes[_class_pos]
classes = classes[_class_pos]
scores = (class_max_score* box_confidences)[_class_pos]
return boxes, classes, scores
def nms_boxes(boxes, scores):
"""Suppress non-maximal boxes.
# Arguments
boxes: ndarray, boxes of objects.
scores: ndarray, scores of objects.
# Returns
keep: ndarray, index of effective boxes.
"""
x = boxes[:, 0]
y = boxes[:, 1]
w = boxes[:, 2] - boxes[:, 0]
h = boxes[:, 3] - boxes[:, 1]
areas = w * h
order = scores.argsort()[::-1]
keep = []
while order.size > 0:
i = order[0]
keep.append(i)
xx1 = np.maximum(x[i], x[order[1:]])
yy1 = np.maximum(y[i], y[order[1:]])
xx2 = np.minimum(x[i] + w[i], x[order[1:]] + w[order[1:]])
yy2 = np.minimum(y[i] + h[i], y[order[1:]] + h[order[1:]])
w1 = np.maximum(0.0, xx2 - xx1 + 0.00001)
h1 = np.maximum(0.0, yy2 - yy1 + 0.00001)
inter = w1 * h1
ovr = inter / (areas[i] + areas[order[1:]] - inter)
inds = np.where(ovr