dl — 深度学习

备注

当前模块只适用于 CoreS3 主机

UiFlow2 应用示例

人脸检测

在 UiFlow2 中打开 cores3_example_human_face_detect.m5f2 项目。

该示例使用人脸检测算法，从摄像头画面中实时检测人脸。当检测到人脸时，会在屏幕上绘制一个边界框以标记人脸的位置，从而直观地显示检测结果。

UiFlow2 代码块：

示例输出：

None

行人检测

在 UiFlow2 中打开 cores3_example_pedestrian_detect.m5f2 项目。

该示例使用行人检测算法从摄像头画面中实时检测行人目标。当检测到行人时，会在屏幕上绘制边界框以突出显示行人位置，从而直观展示检测结果。

UiFlow2 代码块：

示例输出：

None

人脸识别

在 UiFlow2 中打开 cores3_example_human_face_recognition.m5f2 项目。

要运行该示例，你需要 CoreS3 和 Unit Dual Button。

该示例使用人脸识别算法从摄像头画面中实时检测人脸。

通过按下不同的按钮，你可以录入新的人脸数据或进行人脸识别。检测到的人脸和识别结果会以边界框的形式显示在屏幕上。

UiFlow2 代码块：

示例输出：

None

Micropython 案例

人脸检测

MicroPython 代码块：

# SPDX-FileCopyrightText: 2024 M5Stack Technology CO LTD
#
# SPDX-License-Identifier: MIT
import os, sys, io
import M5
from M5 import *
import camera
import dl
import image


img = None
detector = None
detection_result = None
res = None
kp = None


def setup():
    global img, detector, detection_result, kp
    M5.begin()
    Widgets.fillScreen(0x222222)
    camera.init(pixformat=camera.RGB565, framesize=camera.QVGA)
    detector = dl.ObjectDetector(dl.model.HUMAN_FACE_DETECT)


def loop():
    global img, detector, detection_result, kp
    M5.update()
    img = camera.snapshot()
    detection_result = detector.infer(img)
    if detection_result:
        for res in detection_result:
            kp = res.keypoint()
            img.draw_circle(kp[0], kp[1], 3, color=0x0000FF, thickness=1, fill=True)
            img.draw_circle(kp[2], kp[3], 3, color=0x00FF00, thickness=1, fill=True)
            img.draw_circle(kp[4], kp[5], 3, color=0xFF0000, thickness=1, fill=True)
            img.draw_circle(kp[6], kp[7], 3, color=0x0000FF, thickness=1, fill=True)
            img.draw_circle(kp[8], kp[9], 3, color=0x00FF00, thickness=1, fill=True)
            img.draw_rectangle(
                res.x(), res.y(), res.w(), res.h(), color=0x3366FF, thickness=3, fill=False
            )
    M5.Lcd.show(img, 0, 0, 320, 240)


if __name__ == "__main__":
    try:
        setup()
        while True:
            loop()
    except (Exception, KeyboardInterrupt) as e:
        try:
            from utility import print_error_msg

            print_error_msg(e)
        except ImportError:
            print("please update to latest firmware")

示例输出：

None

行人检测

MicroPython 代码块：

# SPDX-FileCopyrightText: 2024 M5Stack Technology CO LTD
#
# SPDX-License-Identifier: MIT
import os, sys, io
import M5
from M5 import *
import camera
import dl
import image


img = None
detector = None
detection_result = None
res = None
kp = None


def setup():
    global img, detector, detection_result, kp
    M5.begin()
    Widgets.fillScreen(0x222222)
    camera.init(pixformat=camera.RGB565, framesize=camera.QVGA)
    detector = dl.ObjectDetector(dl.model.PEDESTRIAN_DETECT)


def loop():
    global img, detector, detection_result, kp
    M5.update()
    img = camera.snapshot()
    detection_result = detector.infer(img)
    if detection_result:
        for res in detection_result:
            kp = res.keypoint()
            img.draw_rectangle(
                res.x(), res.y(), res.w(), res.h(), color=0x3366FF, thickness=3, fill=False
            )
    M5.Lcd.show(img, 0, 0, 320, 240)


if __name__ == "__main__":
    try:
        setup()
        while True:
            loop()
    except (Exception, KeyboardInterrupt) as e:
        try:
            from utility import print_error_msg

            print_error_msg(e)
        except ImportError:
            print("please update to latest firmware")

示例输出：

None

人脸识别

MicroPython 代码块：

# SPDX-FileCopyrightText: 2024 M5Stack Technology CO LTD
#
# SPDX-License-Identifier: MIT
import os, sys, io
import M5
from M5 import *
from unit import DualButtonUnit
from hardware import *
import camera
import dl
import image


dual_button_0_blue = None
dual_button_0_red = None
sys_state = None
FACE_RECOGNIZE = None
FACE_ENROLL = None
FACE_DELETE = None
detector = None
img = None
dl_recognizer = None
detection_result = None
IDLE = None
res = None
kp = None
sys_state_prev = None
frame_count = None
dl_recognize_result = None


def dual_button_0_blue_wasClicked_event(state):  # noqa: N802
    global \
        dual_button_0_blue, \
        dual_button_0_red, \
        sys_state, \
        FACE_RECOGNIZE, \
        FACE_ENROLL, \
        FACE_DELETE, \
        detector, \
        img, \
        dl_recognizer, \
        detection_result, \
        IDLE, \
        kp, \
        sys_state_prev, \
        frame_count, \
        res, \
        dl_recognize_result
    sys_state = FACE_RECOGNIZE


def dual_button_0_red_wasClicked_event(state):  # noqa: N802
    global \
        dual_button_0_blue, \
        dual_button_0_red, \
        sys_state, \
        FACE_RECOGNIZE, \
        FACE_ENROLL, \
        FACE_DELETE, \
        detector, \
        img, \
        dl_recognizer, \
        detection_result, \
        IDLE, \
        kp, \
        sys_state_prev, \
        frame_count, \
        res, \
        dl_recognize_result
    sys_state = FACE_ENROLL


def btnPWR_wasClicked_event(state):  # noqa: N802
    global \
        dual_button_0_blue, \
        dual_button_0_red, \
        sys_state, \
        FACE_RECOGNIZE, \
        FACE_ENROLL, \
        FACE_DELETE, \
        detector, \
        img, \
        dl_recognizer, \
        detection_result, \
        IDLE, \
        kp, \
        sys_state_prev, \
        frame_count, \
        res, \
        dl_recognize_result
    sys_state = FACE_DELETE


def setup():
    global \
        dual_button_0_blue, \
        dual_button_0_red, \
        sys_state, \
        FACE_RECOGNIZE, \
        FACE_ENROLL, \
        FACE_DELETE, \
        detector, \
        img, \
        dl_recognizer, \
        detection_result, \
        IDLE, \
        kp, \
        sys_state_prev, \
        frame_count, \
        res, \
        dl_recognize_result
    M5.begin()
    Widgets.fillScreen(0x222222)
    BtnPWR.setCallback(type=BtnPWR.CB_TYPE.WAS_CLICKED, cb=btnPWR_wasClicked_event)
    camera.init(pixformat=camera.RGB565, framesize=camera.QVGA)
    dual_button_0_blue, dual_button_0_red = DualButtonUnit((8, 9))
    dual_button_0_blue.setCallback(
        type=dual_button_0_blue.CB_TYPE.WAS_CLICKED, cb=dual_button_0_blue_wasClicked_event
    )
    dual_button_0_red.setCallback(
        type=dual_button_0_red.CB_TYPE.WAS_CLICKED, cb=dual_button_0_red_wasClicked_event
    )
    detector = dl.ObjectDetector(dl.model.HUMAN_FACE_DETECT)
    dl_recognizer = dl.HumanFaceRecognizer()
    IDLE = 0
    FACE_ENROLL = 1
    FACE_RECOGNIZE = 2
    FACE_DELETE = 3
    sys_state = IDLE
    sys_state_prev = IDLE
    frame_count = 0


def loop():
    global \
        dual_button_0_blue, \
        dual_button_0_red, \
        sys_state, \
        FACE_RECOGNIZE, \
        FACE_ENROLL, \
        FACE_DELETE, \
        detector, \
        img, \
        dl_recognizer, \
        detection_result, \
        IDLE, \
        kp, \
        sys_state_prev, \
        frame_count, \
        res, \
        dl_recognize_result
    M5.update()
    dual_button_0_blue.tick(None)
    dual_button_0_red.tick(None)
    img = camera.snapshot()
    detection_result = detector.infer(img)
    if detection_result:
        for res in detection_result:
            kp = res.keypoint()
            img.draw_string(10, 10, str("face"), color=0x3333FF, scale=1)
            img.draw_circle(kp[0], kp[1], 3, color=0x3333FF, thickness=1, fill=True)
            img.draw_circle(kp[2], kp[3], 3, color=0x33FF33, thickness=1, fill=True)
            img.draw_circle(kp[4], kp[5], 3, color=0xFF0000, thickness=1, fill=True)
            img.draw_circle(kp[6], kp[7], 3, color=0x3333FF, thickness=1, fill=True)
            img.draw_circle(kp[8], kp[9], 3, color=0x33FF33, thickness=1, fill=True)
            img.draw_rectangle(
                res.x(), res.y(), res.w(), res.h(), color=0x3366FF, thickness=3, fill=False
            )
    if sys_state == FACE_DELETE:
        dl_recognizer.delete_id()
        sys_state_prev = sys_state
        sys_state = IDLE
        frame_count = 15
    elif sys_state != IDLE:
        if detection_result:
            if len(detection_result) == 1:
                res = detection_result[0]
                if sys_state == FACE_ENROLL:
                    dl_recognizer.enroll_id(img, res.keypoint())
                elif sys_state == FACE_RECOGNIZE:
                    dl_recognize_result = dl_recognizer.recognize(img, res.keypoint())
                    if (dl_recognize_result.id()) > 0:
                        print((str("similarity: ") + str((dl_recognize_result.similarity()))))
                sys_state_prev = sys_state
                sys_state = IDLE
                frame_count = 15
        else:
            img.draw_string(104, 10, str("face no detect"), color=0xFF0000, scale=1)
    if frame_count > 0:
        frame_count = frame_count - 1
        if sys_state_prev == FACE_ENROLL:
            img.draw_string(116, 10, str("face enroll"), color=0x33FF33, scale=1)
        elif sys_state_prev == FACE_RECOGNIZE:
            if (dl_recognize_result.id()) > 0:
                img.draw_string(
                    100,
                    10,
                    str((str("recognize id: ") + str((dl_recognize_result.id())))),
                    color=0x33FF33,
                    scale=1,
                )
            else:
                img.draw_string(96, 10, str("no recognized"), color=0xFF0000, scale=1)
        elif sys_state_prev == FACE_DELETE:
            img.draw_string(
                100,
                10,
                str((str("remaining id: ") + str((dl_recognizer.enrolled_id_num())))),
                color=0xFF0000,
                scale=1,
            )
    M5.Lcd.show(img, 0, 0, 320, 240)


if __name__ == "__main__":
    try:
        setup()
        while True:
            loop()
    except (Exception, KeyboardInterrupt) as e:
        try:
            from utility import print_error_msg

            print_error_msg(e)
        except ImportError:
            print("please update to latest firmware")

示例输出：

None

Funtions

dl.ObjectDetector(model) → ObjectDetector

创建一个对象检测器实例。

参数:: model – 参数 model 仅接受以下值：

dl.model.HUMAN_FACE_DETECT 人脸检测
dl.model.PEDESTRIAN_DETECT 行人检测

返回 ObjectDetector 对象.

UiFlow2 代码块：

示例:

detector = dl.ObjectDetector(dl.model.HUMAN_FACE_DETECT)
detector = dl.ObjectDetector(dl.model.PEDESTRIAN_DETECT)

dl.HumanFaceRecognizer() → HumanFaceRecognizer

创建一个人脸识别器

返回 HumanFaceRecognizer 对象.

UiFlow2 代码块：

class ObjectDetector

ObjectDetector 对象由 dl.ObjectDetector(model) 返回

ObjectDetector.infer(img: image.Image) → DetectionResult

返回一个 DetectionResult 实例.

UiFlow2 代码块：

class HumanFaceRecognizer

HumanFaceRecognizer 对象由 dl.HumanFaceRecognizer() 返回

HumanFaceRecognizer.recognize(img: image:Image, keypoint: tuple) → RecognitionResult

人脸识别

img 输入图像
keypoint 人脸关键点数据，详情参考 DetectionResult.keypoint() 解析

返回 RecognitionResult 对象。

UiFlow2 代码块：

HumanFaceRecognizer.clear_id()

清空所有 id

UiFlow2 代码块：

HumanFaceRecognizer.enroll_id(img: image:Image, keypoint: tuple) → bool

注册 ID。

img 输入图像
keypoint 人脸关键点数据，详情参考 DetectionResult.keypoint() 解析

UiFlow2 代码块：

HumanFaceRecognizer.delete_id([id])

删除 id。

id 是一个可选参数。如果提供此参数，则会删除指定的面部信息。默认情况下，它会删除最近记录的 id。

UiFlow2 代码块：

HumanFaceRecognizer.enrolled_id_num() → int

返回已录入的 id 数

UiFlow2 代码块：

class DetectionResult —— DetectionResult 对象

DetectionResult 对象由 ObjectDetector.infer(img) 返回

DetectionResult.bbox()

获取目标检测的边界框。

UiFlow2 代码块：

DetectionResult.x() → int

获取边界框的左上角坐标 x。

UiFlow2 代码块：

DetectionResult.y() → int

获取边界框的左上角坐标 y。

UiFlow2 代码块：

DetectionResult.w() → int

获取边界框的宽度。

UiFlow2 代码块：

DetectionResult.h() → int

获取边界框的高度。

UiFlow2 代码块：

DetectionResult.category() → int

检测到的目标类别。

UiFlow2 代码块：

DetectionResult.keypoint() → tuple

关键点信息（目前只有人脸检测模型输出此数据）：

keypoint()[0], keypoint()[1] 为左眼坐标
keypoint()[2], keypoint()[3] 为嘴巴左角坐标
keypoint()[4], keypoint()[5] 为鼻子坐标
keypoint()[6], keypoint()[7] 为右眼坐标
keypoint()[8], keypoint()[9] 嘴巴右角

UiFlow2 代码块：

class RecognitionResult – RecognitionResult 对象

RecognitionResult 由 HumanFaceRecognizer.recognize(img, keypoint) 返回。

RecognitionResult.similarity() → float

计算人脸相似度，数值越接近 1 表示相似度越高。

UiFlow2 代码块：

RecognitionResult.id() → int

获取人脸ID。大于0的值表示人脸识别成功。

UiFlow2 代码块：