dl — deep learning

备注

当前模块只适用于 CoreS3 主机

UiFlow2 Example

人脸检测

Open the cores3_example_human_face_detect.m5f2 project in UiFlow2.

This example uses a face detection algorithm to detect faces in real time from the camera feed. When a face is detected, a bounding box is drawn on the screen to mark the face’s position, providing an intuitive visualization of the detection results.

UiFlow2 Code Block:

Example output:

None

行人检测

Open the cores3_example_pedestrian_detect.m5f2 project in UiFlow2.

This example uses a pedestrian detection algorithm to detect pedestrian targets in real time from the camera feed. When a pedestrian is detected, a bounding box is drawn on the screen to highlight the pedestrian’s position, providing an intuitive demonstration of the detection results.

UiFlow2 Code Block:

Example output:

None

人脸识别

Open the cores3_example_human_face_recognition.m5f2 project in UiFlow2.

To run this example, you will need the CoreS3 and the Unit Dual Button.

This example uses a face recognition algorithm to detect faces in real time from the camera feed.

By pressing different buttons, you can either enroll new face data or perform face recognition. The detected faces and recognition results are displayed on the screen with bounding boxes.

UiFlow2 Code Block:

Example output:

None

Micropython 案例

人脸检测

MicroPython Code Block:

# SPDX-FileCopyrightText: 2024 M5Stack Technology CO LTD
#
# SPDX-License-Identifier: MIT
import os, sys, io
import M5
from M5 import *
import camera
import dl
import image


img = None
detector = None
detection_result = None
res = None
kp = None


def setup():
    global img, detector, detection_result, kp
    M5.begin()
    Widgets.fillScreen(0x222222)
    camera.init(pixformat=camera.RGB565, framesize=camera.QVGA)
    detector = dl.ObjectDetector(dl.model.HUMAN_FACE_DETECT)


def loop():
    global img, detector, detection_result, kp
    M5.update()
    img = camera.snapshot()
    detection_result = detector.infer(img)
    if detection_result:
        for res in detection_result:
            kp = res.keypoint()
            img.draw_circle(kp[0], kp[1], 3, color=0x0000FF, thickness=1, fill=True)
            img.draw_circle(kp[2], kp[3], 3, color=0x00FF00, thickness=1, fill=True)
            img.draw_circle(kp[4], kp[5], 3, color=0xFF0000, thickness=1, fill=True)
            img.draw_circle(kp[6], kp[7], 3, color=0x0000FF, thickness=1, fill=True)
            img.draw_circle(kp[8], kp[9], 3, color=0x00FF00, thickness=1, fill=True)
            img.draw_rectangle(
                res.x(), res.y(), res.w(), res.h(), color=0x3366FF, thickness=3, fill=False
            )
    M5.Lcd.show(img, 0, 0, 320, 240)


if __name__ == "__main__":
    try:
        setup()
        while True:
            loop()
    except (Exception, KeyboardInterrupt) as e:
        try:
            from utility import print_error_msg

            print_error_msg(e)
        except ImportError:
            print("please update to latest firmware")

Example output:

None

行人检测

MicroPython Code Block:

# SPDX-FileCopyrightText: 2024 M5Stack Technology CO LTD
#
# SPDX-License-Identifier: MIT
import os, sys, io
import M5
from M5 import *
import camera
import dl
import image


img = None
detector = None
detection_result = None
res = None
kp = None


def setup():
    global img, detector, detection_result, kp
    M5.begin()
    Widgets.fillScreen(0x222222)
    camera.init(pixformat=camera.RGB565, framesize=camera.QVGA)
    detector = dl.ObjectDetector(dl.model.PEDESTRIAN_DETECT)


def loop():
    global img, detector, detection_result, kp
    M5.update()
    img = camera.snapshot()
    detection_result = detector.infer(img)
    if detection_result:
        for res in detection_result:
            kp = res.keypoint()
            img.draw_rectangle(
                res.x(), res.y(), res.w(), res.h(), color=0x3366FF, thickness=3, fill=False
            )
    M5.Lcd.show(img, 0, 0, 320, 240)


if __name__ == "__main__":
    try:
        setup()
        while True:
            loop()
    except (Exception, KeyboardInterrupt) as e:
        try:
            from utility import print_error_msg

            print_error_msg(e)
        except ImportError:
            print("please update to latest firmware")

Example output:

None

人脸识别

MicroPython Code Block:

# SPDX-FileCopyrightText: 2024 M5Stack Technology CO LTD
#
# SPDX-License-Identifier: MIT
import os, sys, io
import M5
from M5 import *
from unit import DualButtonUnit
from hardware import *
import camera
import dl
import image


dual_button_0_blue = None
dual_button_0_red = None
sys_state = None
FACE_RECOGNIZE = None
FACE_ENROLL = None
FACE_DELETE = None
detector = None
img = None
dl_recognizer = None
detection_result = None
IDLE = None
res = None
kp = None
sys_state_prev = None
frame_count = None
dl_recognize_result = None


def dual_button_0_blue_wasClicked_event(state):  # noqa: N802
    global \
        dual_button_0_blue, \
        dual_button_0_red, \
        sys_state, \
        FACE_RECOGNIZE, \
        FACE_ENROLL, \
        FACE_DELETE, \
        detector, \
        img, \
        dl_recognizer, \
        detection_result, \
        IDLE, \
        kp, \
        sys_state_prev, \
        frame_count, \
        res, \
        dl_recognize_result
    sys_state = FACE_RECOGNIZE


def dual_button_0_red_wasClicked_event(state):  # noqa: N802
    global \
        dual_button_0_blue, \
        dual_button_0_red, \
        sys_state, \
        FACE_RECOGNIZE, \
        FACE_ENROLL, \
        FACE_DELETE, \
        detector, \
        img, \
        dl_recognizer, \
        detection_result, \
        IDLE, \
        kp, \
        sys_state_prev, \
        frame_count, \
        res, \
        dl_recognize_result
    sys_state = FACE_ENROLL


def btnPWR_wasClicked_event(state):  # noqa: N802
    global \
        dual_button_0_blue, \
        dual_button_0_red, \
        sys_state, \
        FACE_RECOGNIZE, \
        FACE_ENROLL, \
        FACE_DELETE, \
        detector, \
        img, \
        dl_recognizer, \
        detection_result, \
        IDLE, \
        kp, \
        sys_state_prev, \
        frame_count, \
        res, \
        dl_recognize_result
    sys_state = FACE_DELETE


def setup():
    global \
        dual_button_0_blue, \
        dual_button_0_red, \
        sys_state, \
        FACE_RECOGNIZE, \
        FACE_ENROLL, \
        FACE_DELETE, \
        detector, \
        img, \
        dl_recognizer, \
        detection_result, \
        IDLE, \
        kp, \
        sys_state_prev, \
        frame_count, \
        res, \
        dl_recognize_result
    M5.begin()
    Widgets.fillScreen(0x222222)
    BtnPWR.setCallback(type=BtnPWR.CB_TYPE.WAS_CLICKED, cb=btnPWR_wasClicked_event)
    camera.init(pixformat=camera.RGB565, framesize=camera.QVGA)
    dual_button_0_blue, dual_button_0_red = DualButtonUnit((8, 9))
    dual_button_0_blue.setCallback(
        type=dual_button_0_blue.CB_TYPE.WAS_CLICKED, cb=dual_button_0_blue_wasClicked_event
    )
    dual_button_0_red.setCallback(
        type=dual_button_0_red.CB_TYPE.WAS_CLICKED, cb=dual_button_0_red_wasClicked_event
    )
    detector = dl.ObjectDetector(dl.model.HUMAN_FACE_DETECT)
    dl_recognizer = dl.HumanFaceRecognizer()
    IDLE = 0
    FACE_ENROLL = 1
    FACE_RECOGNIZE = 2
    FACE_DELETE = 3
    sys_state = IDLE
    sys_state_prev = IDLE
    frame_count = 0


def loop():
    global \
        dual_button_0_blue, \
        dual_button_0_red, \
        sys_state, \
        FACE_RECOGNIZE, \
        FACE_ENROLL, \
        FACE_DELETE, \
        detector, \
        img, \
        dl_recognizer, \
        detection_result, \
        IDLE, \
        kp, \
        sys_state_prev, \
        frame_count, \
        res, \
        dl_recognize_result
    M5.update()
    dual_button_0_blue.tick(None)
    dual_button_0_red.tick(None)
    img = camera.snapshot()
    detection_result = detector.infer(img)
    if detection_result:
        for res in detection_result:
            kp = res.keypoint()
            img.draw_string(10, 10, str("face"), color=0x3333FF, scale=1)
            img.draw_circle(kp[0], kp[1], 3, color=0x3333FF, thickness=1, fill=True)
            img.draw_circle(kp[2], kp[3], 3, color=0x33FF33, thickness=1, fill=True)
            img.draw_circle(kp[4], kp[5], 3, color=0xFF0000, thickness=1, fill=True)
            img.draw_circle(kp[6], kp[7], 3, color=0x3333FF, thickness=1, fill=True)
            img.draw_circle(kp[8], kp[9], 3, color=0x33FF33, thickness=1, fill=True)
            img.draw_rectangle(
                res.x(), res.y(), res.w(), res.h(), color=0x3366FF, thickness=3, fill=False
            )
    if sys_state == FACE_DELETE:
        dl_recognizer.delete_id()
        sys_state_prev = sys_state
        sys_state = IDLE
        frame_count = 15
    elif sys_state != IDLE:
        if detection_result:
            if len(detection_result) == 1:
                res = detection_result[0]
                if sys_state == FACE_ENROLL:
                    dl_recognizer.enroll_id(img, res.keypoint())
                elif sys_state == FACE_RECOGNIZE:
                    dl_recognize_result = dl_recognizer.recognize(img, res.keypoint())
                    if (dl_recognize_result.id()) > 0:
                        print((str("similarity: ") + str((dl_recognize_result.similarity()))))
                sys_state_prev = sys_state
                sys_state = IDLE
                frame_count = 15
        else:
            img.draw_string(104, 10, str("face no detect"), color=0xFF0000, scale=1)
    if frame_count > 0:
        frame_count = frame_count - 1
        if sys_state_prev == FACE_ENROLL:
            img.draw_string(116, 10, str("face enroll"), color=0x33FF33, scale=1)
        elif sys_state_prev == FACE_RECOGNIZE:
            if (dl_recognize_result.id()) > 0:
                img.draw_string(
                    100,
                    10,
                    str((str("recognize id: ") + str((dl_recognize_result.id())))),
                    color=0x33FF33,
                    scale=1,
                )
            else:
                img.draw_string(96, 10, str("no recognized"), color=0xFF0000, scale=1)
        elif sys_state_prev == FACE_DELETE:
            img.draw_string(
                100,
                10,
                str((str("remaining id: ") + str((dl_recognizer.enrolled_id_num())))),
                color=0xFF0000,
                scale=1,
            )
    M5.Lcd.show(img, 0, 0, 320, 240)


if __name__ == "__main__":
    try:
        setup()
        while True:
            loop()
    except (Exception, KeyboardInterrupt) as e:
        try:
            from utility import print_error_msg

            print_error_msg(e)
        except ImportError:
            print("please update to latest firmware")

Example output:

None

Funtions

dl.ObjectDetector(model) → ObjectDetector

Create an object detector instance.

参数:: model – 参数 model 仅接受以下值：

dl.model.HUMAN_FACE_DETECT 人脸检测
dl.model.PEDESTRIAN_DETECT 行人检测

返回 ObjectDetector 对象.

UiFlow2 Code Block:

Example:

detector = dl.ObjectDetector(dl.model.HUMAN_FACE_DETECT)
detector = dl.ObjectDetector(dl.model.PEDESTRIAN_DETECT)

dl.HumanFaceRecognizer() → HumanFaceRecognizer

创建一个人脸识别器

返回 HumanFaceRecognizer 对象.

UiFlow2 Code Block:

class ObjectDetector

ObjectDetector 对象由 dl.ObjectDetector(model) 返回

ObjectDetector.infer(img: image.Image) → DetectionResult

返回一个 DetectionResult 实例.

UiFlow2 Code Block:

class HumanFaceRecognizer

HumanFaceRecognizer 对象由 dl.HumanFaceRecognizer() 返回

HumanFaceRecognizer.recognize(img: image:Image, keypoint: tuple) → RecognitionResult

人脸识别

img 输入图像
keypoint 人脸关键点数据，详情参考 DetectionResult.keypoint() 解析

返回 RecognitionResult 对象。

UiFlow2 Code Block:

HumanFaceRecognizer.clear_id()

清空所有 id

UiFlow2 Code Block:

HumanFaceRecognizer.enroll_id(img: image:Image, keypoint: tuple) → bool

enroll id

img 输入图像
keypoint 人脸关键点数据，详情参考 DetectionResult.keypoint() 解析

UiFlow2 Code Block:

HumanFaceRecognizer.delete_id([id])

delete id

id 为可选参数，输入 id 为删除指定的人脸信息，默认为删除最近一次录入的 id。

UiFlow2 Code Block:

HumanFaceRecognizer.enrolled_id_num() → int

返回已录入的 id 数

UiFlow2 Code Block:

class DetectionResult – DetectionResult object

DetectionResult 对象由 ObjectDetector.infer(img) 返回

DetectionResult.bbox()

获取目标检测的边界框。

UiFlow2 Code Block:

DetectionResult.x() → int

获取边界框的左上角坐标 x。

UiFlow2 Code Block:

DetectionResult.y() → int

获取边界框的左上角坐标 y。

UiFlow2 Code Block:

DetectionResult.w() → int

获取边界框的宽度。

UiFlow2 Code Block:

DetectionResult.h() → int

获取边界框的高度。

UiFlow2 Code Block:

DetectionResult.category() → int

检测到的目标类别。

UiFlow2 Code Block:

DetectionResult.keypoint() → tuple

关键点信息(当前只有人脸检测模型输出包含此数据)

keypoint()[0], keypoint()[1] 为左眼坐标
keypoint()[2], keypoint()[3] 为嘴巴左角坐标
keypoint()[4], keypoint()[5] 为鼻子坐标
keypoint()[6], keypoint()[7] 为右眼坐标
keypoint()[8], keypoint()[9] 嘴巴右角

UiFlow2 Code Block:

class RecognitionResult – RecognitionResult object

DetectionResult 对象由 HumanFaceRecognizer.recognize(img, keypoint) 返

RecognitionResult.similarity() → float

获取人脸相似度，越接近1表示相似度越高。

UiFlow2 Code Block:

RecognitionResult.id() → int

获取人脸id，大于0表示人脸识别成功。

UiFlow2 Code Block: