MediaPipe結合Grasshopper進行互動設計

互動裝置

現階段Mediapipe的實務應用

姿勢追蹤與改變感知的關係

實作與測試

# source: https://steam.oxxostudio.tw/category/python/ai/ai-mediapipe-pose.html
# source: https://github.com/yucheno/opencv2gh/blob/master/opencv2GH.py
# 按 q 跳出
import mediapipe as mp
import cv2
# 設定 UDPsocket 與 pickle(專門用於python的json)
import socket
import json
def UDP_client(IP, port, message):
    sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
    sock.sendto(bytes(message, 'utf-8'), (IP, port))
設定銜接端口，銜接ghowl，請調整成自己的IP
IP = "127.0.0.1"
port = 5003
# 輸出資料
# 偵測骨架 並輸出 socket 等待接收
mp_drawing = mp.solutions.drawing_utils          # mediapipe 繪圖方法
mp_drawing_styles = mp.solutions.drawing_styles  # mediapipe 繪圖樣式
mp_pose = mp.solutions.pose                      # mediapipe 姿勢偵測
cap = cv2.VideoCapture(0)
#以上cap = cv2.VideoCapture(0)需視鏡頭位置調整(0)(1)(2)
# 啟用姿勢偵測
with mp_pose.Pose(
        min_detection_confidence=0.5,
        min_tracking_confidence=0.5) as pose:
    if not cap.isOpened():
        print("Cannot open camera")
        exit()
    while True:
        ret, img = cap.read()
        if not ret:
            print("Cannot receive frame")
            break
        # webcam 為 4:3 縮小尺寸，加快演算速度
        img = cv2.resize(img, (640, 480))
        img2 = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)   # 將 BGR 轉換成 RGB
        results = pose.process(img2)                  # 取得姿勢偵測結果
        # 根據姿勢偵測結果，標記身體節點和骨架
        mp_drawing.draw_landmarks(
            img,
            results.pose_landmarks,
            mp_pose.POSE_CONNECTIONS,
            landmark_drawing_spec=mp_drawing_styles.get_default_pose_landmarks_style())
        # 使用 pickle 匯出為字串
        all = {'data': []}
        if results.pose_landmarks:
            for id, lm in enumerate(results.pose_landmarks.landmark):
                all['data'].append({
                    'x': lm.x,
                    'y': lm.y,
                    'z': lm.z,
                })
        outputjson = json.dumps(all, indent=4)
        UDP_client(IP, port, outputjson)
        cv2.imshow('oxxostudio', img)
        if cv2.waitKey(5) == ord('q'):
            break     # 按下 q 鍵停止
cap.release()
cv2.destroyAllWindows()

• Fortes (2022). "Deep Learning based Human Pose Estimation using OpenCV and MediaPipe." from https://medium.com/nerd-for-tech/deep-learning-based-human-pose-estimation-using-opencv-and-mediapipe-d0be7a834076.
• Jong-Wook KimORCID, J.-Y. C., Eun-Ju Ha,Jae-Ho Choi (2023). "Human Pose Estimation Using MediaPipe Pose and Optimization Method Based on a Humanoid Model." from https://www.mdpi.com/2076-3417/13/4/2700.
• Körner, R. (2022). "Standing tall and standing wide: Body positions have effects on how people feel." AMERICAN PSYCHOLOGICAL ASSOCIATION.
• Vasileios Moysiadis, D. K., Lefteris Benos,,Patrizia Busato ,Athanasios Anagnostis,Dimitrios Kateris ,Simon Pearson (2022). "An Integrated Real-Time Hand Gesture Recognition Framework for Human–Robot Interaction in Agriculture." applied science 12: 17.
• 法蘭斯 (2023). "Mediapipe Holistic全身辨識體感玩超級瑪莉." from https://www.youtube.com/watch?v=1ZjmQEVsHdE.
• 教育學習網, S. "Mediapipe 全身偵測 ( Holistic )." from https://steam.oxxostudio.tw/category/python/ai/ai-mediapipe-holistic.html.
• 課程學習：TKU=建築系碩士班=BIM工程整合應用實務 授課老師：蕭吉甫