Extract Gesture Embedding

This function extracts a compact feature vector (embedding) from hand landmarks for gesture classification.

Key Implementation

Function: mediapipe_style_gesture_processing in hand_landmark.py

def mediapipe_style_gesture_processing(region, lm_results, gesture_model, classifier_model, gesture_labels_list):
    """Extract gesture embedding and classify using MediaPipe-style logic."""
    if not hasattr(region, 'landmarks') or len(region.landmarks) != 21:
        region.gesture_name = "None"
        region.gesture_confidence = 0.0
        return None
    # Extract hand_landmarks (normalized [0,1] relative to crop)
    hand_landmarks_normalized = np.array(region.landmarks, dtype=np.float32).reshape(1, 21, 3)
    handedness = np.array([[region.handedness]], dtype=np.float32)
    # Extract world landmarks (Identity_3 from landmark model)
    if 'Identity_3' in lm_results:
        world_landmarks_raw = np.squeeze(lm_results['Identity_3'])
    else:
        world_landmarks_raw = None
    try:
        # Pass to gesture embedder model
        embedding = gesture_model([hand_landmarks_normalized, handedness, world_landmarks_raw])
        region.gesture_embedding = embedding
        return embedding
    except Exception as e:
        print(f"Error in gesture embedding extraction: {e}")
        return None

• Inputs:
  • hand: [1, 21, 3] (normalized landmarks)
  • handedness: [1, 1]
  • world_hand: [1, 21, 3]
• Output: [1, 128] gesture embedding

The embedding is stored in the HandRegion object and used for gesture classification and analysis.