10-10 Human Skeleton Keypoint Detection Sample Tutorial

In the previous chapter, 10-9, we used a face recognition model and feature comparison technology to successfully teach AI how to “recognize who someone is,” creating a dedicated VIP welcome system.

But if our application scenario is “medical fall detection” or “athlete posture analysis,” knowing “who the person is” is not the main point. What we really need to know is “what the person is doing right now”!

This chapter takes you into the starting point of behavior analysis : Human Skeleton Keypoint Detection ( Human Skeleton 17KPS ). We will expand the AI's focus from the “face” to the “whole body,” accurately extracting 17 human joint points, including facial points on the head, shoulders, elbows, wrists, hips, knees, and ankles. Even more exciting, we will challenge ourselves with advanced OpenCV drawing techniques and use code to “connect” these joints, drawing a high-tech digital skeleton.

Learning Objectives

Through this chapter, you will learn :

1. Perspective shift : Understand the development logic difference between “feature comparison” and “posture annotation.”

2. Build a skeleton dictionary : Implement a C++ Struct to define the human bone connection mapping table.

3. Apply probability values : Use the fProbability learned in the previous chapter for filtering, preventing skeleton lines from being drawn outside the image.

4. Advanced OpenCV drawing : Combine cv::circle for drawing joints and cv::line for drawing bone connections to complete a dynamic skeleton overlay

Preparation

This model package is slightly different from previous ones because it combines multiple recognition weight files, so there will be more files. Find the Human Skeleton 17KPS model archive, which may be named with HUMAN.SKELETON.17KPS, extract it, and copy QDEEP.OD.HUMAN.SKELETON.17KPS.EX.CFG and all corresponding weight files into the build output directory of your Qt project.

How Should the Core API Be Modified?

In 10-9, we enabled FLAG_FULL to extract face features. But in this chapter, we do not need to recognize who the person is. We only need full-body coordinates, so the API settings return to a simpler usage pattern, while the model configuration value changes significantly:

• Change the model configuration type : In the third parameter of QDEEP_CREATE_OBJECT_DETECT, switch to the skeleton-specific  QDEEP_OBJECT_DETECT_CONFIG_MODEL_HUMAN_SKELETON_17_KEYPOINTS_EX.

• Understand the 17KPS returned structure : Looking back at the previous chapter, the sKeypoints array only contained 5 points for the face. In this model, sKeypoints contains a full set of 17 points. These points follow a fixed order, such as 0 for nose, 5 for left shoulder, 6 for right shoulder, and so on. This time, fProbability, the confidence score inside the structure, will play an extremely important role.

QDEEP_CREATE_OBJECT_DETECT

This is the key API for creating the AI engine and loading the model. Users must initialize the detector through this API.

QDEEP_OBJECT_DETECT_BOUNDING_BOX Structure

After switching to the human skeleton keypoint model, the sKeypoints returned by the AI engine through QDEEP_OBJECT_DETECT_BOUNDING_BOX will have a new meaning.

QDEEP_OBJECT_DETECT_KEYPOINTStructure

An array named sKeypoints[ QDEEP_MAX_KEYPOINY_SIZE ] is unlocked inside QDEEP_OBJECT_DETECT_BOUNDING_BOX. Each element in this array is a structure specifically used to describe a single feature point : QDEEP_OBJECT_DETECT_KEYPOINT.

It contains four highly valuable parameters, defined as follows :

Writing the Core Code

Open your project, and let's make the key code changes.

Define the Bone Connection Dictionary

To draw a skeleton, we cannot randomly connect all 17 points. We must explicitly tell the program that “left shoulder connects to left elbow” and “left elbow connects to left wrist.” At the top of mainwindow.cpp, outside the loop, create a structure and an array to define these connection rules and their dedicated colors.
 

 

 

Modify Model Loading

In the constructor, apply the new skeleton model setting. This time, we do not need face recognition, so dwFlags can simply be set to trajectory tracking.
 

 

OpenCV Dynamic Line Drawing and Fail-Safe Logic

After successfully obtaining the structure returned by the AI, the next highlight is using OpenCV APIs to overlay the skeleton onto the image.

Here, we will use a “draw lines first, then points” approach, and use the continue early-skip technique in programming to implement an elegant fail-safe mechanism. The drawing SOP is as follows :

1. Draw the bone connections first : Use the bones dictionary we defined to draw lines between connected keypoints. There is one extremely important detail here : make sure to draw the lines first! If you draw circles first and then draw lines, the thicker bone lines will cover the delicate joint points, making the result look less three-dimensional.

2. Draw the joint circles using Early Return filtering : Use cv::circle to draw filled circles at each keypoint coordinate. To avoid drawing invalid points caused by occlusion, low confidence, or coordinates equal to 0, we implement continue filtering: if a point does not meet the rules, skip it and do not draw it.

Final Verification

Click “Build and RUN” in the lower-left corner to run the project :

1. Start receiving and detection.

2. Stand in front of the camera, or ask someone nearby to stand there, then wave, lift a leg, or do a few stretching movements.
 

 

➤ Friendly Reminder (Very Important) : The first time you open this software (run the program), the interface may take a little while to appear. Don't worry! This is because we placed the “AI model loading” logic inside the program constructor. During the first run, the model must perform initialization and configuration, and the system is loading the large neural network weight files into the GPU. Once the software opens successfully, it means the AI brain is ready in the background. The next time you open the software, this initialization wait will no longer be needed.

Congratulations! Starting from this step, the data you obtain is no longer just simple X and Y coordinates, but quantified data of “human posture.” These 17 keypoints are the core foundation for building Behavior Analysis. In the future, by further tracking how these keypoints change across continuous frames and over time, you can extend the application to commercial-grade advanced AI fields such as fall detection alerts for smart long-term care or posture analysis for sports training.