RF-DETR Keypoint: seventeen joints per person from one photo, and where to run it

Apache 2.0 through Roboflow's open-source package, so the weights are free and the terms are permissive. At 40.7 million parameters self-hosting means PyTorch, the rfdetr Python package, and a GPU if you want it running on live video. In your browser it runs on CNAPS Studio, included in the basic plan. One caveat belongs in this paragraph rather than buried later: this checkpoint is labelled a preview, so both its behaviour and its accuracy may change as it matures.

A camera sees a person as pixels rather than a body with joints that bend. This model finds every person in a photo and pins down where their joints are: the nose, both eyes and ears, shoulders, elbows, wrists, hips, knees and ankles, seventeen points each. It hands back two things at once, a skeleton picture and a list of coordinates, which matters more than it sounds like because those two outputs feed completely different next steps.

The cost, beyond the preview label, is that it only knows people. There is no option to track a joint on an animal or a machine. One pose estimation model sits in the CNAPS Studio catalog and this is it, which makes the comparison section shorter than usual.

What it actually is

The same architecture as the rest of the RF-DETR family, with a different head bolted on the end. A vision transformer backbone reads the image, a projector merges what it saw at several scales, a deformable decoder finds people in a single pass with no candidate boxes and no cleanup step, and a keypoint head predicts joint positions for each person found.

The seventeen joints are the standard COCO layout, in a fixed order: nose, left and right eye, left and right ear, then shoulders, elbows, wrists, hips, knees and ankles. Multi-person scenes are handled directly, with a separate skeleton per person, and a minimum confidence score from 0 to 1, defaulting to 0.5, decides who counts as detected.

The two outputs are the useful part. The skeleton image is a stick figure on a black background, coloured by the widely used OpenPose convention, which is the format pose-guided image generation expects. The text list gives one line per person with a box and a score, then one line per joint with coordinates and its own confidence, so a clearly visible wrist and an ankle hidden behind a table do not carry equal weight.

The published numbers

Roboflow reports 71.8 average precision on COCO keypoints, out of 100. That measure is not the same as the one used for boxes: it scores how close each predicted joint is to the true one, scaled by how big the person is in the frame, which is the standard way keypoints are graded. It runs at 9.7 milliseconds per image, measured on a small inference GPU with an optimised runtime and one image at a time.

That is roughly twice the cost of the family's plain detector, the price of finding joints rather than just people, and still fast enough for live video.

Two credibility notes. Roboflow measured this in-house with the same tooling used for every other model in its comparisons, better than each vendor quoting its own paper, though still the vendor of this model measuring. And the preview label matters: a checkpoint expected to change has a published score with a short shelf life. The family paper was accepted at a major conference for 2026, so the method is reviewed even if this head is new.

How it compares to the others here

There is nothing to compare it against inside the catalog, because it is the only pose estimation model we carry. Its two siblings answer different questions on the same backbone: RF-DETR Medium returns boxes over 80 kinds of object, RF-DETR Seg Medium returns pixel outlines for the same list. Neither returns joints.

So the choice is what you need out. Reach for RF-DETR Keypoint when the question is how a body is arranged: form in a golf swing, symmetry in a rehab session, a pose to drive an image generator. Reach for RF-DETR Medium when you only need to know people are present and where, since it is twice as quick. Reach for RF-DETR Seg Medium or SAM 3.1 (Scene) when you want the silhouette rather than the skeleton.

What to chain it with

Input is one photo. Output is a skeleton image and a text list of joints, and the two go different places.

Two chains matter. The skeleton image feeds ControlNet XL Union, which uses a pose map to steer an image generator, so you can photograph a pose and generate a new character holding it: the whole character-animation route in two steps. The coordinate list is what analytics needs instead, feeding a multimodal model such as Gemma 4 31B or Qwen3.6-35B-A3B to judge the posture in words. For video, Video Split in front and Video Reassemble behind turns either chain into a clip.

Open RF-DETR Keypoint in CNAPS Studio, run one photo where an arm is partly hidden, and look at the confidence on that wrist before you trust the skeleton downstream of it.

Sources

• github.com/roboflow/rf-detr for the Apache 2.0 license, the 40.7 million parameter count, the 71.8 keypoint average precision, the 9.7 millisecond latency, the preview status, and the note that every published row was measured in-house.
• arXiv:2511.09554 for the RF-DETR architecture and the architecture search behind the family.
• docs.cnaps.ai/ai-model-hub for the CNAPS Studio inputs and outputs, the seventeen joint names and their order, the OpenPose rendering format, and the confidence threshold.