RF-DETR Seg Medium: pixel outlines instead of rectangles, and where to run it

Apache 2.0, weights free from Roboflow, no terms to negotiate. At 35.7 million parameters self-hosting is light work: PyTorch, the Hugging Face transformers library, and a GPU if you want the real-time speeds it was built for. In your browser it runs on CNAPS Studio, included in the basic plan.

It is the cutout version of a detector. Where a detector says there is a dog inside this rectangle, this one says here are the exact pixels that are the dog, and it does the finding and the tracing in a single pass rather than detecting first and segmenting afterwards. Two cats overlapping on a couch come back as two separate masks, not one blob.

The cost is the same fixed vocabulary its detection sibling has: 80 everyday categories and nothing else. If the thing you want to cut out is a specific product, a logo, or anything outside that list, this model cannot see it at all. Five segmentation models sit in the CNAPS Studio catalog and this is the fastest of them by a wide margin.

What it actually is

A vision transformer backbone reads the image, a projector merges what it saw at several scales, a deformable decoder produces a set of objects in one pass, and a mask head on top of that turns each detected object into a pixel outline. The detection half is identical to the plain RF-DETR Medium; the extra two million parameters are the mask head.

Like the rest of the family, the size was not chosen by hand. Roboflow trained one flexible network, searched inside it for configurations balancing accuracy against latency, and published several points on the curve. Seg Medium is the balanced one. Training data is the instance segmentation half of COCO 2017, the same 80 categories.

One setting matters: a minimum confidence score from 0 to 1, defaulting to 0.5. What comes back for each object is four things rather than three, a label, a confidence score, a box, and the mask, delivered as one image-sized label map plus a per-object list so you can pull any single mask out on its own.

The published numbers

On COCO 2017 it scores 45.3 mask average precision, the strict measure averaged over how exactly the mask has to match, and 68.4 on the loose measure that only wants rough overlap. Both are out of 100. It runs at 5.9 milliseconds per image at 432 by 432 pixels, measured on a small inference GPU with an optimised runtime and one image at a time.

It is worth seeing that 45.3 next to the detection sibling's 54.7, because the gap is not the model being worse. Tracing an outline correctly is a harder question than drawing a rectangle around the same object, so mask scores sit below box scores across the whole field. The two numbers answer different questions and should not be read as a ranking.

On credibility this is a better-than-usual case. Roboflow measured every model in its comparison in-house with identical tooling over the full 5,000-image validation split, and states its figures may differ from vendor-reported ones. Uniform measurement beats each model quoting its own paper, though it remains this model's vendor measuring.

How it compares to the other segmentation models here

The five segmentation models in the catalog are scored on four different scales, so ranking them by number would be dishonest. Mask average precision here rewards outlining objects from a fixed category list, cgF1 rewards finding every instance of a concept you described in words, J&F rewards mask overlap through a video, and a structure measure rewards one clean foreground cutout.

The practical split is how you name the target and how fast you need it. Reach for RF-DETR Seg Medium when the object is one of the 80 everyday categories and speed decides, since at roughly six milliseconds it is in a different bracket from everything else here. Reach for SAM 3 or SAM 3.1 (Scene) when the target needs describing in a phrase, SAM 2 (Scene) when you cannot name it and want the whole scene separated, and BiRefNet (Subject) when there is one subject and you want the cleanest cutout.

What to chain it with

It takes one image and returns labels, confidence scores, boxes and masks together, which means it can feed both the box-shaped chains and the mask-shaped ones without a conversion step.

Two are worth naming. For compositing, take a mask into Layer Compose to drop the cut-out subject onto a different background, or hand it to PowerPaint or LatentDiffusion (Object Removal) to erase that object from the original. For anonymising, mask the people or the screens and pass it to Selective Blur, which blurs only inside the outline instead of a rectangle that spills onto everything nearby. That precision is the whole reason to pick this over its detection sibling.

Open RF-DETR Seg Medium in CNAPS Studio, run a photo where two objects of the same kind overlap, and check whether you get two masks or one before you build on it.

Sources

• github.com/roboflow/rf-detr for the Apache 2.0 license, the 35.7 million parameter count, the 45.3 and 68.4 mask scores, the 5.9 millisecond latency and 432 by 432 resolution, the detection sibling's 54.7 for contrast, and the statement that every row was measured in-house over the full validation split.
• huggingface.co/Roboflow/rf-detr-seg-medium and arXiv:2511.09554 for the architecture, the mask head, the training data, and the architecture search behind the family.
• docs.cnaps.ai/ai-model-hub for the CNAPS Studio input, output shape and confidence threshold, and for the other segmentation models in the catalog.