Detection vs Grounding Explained (Why Free-Form Language Changes Everything)

Added: 2026-05-23

[00:00:00]What is the difference between detection or localization and grounding? Now, when we do object detection, we have a class of objects that we want to detect, and [music] we basically train the model, and we get a bounding box and the class label associated with it. YOLO and other models, they solve this problem, right?

[00:00:21]RF Ditter, YOLO, etc., they solve this problem. So, that is the detection problem. Now, grounding is a cousin of this problem, and in grounding, you are trying to actually [music] do localization, but based on language.

[00:00:37]So, you are, for example, you could say that I want the red car in the crowd of cars, right?

[00:00:46]And that localization problem, that is grounding, because it is free-form text, it is matching language to visual reality, right? So, that task is called, you know, grounding. It comes from [music] Steven Harnad. He was a cognitive scientist, and in the 1990s, he came up with this paper where the word grounding was first used in this context. [music] The idea was that grounding basically means stabilizing something, right? When you ground an electrical circuit, you make it more stable. You won't get shocked and stuff like that. Now, when you ground language with visual reality, then you are associating words to something which are abstract, right?

[00:01:27]Words are cooked-up things to something real, [music] like a car is a real thing. It's a real physical thing. Word is an abstract thing. That word car is an abstract thing. So, that process is [music] called grounding, and this comes from 1990s cognitive science, and later in the 2010s or something, um computer vision researchers also started using this phrase to explain what how do you how do you take language and map it to vision reality, right? Visual task. So, that's where the word grounding comes from. And that's the difference between object detection, which basically detects fixed number of classes in a scene, and visual grounding, which basically is free-form language. From free-form language, you're trying to locate things, but based on free-form language, right? So, that's the difference. All right, thanks.