ViLT VQAv2 — Image Q&A AI Model
Ask a picture a question — what color is the car, how many people are there — and ViLT answers in a few words. Small and quick; for open-ended questions reach for a full vision-language model instead. CPU-capable.
What it does
Ask a picture a question - what colour is the car, how many people are there - and ViLT answers in a few words. It is a single transformer reading the image and the question together, small and quick, and it runs on CPU.
The answer comes out of a fixed set learned on VQAv2 rather than being written freely, so this is a short-answer model by design. For open-ended questions a full vision-language model is the better tool.
When to use it
- A short, specific question about an image: colour, count, position
- Describing what is in an image for a reader who cannot see it
- Checking whether an image really contains the expected subject
- Cheap image understanding, on CPU rather than on a GPU
- A question-guided answer instead of a generic caption
Details
- CategoryImage Q&A
- Year2021
- LicenseApache-2.0
Compliance & Provenance
- ProviderOpen-source · Specialized
- EU AI Act RiskMinimal Risk
- Art. 50 TransparencyNot applicable
Inputs & Outputs
- ImageInputimage
Source image to answer questions about
- TextInputstring
Question about the image
- TextOutputstring
Answer to the question about the image
Tags
Alternatives in Image Understanding
- BLIP (Image Description)
Auto image captioning (BLIP Large). Fast.