Multimodal
A model that accepts more than one type of input (text + images, sometimes audio + video). Modern Claude, GPT, Gemini are all multimodal.
This term is also in our AI glossary for Australian business owners, with what it means for your business.
A multimodal model accepts more than one type of input, typically text plus images, sometimes audio and video too. As of mid-2026, all the major frontier models (Claude 4.x, GPT-5, Gemini 2.5) are multimodal by default.
In practice, “multimodal” usually means you can upload a screenshot and ask Claude to describe what’s in it, or paste a chart and ask for analysis, or hand it a photo of a whiteboard and ask for the equation to be transcribed.
What it unlocks for Australian small business
- Receipt + invoice OCR: paste a photo of a receipt, get structured data back (vendor, date, items, AUD total, GST)
- Screenshot Q&A: take a screenshot of your Shopify analytics, ask “what’s going on with conversion this week?”
- Document analysis: PDFs and scanned documents become readable + analysable
- Diagram understanding: feed a hand-drawn architecture sketch, get a written summary
- Brand image review: paste a draft social post graphic, ask “does this match our brand voice notes?”
Cost implications
Image input is billed at a different rate than text:
- Claude charges by image at roughly the equivalent of ~1,500 tokens per typical image (about US$0.003 on Claude Sonnet 5.5 at US$2 per million input tokens, roughly A$0.004, as at October 2026)
- Larger images cost more (up to ~7k tokens for a high-resolution image)
- Resize images before uploading where possible, 1024px wide is usually enough
For occasional use, multimodal is negligible. For heavy batch image processing (e.g. OCR-ing 1,000 receipts), the cost adds up; pre-process with cheaper OCR (Tesseract, AWS Textract) first and only hand Claude the structured output.
What multimodal still doesn’t do well
- Reading dense, small text in screenshots (transcribe with a dedicated OCR first)
- Counting precisely (Claude is okay-but-not-great at “how many people are in this photo?”)
- Generating images (frontier text models like Claude don’t generate images; for that you want Midjourney, DALL·E 3, or Higgsfield)
- Real-time video understanding (still emerging; voice + frame-by-frame works but isn’t smooth)
Related terms
Want this built for your business?
Book a free 30-minute AI audit call with Jenn Yang. We map where your time goes, name the first systems worth building and give you fixed AUD prices. No obligation.
Book a free AI auditOr have us run it for you, end to end: On Autopilot is Australia's outsourced AI department.