100 Points × 100 Days, a Token Bill Under NT$100: How Edge-Cloud Hybrid Crushed the Inspection Bill

Shalun green-energy park (沙崙綠能園區): 5 robots, 5 floors, 100+ inspection points, and patrols that have now been running for 100+ days straight — and over all that time, the AI inference bill hasn’t cracked NT$100.

What we want to share is exactly this: why the bill can be this small. The answer isn’t “we found a cheap model.” It’s the architecture.

Why inspection is harder than delivery

The two most common jobs for service robots are delivery and patrol. What they share: neither really asks the robot to interact with the physical world (opening doors, picking things up) — it just has to drive itself to a target point.

But “arriving at the spot” means completely different things for the two.

For delivery, reaching the spot is basically the end of the job — the package is there, the value is black-and-white, easiest thing in the world to evaluate.

Inspection is the opposite. The robot arrives at the spot, and now the real job begins: is this scene normal or not? And that question gets handed off to image analysis.

The trade-offs across three image-analysis techniques

Image analysis isn’t a new problem — it’s been around for over 40 years. From the apples on a farm to the Apple in your hand, factories have had rock-solid solutions for ages.

But the factory approach has a precondition: the same target, the same lighting, the same background, chasing extreme accuracy under tightly controlled conditions. Drop it into a real environment that changes at random… yeah, good luck.

Then came neural-net algorithms like YOLO, which can detect objects fast and reliably even in a changing environment. That gave everyone dreaming of visual patrol a glimmer of hope — it finally looked like AI could pull its weight on patrol duty.

So this architecture got poured into every kind of AI detection out there: personnel-hazard detection, sky-net systems, hard-hat compliance on job sites, license-plate recognition, vehicle and foot-traffic counting, retail shelf audits… all of it. But it has a fatal catch: YOLO can only tell you what’s in the frame. There’s a cup, a chair, a computer, a backpack — but it can’t tell you “it looks like someone was just working here.”

Image from the web

So how do those real-world hazard judgments actually get made? The answer is surprisingly low-tech — it’s all human ingenuity, rule after rule stacked on top of the detection output. This approach holds the accuracy, but it scales terribly: every new situation means throwing another batch of people at development and maintenance. The architecture is stable, mature, and efficient; its one flaw is the heavy up-front labor, which makes it a great fit for large-scale, same-condition analysis. For anomaly patrol inside a facility, it starts to strain.

Then VLMs arrived.

A VLM (Vision-Language Model) is a bigger model that understands an image and answers questions about it based on whatever you ask.

Cloud models (Google’s Gemini family, for example) can accurately read gauge headers and all kinds of environmental detail, and you can hand them the weirdest requests imaginable — like “this outlet may only be plugged into on the left side.” You can basically type out anything and let the AI judge whether the photo is normal.

So how do you actually choose among the three? Here’s the table:

One line to sum up the table: CV is precise but collapses the moment it leaves the rig, YOLO sees things but can’t read their meaning, and the VLM gets the meaning and swaps rules with a one-line prompt. We’re patrolling ordinary facilities, so the lead is obvious: it’s the VLM.

We went with Google’s offering — natively multimodal, handles both photos and video. It doesn’t just judge whether a spot is clear of clutter or holds a fire extinguisher; it even does a decent job spotting standing water on the floor.

The kicker is the price: one photo, input plus output, runs about 2K tokens. Even judged by the priciest model, Gemini 3 Pro, one shot comes out to roughly NT$1; and because most of those tokens are input rather than output, it actually lands around NT$0.1. In our own testing, Flash performs basically on par with Pro, which drops the price another decimal to NT$0.01.

Hallucinations can be further reined in with structured output — and structured output uses fewer tokens on top of that. Steadier quality, lower price.

VLMs are this strong — why not run everything in the cloud?

If VLMs are this all-powerful, why not just throw everything at the cloud? Fire an endless stream of photos and video at Google to crunch, right? Do the math and the smile fades —

Say a stream uploads at 2 fps. If you point it at the round-the-clock stuff — fire, animal intrusion, all the things that need 24-hour eyes — one month costs 2 × 60 × 60 × 24 × 30 × 0.01 > 50K. Hmm… once the labor shortage gets bad enough, sure, this becomes an option, but it’s clearly not the best one; at this stage plenty of facilities still figure a security guard plus a smoke alarm will do. So we started thinking: why not just run an open model on-device? Every day we watched a fresh wave of open models announce they’d “matched the big models on some benchmark and were about to overtake them on the inside lane,” and our hands got itchy — so we decided to play around too.

Where the gap between big and small models actually is

The conclusion first: small on-device models expose their weakness fast in an inspection setting, and they expose it in a very precise spot — not the “seeing,” the “thinking.”

We tested the usual sub-10B multimodal on-device models — Gemma, Qwen, Llama — and without exception: everyone’s ability to describe a scene is strong. Ask “what’s in this image” and it fluently tells you there are two people, one lamp, a box on the floor, rich enough in detail that you’d mistake it for the same tier as Gemini. But the moment you swap the task from “describe” to “judge” — say you hand it a prompt like this —

There must be no one fighting in this frame, no one who has fallen, and the lights must stay on.

it’ll mostly hand you back a stream of nonsense.

Why? Because “describe” and “judge” are simply two tasks of different difficulty. Description is pure perception: turn what the vision encoder sees into words, something a 3B model does perfectly respectably. But judgment stacks three more layers of reasoning on top of perception: understand the rules, check them one by one, output a verdict. Unpack that prompt above and it’s three conditions, two negations, and one state check — and the small model has three signature ways of dying in front of it:

1. Dropped conditions. Of three rules it checks only one, usually the last, and simply acts as if it never saw the others.
2. Negation flipped. With a negative like “there must be no one fighting,” the small model will readily judge “no one is fighting in the frame” as NG — because it literally sees “fighting” and the frame at the same time and glues the two together. Negation has always been a disaster zone for small models.
3. Verdict drift. Ask the same photo twice and you get OK once, NG the other time, with a perfectly reasoned-sounding rationale each way.

And these three deaths are ones the big cloud models almost never suffer. The root of the gap isn’t hard to grasp: a VLM is a “vision encoder + language model” combo, and for sub-10B open models versus the cloud flagships, the vision half of the gap actually isn’t that large — everyone sees roughly the same thing. What really opens the gap is the language half. Rule comprehension, multi-condition checking, negation logic, verdict consistency — all of it is the LLM backbone’s job, and those abilities are hard-wired to parameter count. The small model isn’t short-sighted — it’s short on brains.

That also explains why those “overtake on the inside lane” headlines feel so far from what we measured. Mainstream multimodal benchmarks (MMMU, MMBench and the like) mostly test description, recognition, multiple choice — precisely the subjects small models are good at. Real inspection tests “open environment + custom rules + negation conditions + forced structured output,” a subject those benchmarks don’t even sit for. A small model tying the big ones on the exam and whether it can pull a shift in your server room are two different things.

Two more jabs:

• Structured-output compliance has a gap too. Ask a cloud model to return {is_NG, Description}, pair it with the API’s schema constraint, and it dutifully returns that JSON. Ask a small on-device model and you’ll get JSON wrapped in markdown, JSON with an enthusiastic paragraph of explanation in front of it, or field names it has renamed to whatever it prefers — so the parsing layer has to grow a pile of defensive code.
• On-device also gets shaved by quantization. Models running on a Jetson are mostly 4-bit quantized (we use AWQ), so already-tight reasoning ability takes another cut — description barely dips, judgment drops hard — and judgment is exactly what we’re after.

So the interim conclusion here is awkward: for a small model to be usable, you have to take “judgment” away from it and fall back to the old YOLO-plus-human-ingenuity road… and if I’m going to hand-carve rules anyway, I’ll just use YOLO — and I save a GPU while I’m at it.

The edge-cloud hybrid architecture

Good thing NVIDIA shipped an interesting architecture: VILA paired with JPS (Jetson Platform Services). This combo essentially concedes the awkward conclusion above, then solves the problem a different way — if a general small model can’t judge, then specialize the model and narrow the task. It has two layers of advantage for “real-time alerting”:

Model layer: VILA reads time. When you use a general VLM to judge single frames one at a time, each frame is seen and concluded on in isolation — “mistaking a fistfight for a game of basketball” comes from exactly this, because “fighting” is an event that only holds across frames, and any single frame just shows two people standing very close. VILA is built for multi-frame / video understanding: it takes the context before and after together and judges “what actually happened over this stretch of time,” not just “what’s in this one frame.” It’s also optimized for the token count per image, which is what lets it run real-time streams on a Jetson. This is another way for a small model to make a living: don’t compete with the big models on general judgment — go deep on the narrow task of “temporal event detection.”

System layer: JPS turns the whole alert pipeline into a service. JPS ingests RTSP streams directly (IP cameras, VST, and other sources all work), lets you set alert rules in natural language (“is there fire in the frame?” “is anyone holding a knife?”), continuously evaluates the stream, and pushes out a True/False alert state. Note the shape of these rules — “is there fire” is a closed, single-condition question, which lands right inside the judgment range a small model can still handle; it never asks the small model to check three rules plus two negations. Flip it around: if all you have is a bare Gemma, then you truly have only a “model” — stream ingestion, frame extraction, continuous inference, alert triggering and state management, deduplication, push notifications, all of it you build from scratch yourself. VILA + JPS is an out-of-the-box microservice optimized for Jetson edge, and it packages all that grunt work for you.

So the division of labor becomes clear: complex, multi-condition, frequently changing rule judgments go to the big cloud model; continuous, narrow-question, real-time-critical event detection goes to the specialized on-device small model. This isn’t a cost compromise of “the big model is better so put it in the cloud” — it’s two kinds of models each standing at the sweet spot of its own capability curve.

The layering principle is a single line: continuous imagery stays on-device; the cloud only judges frames that matter. In practice it lands as three pipelines:

Real-time alerting is the clearest illustration of the split: four event classes — smoke, fire, knife, animal intrusion — are watched continuously against the 2 fps stream by the on-device VLM on the Jetson, detection-to-alert under 2 seconds, pushed straight to LINE/Telegram, and the entire pipeline never touches the cloud. The cloud never sees these streams from start to finish, so it naturally racks up no cost.

Why the bill is this small

Translate the architecture above back into a bill, and what the cloud actually does during a routine patrol is startlingly little:

1. The input is just a single point photo. The robot shoots only once it’s parked, one JPEG per inspection point — not video, not a burst. 100+ points means 100+ single-image judgments, plus one final report-summary generation.
2. The output is squeezed to the minimum by structuring. Each judgment is forced to return the structured JSON {is_NG, Description}; the model writes no prose, output tokens fall naturally, and you skip the brittle text parsing.
3. A narrow question lets the model go light. “Is there a fire extinguisher in this photo” is a closed question, so a lightweight cloud VLM (Gemini Flash-Lite tier) is plenty — no need to call in the flagship. When we said small models can’t judge, we meant multi-condition open judgment; once the question narrows, even the cloud can drop to the cheapest tier.
4. The cost of continuous watching goes to zero. The most expensive part — “watching all the time” — moves entirely to the edge; the cloud only handles the discrete snapshots taken at inspection time.

One inspection point works out to a thousand-odd tokens. Add up 100+ points across 100+ days and the cloud bill still doesn’t crack NT$100 — the cost wasn’t saved, it was decided by the architecture.

The last mile of inspection

Every photo in the patrol got judged precisely, and the bill didn’t blow up. Congratulations all around — feels like we can call it a day! That misses inspection’s most important output: the point of inspection isn’t to do image analysis, it’s to produce a report.

You looked at a pile of fire extinguishers to know whether they meet fire-safety code. You looked at a pile of outlets to know whether they meet electrical regulations. This is the most maddening part of traditional inspection: no matter how flexible the image judgment is, the report’s final schema still has to be hand-carved.

But AI is different — writing reports is its strong suit.

So we can let it write this report automatically And to add or remove any item, you just type

In all my years building robot applications, changing a report format for a customer has never felt this effortless.

A few practical reminders

• The costs above cover AI inference only; the robot itself, subscription, and deployment are separate.
• On-device inference isn’t free — you pay the fixed cost of one Jetson in exchange for real-time response and the privacy of streams that never leave the site.
• Open small models iterate fast; today’s “sub-10B can’t judge” conclusion has a shelf life. But the pattern of “description ability arrives first, judgment ability arrives later” probably won’t change any time soon. When picking an on-device model, don’t be swayed by how fluently it describes — hand it your nastiest rule and put it to the test.