BoxBot: Visual Servoing for the Tape, a VLA for the Flaps

Opening a sealed cardboard box looks trivial until you ask a robot to do it. The tape seam is 1 to 2mm wide and sits somewhere slightly different on every box. The flaps are floppy, they catch on each other, and the right motion depends on how the box was packed. Half the problem rewards precision. The other half resists being a structured set of instructions.

BoxBot is an ongoing Viam research project that treats those as two separate problems and solves each with the tool that fits.

Phase one: cutting the tape

The box cutting logic (packaged as a Viam module) first estimates the box’s dimensions and center, which tells it where the three tape seams should be. Then it uses image-based visual servoing to guide the cutting tip onto each seam. There’s no calibrated workcell. The arm makes millimeter-level corrections based on what the camera sees, so visual feedback closes the loop instead of a carefully measured model of the scene. In the video, the adjustments are almost too small to notice.

This is classical control doing what it does well: a precise, well-defined task where you can state the goal and measure the error.

Phase two: opening the flaps

Once the tape is cut, the arm swaps tools and hands off to a vision-language-action (VLA) model, which predicts the motions needed to peel open and adjust all four flaps. The task is deformable and contact-rich. An analytical controller would have to model cardboard stiffness, friction, and every way a flap can fold back on itself. A learned policy can pick that up from demonstrations instead.

Where the training data came from

The team collected demonstrations by teleoperation: driving the end of the arm with a VR controller and opening box after box. Each episode recorded the arm’s motion synced to two camera streams, one overhead and one eye-in-hand. Two engineers gathered 125 demonstrations in about five hours.

Those episodes fine-tuned SmolVLA, an open source VLA, using LeRobot on Hugging Face Jobs. The policy deployed straight back to the same machine with the same camera inputs and runs inference locally on an NVIDIA Jetson Orin Nano. No custom model architecture, no task-specific engineering. A base model plus 125 episodes produced a policy that opens boxes it never saw during training.

The part that usually eats the schedule

Training was the quick part. Anyone who has put a policy on a real robot knows where the time goes: building the teleop rig, syncing camera frames to joint states, formatting episodes for a training pipeline, then wiring the trained model back into a control stack that looks nothing like the one you collected data on. Then doing it all again for the next arm.

On BoxBot, that layer was already in place. The VR controller connected as a module. Data capture synced the arm and both cameras to the cloud. The trained policy runs through Viam’s ML model service alongside the visual servoing module and classical motion planning, on the same machine, behind the same APIs. The lab setup and the deployment setup are the same setup, and standing up a second station means copying a config fragment.

That’s what we mean by Viam as a harness for physical AI. The model is one component. The harness is everything that connects it to hardware, feeds it data, deploys it, and lets it run next to the parts of the system that aren’t learned. Combining visual servoing, motion planning, and a VLA on BoxBot took configuration, not integration work.

Be deliberate about where

The lesson from BoxBot isn’t “use a VLA.” It’s to choose where. Classical methods still win on precision. Learned policies earn their place on the parts that are hard to specify. Being able to mix both, and swap either one out, is what keeps the project moving: adding a third tool, like a suction cup or gripper to pick what’s inside, is the same kind of change.

Try it yourself

The code and dataset are open on Hugging Face at huggingface.co/viamrobotics. Train on the episodes, benchmark against BoxBot, or reproduce the setup on your own arm.