Measurement, not just detection
Plenty of vendors can put a box around an object. The hard, valuable work is what happens after: tracking the same subject across frames, placing dozens of anatomical keypoints on it with sub-percent error, turning those keypoints into measurements that repeat from one video to the next, and then proving which of those measurements mean anything.
We have done this at scale for the thoroughbred industry: a library of more than 37,000 sale-ring and breeze-up videos, a custom 45-point pose model trained on our own labelled frames, repeatability gates that stop unreliable metrics from ever reaching a user, and outcome validation against a database of millions of race results. The same methods apply to any subject that moves: livestock, athletes, machinery, people on a production line.
Capabilities
- Detection and trackingYOLO object detection, ByteTrack multi-object tracking and segmentation masks with SAM, tuned for your subject and camera.
- Custom pose estimationKeypoint models trained on your labelled data with clean licensing, when open models are non-commercial or simply not accurate enough. Held-out error under one percent of body length.
- Biomechanical measurementStride period and cadence from periodicity analysis, joint ranges of motion, proportions, posture lines and symmetry, each with a repeatability score.
- Identity from visionReading brands, numbers and markings from pose-anchored crops with vision-language models, narrowed by schedule data so the match is 1-in-4, not 1-in-100.
- Capture hardwareRaspberry Pi systems driving IP cameras: low-resolution substream for motion detection, full-resolution main stream for recording, automatic upload to cloud storage.
- Serverless and GPU pipelinesEvent-driven processing on Modal, batch GPU jobs on Vast.ai and Google Cloud spot instances, with cost tracked per video.
A typical pipeline
A camera records a clip and uploads it to a storage bucket. A message triggers a serverless function that scans the clip and decides within seconds whether it is worth keeping. Kept clips are queued for GPU processing, where a detector finds the subject, a tracker follows it, a pose model places keypoints frame by frame, and a measurement stage turns the keypoints into a row of numbers with confidence scores attached. The row lands in PostgreSQL, a dashboard renders it, and an anomaly check compares it against that subject's own history.
Every stage is a separate module with typed inputs and outputs, so the pipeline can be tested against ground truth one stage at a time, and swapped out when a better model arrives. Cost is engineered deliberately: a full re-read of 32,000 videos with the 45-point model ran on spot GPUs for under one hundred US dollars.
Validation is part of the deliverable
A measurement that does not repeat is noise, and a signal that does not survive a held-out test is a story. We build repeatability checks into the pipeline (the same horse filmed twice should score the same), and we validate against real outcomes before any claim is made. In our equine work that discipline has killed as many promising metrics as it has confirmed, and the ones that remain are the ones you can build a product on. Read the detail in the equine video biomechanics case study.