The problem
Every yearling and ready-to-run horse sold at auction is filmed walking and, for breeze-ups, galloping over 200 metres against the clock. Buyers watch the video and form an opinion. Nothing is measured, nothing is consistent between horses, and nobody has checked which visual impressions actually predict anything. We set out to measure every horse the same way from the footage that already exists, and to validate each measure against race outcomes from our racing database.
The pipeline
Videos are collected from the sale companies' catalogues, the horse is detected and tracked, and a pose model places keypoints on every frame. We started with an open quadruped model, but its weights are licensed for non-commercial use only and its accuracy was not good enough, so we trained clean-room models on our own labelled frames: a 29-point side model, a 26-point front and rear model, and the current 45-point side model with 0.63 percent held-out error as a fraction of body length.
From the keypoints the pipeline computes stride period and cadence, tracking-up, swing and reach, joint ranges of motion, topline (croup versus wither height), leg, neck and girth proportions and hoof length. A "balance" drawing of thirteen to fifteen conformation lines, the kind a trainer sketches by eye, is built automatically from the points on hind-stance frames. For breeze-ups, cadence comes from periodicity analysis with a confidence gate, and stride length is derived from the official time and distance.
Scale: 37,861 videos across the three major sale houses from 2013 to 2026. A full re-read of 32,000 parade videos with the 45-point model ran on cloud spot GPUs in about five days for roughly US$93. Around 58,000 catalogue photos have been posed as well.
Repeatability and validation
A measure is only shown if the same horse filmed twice scores about the same. Tracking-up, swing and hoof length pass that gate; cadence and left-right asymmetry do not, so they are never scored, however appealing they look. Breeze-up cadence from video agrees with a sale company's wearable sensor at r .79, and stride length at r .83, once the confidence gate is applied.
Against race outcomes, with held-out horses and permutation nulls:
- Type and distance. One catalogue photo predicts a horse's eventual racing distance at r .22 on 8,071 unseen horses, and adds .18 beyond price and the sire's record. Sire-level shape signals replicate.
- Breeze-up stride. Predicts the horse's later racing stride (+.37) and adds a small amount to rating and earnings beyond price. The clock itself adds nothing once price is known. A "slow clock, long stride" cell won metropolitan races at roughly twice the base rate, replicated in an independent cohort.
- Walk reach predicts the same horse's later breeze stride, and the market does not price it.
- Null. Talent and class beyond price are bounded near zero. Durability, several traditional conformation "rules", and asymmetry showed nothing. Earlier headline claims from older models failed to replicate and were withdrawn.
Live product
The measurements power public horse pages on our equine data venture's site: a points card for 77,000 yearling lots, a balance tab with 24,000 stills, a walk tab for 20,000 horses and sale photos. Private dossiers for individual horses combine walk percentiles, the balance drawing, jump-out cadence, family records and comparables, and each one states its gaps. A race-day gait-screening method developed from the same models, using mounting-yard and barrier cameras with automatic saddlecloth reading (39 of 40 numbers read on broadcast footage), is the subject of a pending Australian patent application.