Results: accuracy of the general models
Measured on data the models never trained on. Every number below was produced on 2026-11-18 from the checkpoints the release ships, with the commands at the end of this page.
| Network | File | SHA-256 (first 12) |
|---|---|---|
| Detector | weights/detector_general.pt |
68639b14eefb |
| Pose (ResNet-18) | weights/pose_general.pt |
459ab81c1044 |
| Temporal | weights/temporal_general.pt |
6c5c6a1ae926 |
| Behavior | weights/behavior_general.pt |
090301599209 |
Summary
| Task | Test data | Result |
|---|---|---|
| Finding animals | CRIM13, top view, 2 mice | AP50 0.905 |
| Finding animals | Rat7M, 6 side/angled cameras | AP50 0.994 |
| Keypoints | CRIM13, top view | [email protected] 0.891, median error 4.9 px |
| Keypoints | Rat7M, angled views | [email protected] 0.967 |
| Keypoints | Mirror mouse, side / bottom view | [email protected] 0.941 / 0.997 |
| Identity over time | CRIM13, 6 videos × 1,500 frames | 92.6 % of frames on the right mouse, 5 swaps |
| 3D (6 cameras) | Rat7M vs motion capture | median 9.0 mm, 79 % within 20 mm |
| Behavior (5 classes) | 1,801 held-out frames of a lab recording | accuracy 0.49, macro F1 0.40 |
Test data
| Set | What it is | Instances |
|---|---|---|
| CRIM13 OOD | Caltech Resident-Intruder Mouse: top camera, a black resident and a white intruder; sessions and intruders not used in training | 800 |
| Rat7M val | one rat filmed by 6 cameras at 8–32° elevation, with motion capture; held-out chunks | 400 |
| Mirror mouse OOD | head-fixed mouse with a side view and a mirrored bottom view (Lightning Pose); held-out sessions | 253 per view |
| Lab recording | 640 × 480 grayscale top camera, one blurred mouse; a held-out section at the end, never trained on | 1,801 frames |
Detection and keypoints
Detection is scored on full frames; keypoints are scored inside the labeled boxes, so the two numbers are independent.
| Test set | AP50 | AP75 | [email protected] | [email protected] | Median error | RMSE | Median box |
|---|---|---|---|---|---|---|---|
| CRIM13 (top, 2 mice) | 0.905 | 0.558 | 0.891 | 0.731 | 4.9 px | 13.0 px | 168 px |
| Rat7M (angled) | 0.994 | 0.642 | 0.967 | 0.861 | 11.3 px | 20.7 px | 539 px |
| Mirror mouse, side | 0.999 | 0.989 | 0.941 | 0.890 | 3.7 px | 18.9 px | 396 px |
| Mirror mouse, bottom | 0.978 | 0.924 | 0.997 | 0.990 | 3.1 px | 6.3 px | 396 px |
On Rat7M, keypoint confidence separates visible from hidden points with an AUROC of 0.978: low-confidence points really are the unreliable ones.
Per keypoint, [email protected]:
| CRIM13 | Rat7M | Mirror side | |||
|---|---|---|---|---|---|
| nose | 0.900 | head_center | 0.925 | nose | 1.000 |
| left_ear | 0.921 | spine_upper | 0.990 | tail_base | 1.000 |
| right_ear | 0.909 | spine_mid | 0.983 | tail_mid | 0.950 |
| neck | 0.930 | spine_lower | 0.968 | left_front_paw | 0.917 |
| tail_base | 0.864 | left / right_shoulder | 0.984 / 0.995 | right_front_paw | 0.952 |
| left_hind_knee | 0.845 | left / right_front_knee | 0.908 / 0.939 | left_hind_paw | 0.868 |
| right_hind_knee | 0.871 | left / right_hip | 0.956 / 0.977 | right_hind_paw | 0.897 |
| left / right_hind_knee | 0.973 / 0.986 |
Each dataset labels only some of the 25 keypoints; the others are not scored.
Identity tracking
Six held-out CRIM13 videos, 1,500 frames each, with tracker.max_animals=2. The two mice differ in
coat (black and white), which identifies them in every frame without manual labels.
| Appearance used for re-identification | Identity purity | Swaps (total) | Coverage | IDs per video |
|---|---|---|---|---|
| Pose-network embedding (default) | 0.926 | 5 | 0.917 | 2.0 |
| Color histogram | 0.863 | 7 | 0.917 | 2.0 |
| None (motion only) | 0.863 | 7 | 0.916 | 2.0 |
3D (multiview)
Rat7M held-out frames: every camera image goes through the detector and pose network, the views are triangulated, and the result is compared with motion-capture markers. 150 frames, 5.8 cameras per frame, 3.3 used per point on average.
| Mean error (MPJPE) | Median error | Within 20 mm | Points triangulated |
|---|---|---|---|
| 13.5 mm | 9.0 mm | 79.2 % | 96.1 % |
| Joint | Error | Joint | Error |
|---|---|---|---|
| spine_lower | 8.7 mm | left_shoulder | 13.6 mm |
| left_hind_knee | 8.7 mm | right_front_knee | 19.6 mm |
| spine_mid | 9.7 mm | head_center | 19.9 mm |
| right_hind_knee | 10.4 mm | left_front_knee | 26.8 mm |
Left and right paws (side view)
In a side view one paw of each pair hides the other. Scored on the mirror-mouse side view (971 paws), with the bottom view as reference:
| Decoder | Paw [email protected] | Both labels on one paw | Left/right swapped |
|---|---|---|---|
| Independent peaks | 0.899 | 16.9 % | 5.8 % |
| Pair decoding (default) | 0.909 | 0.9 % | 10.7 % |
| Pair decoding + bottom view | 0.973 | 0.9 % | 1.5 % |
Over the unlabeled mirror-mouse videos (5,607 scored paw pairs), the side view alone labels left
and right as the bottom view does in 86.7 % of frames, and in 89.5 % after left/right
consistency along the track (pipeline.pair_identity, default motion fraction 0.1).
Image quality
The same held-out images, degraded in fixed ways scaled to the animal's size ([email protected] / AP50):
| Condition | CRIM13 | Rat7M | Mirror side |
|---|---|---|---|
| Clean | 0.894 / 0.923 | 0.957 / 0.992 | 0.941 / 0.999 |
| Animal 32 px long | 0.828 / 0.902 | 0.881 / 0.556 | 0.812 / 0.638 |
| Motion blur, 10 % of body | 0.728 / 0.893 | 0.618 / 0.462 | 0.676 / 0.564 |
| Motion blur, 20 % of body | 0.341 / 0.728 | 0.198 / 0.077 | 0.319 / 0.417 |
| Defocus, 5 % of body | 0.391 / 0.819 | 0.160 / 0.118 | 0.213 / 0.259 |
| Small, blurred, gray, noisy (3-chamber top camera) | 0.238 / 0.596 | 0.138 / 0.014 | 0.213 / 0.129 |
The general models expect a reasonably sharp image with the animal at least ~30 px long. For
blurred or very small animals, use a camera profile (animalpose data audit) and the
degradation-trained models (configs/pipeline_robust.yaml).
Behavior
The 7-class behavior network on the held-out end of a lab recording (1,801 frames), with the shipped
post-processing (5-frame median, prior adaptation). The labels come from rule-named clusters of
the recording's own kinematics (animalpose behavior cluster / name / label), not from human
annotation, and only 5 of the 7 classes occur.
| Class | Share of frames | F1 |
|---|---|---|
| sniffing | 40.3 % | 0.610 |
| walking | 26.4 % | 0.507 |
| resting | 24.9 % | 0.296 |
| grooming | 3.6 % | 0.439 |
| rearing | 4.9 % | 0.133 |
| All (accuracy / macro F1) | 0.494 / 0.397 |
Behavior is the least accurate stage: treat its labels as a first pass. Fine-tuning on a few
minutes of the target recording (animalpose finetune behavior …) is the expected workflow for
measurements.
Not measured here
- Temporal refinement (
temporal_general.pt): its gap-filling test uses a private home-cage dataset not included with this repository. - Experiment measures (time in zones, entries, contact, grooming bouts): not yet compared with human scoring.
Metrics
| Metric | Meaning |
|---|---|
| AP50 / AP75 | detection average precision, a box counting as correct at ≥ 50 % / 75 % overlap with the label |
| PCK@α | share of labeled keypoints within α × the longest side of the animal's box (α = 0.1: 17 px for a 168 px CRIM13 mouse) |
| Median error, RMSE | keypoint distance to the label, in image pixels |
| Identity purity | share of each track's frames on its majority animal, averaged over tracks |
| Swaps | runs of frames where a track sits on the other animal |
| Coverage | share of frames with both animals tracked |
| MPJPE | mean distance between triangulated keypoints and motion-capture markers |
| Macro F1 | F1 averaged over the classes present, each class weighted equally |
Reproducing
From a checkout with the public datasets (animalpose data download, animalpose data prepare),
on an Apple M2 Max:
animalpose eval real --detector weights/detector_general.pt --pose weights/pose_general.pt --crop-size 192
animalpose eval real-tracking --detector weights/detector_general.pt --pose weights/pose_general.pt --crop-size 192
animalpose eval rat7m-3d --detector weights/detector_general.pt --pose weights/pose_general.pt --crop-size 192
animalpose eval mirror-paws --detector weights/detector_general.pt --pose weights/pose_general.pt
animalpose eval mirror-videos --detector weights/detector_general.pt --pose weights/pose_general.pt
animalpose eval blur --pairs weights/detector_general.pt:weights/pose_general.pt --crop-size 192
animalpose eval behavior-sessions --model weights/behavior_general.pt --sessions data/processed/behavior/cai/heldout
The networks are deterministic: rerunning gives the same numbers.