Skip to content

Results: accuracy of the general models

Measured on data the models never trained on. Every number below was produced on 2026-11-18 from the checkpoints the release ships, with the commands at the end of this page.

Network File SHA-256 (first 12)
Detector weights/detector_general.pt 68639b14eefb
Pose (ResNet-18) weights/pose_general.pt 459ab81c1044
Temporal weights/temporal_general.pt 6c5c6a1ae926
Behavior weights/behavior_general.pt 090301599209

Summary

Task Test data Result
Finding animals CRIM13, top view, 2 mice AP50 0.905
Finding animals Rat7M, 6 side/angled cameras AP50 0.994
Keypoints CRIM13, top view [email protected] 0.891, median error 4.9 px
Keypoints Rat7M, angled views [email protected] 0.967
Keypoints Mirror mouse, side / bottom view [email protected] 0.941 / 0.997
Identity over time CRIM13, 6 videos × 1,500 frames 92.6 % of frames on the right mouse, 5 swaps
3D (6 cameras) Rat7M vs motion capture median 9.0 mm, 79 % within 20 mm
Behavior (5 classes) 1,801 held-out frames of a lab recording accuracy 0.49, macro F1 0.40

Test data

Set What it is Instances
CRIM13 OOD Caltech Resident-Intruder Mouse: top camera, a black resident and a white intruder; sessions and intruders not used in training 800
Rat7M val one rat filmed by 6 cameras at 8–32° elevation, with motion capture; held-out chunks 400
Mirror mouse OOD head-fixed mouse with a side view and a mirrored bottom view (Lightning Pose); held-out sessions 253 per view
Lab recording 640 × 480 grayscale top camera, one blurred mouse; a held-out section at the end, never trained on 1,801 frames

Detection and keypoints

Detection is scored on full frames; keypoints are scored inside the labeled boxes, so the two numbers are independent.

Test set AP50 AP75 [email protected] [email protected] Median error RMSE Median box
CRIM13 (top, 2 mice) 0.905 0.558 0.891 0.731 4.9 px 13.0 px 168 px
Rat7M (angled) 0.994 0.642 0.967 0.861 11.3 px 20.7 px 539 px
Mirror mouse, side 0.999 0.989 0.941 0.890 3.7 px 18.9 px 396 px
Mirror mouse, bottom 0.978 0.924 0.997 0.990 3.1 px 6.3 px 396 px

On Rat7M, keypoint confidence separates visible from hidden points with an AUROC of 0.978: low-confidence points really are the unreliable ones.

Per keypoint, [email protected]:

CRIM13 Rat7M Mirror side
nose 0.900 head_center 0.925 nose 1.000
left_ear 0.921 spine_upper 0.990 tail_base 1.000
right_ear 0.909 spine_mid 0.983 tail_mid 0.950
neck 0.930 spine_lower 0.968 left_front_paw 0.917
tail_base 0.864 left / right_shoulder 0.984 / 0.995 right_front_paw 0.952
left_hind_knee 0.845 left / right_front_knee 0.908 / 0.939 left_hind_paw 0.868
right_hind_knee 0.871 left / right_hip 0.956 / 0.977 right_hind_paw 0.897
left / right_hind_knee 0.973 / 0.986

Each dataset labels only some of the 25 keypoints; the others are not scored.

Identity tracking

Six held-out CRIM13 videos, 1,500 frames each, with tracker.max_animals=2. The two mice differ in coat (black and white), which identifies them in every frame without manual labels.

Appearance used for re-identification Identity purity Swaps (total) Coverage IDs per video
Pose-network embedding (default) 0.926 5 0.917 2.0
Color histogram 0.863 7 0.917 2.0
None (motion only) 0.863 7 0.916 2.0

3D (multiview)

Rat7M held-out frames: every camera image goes through the detector and pose network, the views are triangulated, and the result is compared with motion-capture markers. 150 frames, 5.8 cameras per frame, 3.3 used per point on average.

Mean error (MPJPE) Median error Within 20 mm Points triangulated
13.5 mm 9.0 mm 79.2 % 96.1 %
Joint Error Joint Error
spine_lower 8.7 mm left_shoulder 13.6 mm
left_hind_knee 8.7 mm right_front_knee 19.6 mm
spine_mid 9.7 mm head_center 19.9 mm
right_hind_knee 10.4 mm left_front_knee 26.8 mm

Left and right paws (side view)

In a side view one paw of each pair hides the other. Scored on the mirror-mouse side view (971 paws), with the bottom view as reference:

Decoder Paw [email protected] Both labels on one paw Left/right swapped
Independent peaks 0.899 16.9 % 5.8 %
Pair decoding (default) 0.909 0.9 % 10.7 %
Pair decoding + bottom view 0.973 0.9 % 1.5 %

Over the unlabeled mirror-mouse videos (5,607 scored paw pairs), the side view alone labels left and right as the bottom view does in 86.7 % of frames, and in 89.5 % after left/right consistency along the track (pipeline.pair_identity, default motion fraction 0.1).

Image quality

The same held-out images, degraded in fixed ways scaled to the animal's size ([email protected] / AP50):

Condition CRIM13 Rat7M Mirror side
Clean 0.894 / 0.923 0.957 / 0.992 0.941 / 0.999
Animal 32 px long 0.828 / 0.902 0.881 / 0.556 0.812 / 0.638
Motion blur, 10 % of body 0.728 / 0.893 0.618 / 0.462 0.676 / 0.564
Motion blur, 20 % of body 0.341 / 0.728 0.198 / 0.077 0.319 / 0.417
Defocus, 5 % of body 0.391 / 0.819 0.160 / 0.118 0.213 / 0.259
Small, blurred, gray, noisy (3-chamber top camera) 0.238 / 0.596 0.138 / 0.014 0.213 / 0.129

The general models expect a reasonably sharp image with the animal at least ~30 px long. For blurred or very small animals, use a camera profile (animalpose data audit) and the degradation-trained models (configs/pipeline_robust.yaml).

Behavior

The 7-class behavior network on the held-out end of a lab recording (1,801 frames), with the shipped post-processing (5-frame median, prior adaptation). The labels come from rule-named clusters of the recording's own kinematics (animalpose behavior cluster / name / label), not from human annotation, and only 5 of the 7 classes occur.

Class Share of frames F1
sniffing 40.3 % 0.610
walking 26.4 % 0.507
resting 24.9 % 0.296
grooming 3.6 % 0.439
rearing 4.9 % 0.133
All (accuracy / macro F1) 0.494 / 0.397

Behavior is the least accurate stage: treat its labels as a first pass. Fine-tuning on a few minutes of the target recording (animalpose finetune behavior …) is the expected workflow for measurements.

Not measured here

  • Temporal refinement (temporal_general.pt): its gap-filling test uses a private home-cage dataset not included with this repository.
  • Experiment measures (time in zones, entries, contact, grooming bouts): not yet compared with human scoring.

Metrics

Metric Meaning
AP50 / AP75 detection average precision, a box counting as correct at ≥ 50 % / 75 % overlap with the label
PCK@α share of labeled keypoints within α × the longest side of the animal's box (α = 0.1: 17 px for a 168 px CRIM13 mouse)
Median error, RMSE keypoint distance to the label, in image pixels
Identity purity share of each track's frames on its majority animal, averaged over tracks
Swaps runs of frames where a track sits on the other animal
Coverage share of frames with both animals tracked
MPJPE mean distance between triangulated keypoints and motion-capture markers
Macro F1 F1 averaged over the classes present, each class weighted equally

Reproducing

From a checkout with the public datasets (animalpose data download, animalpose data prepare), on an Apple M2 Max:

animalpose eval real          --detector weights/detector_general.pt --pose weights/pose_general.pt --crop-size 192
animalpose eval real-tracking --detector weights/detector_general.pt --pose weights/pose_general.pt --crop-size 192
animalpose eval rat7m-3d      --detector weights/detector_general.pt --pose weights/pose_general.pt --crop-size 192
animalpose eval mirror-paws   --detector weights/detector_general.pt --pose weights/pose_general.pt
animalpose eval mirror-videos --detector weights/detector_general.pt --pose weights/pose_general.pt
animalpose eval blur --pairs weights/detector_general.pt:weights/pose_general.pt --crop-size 192
animalpose eval behavior-sessions --model weights/behavior_general.pt --sessions data/processed/behavior/cai/heldout

The networks are deterministic: rerunning gives the same numbers.