Training and evaluation
Training a network for your own recordings, self-training on unlabeled video, preparing datasets and measuring accuracy. Training runs in either implementation; the Python package (PyTorch, GPU) is the faster choice.
← Overview · Backend · Configuration
Train and fine-tune
animalpose train pose configs/pose.yaml # from scratch
animalpose finetune pose configs/pose_real_r18.yaml weights/pose_general.pt train.steps=2000
<model> is detector, pose, temporal or behavior; trailing key=value overrides any
config key. The checkpoint goes to the config's output: (e.g. runs/pose.pt).
A training config has four blocks:
output: runs/pose.pt
model: {num_keypoints: 25, backbone: {name: tiny, width: 32, out_channels: 64}, embed_dim: 64}
data: {source: synthetic, crop_size: 128, image_size: [320, 256]} # or mixed `sources` with weights
loss: {sigma: 1.5, occluded_weight: 0.5, offset_weight: 0.05, embed_weight: 0.1}
train: {steps: 4000, batch_size: 32, lr: 2.0e-3, workers: 4, log_every: 100}
Optimization is AdamW with a 5 % warm-up then cosine decay and gradient clipping. Datasets are
mixed by weight under data.sources, so synthetic data, public datasets and your own labels can
train one network.
| Config | Trains |
|---|---|
detector.yaml, pose.yaml, temporal.yaml, behavior.yaml |
the synthetic-only networks |
detector_real*.yaml, pose_real*.yaml, temporal_real.yaml, behavior_real.yaml |
fine-tunes on real data (the _robust variants add degraded images) |
behavior_general.yaml |
the general 7-class behavior network |
temporal3d.yaml, behavior3d.yaml |
the 3D networks for multiview |
Use the result in a pipeline config's checkpoints:, or import it into the app
(Settings → General → Import → Model…).
Your own labels
Label in DeepLabCut, then convert to COCO keypoints on the shared skeleton:
animalpose data convert labeled-data/session1/CollectedData_me.csv --out data/processed/mine.json --image-root .
Body parts are mapped onto the 25 keypoints by name; points you did not label carry no loss.
Self-training on unlabeled recordings
label pseudo-labels your own videos with a teacher detector and pose network, keeping only what
it is sure of:
- 200 frames sampled per recording; about 1 in 5 recordings held out for evaluation
(
--no-holdoutto use all). - An animal counts at detection score ≥ 0.5; frames with anything between 0.2 and 0.5 are skipped as ambiguous.
- Keypoints must have confidence ≥ 0.5 and agree with a horizontally flipped pass within 5 % of the box width; an animal needs at least 4 such points.
--boxes blobfinds a single animal by its contrast with the floor instead of the detector.
Then fine-tune on train.json mixed with the original data.
Public datasets
animalpose data download crim13 rat7m_s1 mirror_mouse # about 13 GB for all, MD5-checked
animalpose data prepare --all # → canonical COCO JSON in data/processed/
Available: CRIM13, Lightning Pose mirror mouse, Rat7M (subjects 1 and 2), iRodent, MIT home cage,
ABODe. behavior sessions turns the behavior datasets into labeled session files.
Camera profiles
Measures what a recording does to the image: blur (from the width of the animal's edge against
the floor, and its drift over time), motion smear at the exposure, animal size, contrast and
sensor noise. Set the written profile as data.camera_profile in a training config and clean
training images are degraded the same way, so a network trained on sharp data still works on the
lab's camera (the _robust configs).
Evaluation
| Command | Measures |
|---|---|
eval real |
detection [email protected] and keypoint [email protected] / RMSE on held-out real datasets |
eval unlabeled |
label-free checks on the recordings label held out (share detected, flip agreement, …) |
eval real-tracking |
MOTA, IDF1 and identity switches on CRIM13 two-mouse videos |
eval behavior-sessions |
frame-wise behavior F1 against labeled sessions |
eval benchmark |
every network on held-out synthetic data (results/benchmark.json, .md) |
The Python package adds Rat7M 3D triangulation, mirror-mouse paws and videos, blur sweeps, CRIM13 identity and leave-one-animal-out behavior.
Export
animalpose export safetensors --checkpoint runs/pose.pt --out runs/pose.safetensors
animalpose export import-safetensors --safetensors runs/pose.safetensors --out runs/pose.pt
python -m animalpose export models … # TorchScript / ONNX (needs animalpose[export])
The Rust program and the apps also read .pt directly, so export is optional.