Skip to content

Training and evaluation

Training a network for your own recordings, self-training on unlabeled video, preparing datasets and measuring accuracy. Training runs in either implementation; the Python package (PyTorch, GPU) is the faster choice.

← Overview · Backend · Configuration

Train and fine-tune

animalpose train pose configs/pose.yaml                          # from scratch
animalpose finetune pose configs/pose_real_r18.yaml weights/pose_general.pt train.steps=2000

<model> is detector, pose, temporal or behavior; trailing key=value overrides any config key. The checkpoint goes to the config's output: (e.g. runs/pose.pt).

A training config has four blocks:

output: runs/pose.pt
model:  {num_keypoints: 25, backbone: {name: tiny, width: 32, out_channels: 64}, embed_dim: 64}
data:   {source: synthetic, crop_size: 128, image_size: [320, 256]}   # or mixed `sources` with weights
loss:   {sigma: 1.5, occluded_weight: 0.5, offset_weight: 0.05, embed_weight: 0.1}
train:  {steps: 4000, batch_size: 32, lr: 2.0e-3, workers: 4, log_every: 100}

Optimization is AdamW with a 5 % warm-up then cosine decay and gradient clipping. Datasets are mixed by weight under data.sources, so synthetic data, public datasets and your own labels can train one network.

Config Trains
detector.yaml, pose.yaml, temporal.yaml, behavior.yaml the synthetic-only networks
detector_real*.yaml, pose_real*.yaml, temporal_real.yaml, behavior_real.yaml fine-tunes on real data (the _robust variants add degraded images)
behavior_general.yaml the general 7-class behavior network
temporal3d.yaml, behavior3d.yaml the 3D networks for multiview

Use the result in a pipeline config's checkpoints:, or import it into the app (Settings → General → Import → Model…).

Your own labels

Label in DeepLabCut, then convert to COCO keypoints on the shared skeleton:

animalpose data convert labeled-data/session1/CollectedData_me.csv --out data/processed/mine.json --image-root .

Body parts are mapped onto the 25 keypoints by name; points you did not label carry no loss.

Self-training on unlabeled recordings

label pseudo-labels your own videos with a teacher detector and pose network, keeping only what it is sure of:

animalpose label --root /data/recordings --out data/processed/my_pseudo
  • 200 frames sampled per recording; about 1 in 5 recordings held out for evaluation (--no-holdout to use all).
  • An animal counts at detection score ≥ 0.5; frames with anything between 0.2 and 0.5 are skipped as ambiguous.
  • Keypoints must have confidence ≥ 0.5 and agree with a horizontally flipped pass within 5 % of the box width; an animal needs at least 4 such points.
  • --boxes blob finds a single animal by its contrast with the floor instead of the detector.

Then fine-tune on train.json mixed with the original data.

Public datasets

animalpose data download crim13 rat7m_s1 mirror_mouse     # about 13 GB for all, MD5-checked
animalpose data prepare --all                              # → canonical COCO JSON in data/processed/

Available: CRIM13, Lightning Pose mirror mouse, Rat7M (subjects 1 and 2), iRodent, MIT home cage, ABODe. behavior sessions turns the behavior datasets into labeled session files.

Camera profiles

animalpose data audit myvideo.mp4 --out configs/camera.yaml

Measures what a recording does to the image: blur (from the width of the animal's edge against the floor, and its drift over time), motion smear at the exposure, animal size, contrast and sensor noise. Set the written profile as data.camera_profile in a training config and clean training images are degraded the same way, so a network trained on sharp data still works on the lab's camera (the _robust configs).

Evaluation

Command Measures
eval real detection [email protected] and keypoint [email protected] / RMSE on held-out real datasets
eval unlabeled label-free checks on the recordings label held out (share detected, flip agreement, …)
eval real-tracking MOTA, IDF1 and identity switches on CRIM13 two-mouse videos
eval behavior-sessions frame-wise behavior F1 against labeled sessions
eval benchmark every network on held-out synthetic data (results/benchmark.json, .md)

The Python package adds Rat7M 3D triangulation, mirror-mouse paws and videos, blur sweeps, CRIM13 identity and leave-one-animal-out behavior.

Export

animalpose export safetensors --checkpoint runs/pose.pt --out runs/pose.safetensors
animalpose export import-safetensors --safetensors runs/pose.safetensors --out runs/pose.pt
python -m animalpose export models …                                   # TorchScript / ONNX (needs animalpose[export])

The Rust program and the apps also read .pt directly, so export is optional.