Using the DeepLabCut Model Zoo (30 min)#

Google Colab 🦁🦓🐘: Imagine a digital toolbox, but instead of regular tools, it’s filled with ready-to-use AI models designed especially for animal movement analysis. Born in 2020 and supercharged in 2022, this “Model Zoo” has:

  • 1️⃣ Ready-Made Models: Skip the training step and get straight to analyzing.

  • 2️⃣ Contribute & Grow: A community space where you can share data to make these models even smarter.

  • 3️⃣ Easy Access: Use directly on Google Colab, your browser (DLC GUI), or even the HuggingFace.

  • 4️⃣ SuperAnimals!: These are like the superheroes of AI models, combining various data to work on a range of animals, from dogs to mice, side views to top views.

No need to train anything; just pick a model, plug in your video, and watch it do its thing!

Get Started with Your Video! 🎥#

Have a video of your cat’s curious moments or your dog playing? Load it up! 🐈🐩 Need a video? Check out Pexels for some free options.

And if you’re diving into research, remember we’ve got a top-view mouse model ready for you.

  1. Open the DeepLabCut GUI, look for the Model Zoo button and follow the on-screen steps: Pick your video, choose a model, and watch the magic happen🪄! Once done, don’t forget to peek into the folder where your video is saved to check your results.

  2. You can also use this notebook to analyse your video: DeepLabCut Model Zoo Colab Notebook

Note

Notice how important it is to match the relative size of the individuals in the video. Why is that 🤔?

SuperAnimal Method Highlights 🛠:#

  • Panoptic Pose Estimation: A novel technique introduced to merge and train diverse datasets with unique labels.

  • The SuperAnimal Models cover 45+ mammal species with 27-39 keypoints.

  • Zero-Shot Performance: Demonstrated excellent results without the need for additional training on new datasets.

  • Superior to ImageNet-pretraining: Surpasses ImageNet-pretraining on three benchmarks.

  • Efficiency in Fine-Tuning: These models are 10x more data-efficient, giving a 2x performance boost.

  • Keypoint Matching Algorithm: Features an algorithm to auto-align unfamiliar datasets.

  • Unsupervised Video-Adaptation: Introduced a rapid, unsupervised video-adaptation method enabling model fine-tuning without the necessity of data labeling.

  • Consistency Tools:

    • A spatial-pyramid search method to cater to different video input sizes.

    • Pseudo-labeling to curb temporal jitter in video streams.

Read the cool science staff here: [YFL+23]

References#

[YFL+23]

Shaokai Ye, Anastasiia Filippova, Jessy Lauer, Maxime Vidal, Steffen Schneider, Tian Qiu, Alexander Mathis, and Mackenzie Weygandt Mathis. Superanimal pretrained pose estimation models for behavioral analysis. 2023. arXiv:2203.07436.