The social lives of Siberian Jays

Siberian Jays (Perisoreus infaustus) are group-living corvids living in the Swedish Lapland forests. Based in Luondua Boreal Field Station, I worked on many aspects of the Jay system, from developing computer vision algorithms to automatically quantify who and what the birds are doing, to running social learning experiments to determine how individuals learn from each other.

CHIRP Dataset

Animal-based computer vision datasets are increasingly common. However, solving computer vision problems in biology often requires combining multiple computer vision tasks (e.g. solving behavioural detection is often not enough, if you dont know who is doing the behaviour). Here, we propose a computer vision dataset that aims to create solutions for long-term, individual-level behavioural monitoring. To do that, we need to know "who" is doing "what". We present a task-diverse dataset covering video re-id, action recognition, 2D keypoint estimation, object detection and instance segmentation. Importantly, we introduce an "application specific benchmark", by combining methods to evaluate on biologically relevant metrics, allowing biologists to readily select appropriate models for deployment. 

CORVID: COlour-based Video re-ID

Colour rings are one of the most common ways for brids to be individually identified. As part of the CHIRP paper, we also introduce my first attempt on using computer vision to automatically detect colour rings and infer bird identities.  Check out the paper for detailed benchmark results, but this method is not quite there yet, and further developments would be needed to really deploy this with confidence.

Social learning in Siberian Jays

Finally, over 2023-2025, we ran a social learning experiment in the Siberian Jays, presenting two two-action tasks to determine how juveniles learn a novel foraging task from adults in social groups. Results coming soon!