Computer vision tools for animal behaviour

Computer vision methods have been developing rapidly over the past decade, but selecting and applying the best tools for specific videos and study systems is still challenging. Here are a few tools I have been developing over the years, with the hope of lowering the barrier of entry for biologists who wants to start applying computer vision in their data collection pipeline, and start automating labour-intensive manual annotations.

YOLO-Behaviour

We present YOLO-Behaviour, an easy to implement, flexible and robust method for automated behavioural classification from videos. We used the widely used object detector YOLOv8 to label behaviours within an image. We shocased the robustness of the method across 5 case studies: 1) house sparrows provisoining, 2) Siberian jay easitng, 3) human eating, 4) pigeon behaviours and 5) zebra/ giraffe behaviours.

We show that the method are able to detect behaviours effectively across all these case studies, but if the results are not accurate enough, the method can also be used as a "hybrid" method to first detect behaviours then manually correct them. Check out the preprint and documentation for more details on implementation and guides!