Rescue Dog Action Recognition by Integrating Ego-Centric Video, Sound and Sensor Information

Yuta Ide, Tsuyohito Araki, Ryunosuke Hamada, Kazunori Ohno, Keiji Yanai

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

A dog which assists rescue activity in the scene of disasters such as earthquakes and landslides is called a “disaster rescue dog” or just a “rescue dog”. In Japan where earthquakes happen frequently, a research project on “Cyber-Rescue” is being organized for more efficient rescue activities. In the project, to analyze the activities of rescue dogs in the scene of disasters, “Cyber Dog Suits” equipped with sensors, a camera and a GPS were developed. In this work, we recognize dog activities in the ego-centric dog videos taken by the camera mounted on the cyber-dog suits. To do that, we propose an image/sound/sensor-based four-stream CNN for dog activity recognition which integrates sound and sensor signals as well as motion and appearance. We conducted some experiments for multi-class activity categorization using the proposed method. As a result, the proposed method which integrates appearance, motion, sound and sensor information achieved the highest accuracy, 48.05%. This result is relatively high as a recognition result of ego-centric videos.

Original languageEnglish
Title of host publicationPattern Recognition. ICPR International Workshops and Challenges, 2021, Proceedings
EditorsAlberto Del Bimbo, Rita Cucchiara, Stan Sclaroff, Giovanni Maria Farinella, Tao Mei, Marco Bertini, Hugo Jair Escalante, Roberto Vezzani
PublisherSpringer Science and Business Media Deutschland GmbH
Pages321-333
Number of pages13
ISBN (Print)9783030687953
DOIs
Publication statusPublished - 2021
Event25th International Conference on Pattern Recognition Workshops, ICPR 2020 - Milan, Italy
Duration: 2021 Jan 102021 Jan 11

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume12663 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference25th International Conference on Pattern Recognition Workshops, ICPR 2020
Country/TerritoryItaly
CityMilan
Period21/1/1021/1/11

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint

Dive into the research topics of 'Rescue Dog Action Recognition by Integrating Ego-Centric Video, Sound and Sensor Information'. Together they form a unique fingerprint.

Cite this