Smartphone-based 360? video streaming/viewing system including acoustic immersion

Kenta Niwa; Daisuke Ochi; Akio Kameda; Yutaka Kamamoto; Takehiro Moriya

Smartphone-based 360? video streaming/viewing system including acoustic immersion

Kenta Niwa, Daisuke Ochi, Akio Kameda, Yutaka Kamamoto, Takehiro Moriya

Tohoku Medical Megabank Organization (ToMMo)

Research output: Contribution to conference › Paper › peer-review

3 Citations (Scopus)

Abstract

In virtual reality (VR), 360? video services provided through head mounted displays (HMDs) or smartphones are widely used. Despite the fact that the user's viewpoint seamlessly changes, sounds through the headphones are fixed even when images change in correspondence with user head motion in many 360 ? video services. We have been studying acoustic immersion technology that is achieved by, for example, generating binaural sounds corresponding to the user head motion. Basically, our method is composed of angular region-wise source enhancement using array observation signals, multichannel audio encoding based on MPEG-4 Audio Lossless Coding (ALS), and binaural synthesizing of enhanced signals using head related transfer functions (HRTFs). In this paper, we constructed a smartphone-based real-Time system for streaming/viewing 360 ? video including acoustic immersion and evaluated it through subjective tests.

Original language	English
Publication status	Published - 2016
Event	141st Audio Engineering Society International Convention 2016, AES 2016 - Los Angeles, United States Duration: 2016 Sep 29 → 2016 Oct 2

Other

Other	141st Audio Engineering Society International Convention 2016, AES 2016
Country/Territory	United States
City	Los Angeles
Period	16/9/29 → 16/10/2

ASJC Scopus subject areas

Electrical and Electronic Engineering
Modelling and Simulation
Acoustics and Ultrasonics

Cite this

@conference{22ff88f3a4294b95b9c7a5048ad8caf5,

title = "Smartphone-based 360? video streaming/viewing system including acoustic immersion",

abstract = "In virtual reality (VR), 360? video services provided through head mounted displays (HMDs) or smartphones are widely used. Despite the fact that the user's viewpoint seamlessly changes, sounds through the headphones are fixed even when images change in correspondence with user head motion in many 360 ? video services. We have been studying acoustic immersion technology that is achieved by, for example, generating binaural sounds corresponding to the user head motion. Basically, our method is composed of angular region-wise source enhancement using array observation signals, multichannel audio encoding based on MPEG-4 Audio Lossless Coding (ALS), and binaural synthesizing of enhanced signals using head related transfer functions (HRTFs). In this paper, we constructed a smartphone-based real-Time system for streaming/viewing 360 ? video including acoustic immersion and evaluated it through subjective tests.",

author = "Kenta Niwa and Daisuke Ochi and Akio Kameda and Yutaka Kamamoto and Takehiro Moriya",

year = "2016",

language = "English",

note = "141st Audio Engineering Society International Convention 2016, AES 2016 ; Conference date: 29-09-2016 Through 02-10-2016",

}

TY - CONF

T1 - Smartphone-based 360? video streaming/viewing system including acoustic immersion

AU - Niwa, Kenta

AU - Ochi, Daisuke

AU - Kameda, Akio

AU - Kamamoto, Yutaka

AU - Moriya, Takehiro

PY - 2016

Y1 - 2016

N2 - In virtual reality (VR), 360? video services provided through head mounted displays (HMDs) or smartphones are widely used. Despite the fact that the user's viewpoint seamlessly changes, sounds through the headphones are fixed even when images change in correspondence with user head motion in many 360 ? video services. We have been studying acoustic immersion technology that is achieved by, for example, generating binaural sounds corresponding to the user head motion. Basically, our method is composed of angular region-wise source enhancement using array observation signals, multichannel audio encoding based on MPEG-4 Audio Lossless Coding (ALS), and binaural synthesizing of enhanced signals using head related transfer functions (HRTFs). In this paper, we constructed a smartphone-based real-Time system for streaming/viewing 360 ? video including acoustic immersion and evaluated it through subjective tests.

AB - In virtual reality (VR), 360? video services provided through head mounted displays (HMDs) or smartphones are widely used. Despite the fact that the user's viewpoint seamlessly changes, sounds through the headphones are fixed even when images change in correspondence with user head motion in many 360 ? video services. We have been studying acoustic immersion technology that is achieved by, for example, generating binaural sounds corresponding to the user head motion. Basically, our method is composed of angular region-wise source enhancement using array observation signals, multichannel audio encoding based on MPEG-4 Audio Lossless Coding (ALS), and binaural synthesizing of enhanced signals using head related transfer functions (HRTFs). In this paper, we constructed a smartphone-based real-Time system for streaming/viewing 360 ? video including acoustic immersion and evaluated it through subjective tests.

UR - http://www.scopus.com/inward/record.url?scp=85010791776&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85010791776&partnerID=8YFLogxK

M3 - Paper

AN - SCOPUS:85010791776

T2 - 141st Audio Engineering Society International Convention 2016, AES 2016

Y2 - 29 September 2016 through 2 October 2016

ER -

Smartphone-based 360? video streaming/viewing system including acoustic immersion

Abstract

Other

ASJC Scopus subject areas

Other files and links

Fingerprint

Cite this