Abstract

Sensory substitution contains methods which send perceptible information of a sensory organ to the brain through other sensory modalities, resulting in the rehabilitation of the lost perception. Additionally, they may also be taken into consideration as research devices to examine the brain mechanisms and its cross-modal function. Visual to auditory sensory substitution used in experimentation aimed to provide visually impaired people with environmental perception. According to this paper, vOICe based tones (VBTones) has been introduced as a tool converting visual data into auditory output, where each image row is assigned to a specific sound frequency and the sound amplitude is referred to brightness of the pixel, where the image is scanned, converted the sound is generated column by column. Gradually forming a continuous sound made of single column waves which are the sum of multiple sinusoidal waves of different amplitudes and frequencies related to specific pixels. The objective of this study is to maximize the perception of visual information through audio. Thus, elimination of unnecessary image information is required. By applying anisotropic filtering methods in addition to a Laplacian-Gaussian filter, the produced sound turned out to be finer in texture and more perceivable. Moreover, this study shows analysis of the efficiency of this tool and several enhancements on both sighted participants and the visually impaired ones. The blind scored 87% and the sighted 78% accuracy in recognition in the designed test.

Keyword

Sensory substitution, Image to audio conversion, Blind rehabilitation, Anisotropic filtering.

Cite this article

Refference

[1][1]Murphy MC, Nau AC, Fisher C, Kim SG, Schuman JS, Chan KC. Top-down influence on the visual cortex of the blind during sensory substitution. Neuroimage. 2016; 125:932-40.

[2][2]Capelle C, Trullemans C, Arno P, Veraart C. A real-time experimental prototype for enhancement of vision rehabilitation using auditory substitution. IEEE Transactions on Biomedical Engineering. 1998; 45(10):1279-93.

[3][3]Yang J, Li X, Li Q, Xiao X, Wu Q, Wu J. The effect of visual stimuli on auditory detection in an auditory attention task. In international conference on mechatronics and automation 2017 (pp. 1567-72). IEEE.

[4][4]Ward J, Meijer P. Visual experiences in the blind induced by an auditory sensory substitution device. Consciousness and Cognition. 2010; 19(1):492-500

[5][5]Bach-y-Rita P, Kercel SW. Sensory substitution and the human–machine interface. Trends in Cognitive Sciences. 2003; 7(12):541-6.

[6][6]Abboud S, Hanassy S, Levy-Tzedek S, Maidenbaum S, Amedi A. EyeMusic: introducing a visual colorful experience for the blind using auditory sensory substitution. Restorative Neurology and Neuroscience. 2014; 32(2):247-57.

[7][7]Maidenbaum S, Abboud S, Amedi A. Sensory substitution: closing the gap between basic research and widespread practical visual rehabilitation. Neuroscience & Biobehavioral Reviews. 2014; 41:3-15.

[8][8]Hamilton-Fletcher G, Wright TD, Ward J. Cross-modal correspondences enhance performance on a colour-to-sound sensory substitution device. Multisensory Research. 2016; 29(4-5):337-63.

[9][9]King AJ. Crossmodal plasticity and hearing capabilities following blindness. Cell and Tissue Research. 2015; 361(1):295-300.

[10][10]Kolarik AJ, Scarfe AC, Moore BC, Pardhan S. Blindness enhances auditory obstacle circumvention: assessing echolocation, sensory substitution, and visual-based navigation. PloS One. 2017; 12(4):1-25.

[11][11]Kumar S, Forster HM, Bailey P, Griffiths TD. Mapping unpleasantness of sounds to their auditory representation. The Journal of the Acoustical Society of America. 2008; 124(6):3810-7.

[12][12]Rossovskii LE. Image filtering with the use of anisotropic diffusion. Computational Mathematics and Mathematical Physics. 2017; 57(3):401-8.

[13][13]Black MJ, Sapiro G, Marimont DH, Heeger D. Robust anisotropic diffusion. IEEE Transactions on Image Processing. 1998; 7(3):421-32.

[14][14]Positano V, Santarelli MF, Landini L, Benassi A. Nonlinear anisotropic filtering as a tool for SNR enhancement in cardiovascular MRI. In computers in cardiology 2000 (pp. 707-10). IEEE.

[15][15]Lanzara RG. Webers law modeled by the mathematical description of a beam balance. Mathematical Biosciences. 1994; 122(1):89-94.

[16][16]Wright TD, Margolis A, Ward J. Using an auditory sensory substitution device to augment vision: evidence from eye movements. Experimental Brain Research. 2015; 233(3):851-60.

[17][17]Renier L, Collignon O, Poirier C, Tranduy D, Vanlierde A, Bol A, et al. Cross-modal activation of visual cortex during depth perception using auditory substitution of vision. Neuroimage. 2005; 26(2):573-80.

[18][18]Maidenbaum S, Buchs G, Abboud S, Lavi-Rotbain O, Amedi A. Perception of graphical virtual environments by blind users via sensory substitution. PloS One. 2016; 11(2):1-21.