{"id":42993,"date":"2023-07-07T16:45:07","date_gmt":"2023-07-07T14:45:07","guid":{"rendered":"https:\/\/frh-fondation.ch\/project\/sound-spatialisation\/"},"modified":"2024-11-19T17:57:06","modified_gmt":"2024-11-19T16:57:06","slug":"sound-spatialisation","status":"publish","type":"project","link":"https:\/\/frh-fondation.ch\/en\/project\/sound-spatialisation\/","title":{"rendered":"Sound spatialisation"},"content":{"rendered":"[et_pb_section fb_built=&#8221;1&#8243; _builder_version=&#8221;4.17.0&#8243; _module_preset=&#8221;default&#8221; global_colors_info=&#8221;{}&#8221;][et_pb_row _builder_version=&#8221;4.17.0&#8243; _module_preset=&#8221;default&#8221; global_colors_info=&#8221;{}&#8221;][et_pb_column type=&#8221;4_4&#8243; _builder_version=&#8221;4.17.0&#8243; _module_preset=&#8221;default&#8221; global_colors_info=&#8221;{}&#8221;][et_pb_text _builder_version=&#8221;4.17.0&#8243; _module_preset=&#8221;default&#8221; custom_padding=&#8221;0px||6px|||&#8221; global_colors_info=&#8221;{}&#8221;]\n<h1><strong>Sound spatialisation<\/strong><\/h1>\n<h4><strong>The impact of sound spatialisation in automatic speech recognition for the hearing impaired<\/strong><\/h4>\n<p>&nbsp;<\/p>\n<p><strong>Type of project:<\/strong>\u00a0Exploratory research<\/p>\n<p><strong>Disability concerned:\u00a0<\/strong>Hearing impairment<\/p>\n<p><strong>Topics:\u00a0<\/strong>Autonomy, Communication<\/p>\n<p><strong>Status:\u00a0<\/strong>Completed<\/p>\n[\/et_pb_text][et_pb_text _builder_version=&#8221;4.17.0&#8243; _module_preset=&#8221;default&#8221; hover_enabled=&#8221;0&#8243; global_colors_info=&#8221;{}&#8221; sticky_enabled=&#8221;0&#8243;]\n<h5>This exploratory research aims to assess the usefulness of a digital display with spatialized Automatic Speech Recognition (ASR) for hearing-impaired people. We attempted to develop a quick AR prototype, then proceeded to implement two web prototypes and to test them with actual users.<\/h5>\n[\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row column_structure=&#8221;1_3,1_3,1_3&#8243; _builder_version=&#8221;4.17.0&#8243; _module_preset=&#8221;default&#8221; global_colors_info=&#8221;{}&#8221;][et_pb_column type=&#8221;1_3&#8243; _builder_version=&#8221;4.17.0&#8243; _module_preset=&#8221;default&#8221; global_colors_info=&#8221;{}&#8221;][et_pb_text _builder_version=&#8221;4.17.0&#8243; _module_preset=&#8221;default&#8221; global_colors_info=&#8221;{}&#8221;]\n<p>Hearing-impaired people are isolated from discussions, severely impacting their social and professional lives. In this regard, they could benefit from Automatic Speech Recognition to help catch up during meetings. Currently, ASR is facing a few challenges. Most common systems, such as these embedded within online meeting software, fail to offer significant benefits to hearing-impaired people.<\/p>\n<ul>\n<li>The textual output is usually poorly located at the bottom of the screen, preventing users from seeing the facial expressions of the speakers.<\/li>\n<li>That textual output does not seek to convey a feeling of the location of the participants, and all the interventions are simply stacked one over the others as people speak.<\/li>\n<li>Finally, while remote meetings may benefit from ASR, it is much less common to use similar technologies in co-located meetings. Using ASR in such a context is difficult, mostly because participants cannot be isolated and \u00ab spatialized \u00bb properly with a single input source.<\/li>\n<\/ul>\n<p>Current researches work on spatialization algorithms and in the perspective of successful outcomes, we were interested in anticipating their benefits. We thus designed and implemented a \u201cwizard-of-oz\u201d prototype, along with a \u201cwithin-subject\u201d test protocol to evaluate the interest of hearing-impaired participants for this technology.<\/p>\n<p>The interface itself is a Node.js application using embedded Speech Recognition\u2019s browser libraries and Websockets. It translates actual inputs on-the-go and proved to be sufficiently accurate for us to consider using it in both English and French.<\/p>\n[\/et_pb_text][\/et_pb_column][et_pb_column type=&#8221;1_3&#8243; _builder_version=&#8221;4.17.0&#8243; _module_preset=&#8221;default&#8221; global_colors_info=&#8221;{}&#8221;][et_pb_text _builder_version=&#8221;4.17.0&#8243; _module_preset=&#8221;default&#8221; global_colors_info=&#8221;{}&#8221;]\n<p>The test protocol relies on a single independent variable: the device where the interface is displayed. While we intended to display the interface on AR glasses, we had to settle for tablet vs smartphone. We led two test sessions with 5 users. This study yielded mostly positive feedback, managed to highlight interesting trends and new avenues that we did not think of. However, the limited number of participants, along with further variations introduced during the test (language, smartphone real estate, etc), prevent us from providing proper quantitative results.<\/p>\n<p>Leading such research within a Web Agency is a difficult task, as most employees do not benefit from enough leeway to adapt the impediments of research. As an example, recruiting participants was a much harder task than anticipated, and we had to downsize our expectations.<\/p>\n<p>Implementing a successful interface was quick. We see two issues with Speech Recognition APIs: (1) they are provided by companies with ethical issues, open source technologies significantly lag behind. (2) they usually fail to recognize technical terms and names. The NReal glasses proved challenging. We could not afford sufficient learning time and ultimately failed to use them. Devices backed by robust companies might offer a smoother learning curve.<\/p>\n<p>The exchanges between partners offered invaluable insights. Three examples:<\/p>\n[\/et_pb_text][\/et_pb_column][et_pb_column type=&#8221;1_3&#8243; _builder_version=&#8221;4.17.0&#8243; _module_preset=&#8221;default&#8221; global_colors_info=&#8221;{}&#8221;][et_pb_text _builder_version=&#8221;4.17.0&#8243; _module_preset=&#8221;default&#8221; global_colors_info=&#8221;{}&#8221;]\n<ul>\n<li>Visible devices might increase the feeling of social stigma and be discarded by their users.<\/li>\n<li>Our interface seemed more engaging than some specialised applications (Ava was mentioned)<\/li>\n<li>Reading facial expressions is a must for hearing-impaired people. The larger real estate provided by the tablet was needed, yet the smartphone allowed them to see the speakers.<\/li>\n<\/ul>\n<p>We would like to invest more time to project our interface directly within a set of AR glasses, as planned initially, since we believe this could solve both the real estate issue of the smartphone and the loss of focus of the tablet. The reliance on Microsoft and Google to recognize speech is also a hurdle we\u2019d like to overcome, by furthering our exploration of open source alternatives. Our partner from SignX also mentioned that using generative AIs to summarise live speeches might be very handy, and we\u2019d like to experiment on that point, using open source technologies here too, whenever possible. From a research perspective, fine-tuning our protocol (introducing a distinction between deaf and partially deaf, ensuring the balance of both phases, etc) and finding a sufficient number of participants would offer a clearer understanding of their needs. This understanding would help everyone, from private companies to academic researchers.<\/p>\n[\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=&#8221;4.17.0&#8243; _module_preset=&#8221;default&#8221; global_colors_info=&#8221;{}&#8221;][et_pb_column type=&#8221;4_4&#8243; _builder_version=&#8221;4.17.0&#8243; _module_preset=&#8221;default&#8221; global_colors_info=&#8221;{}&#8221;][et_pb_image src=&#8221;https:\/\/frh-fondation.ch\/wp-content\/uploads\/2023\/07\/group-1825513_1280-1024&#215;456.jpg&#8221; alt=&#8221;Drawing of a group of people with coloured speech bubbles above their heads.&#8221; title_text=&#8221;group-1825513_1280&#8243; align=&#8221;center&#8221; _builder_version=&#8221;4.17.0&#8243; _module_preset=&#8221;default&#8221; global_colors_info=&#8221;{}&#8221;][\/et_pb_image][\/et_pb_column][\/et_pb_row][et_pb_row disabled_on=&#8221;off|off|off&#8221; _builder_version=&#8221;4.17.0&#8243; _module_preset=&#8221;default&#8221; global_colors_info=&#8221;{}&#8221;][et_pb_column type=&#8221;4_4&#8243; _builder_version=&#8221;4.17.0&#8243; _module_preset=&#8221;default&#8221; global_colors_info=&#8221;{}&#8221;][et_pb_text _builder_version=&#8221;4.17.0&#8243; _module_preset=&#8221;default&#8221; text_font=&#8221;|300|||||||&#8221; text_line_height=&#8221;-1.3em&#8221; max_width=&#8221;100%&#8221; global_colors_info=&#8221;{}&#8221;]\n<h4><strong>Contact<\/strong><\/h4>\n<p>Liip SA<\/p>\n<p>Donato Rotunno<\/p>\n<p><a href=\"mailto:donato.rotunno@liip.ch\">donato.rotunno@liip.ch<\/a>\u00a0<\/p>\n[\/et_pb_text][\/et_pb_column][\/et_pb_row][\/et_pb_section]\n","protected":false},"excerpt":{"rendered":"<p>This exploratory research aims to assess the usefulness of a digital display with spatialized Automatic Speech Recognition (ASR) for hearing-impaired people. We attempted to develop a quick AR prototype, then proceeded to implement two web prototypes and to test them with actual users.<\/p>\n","protected":false},"author":41,"featured_media":42454,"comment_status":"open","ping_status":"closed","template":"","meta":{"_et_pb_use_builder":"on","_et_pb_old_content":"","_et_gb_content_width":""},"thematiques":[204],"project_category":[163],"project_tag":[171],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v21.4 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Sound spatialisation - FRH Fondation<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/frh-fondation.ch\/project\/spatialisation-du-son\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Sound spatialisation - FRH Fondation\" \/>\n<meta property=\"og:description\" content=\"This exploratory research aims to assess the usefulness of a digital display with spatialized Automatic Speech Recognition (ASR) for hearing-impaired people. We attempted to develop a quick AR prototype, then proceeded to implement two web prototypes and to test them with actual users.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/frh-fondation.ch\/project\/spatialisation-du-son\/\" \/>\n<meta property=\"og:site_name\" content=\"FRH Fondation\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/IB.TechAndSpecialNeeds\/\" \/>\n<meta property=\"article:modified_time\" content=\"2024-11-19T16:57:06+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/frh-fondation.ch\/wp-content\/uploads\/2023\/07\/group-1825513_1280.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1280\" \/>\n\t<meta property=\"og:image:height\" content=\"570\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:site\" content=\"@IB_SpecialNeeds\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/frh-fondation.ch\/project\/spatialisation-du-son\/\",\"url\":\"https:\/\/frh-fondation.ch\/project\/spatialisation-du-son\/\",\"name\":\"Sound spatialisation - FRH Fondation\",\"isPartOf\":{\"@id\":\"https:\/\/frh-fondation.ch\/#website\"},\"datePublished\":\"2023-07-07T14:45:07+00:00\",\"dateModified\":\"2024-11-19T16:57:06+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/frh-fondation.ch\/project\/spatialisation-du-son\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/frh-fondation.ch\/project\/spatialisation-du-son\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/frh-fondation.ch\/project\/spatialisation-du-son\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Accueil\",\"item\":\"https:\/\/frh-fondation.ch\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Projects\",\"item\":\"https:\/\/frh-fondation.ch\/en\/project\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Sound spatialisation\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/frh-fondation.ch\/#website\",\"url\":\"https:\/\/frh-fondation.ch\/\",\"name\":\"FRH Fondation\",\"description\":\"Fondation pour la recherche en faveur des personnes handicap\u00e9es\",\"publisher\":{\"@id\":\"https:\/\/frh-fondation.ch\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/frh-fondation.ch\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/frh-fondation.ch\/#organization\",\"name\":\"FRH Fondation\",\"url\":\"https:\/\/frh-fondation.ch\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/frh-fondation.ch\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/frh-fondation.ch\/wp-content\/uploads\/2021\/04\/logo-frh-288x110-1.png\",\"contentUrl\":\"https:\/\/frh-fondation.ch\/wp-content\/uploads\/2021\/04\/logo-frh-288x110-1.png\",\"width\":3402,\"height\":1299,\"caption\":\"FRH Fondation\"},\"image\":{\"@id\":\"https:\/\/frh-fondation.ch\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/IB.TechAndSpecialNeeds\/\",\"https:\/\/twitter.com\/IB_SpecialNeeds\",\"https:\/\/www.instagram.com\/IB_TechAndSpecialNeeds\/\",\"https:\/\/linkedin.com\/company\/ib-technology-and-special-needs\",\"https:\/\/www.youtube.com\/channel\/UCZ9Bvy8FfMtEQ3uqrBvIIHg\"]}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Sound spatialisation - FRH Fondation","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/frh-fondation.ch\/project\/spatialisation-du-son\/","og_locale":"en_US","og_type":"article","og_title":"Sound spatialisation - FRH Fondation","og_description":"This exploratory research aims to assess the usefulness of a digital display with spatialized Automatic Speech Recognition (ASR) for hearing-impaired people. We attempted to develop a quick AR prototype, then proceeded to implement two web prototypes and to test them with actual users.","og_url":"https:\/\/frh-fondation.ch\/project\/spatialisation-du-son\/","og_site_name":"FRH Fondation","article_publisher":"https:\/\/www.facebook.com\/IB.TechAndSpecialNeeds\/","article_modified_time":"2024-11-19T16:57:06+00:00","og_image":[{"width":1280,"height":570,"url":"https:\/\/frh-fondation.ch\/wp-content\/uploads\/2023\/07\/group-1825513_1280.jpg","type":"image\/jpeg"}],"twitter_card":"summary_large_image","twitter_site":"@IB_SpecialNeeds","twitter_misc":{"Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/frh-fondation.ch\/project\/spatialisation-du-son\/","url":"https:\/\/frh-fondation.ch\/project\/spatialisation-du-son\/","name":"Sound spatialisation - FRH Fondation","isPartOf":{"@id":"https:\/\/frh-fondation.ch\/#website"},"datePublished":"2023-07-07T14:45:07+00:00","dateModified":"2024-11-19T16:57:06+00:00","breadcrumb":{"@id":"https:\/\/frh-fondation.ch\/project\/spatialisation-du-son\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/frh-fondation.ch\/project\/spatialisation-du-son\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/frh-fondation.ch\/project\/spatialisation-du-son\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Accueil","item":"https:\/\/frh-fondation.ch\/"},{"@type":"ListItem","position":2,"name":"Projects","item":"https:\/\/frh-fondation.ch\/en\/project\/"},{"@type":"ListItem","position":3,"name":"Sound spatialisation"}]},{"@type":"WebSite","@id":"https:\/\/frh-fondation.ch\/#website","url":"https:\/\/frh-fondation.ch\/","name":"FRH Fondation","description":"Fondation pour la recherche en faveur des personnes handicap\u00e9es","publisher":{"@id":"https:\/\/frh-fondation.ch\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/frh-fondation.ch\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/frh-fondation.ch\/#organization","name":"FRH Fondation","url":"https:\/\/frh-fondation.ch\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/frh-fondation.ch\/#\/schema\/logo\/image\/","url":"https:\/\/frh-fondation.ch\/wp-content\/uploads\/2021\/04\/logo-frh-288x110-1.png","contentUrl":"https:\/\/frh-fondation.ch\/wp-content\/uploads\/2021\/04\/logo-frh-288x110-1.png","width":3402,"height":1299,"caption":"FRH Fondation"},"image":{"@id":"https:\/\/frh-fondation.ch\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/IB.TechAndSpecialNeeds\/","https:\/\/twitter.com\/IB_SpecialNeeds","https:\/\/www.instagram.com\/IB_TechAndSpecialNeeds\/","https:\/\/linkedin.com\/company\/ib-technology-and-special-needs","https:\/\/www.youtube.com\/channel\/UCZ9Bvy8FfMtEQ3uqrBvIIHg"]}]}},"_links":{"self":[{"href":"https:\/\/frh-fondation.ch\/en\/wp-json\/wp\/v2\/project\/42993"}],"collection":[{"href":"https:\/\/frh-fondation.ch\/en\/wp-json\/wp\/v2\/project"}],"about":[{"href":"https:\/\/frh-fondation.ch\/en\/wp-json\/wp\/v2\/types\/project"}],"author":[{"embeddable":true,"href":"https:\/\/frh-fondation.ch\/en\/wp-json\/wp\/v2\/users\/41"}],"replies":[{"embeddable":true,"href":"https:\/\/frh-fondation.ch\/en\/wp-json\/wp\/v2\/comments?post=42993"}],"version-history":[{"count":15,"href":"https:\/\/frh-fondation.ch\/en\/wp-json\/wp\/v2\/project\/42993\/revisions"}],"predecessor-version":[{"id":49558,"href":"https:\/\/frh-fondation.ch\/en\/wp-json\/wp\/v2\/project\/42993\/revisions\/49558"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/frh-fondation.ch\/en\/wp-json\/wp\/v2\/media\/42454"}],"wp:attachment":[{"href":"https:\/\/frh-fondation.ch\/en\/wp-json\/wp\/v2\/media?parent=42993"}],"wp:term":[{"taxonomy":"thematiques","embeddable":true,"href":"https:\/\/frh-fondation.ch\/en\/wp-json\/wp\/v2\/thematiques?post=42993"},{"taxonomy":"project_category","embeddable":true,"href":"https:\/\/frh-fondation.ch\/en\/wp-json\/wp\/v2\/project_category?post=42993"},{"taxonomy":"project_tag","embeddable":true,"href":"https:\/\/frh-fondation.ch\/en\/wp-json\/wp\/v2\/project_tag?post=42993"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}