{"id":111862,"date":"2023-04-07T11:06:23","date_gmt":"2023-04-07T15:06:23","guid":{"rendered":"https:\/\/www.hajim.rochester.edu\/senior-design-day\/?p=111862"},"modified":"2024-03-01T15:42:01","modified_gmt":"2024-03-01T19:42:01","slug":"unsupervised-arousal-valence-estimation-from-speech-and-corresponding-discrete-emotion","status":"publish","type":"post","link":"https:\/\/www.hajim.rochester.edu\/senior-design-day\/unsupervised-arousal-valence-estimation-from-speech-and-corresponding-discrete-emotion\/","title":{"rendered":"Unsupervised Arousal Valence Estimation from Speech and Corresponding Discrete Emotion"},"content":{"rendered":"\r\n<h4 class=\"has-text-align-center wp-block-heading\">Enting Zhou<\/h4>\r\n\r\n\r\n\r\n<h2 class=\"wp-block-heading\">Advisor<\/h2>\r\n\r\n\r\n\r\n<p>Zhiyao Duan<\/p>\r\n\r\n\r\n\r\n<h2 class=\"wp-block-heading\">Committee Members<\/h2>\r\n\r\n\r\n\r\n<p>Ross Maddox, Jiebo Luo<\/p>\r\n\r\n\r\n\r\n<h2 class=\"wp-block-heading\">Abstract<\/h2>\r\n\r\n\r\n\r\n<p><span class=\"TextRun SCXW69263389 BCX0\" lang=\"EN-US\" xml:lang=\"EN-US\" data-contrast=\"none\"><span class=\"NormalTextRun SCXW69263389 BCX0\">Data scarcity issues have been a long-standing challenge for speech emotion recognition (SER) tasks. The issue is more severe for dimensional emotion (e.g., arousal and valence) estimation tasks due to the increased difficulty in the annotation of dimensional values. This study proposes a semi-supervised method for obtaining arousal-valence annotations of a speech corpus when only discrete emotion category information is available. Our method proposes to compute the weighted sum of intermediate outputs of large-scale pre-trained speech model <\/span><span class=\"SpellingError SCXW69263389 BCX0\">wavLM<\/span><span class=\"NormalTextRun SCXW69263389 BCX0\"> as utterance-level speech embeddings and combine with a linear MLP to extract speech emotion features. Then, the high-dimensional speech emotion features are mapped to the Arousal-Valence space using a modified version of the dimensionality reduction algorithm UMATO with the aid of speech utterance&#8217;s coarse emotion category label. Results show comparable performance with supervised regression models on the IEMOCAP dataset, and further experiments on other datasets demonstrate the method&#8217;s universal applicability. The proposed method can reduce the labor-intensive task of dimensional emotion labeling and be useful in scenarios where dimensional values are required.<\/span><\/span><span class=\"EOP SCXW69263389 BCX0\" data-ccp-props=\"{&quot;134233118&quot;:false,&quot;134233279&quot;:true,&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559739&quot;:120,&quot;335559740&quot;:240}\">\u00a0<\/span><\/p>\r\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-137812\" src=\"https:\/\/www.hajim.rochester.edu\/senior-design-day\/wp-content\/uploads\/2023\/04\/Enting-Zhou-Design-Day-Poster.jpg\" alt=\"\" width=\"10800\" height=\"8294\" \/><\/p>\r\n","protected":false},"excerpt":{"rendered":"<p>This study proposes a semi-supervised method for obtaining arousal-valence annotations of a speech corpus when only discrete emotion category information is available. <\/p>\n","protected":false},"author":6242,"featured_media":137812,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_coblocks_attr":"","_coblocks_dimensions":"","_coblocks_responsive_height":"","_coblocks_accordion_ie_support":"","_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[96],"tags":[16152,16172,16162],"coauthors":[8612],"class_list":["post-111862","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-csc-archive","tag-automatic-speech-understanding","tag-representation-learning","tag-speech-emotion-recognition"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.5 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Unsupervised Arousal Valence Estimation from Speech and Corresponding Discrete Emotion - Senior Design Day<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.hajim.rochester.edu\/senior-design-day\/unsupervised-arousal-valence-estimation-from-speech-and-corresponding-discrete-emotion\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Unsupervised Arousal Valence Estimation from Speech and Corresponding Discrete Emotion - Senior Design Day\" \/>\n<meta property=\"og:description\" content=\"This study proposes a semi-supervised method for obtaining arousal-valence annotations of a speech corpus when only discrete emotion category information is available.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.hajim.rochester.edu\/senior-design-day\/unsupervised-arousal-valence-estimation-from-speech-and-corresponding-discrete-emotion\/\" \/>\n<meta property=\"og:site_name\" content=\"Senior Design Day\" \/>\n<meta property=\"article:published_time\" content=\"2023-04-07T15:06:23+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-03-01T19:42:01+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.hajim.rochester.edu\/senior-design-day\/wp-content\/uploads\/2023\/04\/Enting-Zhou-Design-Day-Poster.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"820\" \/>\n\t<meta property=\"og:image:height\" content=\"630\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"admin\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"admin\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/www.hajim.rochester.edu\\\/senior-design-day\\\/unsupervised-arousal-valence-estimation-from-speech-and-corresponding-discrete-emotion\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.hajim.rochester.edu\\\/senior-design-day\\\/unsupervised-arousal-valence-estimation-from-speech-and-corresponding-discrete-emotion\\\/\"},\"author\":{\"name\":\"admin\",\"@id\":\"https:\\\/\\\/www.hajim.rochester.edu\\\/senior-design-day\\\/#\\\/schema\\\/person\\\/351018fbcf84ed8cac6d8072ba5b347c\"},\"headline\":\"Unsupervised Arousal Valence Estimation from Speech and Corresponding Discrete Emotion\",\"datePublished\":\"2023-04-07T15:06:23+00:00\",\"dateModified\":\"2024-03-01T19:42:01+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/www.hajim.rochester.edu\\\/senior-design-day\\\/unsupervised-arousal-valence-estimation-from-speech-and-corresponding-discrete-emotion\\\/\"},\"wordCount\":195,\"image\":{\"@id\":\"https:\\\/\\\/www.hajim.rochester.edu\\\/senior-design-day\\\/unsupervised-arousal-valence-estimation-from-speech-and-corresponding-discrete-emotion\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.hajim.rochester.edu\\\/senior-design-day\\\/wp-content\\\/uploads\\\/2023\\\/04\\\/Enting-Zhou-Design-Day-Poster.jpg\",\"keywords\":[\"Automatic Speech Understanding\",\"Representation Learning\",\"Speech Emotion Recognition\"],\"articleSection\":[\"CSC Archive\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.hajim.rochester.edu\\\/senior-design-day\\\/unsupervised-arousal-valence-estimation-from-speech-and-corresponding-discrete-emotion\\\/\",\"url\":\"https:\\\/\\\/www.hajim.rochester.edu\\\/senior-design-day\\\/unsupervised-arousal-valence-estimation-from-speech-and-corresponding-discrete-emotion\\\/\",\"name\":\"Unsupervised Arousal Valence Estimation from Speech and Corresponding Discrete Emotion - Senior Design Day\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.hajim.rochester.edu\\\/senior-design-day\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/www.hajim.rochester.edu\\\/senior-design-day\\\/unsupervised-arousal-valence-estimation-from-speech-and-corresponding-discrete-emotion\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/www.hajim.rochester.edu\\\/senior-design-day\\\/unsupervised-arousal-valence-estimation-from-speech-and-corresponding-discrete-emotion\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.hajim.rochester.edu\\\/senior-design-day\\\/wp-content\\\/uploads\\\/2023\\\/04\\\/Enting-Zhou-Design-Day-Poster.jpg\",\"datePublished\":\"2023-04-07T15:06:23+00:00\",\"dateModified\":\"2024-03-01T19:42:01+00:00\",\"author\":{\"@id\":\"https:\\\/\\\/www.hajim.rochester.edu\\\/senior-design-day\\\/#\\\/schema\\\/person\\\/351018fbcf84ed8cac6d8072ba5b347c\"},\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.hajim.rochester.edu\\\/senior-design-day\\\/unsupervised-arousal-valence-estimation-from-speech-and-corresponding-discrete-emotion\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.hajim.rochester.edu\\\/senior-design-day\\\/unsupervised-arousal-valence-estimation-from-speech-and-corresponding-discrete-emotion\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/www.hajim.rochester.edu\\\/senior-design-day\\\/unsupervised-arousal-valence-estimation-from-speech-and-corresponding-discrete-emotion\\\/#primaryimage\",\"url\":\"https:\\\/\\\/www.hajim.rochester.edu\\\/senior-design-day\\\/wp-content\\\/uploads\\\/2023\\\/04\\\/Enting-Zhou-Design-Day-Poster.jpg\",\"contentUrl\":\"https:\\\/\\\/www.hajim.rochester.edu\\\/senior-design-day\\\/wp-content\\\/uploads\\\/2023\\\/04\\\/Enting-Zhou-Design-Day-Poster.jpg\",\"width\":10800,\"height\":8294},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.hajim.rochester.edu\\\/senior-design-day\\\/unsupervised-arousal-valence-estimation-from-speech-and-corresponding-discrete-emotion\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/www.hajim.rochester.edu\\\/senior-design-day\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Unsupervised Arousal Valence Estimation from Speech and Corresponding Discrete Emotion\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.hajim.rochester.edu\\\/senior-design-day\\\/#website\",\"url\":\"https:\\\/\\\/www.hajim.rochester.edu\\\/senior-design-day\\\/\",\"name\":\"Senior Design Day\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.hajim.rochester.edu\\\/senior-design-day\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/www.hajim.rochester.edu\\\/senior-design-day\\\/#\\\/schema\\\/person\\\/351018fbcf84ed8cac6d8072ba5b347c\",\"name\":\"admin\",\"url\":\"https:\\\/\\\/www.hajim.rochester.edu\\\/senior-design-day\\\/author\\\/seniordesign\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Unsupervised Arousal Valence Estimation from Speech and Corresponding Discrete Emotion - Senior Design Day","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.hajim.rochester.edu\/senior-design-day\/unsupervised-arousal-valence-estimation-from-speech-and-corresponding-discrete-emotion\/","og_locale":"en_US","og_type":"article","og_title":"Unsupervised Arousal Valence Estimation from Speech and Corresponding Discrete Emotion - Senior Design Day","og_description":"This study proposes a semi-supervised method for obtaining arousal-valence annotations of a speech corpus when only discrete emotion category information is available.","og_url":"https:\/\/www.hajim.rochester.edu\/senior-design-day\/unsupervised-arousal-valence-estimation-from-speech-and-corresponding-discrete-emotion\/","og_site_name":"Senior Design Day","article_published_time":"2023-04-07T15:06:23+00:00","article_modified_time":"2024-03-01T19:42:01+00:00","og_image":[{"url":"https:\/\/www.hajim.rochester.edu\/senior-design-day\/wp-content\/uploads\/2023\/04\/Enting-Zhou-Design-Day-Poster.jpg","width":820,"height":630,"type":"image\/jpeg"}],"author":"admin","twitter_card":"summary_large_image","twitter_misc":{"Written by":"admin","Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.hajim.rochester.edu\/senior-design-day\/unsupervised-arousal-valence-estimation-from-speech-and-corresponding-discrete-emotion\/#article","isPartOf":{"@id":"https:\/\/www.hajim.rochester.edu\/senior-design-day\/unsupervised-arousal-valence-estimation-from-speech-and-corresponding-discrete-emotion\/"},"author":{"name":"admin","@id":"https:\/\/www.hajim.rochester.edu\/senior-design-day\/#\/schema\/person\/351018fbcf84ed8cac6d8072ba5b347c"},"headline":"Unsupervised Arousal Valence Estimation from Speech and Corresponding Discrete Emotion","datePublished":"2023-04-07T15:06:23+00:00","dateModified":"2024-03-01T19:42:01+00:00","mainEntityOfPage":{"@id":"https:\/\/www.hajim.rochester.edu\/senior-design-day\/unsupervised-arousal-valence-estimation-from-speech-and-corresponding-discrete-emotion\/"},"wordCount":195,"image":{"@id":"https:\/\/www.hajim.rochester.edu\/senior-design-day\/unsupervised-arousal-valence-estimation-from-speech-and-corresponding-discrete-emotion\/#primaryimage"},"thumbnailUrl":"https:\/\/www.hajim.rochester.edu\/senior-design-day\/wp-content\/uploads\/2023\/04\/Enting-Zhou-Design-Day-Poster.jpg","keywords":["Automatic Speech Understanding","Representation Learning","Speech Emotion Recognition"],"articleSection":["CSC Archive"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.hajim.rochester.edu\/senior-design-day\/unsupervised-arousal-valence-estimation-from-speech-and-corresponding-discrete-emotion\/","url":"https:\/\/www.hajim.rochester.edu\/senior-design-day\/unsupervised-arousal-valence-estimation-from-speech-and-corresponding-discrete-emotion\/","name":"Unsupervised Arousal Valence Estimation from Speech and Corresponding Discrete Emotion - Senior Design Day","isPartOf":{"@id":"https:\/\/www.hajim.rochester.edu\/senior-design-day\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.hajim.rochester.edu\/senior-design-day\/unsupervised-arousal-valence-estimation-from-speech-and-corresponding-discrete-emotion\/#primaryimage"},"image":{"@id":"https:\/\/www.hajim.rochester.edu\/senior-design-day\/unsupervised-arousal-valence-estimation-from-speech-and-corresponding-discrete-emotion\/#primaryimage"},"thumbnailUrl":"https:\/\/www.hajim.rochester.edu\/senior-design-day\/wp-content\/uploads\/2023\/04\/Enting-Zhou-Design-Day-Poster.jpg","datePublished":"2023-04-07T15:06:23+00:00","dateModified":"2024-03-01T19:42:01+00:00","author":{"@id":"https:\/\/www.hajim.rochester.edu\/senior-design-day\/#\/schema\/person\/351018fbcf84ed8cac6d8072ba5b347c"},"breadcrumb":{"@id":"https:\/\/www.hajim.rochester.edu\/senior-design-day\/unsupervised-arousal-valence-estimation-from-speech-and-corresponding-discrete-emotion\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.hajim.rochester.edu\/senior-design-day\/unsupervised-arousal-valence-estimation-from-speech-and-corresponding-discrete-emotion\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.hajim.rochester.edu\/senior-design-day\/unsupervised-arousal-valence-estimation-from-speech-and-corresponding-discrete-emotion\/#primaryimage","url":"https:\/\/www.hajim.rochester.edu\/senior-design-day\/wp-content\/uploads\/2023\/04\/Enting-Zhou-Design-Day-Poster.jpg","contentUrl":"https:\/\/www.hajim.rochester.edu\/senior-design-day\/wp-content\/uploads\/2023\/04\/Enting-Zhou-Design-Day-Poster.jpg","width":10800,"height":8294},{"@type":"BreadcrumbList","@id":"https:\/\/www.hajim.rochester.edu\/senior-design-day\/unsupervised-arousal-valence-estimation-from-speech-and-corresponding-discrete-emotion\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.hajim.rochester.edu\/senior-design-day\/"},{"@type":"ListItem","position":2,"name":"Unsupervised Arousal Valence Estimation from Speech and Corresponding Discrete Emotion"}]},{"@type":"WebSite","@id":"https:\/\/www.hajim.rochester.edu\/senior-design-day\/#website","url":"https:\/\/www.hajim.rochester.edu\/senior-design-day\/","name":"Senior Design Day","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.hajim.rochester.edu\/senior-design-day\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.hajim.rochester.edu\/senior-design-day\/#\/schema\/person\/351018fbcf84ed8cac6d8072ba5b347c","name":"admin","url":"https:\/\/www.hajim.rochester.edu\/senior-design-day\/author\/seniordesign\/"}]}},"_links":{"self":[{"href":"https:\/\/www.hajim.rochester.edu\/senior-design-day\/wp-json\/wp\/v2\/posts\/111862","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.hajim.rochester.edu\/senior-design-day\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.hajim.rochester.edu\/senior-design-day\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.hajim.rochester.edu\/senior-design-day\/wp-json\/wp\/v2\/users\/6242"}],"replies":[{"embeddable":true,"href":"https:\/\/www.hajim.rochester.edu\/senior-design-day\/wp-json\/wp\/v2\/comments?post=111862"}],"version-history":[{"count":8,"href":"https:\/\/www.hajim.rochester.edu\/senior-design-day\/wp-json\/wp\/v2\/posts\/111862\/revisions"}],"predecessor-version":[{"id":137822,"href":"https:\/\/www.hajim.rochester.edu\/senior-design-day\/wp-json\/wp\/v2\/posts\/111862\/revisions\/137822"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.hajim.rochester.edu\/senior-design-day\/wp-json\/wp\/v2\/media\/137812"}],"wp:attachment":[{"href":"https:\/\/www.hajim.rochester.edu\/senior-design-day\/wp-json\/wp\/v2\/media?parent=111862"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.hajim.rochester.edu\/senior-design-day\/wp-json\/wp\/v2\/categories?post=111862"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.hajim.rochester.edu\/senior-design-day\/wp-json\/wp\/v2\/tags?post=111862"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.hajim.rochester.edu\/senior-design-day\/wp-json\/wp\/v2\/coauthors?post=111862"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}