Sophia Chou<br/>周奕慧

Sophia Chou
周奕慧

2nd year graduate student
at CMU LTI

Table of Contents

About Me

  • Hi! I’m Yi-Hui (Sophia) Chou, 周奕慧, a 2nd-year MIIS student at LTI, CMU. I received my Bachelor’s degree in Electrical Engineering (EE) from National Taiwan University (NTU), Taiwan, in 2021.
  • An Applied Science Summer Intern at Alexa, Amazon in 2023.
  • A member of the Speech Processing and Machine Learning Laboratory, National Taiwan, working with Prof. Lin-shan Lee and Prof. Hung-yi Lee from Sep. 2019 to Aug. 2021.
  • A part-time research assistant at Music and AI Lab, Academia Sinica, supervised by Dr. Yi-Hsuan (Eric) Yang, from Nov. 2020 to Aug. 2021.
  • My research interests broadly lie in the field of Speech Processing/Music AI/Natural Language Processing, and I’ve done some interesting projects! Please refer to Publications for more details.
  • Alongside my interests in speech processing and music AI, I’m a huge fan of classical music and have several chamber music collaborations. In this video, my friends and I perform Mendelssohn’s Piano Trio No. 1, and I’m the one playing the piano.

Publications

  • Listener Model for the PhotoBook Referential Game with CLIPScores as Implicit Reference Chain
    Shih-Lun Wu, Yi-Hui Chou, Liangze Li
    [ arxiv | github | slide | poster | short talk ]
    • Proposed and developed the 1st multimodal listener model that can be deployed to real-world gameplay given images and dialogue history.
    • Improved accuracy by 12% with CLIPScores (image captioning metric) as implicit reference chains at all layers.
    • Accepted to ACL 2023 main conference
  • Don’t speak too fast: The impact of data bias on self-supervised speech models
    Yen Meng*, Yi-Hui Chou*, Andy T. Liu, Hung-yi Lee
    [ arxiv ]
    • Accepted by ICASSP 2022 and AAAI 2022 SAS workshop
    • Investigate the impact of data bias, including gender, content, and prosody, on Self-supervised Speech Models (S3Ms)
    • Find that S3Ms have tolerance toward gender bias, and that the content of speech affects little on the performance of S3Ms across downstream tasks, but S3Ms show a preference toward a slower speech rate.
  • MidiBERT-Piano: Large-scale Pre-training for Symbolic Music Understanding
    Yi-Hui Chou*, I-Chun Chen*, Chin-Jui Chang, Joann Ching, Yi-Hsuan Yang
    [ arxiv | github | slide | talk ]
    • Proposed MidiBERT-Piano in PyTorch by extending BERT-like pre-trained models with Masking Language Modeling (MLM) strategy in Natural Language Processing (NLP) to midi-domain
    • Validated pre-training significantly improved over strong RNN baselines on 4 downstream tasks with a 20.5% improvement on average.
    • Received the 2nd prize of the Bachelor Thesis Award, NTU EE in 2021.

Work Experience

  • Applied Science Intern at Alexa, Amazon (May - Aug. 2023)
    • Developed Neural Vector Search approaches and integrated diverse information from queries and entities, effectively closing the gap with the current rule-based production system.
  • Audio AI Engineer at HTC Vive (Sep. 2021 - Jul. 2022)
    • Conducted research on Music Source Separation (MSS)
    • Reduced the model size of the existing algorithm – ResUNet Subbandtime by 50% (394 MB -> 197 MB), with a SDR downgrade of only 0.2%. The model size can be further reduced to 56.2 MB with model quantization technique (LSQ) but at the cost of SDR downgrade of 1.9%.
    • Implemented the scoring algorithm with CREPE, a pitch tracking library, to measure a user’s performance based on timing and pitch.
    • Designed a dynamic demo website to run source separation given a YouTube link, record a user’s singing, and provide score and analysis afterwards.
  • Summer software intern at Ganzin Technology (July - Aug. 2020)
    • Built a voice and eye-gaze controlled IoT device with AWS Alexa voice service and Ganzin’s eye tracking solution [ repo ].
    • Designed and implemented a new feature, brightness adjustment, in addition to the required project ahead of time.
    • Refined the algorithm and improved user experience and stability by reducing latency by at best 60%.
  • Summer software intern at Moxa Inc. (July - Aug. 2019)
    • Studied network intrusion detection system (Snort) for their future product design.
    • Analyzed the efficiency and memory usage by simulating an intrusion environment.

Talk

  • Invited to give a talk (slide) on our ACL paper for the Entity Resolution Team at Alexa during my internship in July 2023.
  • Short video for our short paper accepted by ACL 2023
  • Invited to give a talk about MidiBERT at Music + AI Reading Group @ Mila in March 2023.

Service and Leadership

  • Volunteeer at International Companions for Learning, 2021 spring
    • Paired up with an Indonesian girl to enhance intercultural communication and connect countryside kids to the world.
    • Assist with the preparation, communication and operation during each Skype session.
  • National Taiwan Museum International Docent
    • Gave tours for more than 30 TEDx partners from around the world on TEDxWeekend2019 event, in which Taipei is among five global cities to host a once-in-a-decade celebration of TEDx.
    • Led multilingual tours introducing the museum and the history of Taiwan.
  • Event General Coordinator, Soka Summer Camp (2018 & 2019)
    • Organized a summer volunteer trip for more than 100 primary students in Kaohsiung.

(Updated in July 2023)