SwitchLingua: The First Large-Scale Multilingual and Multi-Ethnic Code-Switching Dataset

Yang Wang (Concordia University) · Hao CHEN (The Hong Kong Univeristy of Science and Technology) · Yangqiu Song (Hong Kong University of Science and Technology) · Peng Xie (The Hong Kong University of Science and Technology) · Xingyuan Liu (The Hong Kong University of Science and Technology) · Yequan Bie (Department of Computer Science and Engineering, Hong Kong University of Science and Technology) · Tsz Wai Chan (The Hong Kong University of Science and Technology) · Kani Chen (Hong Kong University of Science and Technology)

automatic speech recognitioncode-switchingcode-switching asrcross-lingual information retrievalcultural inclusivityevaluation metricslinguamasterlinguistic diversitymonolingual datasetsmultilingual applicationsmultilingual data synthesissemantic-aware error ratespeech technologyswitchlinguatext-to-speech

Code-switching (CS) is the alternating use of two or more languages within a conversation or utterance, often influenced by social context and speaker identity. This linguistic phenomenon poses challenges for Automatic Speech Recognition (ASR) systems, which are typically designed for a single language and struggle to handle multilingual inputs. The growing global demand for multilingual applications, including Code-Switching ASR (CSASR), Text-to-Speech (TTS), and Cross-Lingual Information Retrieval (CLIR), highlights the inadequacy of existing monolingual datasets. Although some code-switching datasets exist, most are limited to bilingual mixing within homogeneous ethnic groups, leaving a critical need for a large-scale, diverse benchmark akin to ImageNet in computer vision. To bridge this gap, we introduce \textbf{LinguaMaster}, a multi-agent collaboration framework specifically designed for efficient and scalable multilingual data synthesis. Leveraging this framework, we curate \textbf{SwitchLingua}, the first large-scale multilingual and multi-ethnic code-switching dataset, including: (1) 420K CS textual samples across 12 languages, and (2) over 80 hours of audio recordings from 174 speakers representing 18 countries/regions and 63 racial/ethnic backgrounds, based on the textual data. This dataset captures rich linguistic and cultural diversity, offering a foundational resource for advancing multilingual and multicultural research. Furthermore, to address the issue that existing ASR evaluation metrics lack sensitivity to code-switching scenarios, we propose the \textbf{Semantic-Aware Error Rate (SAER)}, a novel evaluation metric that incorporates semantic information, providing a more accurate and context-aware assessment of system performance. Benchmark experiments on SwitchLingua with state-of-the-art ASR models reveal substantial performance gaps, underscoring the dataset’s utility as a rigorous benchmark for CS capability evaluation. In addition, SwitchLingua aims to encourage further research to promote cultural inclusivity and linguistic diversity in speech technology, fostering equitable progress in the ASR field. LinguaMaster (Code): github.com/Shelton1013/SwitchLingua, SwitchLingua (Data): https://huggingface.co/datasets/Shelton1013/SwitchLingua_text, https://huggingface.co/datasets/Shelton1013/SwitchLingua_audio