ModaVerse: Efficiently Transforming Modalities with LLMs
Paper
• 2401.06395 • Published
• 3
Audio LLM
Note Multimodal LLM, Using Agent binding to Generation Model for generation
Boosting Large Language Model for Speech Synthesis: An Empirical Study
Paper • 2401.00246 • Published • 14Note Approach A didn't work well so the authors conclude that the speech tokens cannot be treated as a new language
-
An Integration of Pre-Trained Speech and Language Models for End-to-End Speech Recognition
Paper • 2312.03668 • Published • 1 -
Towards General-Purpose Speech Abilities for Large Language Models Using Unpaired Data
Paper • 2311.06753 • Published • 7 -
Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Models
Paper • 2311.07919 • Published • 10 -
Audiobox: Unified Audio Generation with Natural Language Prompts
Paper • 2312.15821 • Published • 17 -
ZMM-TTS: Zero-shot Multilingual and Multispeaker Speech Synthesis Conditioned on Self-supervised Discrete Speech Representations
Paper • 2312.14398 • Published
SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilities
Paper • 2305.11000 • Published • 5Note Using HuBERT speech token directly in the LLM. Train a GAN vocoder (HiFi-GAN) for decoding.
-
OpenMOSS-Team/SpeechInstruct
Viewer • Updated • 6.42M • 71 • 20 -
SpeechTokenizer: Unified Speech Tokenizer for Speech Large Language Models
Paper • 2308.16692 • Published • 1 -
Audiogen/agc-continuous
73.9M • Updated • 3 -
Audiogen/agc-discrete
90.9M • Updated • 310 • 3 -
hubertsiuzdak/snac_44khz
Updated • 1.13k • 11 -
hubertsiuzdak/snac_32khz
Updated • 702 • 12 -
hubertsiuzdak/snac_24khz
Updated • 1.13M • 59 -
advancedcv/Food500Cap
Viewer • Updated • 24.8k • 32 • 6