CORTEXA
← Browse
arxiveess.AScs.AI2026-07-02

An Efficient vLLM-Based Inference Pipeline for Unified Audio Understanding and Generation

Haoran Wang, Jinchuan Tian, Siddhant Arora, Shinji Watanabe

While Large Multimodal Models excel in comprehension, high-throughput inference engines lack native support for multimodal generation. This is severe in Speech Language Models, where generating multi-layered audio tokens via decoupled AR+NAR or synchronous Multi-Token Prediction (MTP) with delay-pattern interleaving conflicts with standard single-stream loops. We present a vLLM-based inference pipeline for unified speech understanding and generation. We extend autoregressive decoding to natively execute delay-pattern de-interleaving and coordinated multi-stream sampling, integrating an on-GPU acoustic decoder for end-to-end waveform synthesis. Crucially, we overcome the shared intuition that Classifier-Free Guidance (CFG) halves throughput. By co-scheduling paired conditional and unconditional requests within a continuous batch, our CFG implementation sustains 80% of non-CFG throughput, absorbing dual-request and logit merging overheads. We open-source our framework.

View free PDFSource page

Related papers

arxivcs.SDcs.AIcs.MMeess.AS2026-06-30

SwiftAudio: Data-Efficient Caption-Only Distillation for One-Step Text-to-Audio Diffusion-based Generation

Binh Mai, Tran Quoc Bao Le, Hung Dinh, Cong Tran

Diffusion-based text-to-audio (TTA) models achieve impressive synthesis quality but suffer from high inference latency due to iterative multi-step denoising. Existing one-step approaches alleviate this issue but still rely on paired text--audio data during distillation. To addres…

View free PDFSource page
arxiveess.AScs.AIcs.LG2026-07-07

TriA Pipeline: A Large-Scale Automatic Audio Annotation Pipeline For Audio Classification In Specific Scenarios

Hong Lyu, Mingru Yang, Qianhua He, Yanxiong Li, Jinxin Huang, Zhengyu Pei

There are some datasets of varying scales for audio classification (AC) applied to different tasks. However, annotated data is limited for most scenarios, such as domestic environments. To address this challenge, we propose an $\textbf{A}$utomatic $\textbf{A}$udio $\textbf{A}$nno…

View free PDFSource page
arxivcs.CLcs.AIcs.LGcs.SDeess.AS2026-07-06

Unified Audio Intelligence Without Regressing on Text Intelligence

Zhifeng Kong, Sang-gil Lee, Jaehyeon Kim, Boxin Wang, Zihan Liu, Sungwon Kim, et al.

Audio intelligence involves understanding, reasoning about, and generating both audio and speech. In this work, we introduce Nemotron-Labs-Audex-30B-A3B (Audex), a unified audio-text LLM built on Nemotron-Cascade-2-30B-A3B, a strong text-only MoE LLM. Audex adopts a simple unifie…

View free PDFSource page
arxivcs.SDcs.AIeess.ASeess.SPeess.SY2026-07-10

A Production-Oriented Framework for Evaluation of SFX Generation

Mélodie Desbos, Yara Bahram, Eric Granger, Mohammadhadi Shateri

Industrial sound design requires audio generation systems that not only produce realistic audio, but also preserve the perceptual identity of a reference, support controllable variation, and remain efficient for practical workflows. Existing evaluations are usually tied to text-t…

View free PDFSource page
arxivcs.SDcs.AIeess.ASeess.SP2026-07-10

ReGen: Hierarchical Multi-Prompt Representation Generation for Efficient Waveform Diffusion Models

Sang-Hoon Lee, Ha-Yeong Choi

Representation alignment (REPA) has been investigated to accelerate diffusion training, but we observe that regularizing intermediate representations in diffusion Transformers (DiT) may implicitly entangle latents and limit generative capacity. To address this issue, we propose R…

View free PDFSource page