Audio VAE roundtrip on highlighted audio

Same path as encode_av_inmem.py / decode_av_inmem.py: load_audio_auto (44.1 kHz mono, 8 s) → ensure_max_of_audio → AudioAutoencoder(/group2/ct/weihanx/vae/vae) encode (latent 64×345) → decode. Two inputs per clip: the raw video audio (src) and the new highlighted wav from 8_sec_highlighted_0822 (target). Metrics are sample-aligned waveform SNR / Pearson corr / log-spectral distance (LSD) between VAE input and decoded output.

Summary (15 clips). Highlighted-target roundtrip: SNR median 13.8 dB (min 6.0, max 33.5), corr median 0.979; video-audio roundtrip SNR median 9.2 dB. RMS is preserved within 0.5 dB everywhere. Lowest-SNR cases are noisy/broadband clips (birds, engine, chickens, crowd, drum kit) — waveform SNR penalises phase differences, so listen to judge; the highlighted target generally round-trips slightly better than the raw mix because the foreground dominates.

id	prompt	src SNR	src corr	src LSD	target SNR	target corr	target LSD	target RMS in→out
`Bm3Lfx58wvo_000200`	train	3.87	0.79	8.22	14.27	0.981	8.05	-14.6 → -14.5
`Bm6puindzsc`	birds	12.32	0.97	13.25	6.56	0.888	10.57	-40.6 → -40.7
`Bm8mxpOSmjw_000070`	melodic percussion instrument	25.53	0.999	8.39	30.42	1.0	9.59	-11.8 → -11.9
`BmBido0MLFk`	idling engine	5.06	0.841	11.7	6.08	0.876	9.89	-16.9 → -17.0
`BmFzGKRojvA`	brass instruments	32.56	1.0	8.72	33.49	1.0	8.8	-14.5 → -14.5
`PifhBU-mSVA_000010`	infant	15.33	0.986	9.0	31.35	1.0	8.97	-12.5 → -12.6
`TOBw7DgXbgg_000047`	musician	14.33	0.987	8.57	23.27	0.998	7.73	-12.7 → -12.8
`xc74uXzCRWk_000013`	harmonica	9.0	0.939	11.85	10.91	0.96	9.87	-17.6 → -17.5
`U-L9YCIdLbg_000080`	electric guitar	14.39	0.982	10.77	16.13	0.988	7.88	-16.8 → -16.7
`R93J7u5EWvk`	drum kit	1.66	0.657	10.56	7.96	0.921	10.36	-18.0 → -17.9
`Kz_gp7UE9IY_000001`	dog	20.61	0.996	6.67	21.58	0.997	10.3	-12.6 → -12.9
`mT49XQjXEvY_000000`	chickens	6.45	0.882	8.88	6.0	0.87	10.1	-16.7 → -17.1
`hqvqPITWcrA_000260`	siren	9.21	0.941	7.93	8.63	0.932	9.7	-17.9 → -17.9
`CsMMU1p3Sik_000030`	crowd	5.51	0.859	10.65	7.0	0.899	8.1	-13.0 → -13.2
`iTdlEJPKT94_000300`	train horn	6.84	0.895	12.62	13.85	0.979	10.44	-22.7 → -22.9

`Bm3Lfx58wvo_000200` — “train” (first5)

video audio roundtrip: SNR 3.87 dB · corr 0.79 · LSD 8.22 dB | highlighted roundtrip: SNR 14.27 dB · corr 0.981 · LSD 8.05 dB · RMS -14.6→-14.5 dBFS

Original video audio (VAE input)

Video audio → encode → decode

Highlighted target (VAE input)

Highlighted → encode → decode

`Bm6puindzsc` — “birds” (first5)

video audio roundtrip: SNR 12.32 dB · corr 0.97 · LSD 13.25 dB | highlighted roundtrip: SNR 6.56 dB · corr 0.888 · LSD 10.57 dB · RMS -40.6→-40.7 dBFS

Original video audio (VAE input)

Video audio → encode → decode

Highlighted target (VAE input)

Highlighted → encode → decode

`Bm8mxpOSmjw_000070` — “melodic percussion instrument” (first5)

video audio roundtrip: SNR 25.53 dB · corr 0.999 · LSD 8.39 dB | highlighted roundtrip: SNR 30.42 dB · corr 1.0 · LSD 9.59 dB · RMS -11.8→-11.9 dBFS

Original video audio (VAE input)

Video audio → encode → decode

Highlighted target (VAE input)

Highlighted → encode → decode

`BmBido0MLFk` — “idling engine” (first5)

video audio roundtrip: SNR 5.06 dB · corr 0.841 · LSD 11.7 dB | highlighted roundtrip: SNR 6.08 dB · corr 0.876 · LSD 9.89 dB · RMS -16.9→-17.0 dBFS

Original video audio (VAE input)

Video audio → encode → decode

Highlighted target (VAE input)

Highlighted → encode → decode

`BmFzGKRojvA` — “brass instruments” (first5)

video audio roundtrip: SNR 32.56 dB · corr 1.0 · LSD 8.72 dB | highlighted roundtrip: SNR 33.49 dB · corr 1.0 · LSD 8.8 dB · RMS -14.5→-14.5 dBFS

Original video audio (VAE input)

Video audio → encode → decode

Highlighted target (VAE input)

Highlighted → encode → decode

`PifhBU-mSVA_000010` — “infant” (multi-source)

video audio roundtrip: SNR 15.33 dB · corr 0.986 · LSD 9.0 dB | highlighted roundtrip: SNR 31.35 dB · corr 1.0 · LSD 8.97 dB · RMS -12.5→-12.6 dBFS

Original video audio (VAE input)

Video audio → encode → decode

Highlighted target (VAE input)

Highlighted → encode → decode

`TOBw7DgXbgg_000047` — “musician” (multi-source)

video audio roundtrip: SNR 14.33 dB · corr 0.987 · LSD 8.57 dB | highlighted roundtrip: SNR 23.27 dB · corr 0.998 · LSD 7.73 dB · RMS -12.7→-12.8 dBFS

Original video audio (VAE input)

Video audio → encode → decode

Highlighted target (VAE input)

Highlighted → encode → decode

`xc74uXzCRWk_000013` — “harmonica” (multi-source)

video audio roundtrip: SNR 9.0 dB · corr 0.939 · LSD 11.85 dB | highlighted roundtrip: SNR 10.91 dB · corr 0.96 · LSD 9.87 dB · RMS -17.6→-17.5 dBFS

Original video audio (VAE input)

Video audio → encode → decode

Highlighted target (VAE input)

Highlighted → encode → decode

`U-L9YCIdLbg_000080` — “electric guitar” (multi-source)

video audio roundtrip: SNR 14.39 dB · corr 0.982 · LSD 10.77 dB | highlighted roundtrip: SNR 16.13 dB · corr 0.988 · LSD 7.88 dB · RMS -16.8→-16.7 dBFS

Original video audio (VAE input)

Video audio → encode → decode

Highlighted target (VAE input)

Highlighted → encode → decode

`R93J7u5EWvk` — “drum kit” (multi-source)

video audio roundtrip: SNR 1.66 dB · corr 0.657 · LSD 10.56 dB | highlighted roundtrip: SNR 7.96 dB · corr 0.921 · LSD 10.36 dB · RMS -18.0→-17.9 dBFS

Original video audio (VAE input)

Video audio → encode → decode

Highlighted target (VAE input)

Highlighted → encode → decode

`Kz_gp7UE9IY_000001` — “dog” (multi-source)

video audio roundtrip: SNR 20.61 dB · corr 0.996 · LSD 6.67 dB | highlighted roundtrip: SNR 21.58 dB · corr 0.997 · LSD 10.3 dB · RMS -12.6→-12.9 dBFS

Original video audio (VAE input)

Video audio → encode → decode

Highlighted target (VAE input)

Highlighted → encode → decode

`mT49XQjXEvY_000000` — “chickens” (multi-source)

video audio roundtrip: SNR 6.45 dB · corr 0.882 · LSD 8.88 dB | highlighted roundtrip: SNR 6.0 dB · corr 0.87 · LSD 10.1 dB · RMS -16.7→-17.1 dBFS

Original video audio (VAE input)

Video audio → encode → decode

Highlighted target (VAE input)

Highlighted → encode → decode

`hqvqPITWcrA_000260` — “siren” (multi-source)

video audio roundtrip: SNR 9.21 dB · corr 0.941 · LSD 7.93 dB | highlighted roundtrip: SNR 8.63 dB · corr 0.932 · LSD 9.7 dB · RMS -17.9→-17.9 dBFS

Original video audio (VAE input)

Video audio → encode → decode

Highlighted target (VAE input)

Highlighted → encode → decode

`CsMMU1p3Sik_000030` — “crowd” (multi-source)

video audio roundtrip: SNR 5.51 dB · corr 0.859 · LSD 10.65 dB | highlighted roundtrip: SNR 7.0 dB · corr 0.899 · LSD 8.1 dB · RMS -13.0→-13.2 dBFS

Original video audio (VAE input)

Video audio → encode → decode

Highlighted target (VAE input)

Highlighted → encode → decode

`iTdlEJPKT94_000300` — “train horn” (multi-source)

video audio roundtrip: SNR 6.84 dB · corr 0.895 · LSD 12.62 dB | highlighted roundtrip: SNR 13.85 dB · corr 0.979 · LSD 10.44 dB · RMS -22.7→-22.9 dBFS

Original video audio (VAE input)

Video audio → encode → decode

Highlighted target (VAE input)

Highlighted → encode → decode

Built by /group2/ct/weihanx/audio_hl_demo/audio_roundtrip.py + build_roundtrip_space.py

Audio VAE encode → decode roundtrip (highlighted targets, 2026-08-22 run)

Bm3Lfx58wvo_000200 — “train” (first5)

Bm6puindzsc — “birds” (first5)

Bm8mxpOSmjw_000070 — “melodic percussion instrument” (first5)

BmBido0MLFk — “idling engine” (first5)

BmFzGKRojvA — “brass instruments” (first5)

PifhBU-mSVA_000010 — “infant” (multi-source)

TOBw7DgXbgg_000047 — “musician” (multi-source)

xc74uXzCRWk_000013 — “harmonica” (multi-source)

U-L9YCIdLbg_000080 — “electric guitar” (multi-source)

R93J7u5EWvk — “drum kit” (multi-source)

Kz_gp7UE9IY_000001 — “dog” (multi-source)

mT49XQjXEvY_000000 — “chickens” (multi-source)

hqvqPITWcrA_000260 — “siren” (multi-source)

CsMMU1p3Sik_000030 — “crowd” (multi-source)

iTdlEJPKT94_000300 — “train horn” (multi-source)

`Bm3Lfx58wvo_000200` — “train” (first5)

`Bm6puindzsc` — “birds” (first5)

`Bm8mxpOSmjw_000070` — “melodic percussion instrument” (first5)

`BmBido0MLFk` — “idling engine” (first5)

`BmFzGKRojvA` — “brass instruments” (first5)

`PifhBU-mSVA_000010` — “infant” (multi-source)

`TOBw7DgXbgg_000047` — “musician” (multi-source)

`xc74uXzCRWk_000013` — “harmonica” (multi-source)

`U-L9YCIdLbg_000080` — “electric guitar” (multi-source)

`R93J7u5EWvk` — “drum kit” (multi-source)

`Kz_gp7UE9IY_000001` — “dog” (multi-source)

`mT49XQjXEvY_000000` — “chickens” (multi-source)

`hqvqPITWcrA_000260` — “siren” (multi-source)

`CsMMU1p3Sik_000030` — “crowd” (multi-source)

`iTdlEJPKT94_000300` — “train horn” (multi-source)