音訊匹配剪輯：在電影和影片中尋找和創建匹配的音訊過渡

Audio Match Cutting: Finding and Creating Matching Audio Transitions in Movies and Videos

August 20, 2024

作者: Dennis Fedorishin, Lie Lu, Srirangaraj Setlur, Venu Govindaraju

cs.AI

摘要

「匹配剪輯」是一種常見的影片剪輯技術，其中一對具有相似構圖的鏡頭能夠流暢地過渡。儘管匹配剪輯通常是視覺上的，但某些匹配剪輯涉及音頻的流暢過渡，不同來源的聲音融合成一個無法區分的過渡，連接兩個鏡頭。在本文中，我們探討自動尋找和創建影片和電影中的「音頻匹配剪輯」的能力。我們為音頻匹配剪輯創建了一種自監督音頻表示，並開發了一個從粗糙到精細的音頻匹配流程，該流程推薦匹配的鏡頭並創建混合音頻。我們進一步為提出的音頻匹配剪輯任務標註了一個數據集，並比較了多種音頻表示的能力來尋找音頻匹配剪輯候選者。最後，我們評估了多種方法來混合兩個匹配的音頻候選者，目的是創建平滑的過渡。項目頁面和示例可在以下網址找到：https://denfed.github.io/audiomatchcut/

English

A "match cut" is a common video editing technique where a pair of shots that have a similar composition transition fluidly from one to another. Although match cuts are often visual, certain match cuts involve the fluid transition of audio, where sounds from different sources merge into one indistinguishable transition between two shots. In this paper, we explore the ability to automatically find and create "audio match cuts" within videos and movies. We create a self-supervised audio representation for audio match cutting and develop a coarse-to-fine audio match pipeline that recommends matching shots and creates the blended audio. We further annotate a dataset for the proposed audio match cut task and compare the ability of multiple audio representations to find audio match cut candidates. Finally, we evaluate multiple methods to blend two matching audio candidates with the goal of creating a smooth transition. Project page and examples are available at: https://denfed.github.io/audiomatchcut/