FilmAgent：一個多智能體框架，用於虛擬3D空間中的電影自動化端對端。

FilmAgent: A Multi-Agent Framework for End-to-End Film Automation in Virtual 3D Spaces

January 22, 2025

作者: Zhenran Xu, Longyue Wang, Jifang Wang, Zhouyi Li, Senbao Shi, Xue Yang, Yiyu Wang, Baotian Hu, Jun Yu, Min Zhang

cs.AI

摘要

虛擬電影製作需要複雜的決策過程，包括編劇、虛擬攝影和演員定位以及動作的精確安排。受到最近在基於語言代理社會中自動化決策方面的進展的激勵，本文介紹了FilmAgent，這是一個新穎的基於LLM的多代理協作框架，用於我們構建的3D虛擬空間中的端到端電影自動化。FilmAgent 模擬了各種工作人員角色，包括導演、編劇、演員和攝影師，並涵蓋了電影製作工作流程的關鍵階段：（1）構思發展將腦力激盪的想法轉化為結構化的故事大綱；（2）編劇為每個場景的對話和角色動作進行詳細說明；（3）攝影確定每個鏡頭的攝影機設置。一組代理通過迭代反饋和修訂進行協作，從而驗證中間腳本並減少幻覺。我們對15個想法和4個關鍵方面的生成視頻進行評估。人類評估表明，FilmAgent在所有方面均優於所有基準，平均得分為3.98（滿分5分），顯示了多代理協作在電影製作中的可行性。進一步的分析顯示，儘管使用較不先進的GPT-4o模型，FilmAgent超越了單一代理o1，顯示了良好協調的多代理系統的優勢。最後，我們討論了OpenAI的文本到視頻模型Sora和我們的FilmAgent在電影製作中的互補優勢和劣勢。

English

Virtual film production requires intricate decision-making processes, including scriptwriting, virtual cinematography, and precise actor positioning and actions. Motivated by recent advances in automated decision-making with language agent-based societies, this paper introduces FilmAgent, a novel LLM-based multi-agent collaborative framework for end-to-end film automation in our constructed 3D virtual spaces. FilmAgent simulates various crew roles, including directors, screenwriters, actors, and cinematographers, and covers key stages of a film production workflow: (1) idea development transforms brainstormed ideas into structured story outlines; (2) scriptwriting elaborates on dialogue and character actions for each scene; (3) cinematography determines the camera setups for each shot. A team of agents collaborates through iterative feedback and revisions, thereby verifying intermediate scripts and reducing hallucinations. We evaluate the generated videos on 15 ideas and 4 key aspects. Human evaluation shows that FilmAgent outperforms all baselines across all aspects and scores 3.98 out of 5 on average, showing the feasibility of multi-agent collaboration in filmmaking. Further analysis reveals that FilmAgent, despite using the less advanced GPT-4o model, surpasses the single-agent o1, showing the advantage of a well-coordinated multi-agent system. Lastly, we discuss the complementary strengths and weaknesses of OpenAI's text-to-video model Sora and our FilmAgent in filmmaking.