JigShape：透過拼圖遊戲評估視覺語言模型中的視覺-幾何推理能力

摘要

拼圖問題的求解需要同時推理視覺內容與幾何約束，然而現有基準採用矩形切割，在紋理重複區域會產生具有歧義的真實值。我們提出了 \ours{} 基準，其使用帶有凸榫與凹槽的互鎖拼圖塊；其中幾何約束提供了強烈的局部相容性要求，與視覺內容結合後可產生無歧義的真實值。在四種網格密度（4×4 至 16×16）的 95,000 個實例中，我們發現零樣本視覺語言模型（VLM）在很大程度上缺乏幾何推理能力：五個前沿模型中只有一個（GPT-5.5）在 4×4 拼圖上超過隨機基線，而其他所有模型的表現均與隨機水平相當。雖然監督式微調在 4×4 上可達到 97% 以上的準確率，但所有模型在更大的網格上都出現崩潰：GPT-5.5 在 8×8 上從 70% 降至接近隨機水平，即便是微調後的模型在 12×12 上也低於 5%。這種「擴展懸崖」表明，當拼圖塊數量增加時，目前的架構無法維持一致的約束滿足能力。該基準確立了可擴展的幾何推理作為視覺語言模型的一項開放挑戰。

English

Jigsaw puzzle solving requires jointly reasoning about visual content and geometric constraints, yet existing benchmarks use rectangular cuts that create ambiguous ground truth in texture-repeated regions. We introduce \ours{}, a benchmark with tab-and-blank interlocking pieces where geometric constraints provide strong local compatibility requirements that, combined with visual content, yield unambiguous ground truth. Across 95K instances at four grid densities (4times4 to 16times16), we find that zero-shot VLMs largely lack geometric reasoning: only one of five frontier models (GPT-5.5) exceeds random baseline on 4times4 puzzles, while all others perform at chance level. While supervised fine-tuning achieves >97\% on 4times4, all models collapse on larger grids: GPT-5.5 drops from 70\% to near-random on 8times8, and even fine-tuned models fall below 5\% on 12times12. This ``scaling cliff'' suggests current architectures cannot maintain consistent constraint satisfaction as the number of pieces increases. establishes scalable geometric reasoning as an open challenge for vision-language models.