DumpsterCluster：从捡垃圾到用60美元GPU服务LLaMA-70B

摘要

随着AI数据中心淘汰功能正常的GPU，大量仍具计算能力的加速器流入二级市场。本文研究这些退役GPU能否迎来富有成效的“第二次生命”，构建一个能够服务现代LLM推理的DumpsterCluster，并探讨在何种条件下此类再利用在经济上可行且在环境上可持续。我们仅使用二手组件从零开始实际搭建了一个128-GPU的DumpsterCluster，并运行了一年。按当前市场价格计算（DumpsterCluster约2.2万美元，而8-GPU B200系统约60万美元），其经济优势十分显著。通过流水线并行优化，我们基于V100的DumpsterCluster实现了具有竞争力的LLaMA-70B吞吐量，验证了生产环境下的可行性。然而，我们的部署揭示了关键的情境依赖性。老旧GPU每token消耗的能量显著更高，使得总拥有成本仅在电价低廉的地区才具有优势。在电网平均碳强度下，与当前代际的硬件相比，二手系统在8B模型上每token产生的总碳排放量约为其4倍，在70B模型上则超过40倍。这些发现表明，GPU的“第二次生命”并非普遍可持续——硬件再利用必须与低碳能源进行战略性结合。当部署在能源经济性优越且电力清洁的地区时，二手GPU为扩大AI算力提供了一条可行路径，同时能够兼顾可负担性、能源安全与环境责任。

English

As AI datacenters retire functional GPUs, vast quantities of still capable accelerators enter secondary markets. This paper investigates whether these retired GPUs can find a productive afterlife to form a DumpsterCluster that can serve modern LLM inference, and under what conditions such repurposing is economically viable and environmentally sustainable. We physically built a 128-GPU DumpsterCluster from scratch using only second-hand components and ran it for one year. At current market prices (\22K for the DumpsterCluster vs. 600K for an 8-GPU B200 system), the economic advantages are substantial. Through pipeline-parallel optimizations, our V100 based DumpsterCluster achieves competitive LLaMA-70B throughput, validating production viability. However, our deployment reveals critical context dependencies. Older GPUs consume significantly more energy per token, making total cost of ownership favorable only in regions with inexpensive electricity. Under grid-average carbon intensity, second-hand systems can produce approximately 4x higher total carbon emissions per token for 8B models, and over 40x for 70B models, compared to current-generation hardware. These findings show that GPU afterlife is not universally sustainable - hardware repurposing must be strategically coupled with low carbon energy sources. When deployed in regions with favourable energy economics and clean electricity, second-hand GPUs offer a viable pathway for expanding AI capacity while advancing affordability, energy security, and environmental responsibility.