LLM生成的JavaScript隱藏DNA：結構模式實現高準確度的作者歸屬

摘要

在本論文中，我們首次進行了大規模研究，探討由大型語言模型（LLMs）生成的JavaScript代碼是否能夠揭示其生成模型，從而實現可靠的作品歸屬和模型指紋識別。隨著AI生成代碼的迅速崛起，歸屬識別在檢測漏洞、標記惡意內容和確保責任追究方面扮演著關鍵角色。儘管AI與人類檢測通常將AI視為單一類別，我們展示了個別LLMs會留下獨特的風格特徵，即使是在屬於同一家族或參數規模的模型之間也是如此。為此，我們引入了LLM-NodeJS，這是一個包含20個大型語言模型生成的50,000個Node.js後端程序的數據集。每個程序都有四種變體，共產生250,000個獨特的JavaScript樣本，以及兩種額外的表示形式（JSIR和AST），以支持多樣的研究應用。利用此數據集，我們對比了傳統機器學習分類器與微調的Transformer編碼器，並介紹了CodeT5-JSA，這是一種基於770M參數CodeT5模型定制的架構，移除了其解碼器並修改了分類頭。它在五類歸屬任務中達到了95.8%的準確率，十類任務中為94.6%，二十類任務中為88.5%，超越了其他測試模型如BERT、CodeBERT和Longformer。我們展示了分類器捕捉了程序數據流和結構中更深層次的風格規律，而非依賴於表面特徵。因此，即使在代碼混淆、註釋刪除和重大代碼轉換後，歸屬識別仍然有效。為支持開放科學和可重現性，我們在GitHub上發布了LLM-NodeJS數據集、Google Colab訓練腳本及所有相關材料：https://github.com/LLM-NodeJS-dataset。

English

In this paper, we present the first large-scale study exploring whether JavaScript code generated by Large Language Models (LLMs) can reveal which model produced it, enabling reliable authorship attribution and model fingerprinting. With the rapid rise of AI-generated code, attribution is playing a critical role in detecting vulnerabilities, flagging malicious content, and ensuring accountability. While AI-vs-human detection usually treats AI as a single category we show that individual LLMs leave unique stylistic signatures, even among models belonging to the same family or parameter size. To this end, we introduce LLM-NodeJS, a dataset of 50,000 Node.js back-end programs from 20 large language models. Each has four transformed variants, yielding 250,000 unique JavaScript samples and two additional representations (JSIR and AST) for diverse research applications. Using this dataset, we benchmark traditional machine learning classifiers against fine-tuned Transformer encoders and introduce CodeT5-JSA, a custom architecture derived from the 770M-parameter CodeT5 model with its decoder removed and a modified classification head. It achieves 95.8% accuracy on five-class attribution, 94.6% on ten-class, and 88.5% on twenty-class tasks, surpassing other tested models such as BERT, CodeBERT, and Longformer. We demonstrate that classifiers capture deeper stylistic regularities in program dataflow and structure, rather than relying on surface-level features. As a result, attribution remains effective even after mangling, comment removal, and heavy code transformations. To support open science and reproducibility, we release the LLM-NodeJS dataset, Google Colab training scripts, and all related materials on GitHub: https://github.com/LLM-NodeJS-dataset.

LLM生成的JavaScript隱藏DNA：結構模式實現高準確度的作者歸屬

The Hidden DNA of LLM-Generated JavaScript: Structural Patterns Enable High-Accuracy Authorship Attribution

摘要

Support