研究者指定共變量下基於LLM文本分析的條件假設生成

摘要

計算社會科學的核心目標之一，是發現語言在感興趣的結果變項（如政治傾向或教學品質）之間如何變化的可解釋差異。近期基於大型語言模型的假設生成方法，雖能以自然語言描述此類差異，但僅選取全域判別模式，而未考慮研究者根據領域知識設定的共變量。忽略共變量時，所選模式可能反映混淆因素，而非實質上值得關注的差異。我們提出條件式假設生成框架，引入研究者指定的共變量，引導假設發現朝向在相關子群體內成立的差異。此過程面臨兩項挑戰：目標子群體可能代表性不足（分層不平衡），且差異方向可能在不同子群體間反轉（符號反轉）。我們提出兩種受計量經濟學啟發的方法：其一引入特徵與共變量的交互作用項以偵測符號反轉；其二應用分層內去均值及逆頻率加權，以平衡代表性不足的分層。合成實驗顯示，每種方法在其目標設定下均優於全域基準；而針對兩個真實世界資料集的專家評估則證實，考量共變量的假設生成能在相關子群體內產出更有用的假設。

English

A core goal of computational social science is to discover interpretable differences in how language varies across outcomes of interest, such as political affiliation or instructional quality. Recent LLM-based hypothesis generation methods describe such differences in natural language, but select for globally discriminative patterns without accounting for covariates that shape the data based on researchers' domain knowledge. When covariates are ignored, selected patterns can reflect confounds rather than differences of substantive interest. We introduce conditional hypothesis generation, a framework that incorporates researcher-specified covariates to steer hypothesis discovery toward differences that hold within relevant subgroups. Two challenges arise: the target subgroup may be underrepresented (stratum imbalance), and the direction of a difference may reverse across subgroups (sign reversal). We propose two econometrics-inspired methods: one introduces feature--covariate interactions to detect sign reversals, and the other applies within-stratum demeaning and inverse-frequency reweighting to equalize underrepresented strata. Synthetic experiments show each method outperforms global baselines in its targeted setting, and expert evaluation on two real-world datasets confirms that covariate-aware generation surfaces more useful hypotheses within relevant subgroups.