表格預測中馬可夫邊界的好、壞與醜

摘要

在標準圖形假設下，目標變量的馬可夫邊界是能夠使其他所有特徵成為冗餘的最小特徵集合。一旦觀測到該邊界，目標變量與表格中其餘特徵便條件獨立。這對表格預測而言極具吸引力，因為它精確標示出模型所需的所有特徵欄位。然而，現代的回歸器仍會在完整特徵集上進行訓練。我們探討在SCM3K（一個包含3,450項任務的合成結構因果模型基準，特徵數量從40到1000不等，涵蓋六類結構因果模型家族，並以六種回歸器進行評估）上，馬可夫邊界對於預測是否確實有用。答案遠比理論所暗示的更為細膩。若將回歸器限制於神諭邊界，通常能顯著提升預測效果，且隨著特徵空間變得更大更稀疏，此改善幅度亦隨之增加。然而，透過因果發現來恢復邊界並在恢復後的遮罩上進行訓練的自然流程，並未能達成預期效果。現有估計器在達到邊界最有效用的區間之前便耗盡了計算預算，即使在能運行的情況下，也鮮少能超越完整特徵集的表現。我們將此歸因於三個原因：因果發現優化的是結構恢復而非預測成效；偽陰性與偽陽性在預測成本上具有極不對稱的影響；精確的馬可夫邊界僅是眾多能勝過完整特徵集的特徵子集之一。我們隨後闡述這些事實對於「與預測對齊的特徵選擇」以及「學習利用因果結構的表格模型」所帶來的啟示。

English

Under standard graphical assumptions, the Markov boundary of a target variable is the smallest set of features that renders every other feature redundant. Once the boundary is observed, the target is conditionally independent of the rest of the table. This is a tempting object for tabular prediction, since it names exactly the columns a model should need. Yet modern regressors are still trained on the full feature set. We ask whether the Markov boundary is genuinely useful for prediction on SCM3K, a 3,450-task synthetic SCM benchmark with feature counts from 40 to 1000 and six SCM families, evaluated with six regressors. The answer is more nuanced than the theory suggests. Restricting a regressor to the oracle boundary often improves prediction substantially, and the improvement grows as the feature space becomes larger and sparser. But the natural pipeline of recovering the boundary with causal discovery and training on the recovered mask does not deliver. Existing estimators exhaust the compute budget before reaching the regime where the boundary helps most, and even where they run they rarely beat the full feature set. We trace this to three causes. Discovery optimizes structural recovery rather than prediction. False negatives and false positives carry sharply asymmetric predictive cost. The exact boundary is only one of many feature sets that beat all features. We then develop what these facts imply for prediction-aligned feature selection and for tabular models that learn to use causal structure.