基于语言嵌入的时间序列分类方法 LETS-C

LETS-C: Leveraging Language Embedding for Time Series Classification

July 9, 2024

作者: Rachneet Kaur, Zhen Zeng, Tucker Balch, Manuela Veloso

cs.AI

摘要

最近语言建模方面的进展显示，将其应用于时间序列数据取得了令人期待的结果。特别是，对预训练的大型语言模型（LLMs）进行微调，用于时间序列分类任务，在标准基准测试中取得了最先进的性能。然而，这些基于LLM的模型存在一个重大缺点，即模型规模庞大，可训练参数数量达到百万级。本文提出了一种利用语言建模成功经验的时间序列领域的替代方法。我们并非对LLMs进行微调，而是利用语言嵌入模型将时间序列嵌入，然后将这些嵌入与由卷积神经网络（CNN）和多层感知器（MLP）组成的简单分类头配对。我们在已建立的时间序列分类基准数据集上进行了大量实验。我们展示了LETS-C不仅在分类准确性上优于当前SOTA，而且提供了一种轻量级解决方案，与SOTA模型相比，平均仅使用了14.5%的可训练参数。我们的研究结果表明，利用语言编码器将时间序列数据嵌入，结合简单而有效的分类头，为实现高性能时间序列分类提供了一个有前途的方向，同时保持了轻量级模型架构。

English

Recent advancements in language modeling have shown promising results when applied to time series data. In particular, fine-tuning pre-trained large language models (LLMs) for time series classification tasks has achieved state-of-the-art (SOTA) performance on standard benchmarks. However, these LLM-based models have a significant drawback due to the large model size, with the number of trainable parameters in the millions. In this paper, we propose an alternative approach to leveraging the success of language modeling in the time series domain. Instead of fine-tuning LLMs, we utilize a language embedding model to embed time series and then pair the embeddings with a simple classification head composed of convolutional neural networks (CNN) and multilayer perceptron (MLP). We conducted extensive experiments on well-established time series classification benchmark datasets. We demonstrated LETS-C not only outperforms the current SOTA in classification accuracy but also offers a lightweight solution, using only 14.5% of the trainable parameters on average compared to the SOTA model. Our findings suggest that leveraging language encoders to embed time series data, combined with a simple yet effective classification head, offers a promising direction for achieving high-performance time series classification while maintaining a lightweight model architecture.