Unified Evaluation of Table Embedding Methods Across Multiple Benchmark Scenarios
Younes, A., Ghoorchian, S., and Schambach, M., and Höhne, J.
ICLR 2026 Workshop on Navigating and Addressing Data Problems for Foundation Models, 2025
We introduce a unified evaluation framework for table-level embeddings, that is, methods that encode an entire table into a single vector. The proposed framework targets operations such as table indexing, clustering, retrieval, as well as data curation primitives for training tabular foundation models with various objectives, including overlap estimation, approximate deduplication, filtering, and sampling. While feature representations in vision and language have enabled scalable retrieval and transfer, tabular representation learning is typically evaluated at finer granularities (rows/cells) or via downstream prediction, leaving table-level embedding quality and robustness under-specified. We benchmark diverse embedding families, including schema and statistical fingerprints, text-serialization encoders, specialized table encoders, and pooled representations from tabular foundation models, on both controlled synthetic table families with known generative factors and real open-source data. Following a set of desiderata, we evaluate consistency under partial views, discriminability across label granularities, robustness to benign perturbations, and efficiency, without downstream fine-tuning. Our results show that simple hashing and lightweight serialization methods are highly competitive and often outperform pooled representations from foundation models. This exposes a representation-prediction tension: strong predictive models do not necessarily yield stable, discriminative table-level geometry after pooling, motivating objectives that explicitly optimize robust table-level embeddings.