SAINTENS: Self-Attention and Intersample Attention Transformer for Digital Biomarker Development Using Tabular Healthcare Real World Data

Background: Deep learning currently struggles with tabular data, but it

can benefit from multimodal learning. SAINT is a deep learning model for tabular

data on which we base our presented developments. Objectives: In this study, we

introduce SAINTENS as a new deep learning method, specifically for the in

healthcare predominant tabular real world data. Methods: For this purpose, we

compare SAINTENS with SAINT and the State of the Art Machine Learning

methods for tabular data. We use tabular data from geriatrics to predict four different

targets (dysphagia, pressure ulcers, decompensated heart failure and delirium). We

determine the relevant feature sets and train the models on these sets. Results: Both

SAINTENS and SAINT models are at least on the same performance level as the

current State of the Art (Gradient Boosting Decision Trees). Conclusion: In

combination with multimodal learning SAINTENS and SAINT may be used on real

world data comprising tabular, text and image data, for discovery and development

of new digital biomarkers.