Add multimodal RNN support #797
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request introduces a new multimodal RNN model to the
pyhealthlibrary, enabling the handling of both sequential and non-sequential input features for clinical prediction tasks. It also provides a comprehensive example script for using the new model on the MIMIC-IV dataset for in-hospital mortality prediction. Additionally, the documentation and model API are updated to reflect these changes.Major changes include:
New Model: MultimodalRNN
MultimodalRNNclass topyhealth.models.rnn, which automatically distinguishes between sequential and non-sequential features and processes them appropriately (sequential features via RNN layers, non-sequential features via direct embedding and pooling). The model concatenates all feature representations for final prediction.MultimodalRNNin thepyhealth.modelspackage init file for public use.MultimodalRNN.Example Usage
mortality_mimic4_multimodal_rnn.pydemonstrating how to use the newMultimodalRNNmodel for mortality prediction with mixed feature types on the MIMIC-IV dataset. The script covers data loading, task setup, model training, evaluation, and sample predictions.Improvements and Bug Fixes
pyhealth.models.rnnto support the new model and feature classification logic.