Add Zero-Shot LLM Evidence Retrieval Pipeline (Ahsan et al. 2024)

docs/api/datasets.rst

@@ -223,6 +223,7 @@ Available Datasets
     datasets/pyhealth.datasets.BaseDataset
     datasets/pyhealth.datasets.SampleDataset
     datasets/pyhealth.datasets.MIMIC3Dataset
+    datasets/pyhealth.datasets.MIMIC3NoteDataset
     datasets/pyhealth.datasets.MIMIC4Dataset
     datasets/pyhealth.datasets.MedicalTranscriptionsDataset
     datasets/pyhealth.datasets.CardiologyDataset

docs/api/datasets/pyhealth.datasets.MIMIC3NoteDataset.rst

@@ -0,0 +1,14 @@
+pyhealth.datasets.MIMIC3NoteDataset
+=====================================
+
+The open Medical Information Mart for Intensive Care III (MIMIC-III) Clinical Notes dataset specialized for NLP and evidence retrieval tasks. This class extends the general 
+:class:`~pyhealth.datasets.MIMIC3Dataset` by always loading the ``noteevents`` and ``diagnoses_icd`` tables and exposing the ``iserror`` flag so that erroneous notes can be filtered downstream.  It is designed to pair with 
+:class:`~pyhealth.tasks.EHREvidenceRetrievalTask` and
+:class:`~pyhealth.models.ZeroShotEvidenceLLM` to reproduce the zero-shot EHR evidence retrieval pipeline of `Ahsan et al. (2024) <https://arxiv.org/abs/2309.04550>`_.
+
+Refer to the `MIMIC-III documentation <https://mimic.mit.edu/docs/iii/>`_ for data access instructions.
+
+.. autoclass:: pyhealth.datasets.MIMIC3NoteDataset
+    :members:
+    :undoc-members:
+    :show-inheritance:

docs/api/models.rst

@@ -201,6 +201,7 @@ API Reference
     models/pyhealth.models.GAN
     models/pyhealth.models.VAE
     models/pyhealth.models.SDOH
+    models/pyhealth.models.ZeroShotEvidenceLLM
     models/pyhealth.models.VisionEmbeddingModel
     models/pyhealth.models.TextEmbedding
     models/pyhealth.models.BIOT

docs/api/models/pyhealth.models.ZeroShotEvidenceLLM.rst

@@ -0,0 +1,17 @@
+pyhealth.models.ZeroShotEvidenceLLM
+=====================================
+
+Zero-shot LLM pipeline for retrieving and summarising clinically relevant
+evidence from unstructured EHR notes.  Implements the two-step prompting
+strategy from `Ahsan et al. (2024)
+<https://arxiv.org/abs/2309.04550>`_ (CHIL 2024, PMLR 248:489-505).
+
+The model requires no task-specific training — it uses instruction-tuned
+models (e.g. Flan-T5 XXL or Mistral-7B-Instruct) in a zero-shot setting.
+A Clinical-BERT dense-retrieval baseline is also available via
+``use_cbert_baseline=True``.
+
+.. autoclass:: pyhealth.models.ZeroShotEvidenceLLM
+    :members:
+    :undoc-members:
+    :show-inheritance:

docs/api/tasks.rst

@@ -206,6 +206,7 @@ Available Tasks
     :maxdepth: 3
 
     Base Task <tasks/pyhealth.tasks.BaseTask>
+    EHR Evidence Retrieval <tasks/pyhealth.tasks.EHREvidenceRetrievalTask>
     In-Hospital Mortality (MIMIC-IV) <tasks/pyhealth.tasks.InHospitalMortalityMIMIC4>
     MIMIC-III ICD-9 Coding <tasks/pyhealth.tasks.MIMIC3ICD9Coding>
     Cardiology Detection <tasks/pyhealth.tasks.cardiology_detect>

docs/api/tasks/pyhealth.tasks.EHREvidenceRetrievalTask.rst

@@ -0,0 +1,18 @@
+pyhealth.tasks.EHREvidenceRetrievalTask
+=========================================
+
+Binary task that pairs a patient's concatenated clinical notes with a
+free-text query diagnosis.  The label indicates whether the patient has been
+assigned any of the specified ICD-9 codes, providing a computable proxy for
+expert ground-truth labels.
+
+This task is designed to be used with
+:class:`~pyhealth.datasets.MIMIC3NoteDataset` and
+:class:`~pyhealth.models.ZeroShotEvidenceLLM` to reproduce the zero-shot EHR
+evidence retrieval pipeline from `Ahsan et al. (2024)
+<https://arxiv.org/abs/2309.04550>`_.
+
+.. autoclass:: pyhealth.tasks.EHREvidenceRetrievalTask
+    :members:
+    :undoc-members:
+    :show-inheritance: