CogPic — A Multimodal Dataset for Cognitive Impairment Analysis

About the Study

What is CogPic?

CogPic is designed to support cognitive impairment screening and subtype research, focusing on multimodal differences between Alzheimer's Disease (AD) patients, Mild Cognitive Impairment (MCI) individuals, and Healthy Controls (HC).

The dataset collects multimodal samples from subjects performing picture description tasks, covering facial frame sequences, speech audio, and transcripts — suitable for multimodal cognitive impairment recognition, speaking style analysis, and interpretable modeling research.

Data organization balances model training and privacy protection. Subject names are anonymized through mapping tables, supporting classification, retrieval, generative analysis, and multimodal fusion tasks.

Explore Modalities

                
                574 Subjects
                AD / MCI / HC groups

                
                Picture Description
                Cookie Theft, Picnic Scene, Accident Scene

                
                Multimodal Capture
                Facial frames, speech audio, and transcripts

                
                Privacy Protected
                All identities are anonymized

Multimodal Data

Three Modalities per Segment

Each sample segment contains face frame sequences, audio files, and transcripts for multimodal fusion research.

Face Frames

JPG frames

Audio

WAV format

Transcript

TXT format

"Through in-depth analysis of multimodal data, we aim to reveal the subtle manifestations of cognitive impairment in facial expressions, language patterns, and acoustic features, providing new scientific evidence for early screening."
— CogPic Research Team

Statistics

Dataset Overview

Overall Scale

Total Subjects 574

Total Segments 1,722

Total Frames 263,536

Avg Frames / Segment 153

Class Distribution

AD 140 subjects

MCI 256 subjects

HC 178 subjects

Train / Test Split

Subject-level split ensuring no subject appears in both training and test sets.

Split	AD	MCI	HC	Total
Train	112	205	142	459
Test	28	51	36	115

Structure

Directory Layout

File Organization

CogPic/
|-- Train/
|   |-- AD/
|   |-- MCI/
|   `-- HC/
`-- Test/
    |-- AD/
    |-- MCI/
    `-- HC/

Sample Example

Train/MCI/MCI_subj_023_1_72_1_28_24/
└── pic_2/
    ├── frames_face/
    │   ├── frame_0001.jpg
    │   ├── frame_0002.jpg
    │   └── ...
    ├── audio.wav
    └── audio.txt

Baseline experiments: https://github.com/CogPic/CogPic_Baseline
— CogPic Baseline

Publications

Citation

If you use the CogPic dataset in your research, please cite the following:

BibTeX

@article{CogPic,
  title={CogPic: A Multimodal Dataset for Early Cognitive Impairment Assessment via Picture Description Tasks},
  author={Wu, Liuyu and Feng, Rui and Li, Jie and Xiang, Wentao and Zhang, Yi and Cao, Yin and Song, Siyang and Gu, Xiao
          and Li, Jianqing and Wang, Wei},
  journal={arXiv preprint arXiv:2604.01626},
  year={2026}
}

Request Access to CogPic

The CogPic dataset is available for research and educational purposes only. Please contact us to request access.

For research and educational use only
Do not attempt to re-identify subjects
Cite the dataset and paper when publishing results

THE COGPIC STUDY