Skills and resources for the multimodal turn: Unlocking audiovisual datasets for research and learning

MULTIDATA will contribute to EU leadership in the transition from analog or textual approaches to digital, audiovisual, and statistical skills for the study of language, communication, cognition, ...

MULTIDATA will contribute to EU leadership in the transition from analog or textual approaches to digital, audiovisual, and statistical skills for the study of language, communication, cognition, and their technological and social applications. We will build an online service to obtain speech and gesture data from video files, offer didactic and learning resources, create an EU-led global community for multimodal data science, and raise awareness of the social implications of multimodality.

We will develop a free online platform with an open-source multimodal processing pipeline to analyze speech and gesture data from videos, alongside learning resources, a user forum, full documentation and code, and tutorials. We will conduct networking across academic organizations, produce publications and presentations at academic events, popularize multimodal data science in the media, and organize local and online outreach events.

An accessible and inclusive website offering the services of the multimodal processing pipeline, tutorials facilitating user experience, comprehensive documentation, and digital resources for learning and teaching. Open-source repositories with all documentation and code. Diverse online and onsite events, publications, presentations, media appearances, and outreach activities to promote and popularize multimodal data science.