HistKI - Investigation, modelling and multimodal AI-based support of image source research and criticism as a complex and fundamental historiographical working technique​​

Photographs and other images of architecture serve as a source and basis for subject- and theory-specific investigations in many historical sciences. For example, historical photographs are used to reconstruct the condition of a building or to identify the formal language of an era. Starting point of these scenarios from architecture, art history and cultural studies is a source research and criticism supported by tools of the respective subjects, upon which further evaluations and uses in the scientific context build.

considerably in recent years, they can only support the process of source research and criticism to some extent, e.g. for the exploration of image repositories or the retrieval of images. This is partly due to the fact that although elementary procedures in this regard are well documented, researchers - as investigated in three dissertation projects under the supervision of the coordinator - proceed very individually. On the other hand, AI image processing has so far been little designed to contextualise image content multimodally, i.e. to combine different source genres such as images and texts. Existing computer vision methods extract purely visual features and classify them, while texts or metadata and the knowledge they contain, such as references to temporal contexts or individual motifs, cannot be linked to the analysis.

The proposed project HistKI aims to research the support and modelling of image source research and criticism as a complex and fundamental work technique in the history sciences through multimodal AI-based procedures. Related sub-questions are: How do historians and other scholars find and evaluate image sources? What generic procedures and sub-problems can be identified for this? How can this be promoted with AI-based approaches? How do AI techniques affect the research process in the humanities?

These questions will be investigated using selected scenarios in which images, texts and 3D models for describing architectural objects and urban ensembles interact synergetically for an analysis process. With the help of machine learning methods, object sources and text sources (e.g. captions) are to be linked in HistKI in order to allow a detailed contextualisation and location of the photographs in the future and thus move a significant step beyond previous methods of distant viewing.

Project duration: 1. Januar 2021 - 31. Dezember 2023

Funding amount: approx. 600.000 EUR

Project participants: Konsortialpartner: LMU München, JMU Würzburg

Funding body: BMBF, Funding code: 01UG2120A