Algorithmic Ways of Seeing: Using Object Detection to Facilitate Art Exploration

This Research through Design paper explores how object detection may beapplied to a large digital art museum collection to facilitate new ways ofencountering and experiencing art. We present the design and evaluation of aninteractive application called SMKExplore, which allows users to explore amuseum's digital collection of paintings by browsing through objects detectedin the images, as a novel form of open-ended exploration. We provide threecontributions. First, we show how an object detection pipeline can beintegrated into a design process for visual exploration. Second, we present thedesign and development of an app that enables exploration of an art museum'scollection. Third, we offer reflections on future possibilities for museums andHCI researchers to incorporate object detection techniques into thedigitalization of museums.