UNIVERSITY PARK, Pa. — Elizabeth Mansfield, professor and head of the Department of Art History, and James Wang, professor of information sciences and technology, have received a Digital Humanities Advancement Grant (DHAG) from the National Endowment for the Humanities (NEH) for a project that will use computer-aided image analysis to examine the depiction of clouds in the paintings of John Constable, a 19th-century European artist noted for his pictorial realism.
The project, “Seeing Constable’s Clouds: An Application of Machine Learning to Art Historical Research,” will use computer vision and machine learning to reveal formal details and patterns that art historians may overlook or be unable to discern, due to their generalized sense of what a “Constable cloud” should look like. Specifically, Mansfield and Wang, who are co-principal investigators on the project, want to determine if a neural network can be trained to recognize the difference between a painted cloud by Constable, a “straight” photograph of a cloud from the repository of the National Oceanographic and Atmospheric Agency Photo Library and an artistic photograph of a cloud from the 1850s by Gustave Le Gray.
“If a neural network can be trained to discriminate between these different kinds of representations of clouds — and not simply to discern imaging artifacts — this training might be applied to data sets composed entirely of images of paintings to see which are classed as more or less ‘photographic’ — a measure of pictorial realism adopted by viewers in Constable’s lifetime,” explained Mansfield. “Such results might enable researchers to understand the visual cues that persuaded Constable’s 19th-century viewers to see in his cloud studies — as viewers continue to see today — persuasive representations of natural phenomena that are virtually impossible to document exactly using brush and paint.”
Computer vision and machine learning have been applied to art historical research since the early 2000s. Current applications of machine learning to art historical research include projects devoted to training computers to recognize specific objects or individuals depicted in various pictorial arts such as prints and paintings. Mansfield and Wang’s approach is to develop machine-learning algorithms to detect the most significant stylistic differences among cloud study paintings. The features that reveal the most distinctive differences among images with clouds will be determined by Convolutional Neural Network (CNN), a deep learning algorithm commonly applied to analyzing digital imagery, which can take in an input image, assign importance to various aspects/objects in the image and be able to differentiate one from the other.