Artificial Intelligence and the Art of Culinary Presentation

On February 16, 2021, Emily L. Spratt, a postdoctoral research fellow at the Data Science Institute, gave an online presentation, “AI and the Art of Culinary Presentation: Gastronomic Algorithms between Alain Passard and Giuseppe Arcimboldo,” sponsored by the Graduate School of Architecture, Planning, and Preservation’s Historic Preservation Program.

Historic Preservation Program Director Jorge Otero-Pailos introduced Spratt, and commended her work for reimagining cultural heritage through the use of artificial intelligence.

“Even if we’re not experts in French cuisine, but just in preservation in general,” Otero-Pailos said, “how can we think of preservation as an expression, as something that needs to be created, and something that we all experience aesthetically?”

Spratt explored the creative process behind the platings, or presentations of food, by Alain Passard, chef and owner of the Paris-based, three-star Michelin restaurant L'Arpège. Her interest in his work was piqued by her observation that his dishes resemble paintings.

A prolific collage artist, Passard is known for employing the medium as an inspiration for his dishes. Computer scientist Thomas Fan and the staff of L'Arpège also contributed to the gastronomic algorithms project, which is notably the first application of Generative Adversarial Networks to the world of haute cuisine. 

“Although the recent developments of artificial intelligence to generate visual content with deep learning techniques, especially GANs, have offered radically new possibilities for its creative applications in the visual medium, there has been little investigation into its use as a tool to explore the engagement of the senses beyond vision alone,” Spratt said. 

Preserving Gastronomy as an Intangible Heritage

Spratt spoke about the preservation of culinary traditions and the history of gastronomic cultural heritage, showing photographs of menus from the Titanic and diagrams illustrating 17^th-century linen folding techniques. Although tangible artifacts such as these reveal associations of a particular culinary experience, she pointed out their insufficiency in capturing the full experience of a meal. 

How can culinary traditions be preserved, Spratt asked, when food is ultimately meant to be consumed? UNESCO recognizes French cuisine as an intangible heritage, which it defines as “not the cultural manifestation itself, but rather the wealth of knowledge and skills that is transmitted through it from one generation to the next.” 

The gastronomic algorithms project, in contrast, emphasizes the cultural manifestation itself. Specifically, the project focuses on the artistic dimension of plating through Passard’s use of collages to visually conceive of actual plates of food. Taking this one step further, the project also explores how fruit-and-vegetable-embellished paintings by the Italian Renaissance artist Giuseppe Arcimboldo (1526-1593) could be reproduced through the use of artificial intelligence tools. 

Spratt then asked the leading question of her research: “How could GANs, a generative form of AI, emulate the culinary images, and would doing so visually reveal anything about the creative process between the chef’s abstracted notions of the plates and collages, and their actual visual execution as dishes?”

Experimenting With Datasets

Although Passard’s collages are a source of inspiration for his platings, a one-to-one visual correlation between the appearance of both does not exist. The dataset initially comprised photos posted by Passard on Instagram, images provided by the restaurant’s employees, and photos captured by Spratt at L'Arpège during each of the different seasons. This was later supplemented by images of vegetables and fruits on plates, as well as sliced variations procured from the internet using web scraping tools. 

To imitate aspects of Arcimboldo’s food-filled portraits, Spratt relied on a dataset of 300 portraits generated by a GANs-based model. Following initial tests, the dataset and algorithms were fine-tuned to increase the differentiation of individual fruits and vegetables characteristic of the platings designed by L'Arpège.

“The GANs-produced dishes have at once both an alluring aesthetic quality and a frightful one, as a tomato carpaccio seems to be able to open eyes and wink, and vegetable flourishes appear to have grown wisps of hair,” said Spratt.

Aiming for a more abstract version, Spratt added collages—created by both Passard and an artificial-intelligence algorithm—to the preexisting image dataset. Unique combinations such as these set the project apart from other AI-generated artworks.  

The Creativity Behind Culinary Art

Otero-Pailos asked Spratt to speak about the division between documentation and artistic expression in culinary art. “Does the line for you necessarily need to be blurred in this form of documentation? Or would you be okay with just a simple photograph of the plate as a record?”

The project, said Spratt, intentionally challenges the UNESCO definition of intangible heritage. Beyond mere documentation, the crux of the project focuses on the creativity behind culinary art. “How could that creative process [undertaken at L'Arpège] be codified as intangible heritage? How do we capture that?”

Shannon Werle is the digital editor in the communications office at GSAPP.