Skip to main content
Artificial Intelligence

Context-aware multimodal AI navigates hidden pathways in five centuries of art evolution.

| Source: Proceedings of the National Academy of Sciences of the United States of America

The rise of multimodal generative AI transforms the intersection of technology and art, offering richer insights into large-scale artworks. While significant research has focused on their creative potential, their ability to represent artworks in latent spaces remains underexamined. We use generative AI, specifically Stable Diffusion, to analyze 500 y of Western paintings by extracting two types of latent information with the model: formal aspects (e.g., colors) and contextual aspects (e.g., sub

The rise of multimodal generative AI transforms the intersection of technology and art, offering richer insights into large-scale artworks. While significant research has focused on their creative potential, their ability to represent artworks in latent spaces remains underexamined. We use generative AI, specifically Stable Diffusion, to analyze 500 y of Western paintings by extracting two types of latent information with the model: formal aspects (e.g., colors) and contextual aspects (e.g., subjects). Our findings reveal that contextual information exhibits stronger vector alignment and orientation with conventional artistic periods, styles, and individual artists than formal elements. Also, we show how artistic expression aligns with historical shifts using contextual keywords extracted from paintings. Our generative experiment, infusing prospective contexts into historical artworks, validates this vector alignment and orientation by synthesizing artworks consistent with the stylistic patterns of target periods. This study demonstrates how multimodal AI expands traditional formal analysis by integrating temporal, cultural, and historical contexts to quantify the latent structure of cultural knowledge.

Read the original source →

Related Stories

Artificial Intelligence

Plant fiber exploitation contributed to the emergence of ground-edged cutting tools in North China.

Ground-edged cutting tools are widely regarded as technological hallmarks of early agriculture in North China, commonly hypothesized to have been developed primarily for cereal harvesting. Yet the functional foundations of this innovation remain insufficiently tested. This study integrates use-wear and microfossil analyses of 140 stone tools from the Peiligang site, with experimental data, to reassess their roles within long-term trajectories of technological change from the Late Paleolithic to

Continue reading
Artificial Intelligence

Neural network-augmented Pfaffian wave-functions for scalable simulations of interacting fermions.

Developing accurate numerical methods for strongly interacting fermions is crucial for improving our understanding of various quantum many-body phenomena, especially unconventional superconductivity. Recently, neural quantum states have emerged as a promising approach for studying correlated fermions, highlighted by the hidden fermion and backflow methods, which use neural networks to model corrections to fermionic quasiparticle orbitals. In this work, we expand these ideas to the space of Pfaff

Continue reading
Artificial Intelligence

Why friends in common reveal network stars.

The Friendship Paradox states that, on average, your friends have more friends than you do. We extend this to common friends-those who appear in multiple people's friend lists. We show that the more people who share a common friend, the more connected that person tends to be, and we derive an expression quantifying this progression. In a regional Facebook network, a common friend to three randomly sampled individuals has on average more friends than 99.9% of the network. In a citation network, a

Continue reading
Artificial Intelligence

Sensory context improves language prediction in humans and LLMs.

Language is a fundamental human capacity. Large language models (LLMs) have presented the first viable model of language outside of humans, yet how these models learn and use language differs significantly from humans. Here, we compare LLMs and humans predicting language with varying levels of sensory information-from disembodied written text to audiovisual videos of speakers-to demonstrate that, in both humans and LLMs, sensory context is critical for optimal performance. We asked human partici

Continue reading
Artificial Intelligence

Treatment Decisions in Multiple Myeloma.

Revolutions in transplantation and targeted and immune therapies have transformed multiple myeloma from a disease with an associated survival of a few years into one for which functional cure is an emerging goal. This abundance of effective therapies has created clinical complexity. Here we provide a practical framework, anchored in trial evidence and informed by emerging biologic discoveries, for the navigation of treatment decisions across the disease spectrum. We outline how cytogenetic and g

Continue reading