Introduction

The contemporary visual arts scene embraces a wide array of artistic forms and creative standards that are gaining popularity through information-sharing platforms. While there have been significant historical examinations of how creative visual art styles have evolved, the quantitative analysis of creative stylization is a relatively recent development. We introduce a research design that aims to document the characteristics of the emergence of artistic movements and their subsequent stylizations. By testing more refined empirical data, we improve upon the previous statistical physics approach. Our interdisciplinary research design incorporates two distinct fields.

First, theoretical definitions of creativity have identified two sequential phases in the recognition of creativity. In the first phase, creativity is considered “the intentional arrangement of cultural and material elements in a way that is unexpected for a given audience1. In the following phase, the creativity finds its “relevance in the new context,” where it contributes to the solution of a problem it assiduously formulated2. While the first phase that “caused scandal” may not necessarily result in the second phase, it often serves as a catalyst for the development of novel artistic styles. Art movements such as Surrealism and Dadaism, among others, have used this disruption to challenge conventional norms and spark creative revolutions3.

Upon the concatenation of the initial disruption and the emergence of a new style, a transformative process takes place. This process influences the creative acts of subsequent artists through the observation, analysis, and internalization of the new style. By assimilating and building upon these innovations, artists contribute to the dynamic evolution of art4,5. Therefore, to gain a comprehensive understanding of the creative process, it is essential to consider the two phases separately. By analyzing the initial novelty and the emergence of a new style as distinct phenomena, we can better appreciate the individualized contributions of each artwork as an original mutation. This analytical approach allows for a more nuanced interpretation of artistic development within an enhanced spatiotemporal framework.

Second, while theoretical studies of creativity in the domain of art can be traced back to earlier eras, contemporary efforts to quantitatively assess and measure relevant characteristics have only recently gained momentum6,7. Through the development of computational and robust methods for analyzing visual arts and the accumulation of large-scale digital art images, scientists have characterized and uncovered various interesting features within a broad range of artworks. Computational studies of visual art have provided both synchronic and diachronic insights into the zeitgeist and evolution of visual arts. Previous studies of visual art styles have been advanced with statistical data of spatial parameters, including chromatic properties8, fractals9, wavelets10, the compression ensemble approach11, and concepts of entropy12,13,14.

In particular, the complexity (C)-entropy (H) plane as a statistical tool has proven to intuitively map a diachronic evolution of art-historical styles13,15. In statistical physics, measurements of C and H based on the distribution of local ordering patterns were initially used to characterize time-series signals; these measurements have since been generalized to analyze patterns of 2D images16. C represents the degree to which the local order patterns deviate from both a random and homogeneous distribution, and H represents the degree of disorder in the image’s pixel organization. C reflects the degree to which the objects within an image are spatially bounded or interrelated (e.g., Impressionist paintings tend to have low C due to the use of different types of brush patterns; different local patterns are detected uniformly on a painting), whereas H reflects the degree to which the objects of an image are more clearly outlined or exhibit fluidity between them (e.g., Mondrian’s minimal geometric paintings tend to have low H) (refer to the Methods: complexity-entropy (C-H) measures of visual artwork images for a detailed description).

Sigaki et al.’s approach13 appraised the localization of the C and H of 137,364 visual artwork images from WikiArt and gained a view of the hierarchical clustering structure of 92 art-historical styles (between Renaissance and Contemporary/Postmodern Art) in the C-H plane. The study regarded the dynamics of transitions of different artistic styles in the C-H plane as evolutionary. This perspective suggests that decoding the latent dynamics that have driven the path of the styles in the C-H plane would reveal the evolutionary process underpinning the emergence and stylization of a specific group of visual arts in terms of the two aforementioned phases of creativity and stylization. Yet, the deterministic properties of the localized information of artworks in the bounded C-H space raise a question of how to estimate evidence of creative stylization of visual arts in the C-H plane.

Here, we consider the indication of diverse image representations of a specific timeframe and groups (i.e., intragroup image diversity) as a conditional phase of the visual artistic stylization in the C-H plane. In this context, we conduct an exploratory data analysis (EDA)17 on the stylization of 149,780 visual art images from quasi-canonical “user-generated arts18” of DeviantArt and Bēhance platforms over a given timeframe (2010–2020) in the C-H plane as an extension of the temporal evolution of art styles by Sigaki et al. Through empirically identifying the relationships between the intragroup image diversity and the corresponding local information of the C-H space, we hypothesize that there will be a particular C-H movement (i.e., mutations) of a specific group of visual artworks over a given timeframe for their stylistic diversity to manifest in the C-H plane. To test this hypothesis, we ask the following research questions:

RQ 1.

Can the C-H plane, which effectively characterized the styles of art-historical paintings, also capture the temporal stylistic evolution (i.e., C-H trajectories) of contemporary user-generated visual arts? If so, what would these trajectories represent?

RQ 2.

How is the average C-H position of user-generated visual arts at a given time related to the intragroup image diversity? Can we comprehend the relationship between C-H positions and the intragroup image diversity through the local information of the C-H space, specifically focusing on the diversity of image representations that can be expressed in particular regions?

To address the issues in the aforementioned questions, we conduct the following analyses.

  1. (1)

    Through validating the applicability of our image set conveyed by the C-H measures and their robustness, we reveal characteristics of temporal stylistic transitions of quasi-canonical user-generated visual arts from DeviantArt and Bēhance in the C-H plane. Moreover, we explore sub-visual art fields within the user-generated visual arts that have significantly influenced the temporal stylistic transitions in the C-H plane.

  2. (2)

    We use two types of similarity measures—cosine similarity and Jaccard similarity—on two multiple image representation spaces—image embeddings through a pre-trained Residual Network (ResNet) architecture19 and Scale-invariant Feature Transform (SIFT) features20, to measure the image diversity/degree of dissimilarity in a specific C-H region. The low- (e.g., visual elements such as lines, contours, height, edges, angles, dots, colors, etc.) and high-level image features (e.g., themes of shapes and objects comprised of low-level features) extracted from the two methods aggregately encompass art style-level explicability of images21.

In addition, we use the autoregressive moving-average (ARMA) model to statistically examine a temporal relationship between the average C-H positions of artworks within a given period and the average dissimilarities of their multi-level image features. Finally, we investigate the diversity of image representations that can be expressed in different areas of the C-H space and unveil an empirical condition for the emergence of styles that are not only novel in the C-H plane but also characterized by greater stylistic diversity.

Results

Stylization of curated visual arts in the C-H plane: DeviantArt and Bēhance (2010–2020)

To account for the stylization of contemporary visual artworks mainly distributed via online communication channels of information, we investigate DeviantArt and Bēhance, which serve as massive online visual art platforms involving various creative fields. Since there are large collections of artworks on both platforms, we specifically focus on representative and quasi-canonical subsets of daily promoted user-generated visual artworks (“Daily Deviation” and “Best of Bēhance”) curated by those platforms. This allows us to process manageable and significant amounts of data22,23. The image set used for complexity-entropy (C-H) measurement is described in Table SI 1.

We map the yearly average C and H values of 149,780 user-generated visual artwork images from DeviantArt and Bēhance (2010–2020) (refer to the Methods: complexity-entropy (C-H) measures of visual artwork images for a detailed explanation of the calculation of C and H values). For comparison, we also overlay the C-H values of conventional art historical paintings from the WikiArt dataset24 as in Sigaki et al.’s experiment13. The WikiArt dataset is composed of 26,415 paintings, primarily collected from Western art history, spanning the period of 1301 to 2016 CE. The art historical periods depicted in our projection of the evolution of visual arts are virtually identical to those identified by Sigaki et al.

Figure 1 demonstrates that both platforms’ C-H trajectories tend toward the upper-left C-H region compared to the previous periods, although their average position shows a slight difference (refer to Table SI 2 for more information regarding the difference in the overall positions of the two platforms in the C-H plane). Also, a closer look at the C-H values of DeviantArt and Bēhance reveals that the averaged positions of the two platforms have varied significantly over time. We cross-validated the temporal difference in C-H values on each platform by evaluating the predictive accuracies of our dataset’s C-H values using four machine learning algorithms. All the classification models predict the curation year of paintings with probabilities (DeviantArt: ~15%, Bēhance: ~13.7%) significantly higher than the chance level (Fig. SI 1).

Fig. 1
figure 1

Group-level temporal transitions of visual art styles in the complexity-entropy (C-H) plane. (a) Contemporary user-generated visual art styles of DeviantArt and Bēhance (2010–2020) compared to the main divisions of art historical periods mapped in the C-H plane. Images with distinct forms and a limited number of ordinal patterns typically exhibit low H values and high C values. Conversely, disordered images comprising more random or interrelated components generally display high H values and low C values. The result reveals that the average C-H values of art historical styles have shifted toward the higher C and lower H areas over time. (b, c) Yearly C-H trajectory of contemporary user-generated visual arts from DeviantArt’s Daily Deviations and Bēhance’s Best of Bēhance (refer to the Data (Materials)). Each C-H plane is composed of multiple elliptical areas, with each ellipse representing 95% confidence interval bounds from the yearly average C-H values of the intragroup visual artworks. The multiple CI ellipses, along with the yearly C-H trajectories connecting them, altogether illustrate the yearly transitions of visual art styles on each platform. Also, insets are included to show the cumulative number of samples over the given time frame.

Full size image

The C-H trajectories of temporal transitions of DeviantArt and Bēhance visual art styles over a decade extend that of the transition between the early 20th century and the 1970s (e.g., from painterly/optic to linear/haptic). Observing the beginning and ending years (Fig. 1b, c), the C of DeviantArt’s visual art increased by 0.002 (from 0.123 to 0.125), while their H decreased by 0.005 (from 0.84 to 0.835). In the case of Bēhance, there was a rise in C by 0.01 (from 0.12 to 0.13) and a decline in H by 0.11 (from 0.81 to 0.7) over the same period. The group-level temporal transitions of visual art styles among the two platforms reveal indirect but eventual tendencies toward styles with higher C and lower H. Consequently, the chronological C-H trajectories of the contemporary user-generated visual arts of both platforms from 2010 to 2020 indicate that their stylizations altered in a macroscopically similar manner, despite the differences in direction details within each platform.

Meanwhile, the yearly average C-H values of Bēhance are notably distinct from those of DeviantArt and the major art historical periods. We explore which sub-visual art fields substantially influenced the C-H positions of Bēhance over time by separately examining the temporal C-H movements of five creative fields that account for the greatest proportion of samples in the Bēhance dataset. We observe the yearly average C-H values of samples from the “Illustration”, “Graphic Design”, “Character Design”, “Animation”, and “Fine Arts” fields in Bēhance. Figure SI 2 shows that all the five fields’ yearly average C-H values are specifically directed toward a similar C-H area (C: 0.1–0.3, H: 0.65–0.85) through a decade of movements. Overall, the C-H movements of the fields reveal a tendency for homogeneous transitions of visual art styles in Bēhance during the time.

Extending Sigaki et al.’s analysis on the evolution of art styles allows us to discover the group-level temporal stylizations of quasi-canonical visual artworks in DeviantArt and Bēhance (2010–2020) aligning with that of the modern art identified by Sigaki et al. Overall, the C-H movements of the platforms are characterized by an increase in the high-level25 visual structure of artworks incorporating geometrical object-oriented patterns; the emergence and taking up of an artistic style with clearer and simpler visual elements.

Through structural notions of creativity and stylization, it is possible to infer the strengthened stylization of the contemporary user-generated visual arts toward a more linear/haptic region in the C-H plane. This trend can be interpreted as a process where artworks exhibit what Birkhoff called a “bijectively homomorphic” (structurally similar)26 stylization. In simpler terms, while the visual representations of artworks remain diverse and random at an individual level, they interact and gradually align with one another, leading to a homogenization of styles. The significance of individual artists’ innovation being proportional to the extent of its influence on others27 fuels a competitive and mimetic process under uncertainty, paradoxically resulting in a cycle of imitation as a necessary step toward creativity28. Also, in any given institution, the number of “insiders” on the curatorial committee is fewer than expected, resulting in “a durable and recognizable pattern of aesthetic choices29”. In response to the inherent “symbolic uncertainty” in the platform of visual arts, individual artists may be subject to “mimetic isomorphism,” or mimicking others’ distinguished artistry as the most cost-effective strategy for gaining insider acceptance30.

The convergence of two distinct groups of images within the C-H framework over a decade exemplifies mimetic isomorphism, where shared environmental factors (e.g., technological trends, aesthetics, or cultural shifts) drive contextual adaptations rather than mere replication. This phenomenon highlights how stylistic and structural similarities can emerge across distinct entities through imitative and adaptive processes in complex creative domains. The rapid dissemination of information and images, driven by platform-based visual content structures, might have accelerated the homogenization of contemporary user-generated visual art styles.

Spatiotemporal relationships between image diversity and the C-H space

We investigate how the average C-H positions of user-generated visual arts from DeviantArt and Bēhance moved over time. In this section, we attempt to reveal the spatiotemporal relationships between image diversity and the local information of the C-H space so that we can understand the previous C-H movements over time in terms of intragroup diversity and stylization.

To assess the intragroup diversity of image representations from the two platforms in the C-H space, we use two different similarity measures on two types of image features. First, we use cosine similarity of the artwork’s image embeddings (IE): multi-level image representations comprising both low- (100-dimensional) and high-level (100-dimensional) image features. Further, we use the Jaccard similarity coefficient of low-level features (i.e., keypoints) of two images matched by the standard SIFT algorithm. Figure 2 shows the detailed pipelines of the two image similarity measures. The analytical independence of the two similarity measures and the distance in the C-H plane verify that their pairwise correlations are fairly weak (range: 0.003–0.07; see Fig. SI 3).

Fig. 2
figure 2

Image representations in the complexity-entropy (C-H) plane and pipelines of image similarity measures through different image representation spaces. (a) Three reputable exemplary masterpieces chosen and localized in the C-H plane according to the given embedding parameters (\(\:{d}_{x}={d}_{y}=2\)), to aid viewers’ understandings of the image representations the plane encompasses. (b) A schematic diagram of an image similarity computation pipeline through a neural network. We measure pairwise cosine similarities of a 200-dimensional embedding vector including both low- and high-level feature maps extracted from individual visual artworks. (c) A schematic diagram of the image similarity computation pipeline through the SIFT descriptor. We measure pairwise Jaccard similarities of the SIFT descriptor matching a 128-dimensional vector of low-level features extracted from individual visual artworks. In addition to the image representations the C-H plane encompasses, we adopt and use two different types of image processing methods to obtain multi-level image features and measure their similarities: the ResNet architecture and the SIFT algorithm. Through our approach, we investigate characteristics of image representations in the C-H plane with image similarity measures that aggregate both low- and high-level features. The implemented images (1–3) are “Composition 2” by Piet Mondrian, 1929 (public domain, retrieved from WikiArt, https://www.wikiart.org/en/piet-mondrian/composition-2), “Rhythm” by Robert Delaunay, 1912 (public domain, retrieved from WikiArt, https://www.wikiart.org/en/robert-delaunay/rhythm-1), and “The Starry Night” by Vincent van Gogh, 1889 (public domain, retrieved from WikiArt, https://www.wikiart.org/en/vincent-van-gogh/the-starry-night-1889). All the image samples used are available in the public domain.

Full size image

We first examine the relationship between the average C-H positions of user-generated visual arts from DeviantArt and Bēhance over the specified timeframe (2010–2020) and their intragroup image diversity. To account for trend components in the time-series data, we use an ARMA model to regress the average values of the dyadic similarities among the artworks on the average values of complexity C and entropy H over the given years.

Table 1 ARMA predicting average similarity among visual arts in Deviantart and Bēhance. Refer to the Methods: statistical regression of visual artwork image similarities for a detailed description.
Full size table

The ARMA model is applied separately to subsampled images from the DeviantArt and Bēhance datasets. The results (Table 1) indicate that the C and H values of a particular year are associated with a decrease in similarity, suggesting that the C and H values can affect the increase in diversity, despite the decrease in average absolute diversity over time. Here, we note that as the analysis predicts a movement of the average value of diversity for a given year with its average value of C and H, the area with large C and H values in the entire C-H space does not represent the area of high diversity. Meanwhile, the variance of H is added as an independent variable and controlled on the one hand, confirming that H variance also significantly has a positive correlation with diversity in all cases.

Higher H is associated with lower degrees of the IE and the SIFT similarity among the artworks over time, as revealed by both platforms in our research. Similarly, higher levels of C are significantly associated with lesser degrees of the IE and the SIFT similarity. The H of artworks in both platforms are skewed to the left (DeviantArt: skewness = -1.95; Bēhance: skewness = -1.04), whereas the C is relatively symmetrical (DeviantArt: skewness = 0.2; Bēhance: skewness = 0.02).

A distribution of given years’ images in the C-H space is likely to include a wider variety of styles along with an increase in image diversity within the given year group upon the following conditions. First, when there is an average movement (indicating increases in C and H) toward the upper-right C-H region, which is the optimal direction of the upper-left (similar, unexplored, sparse) and lower-right (diverse, explored, dense) C-H regions. Additionally, when the distribution (large H variance) is made on the entropy H axis.

We now partially examine the robustness of the ARMA results through patterns of the local stylistic diversity visualized over binned areas of a partitioned C-H region (C: 0–0.31, H: 0.5–1; see Fig. 3). Overall, the sample count plots (Fig. 3a, d) show that the most densely populated area is to the lower-right C-H region with given embedding parameters,\(\:\:{d}_{x}={d}_{y}=2\). This finding confirms that there is likely to be a strong correlation between image diversity and the highly concentrated C-H region, where previous art-historical styles with great image diversity are positioned. Hence, the more artworks where entropy H is greater, the greater image diversity there is likely to be.

In terms of observing image diversity manifested in the C-H region, we first measure the mean value of pairwise cosine similarity of the IE of all the samples from each accountable C-H bin. Subsequently, we measure the mean value of pairwise Jaccard similarity of the SIFT features of randomly subsampled artworks from each accountable C-H bin. The results altogether reveal that the highly dense C-H area tends to reflect less image similarity. The IE similarity (Fig. 3b, e) and the SIFT similarity (Fig. 3c, f) plots demonstrate that at a given level of entropy H, higher complexity C is likely to be associated with greater intragroup image diversity (see Fig. SI 4 for the supporting results of the WikiArt images previously mapped for Fig. 1a).

Fig. 3
figure 3

Visualization of spatial relationships between the C-H space and the observed image diversity in DeviantArt and Bēhance data as measured by the image similarity. (a, b, c) Degrees of sample density, the IE similarity (sample mean), and the SIFT similarity (subsample mean) of the DeviantArt images per bin in the C-H region. (d, e, f) Degrees of sample density, the IE similarity (sample mean), and the SIFT similarity (subsample mean) of the Bēhance images per bin in the C-H region. Each C-H bin (C ≈ 0.01, H ≈ 0.02) per platform occupies more than 50 images.

Full size image

As observed in Fig. 1a, our results uncover the C-H movements of quasi-canonical visual artworks in contemporary online platforms, which ultimately trended to the upper-left C-H region. The C-H movements begin and propel away from the highly dense and stylistically diverse areas (low C and high H) and move toward sparser and stylistically homogeneous areas (high C and low H), forming a process of a particular stylization. Such a stylization (toward higher C and lower H) is consistent with the broader movements in Western art history14, which ranged from classical to modern times. However, the region with high C and low H has the lowest diversity (i.e., high image similarity) of image representations in artworks. Therefore, even though propelling toward the area is a new artistic endeavor in terms of C and H, there are few artworks that could be expressed with the given C-H values.

Based on our analyses, we suggest conditions for the groups of visual artworks to have high intragroup diversity and shed light on an evolutionary process in which the intragroup diversity gives rise to styles. It is confirmed that when the average C-H movement of the groups is balanced toward higher C and higher H rather than the extremes in the C-H space, the stylistic diversity of the groups shows a significant tendency to increase more compared to the previous period. Our findings also suggest that a visual artistic stylization process occurs as a result of the cumulative diversity of individualized artworks contributing as original mutations over time, to the transformative process of the prior narratives of styles and the emergence of a new style.

Discussion

Beyond computationally observing the macroscopic evolution of visual art, opportunities for analytically mapping its latent dynamics in a creativity framework remain. Notably, Sigaki et al.13 have performed a seminal work that demonstrates the feasibility of using the C-H plane to intuitively map the diachronic evolution of visual art styles. The study has recognized “a natural law in the same way as physical growth (Wölfflin)” when affiliating historical alterations in style, particularly from linear to painterly. This remark is similar to David Bohm’s belief about artistic creativity being related to “a harmony parallel to that of nature (Cézanne)31”.

This study quantitatively tracks user-generated artworks of the contemporary visual art disciplines within the C-H space and interprets their stylization from the perspective of diversity within groups. Apart from the findings of the study by Sigaki et al. providing useful insights into the evolution of visual art styles and their potential relationship to natural laws and creativity, we take an alternative strategy to explore the evolutionary dynamics of creative visual arts in the C-H plane. We look at the evolutionary process underpinning a stylization through the lens of selection, in which various competing agents interact. This state arises from the emergence of novel and different mutations during the reproduction process32. A stylization of visual arts entails the popularization of diverse artistic forms that are influenced by novel stylistic canons. Therefore, considering diversity as a prerequisite for the emergence of creativity and subsequent stylization, we investigate the C-H plane to empirically identify the optimal C-H condition by which stylistic diversity is most pronounced while visual artworks of a specific timeframe and groups continue migrating toward homogeneous styles.

We find that as the average entropy H rises, so does the intragroup diversity. Our analyses indicate that a novel style emerges in C-H regions where random and diverse agents coexist with strong individual innovativeness (Fig. 1). This finding echoes Eric Hobsbawm, who observed that during the period of highly unpredictable social and technological changes (i.e., the avant-garde prior to 1914), modern art (low C and high H; Fig. 1a) was “not to claim that it displaced the classic and the fashionable, but that it supplemented both3”.

The ARMA analysis (Table 1) supports our viewpoint by demonstrating a substantial increase in the respective group’s intragroup diversity when the group’s C-H movement is directed toward a balance (synchronous increases in C and H). Moreover, the results of this study empirically confirm that the intragroup image diversity varies depending on the locations in the C-H plane, further indicating the heterogeneity of image representations and local stylistic diversity of the C-H space. Therefore, from a collective rather than an individualist perspective, we assume that the optimal C-H condition for the greatest intragroup diversity and emergence of a certain style will be revealed when a group sets a balance between attempts to escape the conventionally dense C-H area (moving toward high C and low H areas) and attempts to remain in the C-H area where diversity can be expressed (low C and high H).

Building on the study by Sigaki et al.13, our research provides a novel perspective on contemporary visual art disciplines by positioning user-generated artworks within the complexity-entropy (C-H) space. This framework quantifies stylization while revealing diversity and coherence within artistic groups. Our work extends the analysis of C-H trajectories of visual art stylization from the early 20th century to the 1970s, confirming an intensified tendency of contemporary visual artworks to shift toward the upper-left region within the given C-H framework. By situating our findings within art history, parallels can be drawn with earlier work that used neural networks to trace artistic influences across historical movements33, as both approaches highlight how stylistic diversity reflects creative innovation and adherence to group norms. Similarly, our entropy-based analysis aligns with a study that applied spatial entropy to landscape analysis34, underscoring entropy’s utility in analyzing structural and compositional diversity across different contexts.

Our research also complements the findings of Valensise et al.35, who used the C-H plane to analyze meme evolution, highlighting how community-driven curation fosters a balance between novelty and conformity. This dynamic is evident in curated subsets like DeviantArt’s “Daily Deviation” and Bēhance’s “Best of Bēhance,” which mirror the evolutionary patterns of digital culture. Additionally, our work aligns with Deng et al.36, who examined representativity in art paintings. Their study underscores the challenges of quantifying art’s diversity while preserving its nuanced cultural significance. By extending these efforts to contemporary digital art, we bridge historical and modern perspectives, demonstrating how user-generated art mirrors broader trends in collective artistic evolution.

Meanwhile, this study has several limitations, including potential selection bias in our data and the methodologies used for measuring image similarity. While our dataset includes works by artists from around the world, it is predominantly composed of two-dimensional visual artworks created by Western artists. Furthermore, the curated user-generated artworks on DeviantArt and Bēhance during the specified time likely represent only a fraction of the vast and dynamic evolution occurring within contemporary visual art disciplines.

As platforms showcasing global creativity, their curated collections are expected to result in cultural and social diversity. However, many independently operated platforms within specific cultural contexts remain unexplored, presenting valuable opportunities for further research. Future studies could address these limitations by incorporating a broader range of data sources, including other social media and regional art-sharing platforms, to enhance the representation of global artistic and cultural diversity and to validate and expand upon our findings.

Regarding our methods for extracting image features, although the ResNet-18 architecture and the SIFT altogether account for multi-level image features, there are numerous other possible combinations of methods that capture various spatial image characteristics. Understanding the limitations of our methods for measuring image similarity and how they impact our analyses is also crucial. The SIFT algorithm is sensitive to image variations (e.g., significant affine transformations, illumination changes, and image noise) possibly affecting the detection and description of keypoints. Possible loss of subtle yet important features or fine-grained details cannot be ignored due to reduced dimensionality in latent PCA embeddings from the ResNet. Therefore, exploring alternative or complementary visual features in future experiments could provide more robust and generalizable insights.

The rapid advancements in computer vision have introduced many image processing models, each offering unique capabilities for analyzing visual data. Different choices of feature extraction models may produce moderately varying similarity outcomes, thereby shaping diversity assessments and influencing our understanding of the evolution of art styles. Notably, transformer-based models, such as Vision Transformers (ViTs)37, have demonstrated potential for capturing global context across entire images, providing richer and more nuanced feature embeddings. Specifically, CLIP (Contrastive Language-Image Pretraining)38 and DINO (Distillation of Non-contrastive Image Representations)39 provide embeddings aligned with semantic meaning, which could improve high-level similarity detection. While our research focused on assessing the “diversity of visual forms” by focusing more on visual information itself, future studies could explore the diversity and evolution of contemporary visual arts from a new perspective. This could involve utilizing state-of-the-art models to measure artwork similarity based on semantic and contextual information, offering a deeper understanding of artistic trends and styles.

Future research could also investigate the properties of images generated by artificial intelligence (AI) models. Recently, state-of-the-art AI models (e.g., DALL-E240, stable diffusion models41, etc.) have made enormous strides in text-to-image art generation. With growing interest in AI-generated images, the relationship between AI and human creativity has come to the forefront of research. A recent study of images generated by a stable diffusion model revealed that AI-generated images tend to have a narrower distribution of entropy and complexity when compared to human drawings; this may lead to less diversity and creativity in their visual artistic expression42. In addition, another study addressed the difficulties of evaluating interactions between human and computational creativity in online creative ecosystems43. Even though the development of AI art is rapidly advancing and attracting a great deal of attention, we cannot overlook the need to investigate and comprehend its characteristics in a more systematic manner. One could pose queries about the potentials of AI art with a desire to investigate its representations and inspirations. As our methods are readily applicable to any corpus of two-dimensional digital images and their resulting knowledge maps, we believe the framework of this study can facilitate a better understanding of diversity and stylization in the emerging field of AI art.

Data (Materials)

Quantitatively analyzing raw cultural data of user-generated content to capture their similarities on many possible dimensions can yield insights into the diversity of data’s visual organization in multidimensional spaces44. On this account, we assessed popular online platforms as sources of artistic creativity and diversity—DeviantArt and Bēhance, which have become increasingly popular among artists who exhibit and share their creative works online. Accessibility, availability, and direct observation of a large corpus of user-generated visual art images are notable benefits of these platforms.

Meanwhile, the vast size of the entire collection of user-generated artworks on these platforms makes comprehensive analysis impractical. Therefore, we processed a more manageable and meaningful dataset of high-quality artworks broadly recognized as influential within their respective artistic communities. To ensure that the images analyzed represented broad trends in contemporary user-generated visual art, we specifically focused on subsets of artworks curated by the platforms themselves: “Daily Deviation” on DeviantArt and “Best of Bēhance” on Bēhance. These subsets are considered quasi-canonical, as they are selected and promoted daily by the platforms based on quality, innovativeness, and relevance—reflecting current trends in user-generated art.

We used the 149,780 images with the C-H values (Table SI 1) during the IE obtaining process. However, the computing process of extraction and pairwise comparison of SIFT features between images consumes considerably more time compared to that of the image embeddings through convolutional neural networks (CNN). Therefore, we implemented two separate subsampling procedures for EDA on spatiotemporal relationships between local information in the C-H space and image diversity. (1) We randomly subsampled images for both IE and SIFT similarity measures for the ARMA model (Table SI 3). (2) Considering each C-H bin sample as a population, we additionally used the sample size formula (confidence level: 95%, margin of error: 5%) to determine and randomly subsample the minimum number of necessary samples to meet the desired statistical constraints and to draw proper inferences from respective C-H bins in Fig. 3.

Methods

Complexity-entropy (C-H) measures of visual artwork images

Sigaki et al.13 observed that changes in complexity (C) and entropy (H) of paintings over time could reflect the evolution of art historical styles. Here, we explain how a two-dimensional image is represented in the C-H space. We calculate the normalized permutation entropy H and the statistical complexity C of an image based on its matrix representation. The original image files were in JPEG and PNG formats, represented in a 24-bit RGB color space (8 bits each for the red, green, and blue channels). We obtain a simpler matrix representation of each image by averaging the three color channels of every pixel.

Next, we examine submatrices with ordinal patterns for each image using embedding dimensions\(\:\:{d}_{x}\times\:\:{d}_{y}\) (in our case, \(\:{d}_{x}={d}_{y}=2\)), leading to \(\:(\left({d}_{x}{d}_{y}\right)!=24)\) possible ordinal patterns. By sliding partitions of size \(\:\:{d}_{x}\times\:\:{d}_{y}\:\)pixels across the entire matrix, we obtain a distribution of ordinal patterns P for the image. Here, \(P={p_i; i=1, \ldots,n}\) can be viewed as a 24-dimensional vector whose elements sum to 1.

From the probability distribution P, we calculate the normalized Shannon entropy H(P):

$$\:H\left(P\right)=\:\frac{1}{\text{l}\text{n}\left(n\right)}\:\sum\:_{i=1}^{n}{p}_{i}\text{l}\text{n}(1/{p}_{i})$$
(1)

where \(\:n=\left({d}_{x}{d}_{y}\right)!\).

The entropy H approaches 1 if the pixel order appears random, and H ≈ 0 if the pixel order is highly regular and always appears in the same configuration. The value of H signifies the degree of ‘disorder’ in the configuration of the pixels in an image, as represented by its matrix representation.

Furthermore, C(P) is calculated to investigate the degree of structural complexity present in the submatrices. C(P) is defined as:

$$\:C\left(P\right)=\frac{D\left(P,U\right)H\left(P\right)}{{D}^{*}},$$
(2)

where \(\:U=\{{u}_{i}=1/n;\:i=1,\ldots,n\}\) represents the uniform distribution, D(P, U) is the Jensen-Shannon divergence between P and U, and D* is the maximum Jensen-Shannon divergence.

The Jensen-Shannon divergence D(P, U) is given by:

$$\:D\left(P,\:U\right)=S\left(\frac{P+U}{2}\right)-\:\frac{S\left(P\right)}{2}-\:\frac{S\left(U\right)}{2},$$
(3)

where \(\:(P+U)/2\:=\:\left\{\right({p}_{i}+1/n)/2,\:i=1,\ldots,\:n\}\). The maximum divergence D* can be calculated as:

$$\:{D}^{*\:}=\text{max}D\left(P,U\right)=\:\frac{1}{2}\:[\frac{n+1}{n}\text{ln}\left(n+1\right)+\text{l}\text{n}\left(n\right)-2\text{ln}\left(2n\right)]$$
(4)

.

This is obtained when \(\:P=\{{p}_{i}=\:{\delta\:}_{1,i}\:;i=1,\ldots,n\}\), where \(\:{\delta\:}_{1,i}\:\)is the Kronecker delta.

C(P) increases as the distribution P of the local order patterns deviates from the uniform distribution U or as the entropy of P increases. Specifically, C(P) reaches its maximum in a state that is neither completely uniform nor entirely homogeneous. Stylistically, C(P) reflects how much the objects within an image are spatially circumscribed or interrelated, while H(P) reflects how distinctly the objects are outlined or how fluidly they are intertwined15.

In practice, we calculate the C-H values of artworks using the Ordpy module45, a simple and open-source Python module that implements permutation entropy and several principal methods for analyzing time series and two-dimensional data using complexity parameters16,46,47. To obtain the upper- and lower-boundary curves in the C-H plane for our 2D images, we employed two functions, maximum_complexity_entropy and minimum_complexity_entropy in the Ordpy module45. These functions systematically generate multiple probability distributions over the possible ordinal patterns and compute both H(P) and C(P). They do so by assigning probability mass to one or a small number of ordinal patterns while distributing the remaining probability evenly among the other patterns, scanning through this space of distributions for H and C values. Collecting and sorting the H and C points yields approximate upper- and lower-boundary curves that envelop the values found in experimental or simulated data, thus delineating the limits in the C-H plane.

Lastly, for the choice of embedding dimensions, we set \(\:{d}_{x}={d}_{y}=2\). While these values are tuning parameters that can be adjusted, it is known from previous studies that \(\:\left({d}_{x}{d}_{y}\right)!\) \(\:\ll\:\) (width × height) must hold to ensure reliable results16. Given that the average image width and height of 149,780 images used in this study are approximately 1000 pixels (average width: 1011.61px, average height: 953.91px; refer to Table SI 1), we chose \(\:{d}_{x}={d}_{y}=2\) as the embedding dimensions.

Statistical regression of visual artwork image similarities

Among typical statistical models for time series analysis (e.g., the Bayesian network, the hidden Markov model, etc.), we used the ARMA model to regress the dyadic similarities among the artworks on the degree of complexity C, entropy H, and their variances over the respective years from 2010 to 2020 (Table 1):

$$\:{y}_{t}={X}_{t}\beta\:+\sum\:_{i=1}^{p}{\phi\:}_{i}{y}_{t-i}+\sum\:_{i=1}^{q}{\theta\:}_{i}{\epsilon\:}_{t-i}+{\epsilon\:}_{t}$$
(5)

where \(\:{y}_{t}\) is the average value of the dyadic similarities in DeviantArt and Bēhance at year t, and Xt is the vector of C-H covariates at time t. Variables p and q are the number of lags for the autoregressive and moving-average components, respectively. In order to obtain a stationary series of our data, we used the modified Dickey-Fuller unit-root test48, which determined the parameter p. This results in p = 3 for IE models and p = 1 for SIFT models. q was determined to be 1 for both IE and SIFT models using Bartlett’s approximation49.

Multi-level image features for visual artwork similarity measures

The Euclidean distance in the C-H space (Fig. 2a) is a straightforward metric for measuring the stylistic dissimilarity between images. In addition, we further adopt and utilize both global and local descriptors to extract multi-level image features and measure their similarity: the ResNet architecture and the SIFT algorithm. By utilizing the multi-level image features that aggregate both low- and high-level features, we disclose stylistic characteristics of image representations in the C-H plane.

Recent analyses of visual art styles use style vectors based on deep CNN to extract stylistic (i.e., high-level) features and information from paintings50. On the other hand, the SIFT feature descriptor, as a well-established CV technique, has been demonstrated to be effective for various objectives related to image matching and object recognition. CNN filters by themselves perform similarly to the SIFT in detecting low-level and straightforward (i.e., handcrafted) local invariant features, while outperforming the SIFT in detecting high-level and complex features51. The combined usage of CNN and SIFT features has proven to result in discriminative multi-level image features using the best of both52.

We initially built on the ResNet-18 architecture, which is pretrained with the ImageNet database, to construct multi-level image features in our image set. The ResNet pretrained with ImageNet has also been shown to provide sufficient feature representations for paintings19, making it appropriate for extracting latent visual features—i.e., image embeddings (IE)—from heterogeneous visual artworks.

Inspired by Liu et al.’s approach21 of extracting multi-level features of artworks to represent art styles of images, we obtained the embeddings of our images in the following manner (see Fig. 2b for a detailed technical pipeline). (1) As combining both low- and high-level features is effective for encoding art styles, the max (front) and average (back) pooling layers of the ResNet-18 are chosen as the convolutional layers to extract hidden feature maps from each input image (224 × 224 × 3). (2) We then reduced the dimensionality of the hidden low-level feature map (56 × 56 × 64) extracted from the maximum pooling layer to a one-dimensional feature map (56 × 56 × 1). (3) We also extracted a hidden high-level feature map (1 × 512) from the average pooling layer just prior to the fully connected layer, where no feature map exists. (4) We then ran a Principal Component Analysis (PCA) on both low- and high-level feature maps to derive 100-dimensional embeddings from each, thereby creating a 200-dimensional embedding vector for each user-generated artwork. (5) Lastly, we measured pairwise cosine similarities between embeddings of the artworks.

Following the neural network-based image similarity measurement, we utilized the SIFT feature matching technique (see Fig. 2c for a detailed technical pipeline). Specifically, the SIFT uses the local histogram of oriented gradients to match the local features of an image with those of other images53. The SIFT features are extracted from keypoints detected between two distinct images while preserving their original data dimensions; gradient orientation histograms are extracted from quadrants of points of interest in an image. They are then merged into a normalized histogram (a SIFT descriptor) that is invariant to image location, scale, and rotation20. Each SIFT descriptor in an image is compared to its counterparts in other images.

We implemented the standard SIFT feature indexing and matching technique for obtaining the desired degree of similarity between images based on its highest matching accuracy compared to the performance of other succeeding feature descriptors (e.g., PCA-SIFT, SURF, BRIEF, ORB, etc.)54. As for the SIFT feature extractor, we used the xfeatures2d module from the cv2 interface in OpenCV versions for Python55 to calculate the SIFT matching degree between images. (1) We first detect the local extrema in a single image as a potential keypoint detected in each pixel compared with its neighboring pixels. Each accurate keypoint selected from an image then generates an orientation histogram of 8 bins for the keypoints’ neighboring 16 × 16 blocks, which are then subdivided into 16 sub-blocks of size 4 × 4. The respective keypoints are subsequently allocated 128 (= 8 × 16) bin values as a vector56. (2) We then calculate the number of acceptable matches between keypoints from a pair of input images by the Fast Library for Approximate Nearest Neighbors (FLANN) based on a k-nearest neighbors (KNN) matcher57. (3) Using the number of keypoints and their good matches from each pair of images, we calculate the intersection (n of matching keypoints) divided by the union (n of total keypoints from a pair of images) of the images’ keypoints.