Introduction

Since the early nineteenth century, scholars have examined the relationship between the perception of color and levels of civilization. In his Zur Farbenlehre (1810), the German poet and physicist Johann Wolfgang von Goethe noted that “uncivilized” nations, children, uneducated people, and southern Europeans (especially women) had “a great fondness for colors in their utmost brightness.” In contrast, he claimed that the people of Northern Europe mostly avoided vivid colors (Goethe and Eastlake, 1970). This assertion prompted subsequent scholars to empirically explore the relationship between culture and Farbensinn [color sense].

In an overview of this kind of work, German ophthalmologist Hugo Magnus argued that varying Farbennomenclatur [color vocabularies] were prevalent among different societies, with the so-called uncivilized societies having a less developed color vocabulary. Magnus noted that indigenous peoples often had no words for blue and green. The lack of these color terms, however, was not caused by a lack of diversity in language. He observed that the Sotho people of present-day South Africa had more than “26 different words for the color of cattle.” In his view, even a “European eye” would be hard-pressed to separate all these shades of brownish gray. Why did the language of the Sotho, capable of such nuance, not contain “at least one word” for “sensations as sharp as blue and green” (Magnus, 1880)?

In 1969, Berlin and Kay ([1969]1991) provided the commonly accepted answer to Magnus’ question. Based on a large-scale study of different languages, they found that the color vocabularies of different peoples are consistent in how they partition the color space, which means that color sense is universal and evolutionary in nature. The primary color terms of a society were directly related to the total number of color terms in its language. Almost all indigenous societies had words for black/dark and white/bright. In line with the cultural development of a people, new colors were added in a predictable order: first, red, then yellow or green, then blue, then brown, and finally, purple, pink, orange, or gray.

In the 1980s, semiotician Umberto Eco gave a different answer to Magnus’ question. While he agreed that color sense was tied to language, he dismissed the notion that this connection was evolutionary or universal (Eco, 1985). While recent work often approaches color sense as a compression problem (Regier et al. 2007; Komarova et al. 2007) — how do humans efficiently map the vast space of perceivable colors to a small set of color terms?— Eco focused on the wide variety of cultural systems that could guide the making of this map. He argued that the color system of a society, the ontological structure behind the divisions of the color space into distinct color terms, depended on the usefulness of these terms in everyday life. In this theory, the twenty-six different terms that the live-stock herding Sotho used to describe cattle made perfect sense. Based on the same data as Berlin & Kay, recent studies have provided empirical evidence for this cultural relativist view of color sense: “communicative needs”—the need to be able to describe objects of a particular color—are presented as explaining differences in the number and combination of color terms between different languages and different peoples (Gibson et al. 2017; Twomey et al. 2021).

Although diametrically opposed in many aspects, the universalist and cultural relativist views of color sense share a common characteristic: both theories directly relate the colors that people see to closed-off cultural units. However, since the mid-nineteenth century, as a result of a set of historical phenomena described as globalization, different cultures, their color systems, and their color terms have come into contact with each other (Osterhammel, 2014). In addition, in the same period, the industrialized world experienced a “color revolution” (Blaszczyk, 2012): artificial dyes, and the new color media that made use of them, made it feasible to color in the world in new ways and en masse. All over the globe people started to use these new color media to imagine their own world as well as the world(s) of others. Following the work of Gibson et al. (2017), we might argue that they had the communicative need to give color to worlds that were not their own. This process was, of course, constrained by the possibilities of different color media, such as painting, illustration, or (color) photography: different media made different color palettes available.

How can we investigate the influence of color media on historical color sense? Two issues impact our research. First, the colors of historical visual sources, such as paintings, illustrations, and photographs, change over time. Digitization of these sources might further alter the original colors that were used. Second, the ways in which we map the color space to color terms might have changed considerably. Rossi (2019) notes that we have no way of knowing if our word for yellow describes the same range of physical sensations as it did around 1900. Partly a result of these methodological difficulties, historical color sense has received little scholarly attention: a lacuna which Kalba (2021) describes as a form of “historiographical color blindness.” This article shows, however, that by comparing the colors of groups of images produced by different color media we can learn a lot about how these palettes influenced what kind of world people in the past could see. More specifically, building on the work of Said (1978), we examine how the palettes of specific color media impacted the production of two particular cultural categories: the “imagined geographies” of the Occidental and the Oriental worlds.

This article harnesses machine learning and explainable AI (xAI) techniques to investigate how different media shape our ability to color in the world of others. We analyze the use of color in two turn-of-the-twentieth-century collections of color(ed) photographs: photochromes and autochromes, representing the “Orient” and the “Occident”. In the case of photochromes a printer added color, while the colors of autochromes were derived from the interaction between the photographic plate and light during exposure. Using dominant colors as input features for a random forest classifier, we answer the following three research questions: Do specific media affect the use of color? Can we use dominant colors to differentiate between visual representations of the Orient and Occident across different color media? Does the presence or absence of specific colors define visual Orientalism? By addressing these research questions, we bridge the gap between color perception and historical color representation in different cultures and contribute to a more comprehensive understanding of historical color sense.

Background: orientalism and (color) photography

In Orientalism (1978), literary scholar Edward Said posits that the West defines itself by contrasting its own image with the imagined world of the “Oriental Other”. In the nineteenth century, colonial powers started to study, describe, and represent the Orient, which refers to the countries we now primarily associate with the Middle and Near East. To justify colonial subjugation, the scientific and artistic field of Orientalism produced a “contrasting image” of the Occidental, industrialized, western world. The Orient was represented as static, un- or underdeveloped, and non-rational. More positively, at least for some writers and painters, it was also seen as mysterious, sensual, and sexually inhibited: a “living tableau of queerness” in the words of Said. A world of strange customs, strong smells, and, most importantly for our research, intense colors (Oueijan, 2006).

While Said focused on text, other researchers extended his theory to visual culture. In her groundbreaking essay ‘The Imaginary Orient,’ Nochlin ([1989]2018) examines the orientalism of so-called orientalist painters. Famous artists, such as Eugene Delacroix (1798–1863), claimed to provide a realistic and authentic picture of countries in the Near- and Middle East. In reality, their depiction of the Orient as an exotic, backward, and uncivilized place served as a mirror image of the West’s self-image. Others have extended Said’s theory to photography. While we might expect this medium to provide a more objective picture of the world, scholars have argued that it did not fundamentally change how the Orient was depicted. By capturing the same visual “phantasms” of Orientalist painters as “objective” visual facts, photography “fixed and stabilized” patterns of Orientalist representation (Alloula, 1986). More importantly, cheaply produced and widely distributed pictures, such as postcards, distributed visual Orientalism to a mass audience and ‘democratized’ access to its fictional world (Behdad and Gartlan, 2013; Behdad, 2016).

Most studies of Orientalist visual culture focus on content: the scenes in the harem and bathhouse, palm trees, camels, minarets and mosques, slave markets, and arabesque decoration that made up the visual Orientalist world. However, visual Orientalism also found expression in a specific “Orientalist aesthetic” (Benjamin, 2003): an aesthetic in this context refers to a distinctive set of visual principles, stylistic choices, and artistic techniques that collectively define the look and feel of Orientalist artworks. Traveling to countries in the Near- and Middle East, painters became fascinated by the ”brilliant colors” they saw. Accurately capturing them became the “key aesthetic challenge” of Orientalist painters and photographers (Benjamin, 2003).

Since the invention of photography in the 1830s, it has been inextricably tied to Orientalism. Behdad and Gartlan (2013) even argue that the development of photographic technology was partly fueled by an “exoticist desire for visual contact with the rest of the world”. The aesthetic challenge of the Orientalist painters—the accurate depiction of color—played an important role here. For example, the well-known mid-nineteenth century photographer Francis Frith (1822–1898) complained that his “colorless pictures” could not truly capture the “sunny East” (Jay and Frith, 1973).

Around the turn of the twentieth century, two new photographic media made it possible to represent the Orient in color. In 1888, the Swiss printing company Photoglob patented Photochrome: a new lithographic printing technique for mass-producing realistically colored in photographs. After hand-coloring a black-and-white negative, lithographic stones were made by letting daylight pass through different color filters. These tint stones were used to mass-produce colored photographic prints that contained combinations of four to fourteen different colors—depending on how many tint stones had been applied. Despite claims of the company that photochromes represented “photography in the colors of nature”, the colors were added to black-and-white negatives and thus reflected the printer’s color sense (Arqué, 2009).

In 1903, the French Lumière brothers, famous for their invention of the Cinematograph, announced a new color medium: autochrome. This technique captured color during the exposure of the photographic plate. In the process, a glass plate was coated with small grains of red-orange, green, and blue-violet potato starch. For the human eye, the light coming through all the differently exposed grains blends together, resulting in a wide range of colors on the photograph (Lavédrine and Gandolfo, 2013). Orientalist photographers quickly embraced this new medium. An advertisement for a presentation of Orientalist autochromes in Paris underlined the realistic colors of the pictures: “A plate sensitive to colors, which it registers by fixing them forever! Formerly an unrealizable dream, today a reality […] All the colors, the most delicate nuances are registered by the autochrome plate … the plate faithfully returns what nature offered her” (Boulouch, 1996). In contrast to these claims about the objective nature of autochrome, scholars have argued that these pictures still bear the subjective interpretations of their creators and remain embedded in the tradition of Orientalist representation.

The photochrome and autochrome were used for different purposes. Because color was added during printing, photochromes could be mass reproduced. As a result, the technique was widely used in the tourist industry for prints, postcards and small brochures, which were sold at tourist sites and through mail order catalogs. One of the largest sellers of photochromes, the American Detroit Publishing Company, offered between ten to thirty thousand different images and produced up to seven million prints in some years (Walter and Arqué, 2020). In contrast, autochromes were difficult to reproduce. To print them, the original photographs had to be converted into halftone plates, which could only approximate the colors of the autochrome. Consequently, the autochrome process was mainly used by artists and amateurs (Langford, 2022). However, as the advertisement quoted above shows, collections of autochromes were frequently exhibited to larger audiences.

Materials and methods

Data

To study the impact of color media on historical color sense, this article uses AI techniques to compare how photochrome and autochromes were used to color in the Oriental and Occidental worlds. We rely on two digitized collections: 6500 photochromes (1890–1910) and 65,000 autochromes (1909–1931). The photochromes originate from the Library of Congress’s Photochrome Print Collection, produced by Photoglob and the American Detroit Photographic Company (Walter and Arqué, 2020). The autochromes are from the Albert Kahn Archives de la Planète collection, which features pictures taken by photographers commissioned by French banker Albert Kahn (Bjorli and Jakobsen, 2020). Kahn’s collection aimed to foster mutual understanding among different cultures and to document fading traditions (de Luca, 2022).

Both collections were digitized in the early 2000s. According to the National Library of Congress website, “views of Europe and the Middle East” in the photochrome collection were digitized by staff in 2001–2002 using a local flatbed scanner at a resolution of 400 pixels per inch (National Library of Congress, 2010). Information on the digitization of the Albert Kahn collection is more elusive, but it is known that the process was conducted locally at a slow rate of 90 images per day (Texier, 2010). As it was standard practice at the time, both processes likely involved color calibration via a color chart, such as ColorChecker. However, there are no color charts visible on the digitized images. To minimize differences in scanning quality and color degradation, we have implemented a clamping method that normalizes differences in brightness.

We took several steps to preprocess the images for our analysis. Geographic entities were aligned to accommodate both modern and historical naming conventions. A rule-based approach was used to automatically crop parts of the images. For the photochromes, which contained a small white border, we applied a crop ratio of 0.85 to remove it. The digitized autochromes featured a watermark from the Albert Kahn Museum in the lower-left corner, along with textual descriptions on white paper that often appeared on the left, right, or top borders. To remove these areas, we used crop ratios of 0.75 for the bottom and 0.92 for the left, top, and right edges. Because the cropping was applied uniformly, it will not affect the computational analysis of color.

For our analysis, we categorized the two collections based on the countries they depict, dividing them into Orient and Occident groups (Table 1). This resulted in 6685 Occident and 7024 Orient autochrome images, and 4435 Occident and 371 Orient photochrome images.Footnote 1 This classification, while acknowledging that the cultural category of the Orient has no clear geographic borders, facilitates the computational analysis of the cultural production of the Orient as an “imagined geography” through different kinds of color(ed) photographs (Gregory, 2000). The division of countries into Oriental and Occidental is purely for analytical purposes and is meant to reflect the historical cultural category of Orientalism. While Orientalism has served as a divisive mechanism with profound societal effects, such as colonialism and Islamophobia, the dichotomous separation between Oriental and Occidental countries has never been warranted by observable facts.

Table 1 Number of images per country for Autochromes and Photochromes.
Full size table

Method

The digitization of historical visual sources allows us to examine color use at scale. Manovich (2012), for example, used image features, such as color, contrast, and hue, to visualize large collections of images. Computational methods have also been applied to investigate color in early cinema (Olesen et al. 2016; Flueckiger, 2017). K-means clustering has been used to extract dominant colors from images, offering a novel way to explore digitized collections (Wrubel, 2021).

Our method comprises five steps:Footnote 2

  1. 1.

    K-means Clustering: We apply K-means clustering to extract eight and sixteen dominant colors from each image. This method clusters the red, green and blue (RGB) values of each pixel into n clusters, based on the mean pixel value proximity.Footnote 3 Although these dominant colors might not align with historically common colors, they enable us to quantitatively explore if color could be used to imagine different worlds.

  2. 2.

    Frequency Calculation: We determine the relative frequency of the dominant colors within the image as weights (Fig. 1).

  3. 3.

    Color Bucketing: For every image, we place the dominant colors and their weights into equally sized color buckets based on the RGB color space. As such, similar colors will be grouped together. We divide the dimension (0–256) into 4, 6, or 8 parts, resulting in n_division^3 buckets.

  4. 4.

    Random Forest Classification: Using the color buckets and their weights as an input feature, we employ random forest classification to predict if an image is a photochrome or autochrome, and if it represents the Occident or the Orient. During training, a Random Forest algorithm constructs a multitude of decision trees (is this A or B)? During classification, the output of the classifier is the class selected by the most trees. Applying this simple algorithm allows us to assess whether dominant colors can be used to distinguish between the two color media and/or images of the Orient and Occident in the photochrome and autochrome collections.

  5. 5.

    Interpretation with SHAP: We use SHAP (SHapley Additive exPlanations), an explainable AI (xAI) method, to interpret the output of the classifiers. This method allows us to understand how a specific feature—in this case a specific color bucket—impacted the classifiers’ predictions for specific images. The blue-red color bar in Figs. 2 and 3 indicates the absence or presence of the feature and the x-axis what the impact is on the model’s prediction. For example, in Fig. 3 an absence of color 447 (blue dots) has a negative impact on the prediction, thus pointing toward “Occident”. In contrast, a strong presence of the same bucket (red dots) strongly predicts that the image should be classified as “Oriental”.

    Fig. 1: Autochromes and photochromes of the Orient and Occident.
    figure 1

    The histogram on the bottom shows the 16 dominant colors and their relative frequency.

    Full size image
    Fig. 2: SHAP plot for autochrome vs photochrome classifier (16 colors, 512 buckets), demonstrating the impact of various color features.
    figure 2

    Each row corresponds to a feature, represented by a number that indicates the color bucket. The x-axis shows the SHAP value (impact on model output), while the color gradient from blue to red represents low to high feature values within each bucket. The color swatches display the central RGB values (color) for the top five most influential color buckets.

    Full size image
    Fig. 3: SHAP plot for Occident vs Orient classifier in photochromes (16 colors, 512 buckets).
    figure 3

    See Fig. 2 for a detailed explanation.

    Full size image

Results

Our study has yielded three principal findings. The first result addresses research question one: do specific media affect the use of color? This question relates to the classifier’s ability to use dominant colors to distinguish between photochromes and autochromes with an accuracy of 0.95. This distinction was achieved by calculating 16 dominant colors per image, which were placed in 512 color buckets. Increasing the number of dominant colors per image or the number of color buckets only slightly improves accuracy (Table 2). The biggest increase can be found in photochromes, which might result from the lower number of oriental images in this category.

Table 2 Accuracy of classifiers.
Full size table

Given the unique production processes of photochromes and autochromes, the ability of the classifier to separate based on color might be unsurprising. However, the result empirically demonstrates our thesis that different color media represent identical subjects and scenes in varied colors. It underlines that, around 1900, color media started to significantly influence how the color space was divided into color terms. In this period, people all over the world gradually transitioned from only directly perceiving the colors of the world around them to experiencing their environment, and that of others, through color(ed) photographs. In other words, the perception of color became a mediated phenomenon. While recent literature on color sense investigates the impact of increasing industrialization on color perception, this mediatization of color sense has remained unaccounted for.

Using the SHAP methodology, we can examine which colors are relatively important for the classification algorithm in making a distinction between photochromes and autochromes. Figure 2 shows that the absence of certain colors can contribute just as much to a classifier’s output as their presence. In the humanities, this kind of pattern is often ignored. Art historical research seldom describes formal aspects or style by noticing and describing that which is conspicuously not there. The absence of the dominant colors stored in color buckets 146, 438, 365, and 73 is a strong predictor of an image being an autochrome. At the same time, the absence of the colors of bucket 229 strongly indicates that an image is a photochrome. Because the colors of photochrome are derived from artificial dyes added to the picture by a printer, the output of our classifier suggests that this photomechanical process is unable to produce the colors of the first four buckets. Similarly, the colors stored in bucket 229 were probably not a part of the palette of the autochrome printers.

Our second finding addresses research question two: can we use dominant colors to differentiate between visual representations of the Orient and Occident across different color media? We demonstrate that we can use dominant colors to distinguish between representations of the Occident and Orient in the photochrome collection (accuracy: 0.93). However, for the same task in the autochrome collection, our classifier only achieves an accuracy of 0.67. Here, our classifier only slightly outperforms a random guess in determining whether an autochrome portrays an Oriental or Occidental scene, as delineated by fin de siecle contemporaries (Table 2). For photochromes, as mentioned earlier, printers added colors. As a result, we might suspect that their idea of how the Occident and the Orient should look influenced the colors they used. The accuracy of our classifiers demonstrates that autochromes, where color was derived from the interaction between light and the photographic plate, presented a more neutral or complex perspective of the Orient and Occident. This proves that, as far as aesthetics are concerned, it became harder to maintain the imagined separation between the Orient and the Occident after the introduction of the autochrome. This, of course, does not mean that distinctions between the two imagined geographies were no longer being represented in stereotypical content. For example, in Fig. 1, we immediately note the difference between the mosques and the Swiss mountains. These aspects are not picked up by our classifier.

Our third finding relates to research question three: Does the presence or absence of specific colors define visual Orientalism? The SHAP methodology also supports this finding. We observe that in the Photochrome collection, the presence and absence of specific colors can be potent predictors of visual Orientalism—the way the “West” represented the Oriental Other to establish its own identity. More specifically, we find that bucket 447 favors Oriental predictions, while its absence strongly points towards the Occident (Fig. 3). Similarly, color bucket 1, when present, weakly indicates Occidental images, whereas its absence weakly suggests Oriental pictures. Notably, the presence of the green color associated with bucket 18, despite not being readily linked by the authors with Orientalist aesthetics, indeed predicts this category.

Before presenting our three main conclusions, we want to explain why we did not include a close reading of one or a small number of images in our analysis. While digital humanities (DH) research often oscillates between distant and close forms of analysis, this study intentionally avoids close reading. Doing so would distract from our central argument: that color serves as a subtle but powerful predictor of visual orientalism in photochromes. Our aim is to demonstrate that it is not the specific color of a mosque, camel, or Swiss lake that matters, but rather the broader, perhaps even unconscious, decisions to use different colors for the depiction of oriental and occidental scenes. We argue that these patterns are most effectively observed when we take a distant view and might even be lost when individual images are examined.

Our results inform three interconnected conclusions: (1) color sense became mediated in the late nineteenth century, (2) in photochromes, the presence and absence of specific colors was a vital aspect of visual Orientalism, (3) the autochrome, where color was derived from light, provided a more objective picture of countries in the near and middle East than the photochrome.

Discussion

A recent study described the cattle color terms of the Sotho people as an “indigenous knowledge system which is clearly in decline” (Taljard, 2015). As a result of colonialism, industrialization, and globalization, it seems reasonable to assume that this phenomenon is part of a wider pattern: the homogenizing force of globalization has resulted in a smaller number of culturally specific perceptions of color. Despite this decline, research on color sense has predominantly centered on linguistic variations in color vocabularies, attributable either to evolutionary stages (the universalist perspective) or cultural contexts (the cultural relativist standpoint). Shifting our attention to the impact of visual media, we have studied the color palettes that two different media enabled at the turn of the twentieth century. Our findings indicate that the possibilities and constraints inherent in diverse visual media played an instrumental role in shaping historical color sense. Essentially, our collective interaction with color has become increasingly mediatized, i.e., shaped by the affordances of different media. While other research has pointed out that the usefulness of colors in daily life shapes the color space in closed-off cultures (Gibson et al. 2017; Twomey et al. 2021), we add the role of media across cultures as a factor. Consequently, we advocate for a comprehensive approach in color perception studies that goes beyond linguistics, incorporating the material circumstances that influenced people’s engagement with color.

Even though it might be impossible to directly measure which colors were perceived in the past, our investigation corroborates the existence of two distinct color palettes belonging to two imagined geographies—the Orient and the Occident. We underline the vital role of color, an important formal element of aesthetics, in the cultural production of these worlds, thereby broadening existing studies of visual Orientalism that have mainly concentrated on content. As the nineteenth century drew to a close, photochromes might have seemed more “real” because of their color’s vivacity. However, as our research shows, their production method facilitated the subjective creation of fictitious color universes. Our research thus underlines that the perceived realism of photochromes made them more fictious. While photochromes both mirrored and shaped the cultural production of the Orient and Occident, autochromes, recognized as the first photomechanical color medium, captured the world’s colors more neutrally, thereby limiting the ability of contemporaries to color in the world of others.

This article hopes to provide a starting point for research that studies how the mediatization of color has influenced color perception—the colors we see and use to interpret our world—over the last two centuries. We examined one transition, from photochromes to autochromes, but numerous other historical transitions exist. Think, for example, of Dufaycolor (1935), Kodachrome (1935), Agfacolor-NEU (1936), and digital techniques for color photography or the different systems, like Technicolor (1922), used for moving pictures. More recently, visual material generated by AI has presented viewers with idiosyncratic color schemes in imagined worlds, signifying yet another area of interest for future research.