A robot-assisted pipeline to rapidly scan 1.7 million historical aerial photographs

Abstract

During the 20th Century, aerial surveys captured hundreds of millions of high-resolution photographs of the Earth’s surface. These images, the precursors to modern satellite imagery, represent an extraordinary visual record of the environmental and social upheavals of the 20th Century. However, most of these images currently languish in physical archives where retrieval is difficult and costly. Digitization could revolutionize access, but manual scanning is slow and expensive. Automated scanning could make at-scale digitization feasible, unlocking this visual record of the 20th Century for the digital era. Here, we describe and validate a novel robot-assisted pipeline that increases worker productivity in scanning 30-fold, applied at scale to digitize an archive of 1.7 million historical aerial photographs from 65 countries.

Introduction

Aerial photography revolutionized the speed and cost of cartography in the 20th Century. A single flight and a week of laboratory processing could map the same area as a team of ground surveyors in a whole year¹. Around the world, pilots logged millions of flight hours to capture hundreds of millions of aerial photographs of the Earth’s surface. While some of these photographs were lost or destroyed, a survey reveals that more than 140 million have been carefully preserved in dozens of archives worldwide (Tab S1). This extraordinary visual record holds the potential to revolutionize our understanding of the 20th Century, just as modern satellite imagery has transformed our understanding of the world in the 21st Century². With only relatively small historical aerial photography datasets, scientists have tracked glacier change³, shed new light on the impacts of sea level rise⁴, rediscovered indigenous land management techniques⁵, and learned about how to prevent deforestation⁶.

Access to physical archives remains extremely limited, however. Storing a million photographic prints requires approximately 1.4 km of shelf space. Most of the larger aerial photography archives are thus kept in off-site storage, making it costly to search for, extract, and view any images. The negatives and prints in these archives are also, inevitably, ageing and fading⁷. Storage in conditions that are too warm or too humid hastens the rate of decay⁷. To unlock the data stored within these archives for widespread use, and to ensure the preservation of these archives for future generations, many archives are seeking to create digital copies of the physical images. Available in digital form at scale, historical aerial photographs could open the door to an entirely new quantitative understanding of the extraordinary environmental and social changes that characterized the 20th Century.

The manual approach to scanning is, however, prohibitively costly and time-consuming. As a result, although most archives have begun to digitize their physical archives, few approach completion⁸. Here, we describe a robot-assisted pipeline that increases scanning rates per full-time worker more than thirty-fold without compromising the physical integrity of the historical photographs, applied to an archive of 1.7 million historical aerial photographs covering 65 countries worldwide.

Methods

Images of a third of the world’s countries throughout the 20th Century

We developed the pipeline to digitize an archive of 1.7 million images, held and curated since 2012 by the National Collection of Aerial Photography (NCAP) in Edinburgh, UK (part of Historic Environment Scotland). The archive was created by a government unit, originally part of the Colonial Office, established in 1946 as the Directorate of Colonial Surveys. One of its central objectives was to produce accurate topographical maps of the “Colonial Empire”⁹. The unit outlived the decline of the British Empire, being renamed the Directorate of Overseas Surveys (DOS) and widening its remit to developing countries more broadly. The DOS survived in some form—eventually incorporated into the Ordnance Survey, the British national mapping agency—until 2001⁹, well into the era of modern satellite imagery.

Over its history, the DOS mapped more than 2 million square miles (more than 5 million km²) of the earth’s land surface⁹. To produce these maps, the organization commissioned aerial surveys, building on advances in aerial photogrammetry during the First and Second World Wars⁹. The first million square miles of coverage were flown by the British Royal Air Force (RAF)⁹. The earliest RAF-flown missions used the unforgiving K17 Fairchild camera, which required a vacuum to flatten film before exposure and produced inadequate quality images when handled with insufficient care, later superseded by the more reliable Williamson F49^9,10. Later surveys were procured commercially from independent firms, who used a wide variety of aircraft and cameras.

Each survey or contract set out to cover an entire country or region in a particular window of time. To complete a survey, reconnaissance aircraft flew sorties or missions, in this context to photograph a specific area for mapping, recording images on film. In the earliest surveys, in the absence of any geographical information with which to navigate, pilots flew in concentric circles around radar beacons⁹. Later, as navigation instrumentation improved, pilots flew in long parallel swathes, known anecdotally as “mowing the lawn”.

The exact geographical coverage of images in the DOS archive is difficult to confirm. Matching placenames on box labels to modern day countries yields the estimate of 65 countries. For an exact coverage map, one would need to georeference each image in the archive. To do so manually for the entire DOS archive would require approximately 60 person-years. Automated georeferencing processes are still under development¹¹, but we obtain a preliminary coverage estimate by digitizing maps of photographic cover from DOS annual reports (Fig. 1). To the extent that not all photographed areas were mapped in annual reports, the map understates the true extent covered by the archive.

Fig. 1: Approximate coverage of the photographs in the DOS archive.

Estimated based on digitization of coverage maps that were published in DOS Annual Reports between 1951 and 1984. Shading indicates the approximate year of earliest coverage in the archive. Coverage from between 1946 and 1951 is shown as from 1951, based on the earliest available coverage map, published in 1951 and showing coverage to that date. Between 1951 and 1966, maps of coverage were cumulative, i.e., displaying all coverage up to and including the report year. We differenced the coverage areas between years to obtain newly covered areas. From 1967, maps showed recent coverage in a different color, allowing us to observe recent coverage directly. Land in grayscale, modern country boundaries displayed in white. Mollweide projection.

In the process, the organization accumulated an extensive and meticulously curated archive, including two libraries of photographic prints: a master copy and a second, working copy. As the unit wound down and finally closed, the archive was dispersed. While film negatives were transferred to the relevant national governments or mapping agencies of the covered countries, the master print library was entrusted to the British Empire and Commonwealth Museum in Bristol, England¹². Unfortunately, however, storage conditions at the museum proved damp and inadequate, and when the museum later closed—due to financial difficulties¹³ and amid scandal over unauthorized sales of objects¹⁴—the archive was on the verge of being destroyed. NCAP secured permission to take custody of the archive and transferred it to controlled and consistent storage conditions.

All prints in the archive are contact prints, produced by laying film negatives flat on photographic paper and exposing to light. The resulting printed image has exactly the same size as the original negative. DOS surveys typically used 9 × 9 inch negatives, although the archive also contains prints from other negative sizes (7 × 9, 5 × 5, and 22 × 11.5). Prints display both the photographic image and any marginalia, including fiducial marks and metadata recorded contemporaneously, such as the date captured, the general geographic location, the sortie, the frame number, the image type (typically vertical, when captured for cartographic purposes), and the scale (see, e.g., Fig. S1). The prints themselves vary in shape and margin width, many having been hand cut. Most are black and white prints from panchromatic negatives; some are black and white prints from color or color infra-red negatives; and a few are color prints. Prints are organized by survey or contract, and then sortie or mission. Sorties are labeled following an established standard (Tab. S2).

Prints were originally stored in plain brown cardboard boxes, or, occasionally, commercial print boxes from the photographic paper manufacturer. The original boxes have printed or hand-written notations on the outside edge, detailing the country, air force or company, year, sortie code, print numbers, and other information. All boxes were photographed on accession to the NCAP collection.

Missions were flown at a range of altitudes to create maps at different scales. We randomly sampled 124 images from the archive, stratifying the sample to sample one image at random for each country and decade of coverage, if we had collated data on the year of coverage, and one image at random for each country otherwise. For each image, we recorded the focal length (f) and the altitude (H), or the scale (S), as shown in the metadata (see, e.g., Fig. S2). Weighting images according to the share of the archive in each country or country-decade stratum, the mean altitude in meters was 5543 (median 6262; interquartile range 5722–8308, see also Fig. S3). Most cameras had a focal length of about 152 mm, approximately 6^″. Approximating scale S as f/H where scale was not separately specified, we estimate a range of scales from about 1:5000 to 1:80,000 (median 42,579; interquartile range 36,510–62,403). These approximations do not account for the local difference between ground level and sea level—to account for this, we would need to georeference each image, which is difficult without contextual information—so the estimated denominators are an upper bound.

Each contact print has a resolution, typically measured in line-pairs per millimeter (mm) that can be visually resolved¹⁵. The resolution depends on the intrinsic properties of the photographic film and paper, but also on blur and the speed of movement of the aircraft^15,16. Estimates of the resolution of aerial photographic prints in practice vary between 10 and 14 line pairs/mm in the 1940s¹⁶ to 27 line pairs/mm later in the 20th Century¹⁷.

The ground resolved resolution—the size of the smallest object on the ground that can be distinguished on the print—is given by dividing the smallest resolvable feature in the print by the scale. At resolution of 10 line pairs/mm, the smallest resolvable object in a print would be 50 μm; at 27 line pairs/mm, it would be 18 μm. At the median scale in our data, these correspond to ground resolved resolutions of 2.1 m and 0.8 m, respectively.

A robot-assisted preservation and scanning pipeline

Scanning the DOS archive manually would require approximately 10 person-years of tedious, physically demanding labor, even with fast, modern scanners. But the labor required is extremely monotonous and physically demanding, and workers bear a high risk of repetitive strain injury. To make digitization at scale feasible, we developed a robot-assisted pipeline. The pipeline is an example of a cobot¹⁸ or collaborative robotic¹⁹ system. The pipeline leverages the nuance and judgement of human workers during the “preservation pipeline”, in which prints are retrieved from storage and prepared for scanning, at which stage the range of tasks required are varied and complex. The pipeline then leverages the precision and untiring capacity for repetitive tasks of the robot during the “scanning pipeline”, in which each print is scanned to create a digital copy. The physical prints are returned to storage, while the digital copies are stored on a server after post-processing (Fig. 2).

Fig. 2: Overview of robot-assisted pipeline.

LtR: Prints in storage cartons in temperature and humidity-controlled in-house storage, after retrieval from long-term storage. Inspecting prints to evaluate condition and necessary remediation. Robotic arm lifting new print onto scanner bed. Quality control checks during post-processing. All individuals shown have provided informed consent for publication of their images. ⓒNational Collection of Aerial Photography.

The manual preservation pipeline

Before prints can be scanned, workers need to evaluate the condition of each photograph, undertake remedial work where necessary, and prepare the images for scanning (Fig. 3). The prints are silver gelatin, made up of several layers: a paper backing; a white pigmented layer; the photosensitive layer, a suspension of light-sensitive silver salts in gelatin, referred to (albeit erroneously) as the emulsion; and a protective coating^7,20. Due to inadequate storage conditions in the past, some prints have sustained damage. Evaluating and remediating damage requires extensive skill and discretion. The range of preservation issues that can occur is substantial. Table S3 summarizes the total number of boxes and the share affected by specific preservation issues. The most commonly-encountered issues are as follows.

Fig. 3: Overview of preservation pipeline.

First row, LtR: Checking for active mould using UV light. Blocked/adhered prints. Silver dust on surface of print. Second row: Removing surface mould using museum vacuum with HEPA filter and brush. Separated blocked prints. Dry cleaning silver dust with brush using a museum vacuum with HEPA filter to collect particulate matter. Third row: Print with extensive emulsion peeling. Curling prints. Chinagraph annotations. Fourth row: Vacuum packing stacked prints, interleaved with steel plates, ready for scanning. Protecting a print with emulsion peeling using a polyester sleeve. Preparing prints for the humidification chamber before flattening. Cleaning using PEC-12 solvent and PEC Pads. © National Collection of Aerial Photography.

When stored in conditions that are too humid—as they were prior to NCAP’s custody—mould spores can infest prints, feeding off the gelatine and sometimes the paper itself⁷, leaving spores or stains. We remove surface mould using a museum vacuum with a HEPA filter and a brush. Much of the mould is dormant, but we evaluate activity by checking for fluorescence with an ultraviolet torch. If mould is active, we may aqueous clean the prints with a solvent and isolate the affected prints in conservation grade polyester sleeves. If the original box is mouldy, we discard it—noting that information from notations on the box has previously been stored photographically—and transfer the prints to a new archival quality box. Evidence of mould necessitates an entirely separate processing pipeline, avoiding all contact with moisture, which could reactivate dormant mould. Boxes of photographs that are affected by mould must also be scanned separately, and any contact surfaces disinfected afterwards, to avoid cross-contamination of mould strains across boxes of prints.

Blocking occurs when two or more images are stuck together. Blocking typically results from exposure to humidity: the gelatine layer absorbs water, becoming tacky and adherent²¹. Blocked prints can sometimes be separated by gently easing them apart at the corner, especially if the gelatine retains some humidity. Prints are harder to separate when the gelatine is dry²¹. When physical separation is not initially successful, prints are soaked in filtered water for several hours and then gently separated using a bamboo spatula. These prints must then be dried in a nipping press over several days, with repeated changes of blotting paper.

Some prints were annotated, typically to highlight visual points of significance such as roads, rivers, vegetation, or other landmarks that a surveyor might need to use as a control point. However, these annotations obscure the features on the image. Annotations were usually made using a grease, or chinagraph, pencil, and can be straightforwardly removed by aqueous surface cleaning with PEC-12 solvent and PEC Pads. Other prints are also affected by silver migration, which occurs as the emulsion breaks down, allowing silver to migrate through the paper to its surface⁷. This results in a sparkly dust that is highly visible on the fingertip of a black nitrile glove and can include masses up to 1mm in length. This dust can be gently brushed or wiped off with a brush or PEC pad, using a museum vacuum with HEPA filter to collect particulate matter.

The layers of the photographic paper can expand or contract at different rates when exposed to different humidity levels, resulting in a tendency to curl²². Curled prints are vulnerable to damage during robotic handling and scanning. They can fold and crack or crease under the weight of the scanner lid or the steel plates. Additionally, curled areas that do not lie flat against the scanner glass may fall outside the scanner’s focal depth of field. Creasing can also arise if images have not been stored flat. Curled or creased images cannot simply be mechanically flattened without risking cracks in the emulsion. Prints with substantial curling or creasing are thus first humidified for 2-3 hours in a large humidifying chamber. After humidification, prints can be pressed flat in a cast iron nipping press between sheets of aluminium composite material (ACM) and wooden boards, with alternating layers of blotting paper, wax paper, or bondina—a non-woven polyester material—to absorb moisture. The drying and pressing process takes several days. Prints that are both curled and affected by mould are humidified and flattened separately without direct contact with water, cleaning or disposing of any materials with which they come into contact.

Some prints have physical damage, including peeling of the emulsion away from the backing paper²³. Prints with significant rips or peeling emulsion are placed in protective polyester sleeves for transport and scanning. Rarely, sorties have such substantial emulsion peeling that they cannot be scanned as so little information is left on the prints. The affected prints are typically from the older sorties, mostly flown by the RAF. In some of these cases, NCAP held the original film in its archive, which could be scanned in lieu of the prints.

The output of the preservation pipeline is a stack of prints ready to scan, vacuum-packed for safe temporary storage while awaiting scanning.

The fully-automated scanning pipeline

In contrast to the preservation pipeline, scanning involves a very large number of identical repeated tasks. Beyond loading and unloading the prints, the scanning pipeline is entirely automated. We developed the scanning pipeline through extensive piloting and refinement, including a full pilot trial digitizing 8000 images from The Gambia.

Each scanning station houses a multipurpose robotic arm with a bespoke end-of-arm tool designed to lift and release images. The robotic arm is a Sawyer and Sawyer Black robotic arm²⁴, adapted for our task by CBM-Logix, an automation engineering firm specializing in robotics and system integration. Each robot arm houses an optical sensor, four small electromagnets, and a vacuum attachment with four suction cups. The vacuum attachment is calibrated to lift exactly one image from the stack on the input hopper. Each robotic arm operates a pair of Contex IQ Flex scanners (Fig. 4).

Fig. 4: Robotic arm and scanner system.

Figure shows system as robotic arm (3) lifts a print from the input hopper (1) to load it onto the scanner bed (6). The scanner lid (5) will close automatically and the scan initiates. While this scan completes, the second scanner lid (7) lifts and the robot arm will load a print from the second scanner’s input hopper (9) onto the second scanner bed. Once the first scan is complete, the scanner lid lifts, and the robot arm lifts the print from the scanner bed to the output hopper (8). ⓒNational Collection of Aerial Photography.

Workers place prints face down in the input hopper, correctly oriented so that each print is placed consistently on the scan bed. A printout summary of the sortie and frame information is placed on top of each stack of prints from a box. Prints are interleaved with matte black or blue powder-coated mild steel plates to maintain flatness of prints while stacked and to ensure stack stability. To further reduce instability, workers only stack prints of the same size. Approximately 300 prints can be loaded at one time. Workers replenish hoppers throughout the day. After loading prints into the input hopper, workers initiate the robotic program.

Upon initiation of the robotic program, the robot arm moves to the input hopper and its optical system detects whether the target is the reverse side of a print (light in color) or a steel weight (dark in color).

If a print is detected, the arm initiates the vacuum and contacts the print until the pressure changes, indicating good contact. If no pressure change occurs, the robot arm makes two further attempts, applying a small additional downward force before initiating the vacuum. This force ensures that any curled prints will be flattened and in good contact with all four suction cups. The robot arm then lifts the print from the hopper and moves it to the scanner bed, slowly lowering the print onto the bed until it detects a predetermined level of resistance indicating contact with the scanner bed. The vacuum is turned off and the arm moves clear. If a weight is detected, the arm aligns centrally with the weight and lifts it using the electromagnets. The robot arm then transfers the weight to the output hopper. If unsuccessful, the robot arm makes two further attempts before stopping and reporting an error.

The scanner lid is automated using a digitally controlled modified gate opener to cycle the lid. Once the robot arm is clear, it signals to the pulley system to lower the scanner lid. Once the lid is lowered, the lifting system in turn signals to the scanner to initiate a scan.

Once a successful scan is confirmed, the process is reversed. The successful scan triggers lifting of the lid. After the lifting cycle is complete, the controller signals to the robot arm to lift the scanned print from the scanner bed and move it to the output hopper, and then to reload the scanner. While each scan is underway, the robotic arm rotates 180° to unload and reload a second scanner. After the last image is removed from the input hopper, a small lamp lights up. When the robot arm next attempts to pick a print from the input hopper, the optical sensor detects the light, and production stops on that scanner awaiting reloading. The robot arm continues to load the other scanner.

All loading cycle transitions are triggered using absolute references, such as successful lifting of a print or completion of a scan. This eliminates the need for error margins, maximizes efficiency, and ensures that movement cycles only initialize when it is safe.

The scanning pipeline operates in a controlled air environment, maintained at a weak positive pressure through a system of airlocks. Humidity and temperature control are vital when working with archival prints. Additionally, prints shed fibers which can contaminate scanning work spaces, creating tiny artefacts on scanned images. In operation, the controlled air environment successfully reduces fiber spread and greatly delays the onset of print curl.

The scanners are integrated with Nextimage software and set to scan at 1200 ppi, the highest optical resolution of the scanners. The choice of 1200 ppi is partly driven by practical considerations, including the limited availability of scanners that undertake higher-resolution scanning at speed, but it also compares quite favorably to optimal scanning resolutions derived from sampling theory¹⁷. For a print with resolution R defined in line pairs per millimeter, the optimal scanning pixel size lies between $1/2\sqrt{2}R$ and 1/2R. For 27 line pairs per millimeter, the acceptable pixel range is between 13 and 18 μm; for 10 line pairs per millimeter, the acceptable range is between 35 and 50 μm. At 1200 ppi resolution, pixels are approximately 21 μm, more than sufficient for 10 line pairs per millimeter and slightly higher than ideal for 27 line pairs per millimeter. At 1200 ppi, we thus largely preserve the information in the original photographs.

The manufacturer reports a scanning accuracy of 0.1% +/− 1 pixel. Each scanner bed is pre-loaded with a 250 × 25 mm calibration target, displaying a measure scale, tonal step wedge, and resolution target (Fig. S5). The pre-set scan area is 340 × 315 mm, except for the largest 22 × 11.5 inch prints. The scan area includes a margin around the print and the calibration target (see Fig. S1). Except for the few color prints, we scan at 8-bit grayscale.

Post-processing, quality control, validation, and distribution

Once a sortie is completely scanned, it is registered as complete in the database, and scanning metadata is recorded, including the scanning date and the scanner. Scans are checked and renamed to correspond to the print numbers. Workers identify missing scans or prints that were flagged during preservation as too damaged for robot handling. They then locate the corresponding prints and scan them manually on a dedicated flatbed scanner. An automated procedure crops the NCAP calibration target and the surrounding black background, leaving the print and a 5 mm black border.

A worker later checks the quality of the scan and the cropping. They identify any folds or dust particles obscuring the image surface, issues with file naming—either at sortie level or frame numbers—and confirm that none of the image has been cropped. Any prints that fail these quality checks are renamed, re-cropped, or recalled for re-scanning. Material is returned to the off-site storage facility after completing the quality checks.

Capturing the calibration target allows us to later verify the size, tonal values of the image, and capture resolution. In our stratified sample of 124 images, we manually labeled the calibration targets to evaluate the consistency of the results. First, we labeled the beginning and end points of the measure scale in inches and calculated the length of the scale in pixels. Second, we labeled the approximate centroids of the first and last bar of the tonal step wedge, interpolated points between them to locate the centroids of the intermediate bars, and recovered the grayscale pixel value for each bar’s centroid on each image. Lastly, we inspected the resolution targets to identify the last group of bars that could be seen and correctly counted, which reveals the size of the smallest distinguishable feature, to compare to the theoretical resolution of the scan.

The primary uncropped archival versions created by the scanner are uncompressed, full-resolution TIFFs, each approximately 250 MB in size. The entire digitized DOS archive thus constitutes around 425 TB. These versions are stored in archival-grade digital storage optimized for long-term preservation and infrequent access. We also create cropped, 1200 dpi, 50% compressed JPEG versions of each image for distribution to users. Once georeferenced, these versions will be searchable via a mapping interface and available for download on NCAP’s website (https://www.ncap.org/). Since NCAP is a public organization that operates on a cost recovery basis, website subscriptions and download fees fund the costs of site maintenance and service.

Results

A thirty-fold increase in worker productivity

Human workers can, in principle, achieve higher scanning rates per scanner-hour than the automated system. Each scanner can produce a 1200 pixel per inch (ppi) scan in 45 s, whether operated by a human worker or a robot. A human worker continuously operating 2 scanners can achieve a scanning rate of 100 scans/hour, or 50 scans/scanner-hour. While the average time for a loading cycle in the automated system varies, as a result of the use of absolute references to trigger transitions, we estimate using several days of hourly production records that a robot operating two scanners can on average achieve scanning rates of 54 scans/hour, or 27 scans/scanner-hour, 54% of the theoretical maximum human worker rate.

One human worker, however, can only load and unload two scanners manually and only operates scanners during working hours. Assuming a 35-h work week gives productivity of 1750 scans/scanner-week and 3500 scans/worker-week. Each human worker can load and troubleshoot four robots and eight scanners in only 2 h of their working day, leaving the remaining hours free for other tasks, such as quality control. The robotic systems can also, in principle, load and unload scanners 24 h a day, 7 days a week. Assuming 24-hour production 7 days a week gives weekly productivity of 4536 scans/scanner-week and 127,008 scans/full-time-worker-week. Per scanner, the increase in productivity of the robotic system over the manual system is 2.6-fold. Per full-time-worker-week, the increase in productivity is more than thirty-fold.

In practice, neither the manual system nor the robotic system will consistently achieve these theoretical maximum production rates. Humans can be distracted or need to take breaks and sick days, and replacement workers need training and time to learn on the job. If the robotic system encounters a problem—a failed lift of a print, for example—it pauses and returns an error, which may not be resolved until the next working day if it happens at night.

The equipment for this project comprised 7 robots and 14 scanners, giving a theoretical maximum productivity—if all 14 scanners operated continuously for 24 h a day and 7 days a week—of 9072 scans/day and 63,504 scans/week. While the maximum 24-h scanning rate achieved in this project was almost exactly equivalent to this theoretical maximum, the maximum weekly scanning rate in this project was just over half the theoretical maximum. Team members recorded daily and weekly production statistics, annotated to record reasons for anomalous events. Production statistics were only recorded on weekdays, so statistics recorded on Mondays include production over the weekend. Excluding Mondays, the maximum daily scanning rate was 9090 scans, effectively identical to the theoretical maximum. The maximum weekly scanning rate was 36,084, 56% of the theoretical maximum. There was, however, significant learning by doing: scanning rates increased by an average of 13% week-on-week for the first eleven months of this project. After this point, scanning productivity exceeded preservation productivity, creating a bottleneck in the process.

Safe for workers and protective of archival material

The pipeline was both safe for workers and protective of the irreplaceable archival material.

No worker sustained any more than minor injuries during the project. Workers wore gloves, masks, and protective lab coats, and they cleaned prints in fume cabinets to allow the safe removal of mould without exposure to mould spores and to avoid exposure to fumes from solvent cleaners. The Sawyer robotic arm is intrinsically designed for safe use as a collaborative robot, with limited power and force capabilities²⁴. Design features and operational specifications, including shock-absorbing springs in every joint and speed limits on robotic movement, minimize the force associated with any human-robot collision. Additionally, the motion footprint of the robotic arm is designed to minimize the risk of collision. During this project, we recorded one incident of a human-robotic arm collision, which resulted, as designed, in no significant injury.

We estimate that more than 99.9% of prints were scanned without damage. Any damage incurred was minor, limited to print corners or curled prints being caught and folded by the scanner lid or steel plates, and some prints coming into contact with the compressor fluids from the robotic arm. All damage incurred could be remediated using standard preservation techniques and did not affect the fidelity of the final scan.

Fidelity of reproduction

The digital images faithfully reproduce the printed images. Inspecting the calibration targets shows that scanning accuracy, resolution, and grayscale tones meet or exceed the reported manufacturer standards and are consistent across scans.

All but three images correctly displayed the calibration target as designed. The scanning accuracy compares well to the manufacturer’s specifications (Fig. S6). Tones are also clearly distinguished and consistent across images (Fig. S7). The smallest consistently distinguishable feature is on average approximately 46μm in width, approximately twice the pixel size (Fig. S8).

In some cases, the scanned images contain features that reflect not ground conditions in the area depicted but artefacts created during exposure, printing, or storage that cannot be remediated (see Figs. S9a to h). Awareness of these potential artefacts is particularly important for users who wish to take advantage of computer vision techniques to automate feature identification in the scanned images, because of the potential for confusion with ground features. In many other cases, preservation successfully remediates damage incurred during inappropriate storage in the past so that images are restored to close to their original condition (Fig. S10).

Cost-effectiveness

Automation requires a considerable up-front investment in infrastructure. We estimate that automation becomes cost-effective when scanning more than 2.4 million images, which takes only 65 working weeks if 24/7 production can be sustained. Automation more than halves costs compared to the manual approach when scanning more than 5.0 million images.

Our benchmark estimates are based on the following assumptions, as well as the productivity estimates we reported above. For the robot-assisted pipeline, we assume 4 robots and 8 scanners housed in 4 tables, with one additional scanner for manual scanning as required, e.g., for prints that are too fragile for robotic handling. The worker in this scenario dedicates 2/7 of a full-time equivalent position to loading and troubleshooting the robot pipeline, and receives 31,177 GBP in annual compensation²⁵. For a manual pipeline, we assume one worker operates two scanners. The worker in this scenario works full-time and receives 25,325 GBP in annual compensation²⁵. We assume the following unit costs of capital equipment: engineered table to house robotic arm and scanner units 80,000 GBP; robotic arm 36,000 GBP; scanner 5000 GBP; scanner-lid lifting automation 350 GBP. We abstract from rent and energy costs, which are difficult to calculate and compare: the robotic pipeline has a somewhat larger footprint, but the manual pipeline takes much more time. The fixed cost of establishing the robotic pipeline is thus 512,150 GBP and the cost per scan thereafter is 0.0075 GBP, while the fixed cost of establishing the manual pipeline is 10,000 GBP and the cost per scan is 0.22 GBP. With these estimated costs, we can trace out how cost per scan evolves with the total number of scans, yielding the estimates in the previous paragraph (Fig. S11). An important caveat to these estimates is that they are based on UK-specific capital and labor costs.

Discussion

The pipeline we describe here exemplifies an approach to collaboration between humans and robots that draws on the strengths of both: the unique abilities of humans to handle complex, unstructured, and varied tasks, and the capacity of robots for unerring, untiring repetition¹⁹. The consequence is a massive expansion in feasible output per worker.

The reallocation of tasks creates new, varied, and complex jobs that require high levels of skill and experience. Automating one task in the digitization process—scanning—massively decreases the total cost of digitization, increasing demand for digitization programs and in turn for workers to carry out the tasks that cannot be automated (Fig. 2). A direct consequence of the development of the automated pipeline is the creation of seven full-time, permanent positions. These workers are engaged across the full range of tasks involved in digitizing the other archives in NCAP’s aerial photography collection.

One might be concerned nonetheless that automating the scanning process displaces greater numbers of less-skilled workers. However, jobs for these workers were, in practice, largely hypothetical. The costs of manual digitization are high enough that NCAP, like most archives, was simply not undertaking digitization at scale. While global data on digitization progress is not available, two of the largest archives worldwide—NCAP, and the United States National Archives and Records Administration (NARA)—had respectively digitized only about 2% and 0.5% of their aerial photography holdings before this project began.

No previous technology, to our knowledge, achieves the same degree of automation for the scanning of photographic prints. Photographic films with fiducial marks and consistent frame spacing—in widespread use since the 1970s—can already be automatically scanned at rates that are comparable to those we achieve here^26,27. New techniques are currently being developed to apply edge detection to automate the scanning of older films with uneven frame spacing.

Lessons from our approach are applicable to many other aerial photography archives. Even subtle differences in archival material may, however, require modifications. In ongoing work, NCAP are applying techniques developed for this project to the Allied Central Interpretation Unit archive dating from the Second World War. Given wartime supply constraints, these images were printed on cheaper, shinier, and more slippery photographic paper, creating new challenges, such as prints that slip rather than lying still on the scanner bed after placement by the robotic arm, resulting in incorrectly oriented scans. Many of the auxiliary techniques we develop here to digitize prints can also be adapted for archives of photography film.

Beyond the considerable up-front investment in infrastructure, automation also requires preservation and post-processing capacity to match the scanning capacity in order to sustain high production rates. This project was made possible due to research funding that allowed us to undertake scanning at a sufficiently large scale to justify the up-front investment required. Automation will not be individually cost-effective for small archives. New partnerships and collaborations may be needed to completely digitize the entire global patrimony embodied in aerial photography archives.

Beyond automating scanning, there remains much to do to increase access to aerial photography archives. To use digital images that are not yet georeferenced, users must follow a complex trail of historical documentation to locate each image, for example, on the hand-drawn sortie plots that sketched out the coverage of each aerial photography mission, or painstakingly identify the individual features in each image if such documentation has not been preserved. New techniques are also being developed to automate georeferencing and reduce dependency on historical documentation¹¹, creating footprints for each image that greatly increase the efficiency of searching within an archive for coverage of a given location. Creating digital copies of physical images is, however, a critical first step in increasing access to the many millions of aerial photography images currently stored in archives.