ISSN 1753-7223

home | issues | authors | alerts | about | panel | subscriptions

issue 3, 2008 << back to contents page

The Interaction of Art, Technology and Customers in Picture Making

John J McCann

McCann Imaging, Belmont, MA 01478, USA

Summary

Introduction

Pictures can be paintings, prints, lithographs, silver-halide photographs, video images, digital stills and movies, or phone displays. Pictures are the result of both technology and consumers’ use of images. We know of pictures made as long ago as 14 000 BC in the Lascaux cave paintings. Most images up to the 13th century were narratives of objects of interest, without much thought devoted to scene reproduction. With the adoption of Brunelleschi’s geometrical perspective in 1400 AD, pictures added more realistic shapes and sizes to the rest of the scene around these objects. With the addition of chiaroscuro, around 1500 AD, the illumination became as important as the objects. As well, chiaroscuro introduced rendering high-dynamic-range scenes in low-dynamic-range media.

The advance of photo technology since Daguerre and Fox Talbot (1840s) has been remarkable. Photography has reinvented itself every generation over the last 170 years, and it continues to do so. With the increase in use of photography in the last half of the 18th century, photography moved into the painter’s space of scene reproduction, and painters moved elsewhere. However, technology provides a fraction of the story that has controlled the recent history of pictures. The artist-industrialist J C LeBlon invented commercial colour printing in about 1700, and hand-painted colour photographs were common in 1861 when James Clerk Maxwell made the first photographic colour separation images using colour filters. Colour printing preceded its photomechanical implementation by one and one-half centuries. The legacy of making colour photographs includes many ingenious cameras, subtractive dye-transfer systems, and additive colour films. It was not until the 1940s that the subtractive dye-coupler process became the universal colour capture technique. With this highly sophisticated, multilayered film came a universally-used, low-cost, high-volume, product with multi-national corporations as film suppliers, replacing amateur and small business sources.

The digital picture added computation, instant global transmission advantages, and long-term storage problems. Computation allows image processing for high-dynamic-range scenes, as well as aesthetic modification. Image processing algorithms can perform tasks with digital images that artists used to do. In addition, digital manipulation has become a principal tool of artists. Instant transmission over the Internet changed both accessibility and the standards of information content.

In 1990 the universally accepted standard for a photograph was the information captured in a 35 mm negative, or about 9 megabytes. Today, the typical web image is less than 0.1 megabytes in size (JPEG). Daguerreotypes made in the 1850s are as vibrant today as when they were made. However, digital images are easily lost in media that is easy to use, but is inherently unstable because of obsolescent storage devices and unstable media. The majority of cell phone images have only the lifetime of a telephone call.

The artist, the patron, the dedicated amateur hobbyist, the commercial service provider, and recently multi-national corporations are all image makers. Technology has changed who makes and who uses pictures. It is the combination of technology and social use that controls the lifetime of an imaging technology, as well as the lifetime of individual images. This article will describe the details of many advances in imaging technology and how they were replaced by further advances, often dictated by customer preferences.

Painting

The earliest examples of painting are preserved in rock art. Lascaux cave painting have been dated to between 15 000 and 13 000 BC. As best we can tell, these are the artistic expressions of individuals, with possible social implications.

In North America petroglyphs are as old as 3000 BC. Anasazi Indians in Chaco, New Mexico around 1000 AD used a camera-like cave with a small slit so as to project light onto a spiral petroglyph to identify summer solstice, equinox and the cycles of the moon. [1–3].

For centuries the picture makers were artists who painted them for themselves, or for noble clients. Pictures were rare; their users were few. Figure 1 shows a thousand year old Sung dynasty scroll [4]. The children are seen on a plain background in uniform illumination.

In the Italian renaissance, art became more public. Family dynasties such as the Medicis supported public art as today benefactors support public parks. This led to artists’ workshops that resembled 20th century research laboratories. These workshops were well funded and well staffed and generated a considerable volume of images available to a wider range of viewers.

The practice of portraying people and objects without an accurate rendition of the surrounding scene and lighting environment changed in the Renaissance. First, Brunelleschi’s perspective rendered the geometry of the spatial environment. Later chiaroscuro rendered the high-dynamic-range (HDR) illumination environment (Footnote 1).

Pre-Renaissance paintings render people and scenes in uniform illumination. Leonardo da Vinci is credited with the introduction of chiaroscuro, the painting of light and dark. His portraits of Benois Madonna in 1478 and Lady with an Ermin in 1483–90 capture the illumination as well as the figures. One sees that the illumination comes from a particular direction and that there are highlights and shadows. Caravaggio’s paintings, such as The Musicians from 1595–96 portray people and illumination with equal importance. In turn Caravaggio influenced several Dutch painter, among them Gerrit van Honthorst. In the latter’s painting The Childhood of Christ the boy holding the candle has the lightest face; the father, further from the light, is darker, while the other children, progressively further from the light, are slightly darker still (Figure 2).

Artist’s Rendering, Luminance and Appearance

In recent experiments the appearance of four identical transparent targets have been measured with four levels of illumination in the same scene in a black surround (Figure 3) [5]. Four identical pie-shaped targets with ten different transmissions were mounted on a light box. The top (A) had no neutral density behind it, B on the left had 1.0, C on the bottom had 2.0, and D on the right had 3.0 ND filters. The surround was opaque. In total the target had 40 test areas with a luminance dynamic range of 18 619:1. Figure 3 plots the average of observers’ magnitude estimates of the appearance of the 40 test areas versus luminance. The test target is shown in the bottom right. This is the experimental equivalent of van Honthorst’s painting. Observers measured appearance by making magnitude estimates (MagEst) between white and black. They were asked to assign 100 to the whitest area and 1 to the blackest areas in the scene.

Average observer estimates are plotted in Figure 3. The horizontal axis plots luminance measured with a spot photometer (candelas per square metre). The vertical axis plots appearance (magnitude estimate value). The top target A has the highest luminance. It generates MagEsts from 100 to 11. The left target B, viewed through a 1.0 ND filter, has uniformly 10 times less luminance than A. It generates MagEsts from 87 to 10. The bottom target C, viewed through a 2.0 ND filter, has uniformly 100 times less luminance than A. It generates MagEsts from 79 to 6. The right target D, viewed through a 3.0 ND filter, has uniformly 1000 times less luminance than A. It generates MagEsts from 68 to 4.

If we look along the horizontal line at MagEst 50, we see that four different luminances (1.06, 8.4, 64 and 414 cd/m²) generate the same appearance. If we look at luminance 147 cd/m2 we see that it generated both MagEst = 17 (near black) in A, and MagEst = 87 (near white) in B. Similar examples of near white and near black appearances are found at luminance 15 (B and C), and 1.8 cd/m² (B and C).

Magnitude estimates of appearance in complex images do not correlate with luminance. The experimental data in Figure 3 shows great similarity to Gerrit van Honthorst’s figures in the painting Childhood of Christ. Each Scale (A, B, C, D) is the analogue for one of the four figures in the painting. As the distance between the candle and the faces grew, the tones rendering the faces got slightly darker. Each person is rendered slightly darker, but the spatial contrasts for each are very similar. Scales A, B, C and D behave in the same manner. The only difference was that each started and ended a few percent lower in magnitude estimates, despite the substantial decreases in luminance. Figure 3 just assigns numbers to 17th century observations. Chiaroscuro painters did not render luminances; rather they rendered what they saw.

Printing

Printing is photography’s older and bigger brother. Where photography is the technology that gives the individual a record of a scene, printing is the technology of efficiently making many copies of a single image. Printing, as the distribution of information to many, was a problem that was addressed earlier and led to much larger commercial markets.

There are reports of cuneiform stone cylinders for making clay tablets as early as 3000 BC. Chinese had paper, ink and relief pillars in the 2nd century AD. Wood-block xylography has been traced to the 6th century. Paper was found in Asia, the Arab world and Europe in the 12th century. Gutenberg’s press with moveable type printed bibles in 1455. The history of printing is so vast that the intention in this paper is simply to make relevant comparisons where appropriate.

19th Century Photography

Although photography is a smaller industry than printing, it deals with the interface between light in the world and image capture. Two very interesting subtopics are the study of capture/reproduction of scenes’ dynamic range and the capture/reproduction of colour. This paper will discuss these topics sequentially in the 19th and 20th centuries.

The Era of Professional Amateurs

There are so many excellent histories of early photography that it would be foolish to restate the rich and varied history of writing images with light. A particularly good simple summary has been published by Mees [6], based on a course of lectures given at the Royal Institution. It summarises developments starting in 1727, when Schultze experimented with light-sensitive silver salts, through Thomas Wedgewood’s and Sir Humphrey Davy’s work on silver nitrates, and through the public and private studies of Niepce, Daguerre and Fox Talbot, including Sir John Herschel’s major contribution of hypo, the silver-salt clearing agent. More detailed histories, including ancient Greek references to light-sensitive matter and descriptions of camera obscura are available [7–11].

In the late 1830s silver halide photography took a great leap forward from the research on light-sensitive material to the development of practical photographic systems. The key advance was Daguerre’s discovery of silver development by mercury vapour. In 1839 Daguerre made public disclosure of his technique to the Academie des Sciences, Paris, in exchange for 6000 francs per annum for life from the French government.

One of the early adopters of photography was Samuel Morse, inventor of the telegraph, who bought a daguerreotype camera and obtained a full description of the process. He transferred the technology to the United States by teaching the process to American photographers, such as Samuel Broadbent, Albert Southworth, Edward Anthony and Mathew Brady. In 1844 Brady opened a Gallery in New York featuring family portraits and jewellery. During the American Civil War (1861–65) Brady became the portrait maker of the war, depicting famous generals, politicians as well as the common soldier.

Photographic jewellery was an early business that flourished in the mid-19th century [12]. It became very fashionable in England when Queen Victoria began to wear portraits of her husband Prince Albert after his death in 1861.

Today’s negative–positive photography descends from the experiment of 1835 Fox Talbot’s Calotype process [9,10]. In the daguerreotype the silver plate was developed to a positive image with fuming mercury. With the Fox Talbot’s Calotype the silver salts on paper were developed in a water bath to a negative image. This negative, when printed on a second silver chloride (AgCl) paper, made the positive print when developed.

In 1851 Scott Archer described a wet collodion process with which photographers made their silver halide negatives at the time of exposure. Landscape photographers carried their darkrooms with them to process negatives in the field (Figure 4).

By the 1850s the fascination with making pictures led to societies for the discussion of techniques. An excellent example is the 1854 Journal of the Photographic Society of London containing the transactions of the Society and a general record of photographic art and science [13]. It, along with many other documents in the collection of the Royal Photographic Society of Great Britain, provides a time capsule view of life in the 1850s.

Lewis Carroll’s Photography

Like most people, I first learned of Lewis Carroll as the Alice in Wonderland. Under his real name of Charles Lutwidge Dodgson he was also an accomplished lecturer in mathematics at Oxford (1856–81) and a skilled photographer [14]. Dodgson’s diary discusses photography in the years between 1856 and 1880. Figure 5 shows his portraits of Alfred Lord Tennyson and Hallam Tennyson. It is easy to understand that Dodgson had the ability to master the skills required, but what I find fascinating is he could find the time to make so many remarkable portraits using 1850s technology.

H P Robinson

Gernsheim reports that Dodgson admired the photographs of H P Robinson [14]. Robinson’s work is remarkable because it remains of great interest today for two completely different reasons. Firstly, it competed directly with painters in the creation of fine art. It went beyond scene reproduction to creating images designed to evoke an emotional response. Secondly, to be truly competitive, it had to overcome the technical limitations of image rendering. It had to solve all the problems of today’s HDR imaging.

In the mid 19th century HDR scenes presented a severe problem for films available at that time. Multiple exposure techniques for rendering HDR scenes go back to the earliest days of negative–positive photography. Robinson’s (1858) composite print Fading Away (Figure 6) was made using five differently exposed negatives [15]. This dramatic still life was staged using actors.

This multiple negative process was described in detail by Robinson and Abney [16]. The negative capture and positive print is very important to this process. When the photographer developed his negative of the scene exposed for the areas with the most light, the well-exposed areas were darkened with developed silver. The unexposed areas in the negative, corresponding parts of the scene with much less light, were clear glass. That meant that the photographer could take a second, longer exposure to record details in the shadows, develop it and superimpose the two negatives. The shadow details were added through the nearly clear glass of the first negative. This sandwiching of negatives combined with cutting and juxtaposing different negatives of the same scene produced HDR rendition as good as today’s digital techniques.

Robinson’s techniques were empirical. Later in the 1870s and 1880s Hurter and Driffield established the field of photographic sensitometry [17]. They measured the sensitivity function of silver halide films. Mees repeated this work in his thesis at University College London [18].

Nineteenth century photography started with a few scientists experimenting with light changing matter. It ended with the beginnings of the massive industrialisation of photography. Over most of the century photography was developed by thousands of individuals and small groups who painstakingly made each photograph from start to finish. Figure 7 is the frontispiece from Mees’s 1937 book, showing the coating of a gelatin dry plate by hand. It illustrates beautifully one of many steps needed to make a picture; many of these steps were performed in complete darkness. It also illustrates the commitment of time, facilities and equipment beyond film and camera that were required. By the end of the century there were innumerable small businesses, each in unique niches, finding independent solutions to common problems. There is no better example of the vast diversity of 19th century photography than the search for colour photography.

Colour Photography

Thomas Young in his Bakerian Lecture of 1801 at the Royal Society proposed that human vision is limited to three types of colour detectors. Later in 1861 James Clerk Maxwell described the invention of colour photography using red, green and blue filters to make three black and white colour separation photographs. The principle Maxwell described was hugely significant and stimulated a vast number of techniques for making colour photographs. Various publications have described colour techniques arising from this pioneering work in greater detail [19–21].

LeBlon introduced colour printing in Paris about 1700 [22]. His three-colour mezzotint process was quite remarkable. He drew multiple-colour separation mezzotint plates, assigning the amounts of each ink in each area by visualising the desired colour. Figure 8 shows a magnification of a woman’s eye in a print bound into an original copy of Coloritto in the British Art Center, Yale University. One can see the structure of the individual colour inks from the multiple colour separations.

Fine art colour printmaking made remarkable progress in 18th century France as demonstrated in the 2003 exhibition’ Colorful Impressions’ at the National Gallery, Washington [23]). So 160 years before Maxwell worked out the theory of colour photography, commercial printers had found means to manufacture coloured images. Making three or four continuous tone printing plates by hand required great skill. Hand colouring of the mechanically printed lithograph was easier. The high level of people’s interest made such processes into successful businesses. A primitive form of colour photograph was common even in 1861, in which a technician simply painted a low-spatial resolution colour wash on top of a black and white photograph.

One of the most interesting anecdotes of the 1890s is Ives’s experiments with ‘natural colour’ reproduction. Frederick Ives tried to combine Maxwell’s colour camera idea with Maxwell’s measurements of the colour spectral sensitivity curves [24]. He proposed that colour separations, used to make colour reproductions, would have the most ‘natural colour’ if they had the same spectral sensitivities as humans. He published a book on the subject and filed a US patent [25,26]. As described by Friedman [27]:

The Ives disclosure let loose a rather lengthy discussion from which very little seems to have resulted. The poor results that were obtained when using filters for colour separation whose transmissions corresponded to colour-mixture curves, soon forced the subjective idea of colour reproduction into the discard, and the further development of the subject of colour photography appears to have proceeded along objective lines.

Despite a significant investment in the idea of using broad sensitivity functions, Ives abandoned the idea and used narrow-band filters in his 1895 commercial Kromstop colour cameras. This assumption, that films should have the same spectral sensitivity as human cone pigments, persists today among theoreticians, but not among colour imaging manufacturers.

The Century of Corporate Photography

In 1900 Eastman Kodak introduced the Brownie camera. It cost $1 ($0.15 for film). The advertisements for it read ‘You press the button, we do the rest’ [28]. Kodak was the corporate organisation that put simultaneous advertisements in every US town’s newspaper that changed photography. It universalised the photographer. Up until then photography was a specialised occupation, or a serious avocation. The new Brownie photographer clicked the shutter, mailed the camera to Kodak, and received by mail his photos and another camera loaded with fresh negatives. All the dangerous chemicals and bulky dark tents were taken away by a large corporation.

Just after the introduction of the Brownie, George Eastman went on a different quest, a research laboratory. It took several years, but in 1912 he persuaded C E K Mees to move from London to Rochester and become Kodak’s director of research.

Control of Dynamic Range

Silver Halide

Over the years the science of silver-halide imaging improved rapidly. Mees established standards for high-dynamic-range image capture on the negative, and high-slope rendering on prints [29]. Negatives are designed to capture all the information in any scene. The negative response function changes very slowly with change in exposure (low-slope film). This property translates into the fact that negatives capture a wide range of scene luminances. Further, it relaxes the requirements for cameras to make accurate film exposures. Once the scene is captured and the negative is developed, the final print can be made under optimal conditions at the photofinishing facility. The print paper has an S-shaped nonlinear response to light (high slope). The resulting positive print is higher in contrast than the original scene. The loss of scene detail occurs in this high-contrast print rendering.

In 1920 Mees published a print made with multiple negatives with different exposures (Figure 9) [30]. This example of multiple exposures was not so much an artistic technique, as it was a demonstration of an improvement in image quality. Mees, as director of Research at Kodak for half a century, led the development of negative films that can capture a greater range of luminances than possible on camera image planes for the vast majority of scenes [31]. This film design was the result of extensive photographic research [18,29]. This work led to a single tone-scale reproduction function (Footnote 2) for colour prints. It was used in all manufacturers’ colour films for the second half of the 20th century. Innumerable experiments in measuring users’ print preferences led the massive amateur colour print market to use a single tone-scale system response. Even digital camera/printer systems mimicked this function [32]. It is important to note that this tone-scale function is not slope 1.0. It does not accurately reproduce the scene. It compresses the luminances in both whites and blacks, enhances the mid-tones (increased colour saturation) and renders only skin tones accurately. This ‘one tone-scale fits all scene’ approach was very successful and became the basis of all colour prints made in the second half of the 20th century.

Adams’s Zone System

In 1939 Ansel Adams first described the zone system for photographic exposure, development and printing. It described three sets of procedures: first, for measuring scene radiances; second, for controlling negative exposure to capture the entire scene range; and third, spatial control of exposure to render the high-range negative into the low-range print [33].

Adams used a spot photometer to measure the luminances of image segments and assigned them to zones in the scale from white to black in the final photographic print. The zone s ystem imposed the discipline of visualising the final image by assigning image segments to different final print zones prior to exposing the negative. Adams was a professional performing pianist. He often described the negative as the analogue of the musical score and the print as the performance. It was essential that the negative recorded all the information in the scene and that the printing process rendered this information in the print.

Photographic contrast is the rate of change of density versus exposure. In the negative, the low-zone values are controlled by exposure, and the high-zone values are controlled by development and exposure. The zone system provided the necessary information to select appropriate exposure and processing for each scene’s dynamic range.

The final stage was to control the amount of exposure for each local part of the image (dodge and burn) to render all the desired information from a high-dynamic-range scene into a low-dynamic-range print. This process starts with a preliminary test print using uniform exposure. Examination of the print identifies the areas with overexposed whites and underexposed blacks that have lost spatial detail. Dodging refers to holding back exposure from areas that are too dark. Less exposure lightens this local region of the negative-acting print paper and gives better rendition in the blacks. Burning refers to locally increasing the exposure to make an area darker. This is a local spatial manipulation of the image. Not only can these techniques preserve detail in high and low exposures, they can be used to assign a desired tone value to any scene element.

Adams described the dodging and burning process in detail for many of his most famous images. He executed remarkable control in being able to reproducibly manipulate his printing exposures so that the final print was a record of his visualisation of his desired image, not a simple record of the radiances from the scene. In fact, Adams’s zone system process was the 1940s equivalent of an all-chemical Photoshop. It combined the chemical achievements of capturing wide ranges of luminances in the negative with dodging and burning to synthesise Adams’s aesthetic intent. Controlling exposure and development capture all the desired scene information. Spatial manipulations (dodging and burning) fit the captured range to the limited dynamic range of prints.

Dynamic Range of Scenes

A 1940s study of 128 outdoor scenes by Jones and Condit provided two important benchmarks in HDR imaging [34]. It compared photographic images (measured camera luminances) and spot photometer measurements from scenes as well as analysing flare in the image falling on the camera’s image plane. The limit of film response to no light exposure is called the fog level of the film. This is the equivalent of the various noise limits in blacks in CCD and CMOS sensors. Jones and Condit showed that the fog limit was significantly lower than the camera flare limit. Although many papers discuss digital camera noise limits, few discuss flare limits. Flare, not sensor signal-to-noise ratios of noise limits, sets the usable dynamic range of cameras.

Colour Photography

Silver Halide

In the late 19th century colour photography used large beam-splitting cameras to make three simultaneous colour separation images. In the early 20th century these bulky cameras could be replaced by additive colour transparent films. They worked on the same principle as our LCD laptop displays. Very small adjacent patches of R, G and B images viewed at a distance merge into a three-colour image, hence the name ‘additive’ [35]. John Wood’s The Art of the Autochrome [36] and Brian Coe’s Colour Photography [21] display many fine examples of single-sheet additive photography.

The disadvantage of these processes was that they had to be illuminated by a bright light and lacked the convenience of colour prints. Four-colour subtractive printing, using yellow, magenta, cyan and black inks, was universally available and, more importantly, universally viewable because they were prints. Whites in subtractive colour images reflect all incident light. Since additive films remove all the green and blue light to make red, remove all the red and blue light to make green, and remove all the red and green light to make blue, their combination to make white is only one third of the incident light. Subtractive colour transparencies are much brighter than additive ones; subtractive colours are essential for prints.

In 1935 Kodak introduced the Kodachrome process that set the standard for all colour films for the rest of the 20th century. The project was led by Mannes and Godovsky. Kodachrome was made up of multiple layers of R, G and B emulsions to capture three-colour separations in a single layer. The film was processed to form a different colour dye image by reacting with the silver in each colour separation layer using coupler developers. Mees described a coupler developer as ‘one that in which the oxidation product of the developing agent combines with a chemical agent in the solution to form an insoluble dye’ [37].

Figure 10 is a diagram of the Kodachrome process. The lower part of the figure shows a cross-section micrograph of a Kodachrome transparency, in which light passes from the top through the layers to the observers’ eye at the bottom. This cross-section is of black area in the photograph and shows the superimposed yellow, magenta and cyan dye layers.

The process was so complex and the processing conditions so exact that only large high volume chemical processing facilities could achieve satisfactory results. The 19th century style professional amateur continued to enjoy his dark room making black and white imagery, but convenient subtractive colour processing became the job of large chemical facilities.

Instant and Electronic Imaging

Many technologies invented in the first half of the 20th century became very successful businesses in the second half. Baird’s mechanical television in 1925, Farnsworth’s broadcast television in 1927, and Zworykin’s all-electric camera tube (Iconoscope) in 1929 became the basis of today’s television. Carlson first described Xerography in 1937; Land introduced instant photography in 1947. Colour versions followed in shortly afterwards in each case [38].

Human Vision is a Spatial Mechanism

Over the past century psychophysical and physiological experiments have provided overwhelming evidence that human vision is a result of spatial processing of receptor information. Hecht and others showed that threshold detection mechanism use pools of retinal receptors [39]. Kuffler and Barlow showed that the signal travelling down the optic nerve has spatial-opponent signal processing [40,41]. In one example, the centre of the cell’s field of view is excited by light (more spikes per second). The receptors in the surround of the cell’s field of view are inhibited by light (fewer spikes per second). The net result is the cell does not respond to uniform light across its field of view and is highly stimulated by edges. It has the greatest response to a white spot in a black surround. Dowling showed pre- and post-synaptic behaviour of the retina establishing post-receptor spatial interactions in mammals [42].

In 1964 Land proposed his retinex theory, asserting that the three cone types act as sets, where the response was determined by their spatial interactions [43]. The phenomenon of colour constancy is best explained by independent long-, middle- and short-wave spatial interactions. Zeki found colour constant cells in V4 with predicted spatial properties [44]. Hubel and Wiesel have recounted their study the organisation of the primary visual cortex’s response to stimuli projected on a screen in front of the animal [45]. In each small region of the cortex they found a three-dimensional array of different representations of the visual field. Each segment of the visual field has columns of cortical cells that report on the left-eye image next to a column for the right-eye image. The cells perpendicular to the left/right eye columns respond to bars of different orientations. The third dimension has cells with different retinal size segments of the field of view. Campbell and colleagues showed that there are independent spatial-frequency channels corresponding to bar detectors of different visual angle. Gibson described the importance of bottom-up spatial image processing [46].

Control of Dynamic Range

Electronic HDR Rendering

Land and McCann demonstrated the first electronic (analogue) HDR rendering (Figure 11) [47,48]. Here, the intent was to render HDR images using spatial comparisons that mimic human vision. This developed the idea of Wallach that suggested that lightness correlated with spatial ratios [49], and expanded it beyond the restraints of uniform illumination. The idea was that what we see was synthesised from the ratio at an edge multiplied by the ratio at all other edges. This process synthesised an image based on the relationship of all edges in the scene, independent of the luminances of each. The history of the development of this idea was described by Land [50].

Digital Electronic Rendering

The practical embodiment of the principles articulated by Land and McCann needed two technological developments: the digital image processing hardware and an efficient algorithmic concept that reduced the enormous number of pixel- to-pixel comparisons to a practical few, enabling rapid image synthesis. The hardware became commercially available in the early 1970s for the display of digital satellite and medical images. The efficient image processing began with a 1983 patent using I2S image processing hardware with multi-resolution software [51]. This work and its relation to other multi-resolution and pyramid processing has been explained previously [52].

Figure 12 shows an example of a very efficient digital, multi-resolution HDR algorithm, using spatial-comparisons first shown at the Annual Meeting of Society of Photographic Scientists and Engineers in 1984. Here, spot photometer readings show that the illumination in the sunlit foreground is 32 times brighter than in the shade under the tree. That means that the sunlit black square has the same scene luminance as the white card in the shade. Prints cannot reproduce 32:1 in sun, plus 32:1 in shade, (dynamic range 32²) because the entire print range is only 32:1 in ambient light. Using the spatial comparison algorithms, described in detail by Frankle and McCann [51], it is possible to synthesise a new 32:1 image that is a close estimate of what we see.

The retinex theory, starting with analogue electronics and quickly expanding to digital imagery, used a new approach. It assumed the initial stage of Mees’s and Adams’s wide-range information capture for its first stage. Instead of using Adams’s aesthetic rendering, it adopted the goal that image processing should mimic human visual processing. The retinex process writes calculated visual sensations on print film, rather than a record of scene luminances [53]. To this aim, retinex substitutes the original pixel luminance values with the results of a spatial computation that takes into account ratios among areas. In computing these spatial relationships the reset step is essential to mimicking vision. It is a powerful non-linear operator that applies the equivalent of a scene-dependent spatial frequency filter [54].

Stockham’s model of spatial filtering of low-spatial frequency image content intended to combine Land’s black and white Mondrian experiments with Campbell’s multi-channel spatial frequency model of vision [55]. This concept was the basis of a great many image processing experiments and algorithms. It differs from the original retinex algorithm because it lacks the non-linear reset, which locally normalises images to maxima and generates the equivalent of automatic image-dependent spatial filtering [54].

The use of multiple exposures in electronic imaging for HDR scene capture was described by several workers [56–60]. Debevec and Malik had a new and different rendering intent, namely accurately record the scene luminance. This led to proposals for digital image files covering extended dynamic ranges up to 76 log units [61]. It also led to the development of Brightside technology with a modulated DLP projector illuminating an LCD display [62]. This raised the luminance level of display whites. Raising the luminance of white increases the display’s range between white and ambient black. By increasing the range of luminances of the display one can make use of the extended range from HDR capture. There is a simple tautology, namely a display that accurately reproduces all scene radiances must look like the scene.

Recalling Figure 3 is very helpful here. There was no correlation between luminance and appearance in a complex image. Pixel-based global tone-scale functions cannot improve the rendition of both the black and the white areas in targets A, B, C and D with the same luminances. Tone-scale adjustments designed to improve the rendering of one luminance region make other luminance regions worse. Land and McCann made the case that spatial algorithms can automatically perform spatial rendering, doing what Adams did to compress HDR scenes into the limited range of prints [48]. Such spatial rendering is not possible with tone-scale manipulations. By their design, global tone-scale functions have the same effect on all pixels with the same digital input value.

Electronic imaging made it possible and practical to spatially manipulate images. Such spatial processing is not possible with chemistry in silver halide photography (Footnote 3). Quanta catch at a pixel determines the system response, namely density of the image. Digital imaging processing, or its equivalent, had to be developed in order for each pixel to be able to influence each other pixel. With the advent of digital processing imaging was no longer bound to universally responsive pixels. Spatial interactions, by computational means, became technologically possible. Ironically, recent HDR tone-scale processes impose pixel-value-dependent global restrictions on digital systems.

Details in the shadows are necessary to render objects in shade to humans. Clearly, the accuracy of their luminance record is unimportant. The spatial relationships of objects in shadows are preserved in multiple exposures. Spatial-comparison image processing has been shown to generate successful rendering of HDR scenes. Such processes make use of the improved differentiation of the scene information. Therefore one can make the case that improved quantisation is key to successful HDR image processing [31]. By preserving the original scene’s edge information, observers can see details in the shadows that are lost in conventional imaging. Spatial techniques have been used by painters since the Renaissance, while photographers have used multiple exposures and dodging/burning for 160 years.

There have been many different examples of spatial algorithms used to synthesise improved images from captured image plane luminances [63]. Digital spatial algorithms, such as that of Frankle and McCann [51], have been used to display high-range scenes with low-range media. HDR imaging is successful because it preserves local spatial details. This approach has shown considerable success in experimental algorithms [5,64], and in commercial products [65,66]. Figure 13 shows the results of spatial image processing from a single exposure using automatic firmware in an amateur camera. The pictures are hand-held single-exposure images. The image on the right uses retinex-based adaptive lighting/digital flash camera options. The spatial processing removes the over-exposure of the windows while lightening the red rug, white marble altar and pews.

More on Electronic Imaging

In September 1981 Aiko Morita, chairman of Sony Corporation, gave a press conference at the Plaza Hotel in New York City announcing Mavica, the first digital camera [67]. He took a colour digital still picture, displayed it on television and then the picture was transmitted across the room to a second computer. He also made a thermal print. The camera was a machine shop prototype, not a production camera. The transmission took many minutes to cross the room and the colour saturation of the print was very poor compared with that of AgX prints. Nevertheless, this presentation changed imaging. Immediately following that press conference all imaging companies began to spend more than 50% of their research budgets on digital imaging.

That year, IBM announced the first PC. In 1984 Apple introduced the Macintosh. These digital technologies have expanded to the point of crowding out AgX photography. Digital printing replaced optical print by the end of the 20th century. Even high-resolution Ektachrome transparency AgX films are printed with digital printers today. In addition to colour film printers, other technologies introduced were electrographic in 1955, ink-jet in 1976, thermal in 1980 and Vaught and Endo’s thermal ink-jet in 1987 [35,38].

Cellular Digital Imaging

Photography is in the middle of another massive change caused by technology and the lives of its users. Digital cameras have been added to the feature set of most new cellular phones. The techniques of today’s electronics manufacturing make it very difficult to add or subtract features from a mass-produced product. It is far simpler to include all features in a device and make the user ignore the ones not wanted than to make, market and train users to select what they want. The world is well on the way to universal cell phone usage, and that includes a camera available at all times, whether or not it is wanted. That creates an enormous problem for camera manufacturers. If every one has been given a camera by their cell phone makers, how do they sell cameras? The only cost for this camera is the time to learn to use it and the problem of what to do with the images.

The use of cell phone images is interesting. The price is nil, the convenience is very high, and the quality ranges from poor to good enough. Even at these low resolutions and high compression, the storage and transmission costs of images, still and video, are much higher than for audio. The cell phone is not the photo album. It seems unreasonable to spend much more time to put the image somewhere than to make it. And when you put it somewhere, how do you find it?

Archival Images

Daguerreotype images developed in fuming mercury are remarkably stable. With only reasonable care they are as vibrant today as the day they were made. Black and white silver halide prints have a life expectancy of hundreds of years. Colour films are somewhat more likely to fade. Electronic prints vary from unstable thermal prints to images made with high-stability pigments. Nevertheless, the trend is unmistakable. The more recent the technology, the less easy are its images to archive. In the majority of uses this is not a problem. The image stability meets the users’ needs. Ironically, the more convenient the process is, the less valuable the result. It is so easy to take a picture automatically that it is difficult to devote attention to the composition, lighting and perspective of the image. We are far more likely to take the instantaneous picture and see if it is good enough. Great convenience makes it harder to make pictures that are as good as they can be.

Discussion

This paper is a brief and selective survey of picture making technology, with a few passing references to its older sibling: printing. The 200 year history began as the work of a very few who formed small groups of like-minded colleagues. As the number and size of these groups grew they spawned hundreds of cottage industries that prospered in small niche markets. The user of photography began with a few chemists, spread to devoted amateurs and a small number of professionals. Gernsheim reports that London had three professional photographers in 1841, 51 in the whole of Great Britain in 1851, which rose to 534 by 1861. He comments that this list of registered photographers ‘is certainly an understatement, since it does not take account of the legion of petty dabblers, or the thousands of employees engaged in the trade’ [14].

In comparison, today’s almost universal imaging technology is breathtakingly beautiful. The displays are very bright; the prints are clean and sharp. The colours are fully saturated. The process is instantaneous. The availability is global. Who could possibly complain?

Imaging in the 21st century imaging is becoming increasingly democratised and less individualistic. Today’s cell phone has in it CMOS sensors, digital image processing, storage, display and transmission functions, each the results of sizable development teams. It is not possible to build any of these devices in a small laboratory. It is the result of large teams making tiny modular components integrated into a single device. The more universal the device is, the larger the market, the greater the profit, the more successful the endeavour. But, what would H R Robinson think about replacing his AgX film and camera with a cell phone? Colour would be an advantage, but he, as Ansel Adams, might be uncomfortable with the assigned fixed colour rendering of the scene. Multiple exposure techniques might emulate those he developed in the 1850s, but the dynamic range of light on the camera’s image plane is limited by veiling glare [31]. Glare is dependent on camera size and aperture so that the cell phone’s small, fast lenses will limit dynamic range more than his old cameras. Multiple exposures give improved quantisation of details, but do not affect the camera’s veiling glare limit. Robinson could use Photoshop to fuse and manipulate his multiple exposures. That certainly would be an advantage over cutting and fitting glass plates by hand.

Are there concerns about the development of future technology? An interesting example is the perpetual debate between marketing managers and engineers about the number of pixels. The best pixels have large areas, to capture many photons, and improve signal-to-noise ratios and dynamic range. The problem is that resolution and cost both increase with the number of pixels in a sensor. If not limited by cost, we could have large arrays of large-area pixels capable of capturing the entire dynamic range of light falling on the camera image plane. But cost is a very big problem. Then, for the same size sensor, we have to compromise between resolution (number of pixels) and dynamic range (area of each pixel).

Think back to the debates in the 1980s between AgX film/camera engineers and digital camera engineers, both competing for R&D dollars from perplexed managers and marketing strategists. These debates happen in all imaging companies. The simplest take-home message at the time for the AgX proponent was that digital was inferior because it had such poor resolution. It took 15 to 20 years for digital resolution in amateur cameras to be equivalent to AgX. In those years the marketing managers and the public have all ‘learned’ the popular misconception that image quality depends almost exclusively on the number of pixels. Engineers are now confronted with the difficult task of raising the level of the discussion to re-evaluating the trade-offs between resolution and dynamic range. Number of pixels does not determine image quality.

In a cell phone the cost and size of the camera chamber, the sensor, and the lens all work against capturing the same range of light as a 35 mm camera and negative. The range of illumination from night-time camp fires to sunny beaches also makes the problem difficult. If one’s glass is half-empty, the 21st century may replace the convenient 20th century pictures with still more convenient, but lower quality, images. If one’s glass is half-full, the appreciation and need for quality in photography will lead us along a path, so far unanticipated, to many new advances in picture taking. That is what has happened in the past. We will have to wait to see, or better yet, participate in the development of pictures in the 21st century.

Acknowledgements

The author thanks the British Art Center, Yale University for their help with Le Blon’s mezzotints. The author also thanks Carinna Parraman, director of CREATE, for the invitation to present this talk, and Mary McCann for her micrograph and for years of discussions on the material in this paper.

References

Figures and Tables

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Copyright © Society of Dyers and Colourists 2008