IEEE-CS   
Venue


Home > Program > Tutorials > AM-01 & PM-01


ICPR2012 Tutorials AM-01
Computer vision and image analysis in the study of master drawings and paintings (Part I)

ICPR2012 Tutorials PM-01
Computer vision and image analysis in the study of master drawings and paintings (Part II)

Lecturer:
David G. Stork
Rambus Labs, USA

 

Abstract:
These half-day tutorials will apply methods from image processing, computer vision and pattern recognition to problems in the history and understanding of master paintings. Some of these analysis techniques are built upon methods used in forensic image analysis of photographs but are tailored to specific contingencies of painting. Questions addressed include: How do we judge the sizes and positions of objects depicted and the geometry of structures such as architecture? Was the image created using a mechanical or optical aid? What were the sources of illumination and their color? What form of perspective was used? What is revealed by shadows and reflections depicted within a painting? Some of the analysis techniques require nothing more than a tutored and perceptive eye; others merely a ruler and pencil; yet others require advanced statistical estimation procedures and computer analysis. This course is based almost entirely on the analysis of images, not the physical or chemical analysis of pigments and media, the purview of traditional art conservators.

 

Course description

AM-01 [09:30 – 13:00]

Color and point-wise operations
Modelling pigment bleaching to digitally rejuvenate the color schemes in aged paintings; rejuvenating faded tapestries based on (unfaded) reference paintings; image processing for revealing

 

Analysis of geometrical perspective
Single point and multiple point perspective, cross ratios, Desargue’s theorem, detecting perspective inconsistencies, inferring viewing position and relative depth, inherent scale

 

Anamorphic art
Slant, conical and cylindrical anamorphic art; distorted perspective; the mathematics of perspective distortions

 

Metrology from images
Estimating size, location from uncalibrated and from single “calibrated” images (i.e., ones containing objects of known size or angular size and/or position, e.g., sun, moon, reference objects)

 

Analysis of shadows and scattered light
Cast shadows (cast onto another object) and form shadows (on the object itself). Inferring the number, position and color of illuminants. Detecting lighting inconsistencies. Estimating time of day or geographical latitude from shadow analysis.

 

Analysis of depictions of mirrors
Reflections in plane, convex, concave mirrors; reconstructing the tableau using multiple viewpoints provided by mirrors.

 

Analysis of luminance
Computation of brightness in projected images; atmospheric perspective

 

PM-01 [14:00 – 17:30]

Computer graphics reconstructions of tableaus to infer artists’ working methods
Reconstruction of geometry, illumination, compositions


Elementary optical systems and camera model
Elementary image forming systems (concave mirror, converging lens); the lens and mirror equation; focal length, angle of view, depth of field/depth of focus

 

Introduction to historical drawing and copying aids
Camera obscura, camera lucida, pantograph, compasso da reduzione, Claude mirror, Alberti’s screen, Albrecht Dürer’s drawing machines, and their effects in drawings and paintings.

 

Analysis of common visual illusions in paintings
Qualitative and quantitative analysis of the Poggendorff illusion, Ponzo illusion, lightness contrast, …

 

Outstanding problems in the image analysis of master paintings
                                     

 

 

Relevant References:

Books/proceedings

  1. David G. Stork and Jim Coddington (eds.), Computer image analysis in the study of art (SPIE, 2008)
  2. David G. Stork, Jim Coddington and Anna Bentkowska-Kafel, Computer vision and image analysis of art (SPIE, 2010)
  3. David G. Stork, Jim Coddington and Anna Bentkowska-Kafel, Computer vision and image analysis of art II (SPIE, 2011)
  4. Antonio Criminisi, Accurate visual metrology from single and multiple uncalibrated images (Springer, 2001)
  5. David Falk, Dieter Brill and David G. Stork, Seeing the Light: Optics in nature, photography, color, vision and holography (Wiley, 1986)
  6. Ernst Gombrich, Art and Illusion: A study in the psychology of pictorial representation (Princeton U. Press, 1961)
  7. Richard Hartley and Andrew Zisserman, Multiple view geometry in computer vision (Cambridge U. Press, 2004)
  8. Martin Kemp, The Science of Art (Yale U. Press, 1997)

Papers on image analysis of master paintings

  1. Antonio Criminisi, “The virtual Trinity,” Proc. Workshop on Art, Science and Techniques of Drafting in the Renaissance, May, 2001, Florence, Italy
  2. Antonio Criminisi, Martin Kemp and Andrew Zisserman, “Bringing Pictorial Space to Life: Computer Techniques for the Analysis of Paintings,” CHArt Annual Conference 2002: Digital Art History? Exploring Practice in a Network Society, November, 2002
  3. Antonio Criminisi and David G. Stork, "Did the great masters use optical projections while painting? Perspective comparison of paintings and photographs of Renaissance chandeliers," in J. Kittler, M. Petrou and M. S. Nixon (eds.), Proceedings of the 17th International Conference on Pattern Recognition, Volume IV, pp. 645-648, 2004
  4. Antonio Criminisi and David G. Stork, "Did the great masters trace optical projections? Machine vision techniques address questions in art history," Electronic Imaging, 2005, submitted for publication
  5. Antonio Criminisi, Martin Kemp and Sing-Bing Kang, “Reflections of reality in Jan van Eyck and Robert Campin,” Proc. Measuring Art: A Scientific Revolution in Art History, May-June 2003, Paris David Hockney and Charles M. Falco, “Optical perspectives on Renaissance art,” Optics and Photonics News, 11:52, 2000
  6. Thomas Ketelsen, Olaf Simon, Ina Reiche and Silke Merchel, “Als Ixh Xan: Zum zeichnerischen Kalkül Jan van Eycks: New insights by a co-operation between natural science and the history of art,” The Burlington Magazine, 2005
  7. David G. Stork, "Color and illumination in the Hockney theory: A critical evaluation," Proceedings of the Color Imaging Conference (CIC11), Scottsdale AZ, November 2003, pp. 11-15
  8. David G. Stork, "Were optical projections used in early Renaissance painting? A geometric vision analysis of Jan van Eyck's 'Arnolfini portrait' and Robert Campin's 'Mérode Altarpiece'," SPIE Electronic Imaging, Vision Geometry XII, L. J. Latecki, D. M. Mount and A. Y. Wu (eds), pp. 23-30, 2004
  9. David G. Stork, "Did Jan van Eyck build the first 'photocopier' in 1432?" SPIE Electronic Imaging Color Imaging IX: Processing, Hardcopy, and Applications, R. Eschbach and G. G. Marcu (eds.) pp. 50-56, 2004
  10. David G. Stork, "Optics and the old masters revisited," Optics and Photonics News, 15(3):30-37, March 2004
  11. David G. Stork, "Did Hans Memling employ optical projections when painting Flower still-life?"
  12. Leonardo, January 2005
  13. David G. Stork, "Did Georges de la Tour use optical projections while painting Christ in the carpenter's studio?" SPIE Electronic Imaging 2005
  14. David G. Stork, "Did early Renaissance painters trace optical projections? Evidence pro and con," SPIE Electronic Imaging 2005
  15. David G. Stork, "Asymmetry in 'Lotto carpets' and implications for Hockney's optical projection theory," SPIE Electronic Imaging 2005
  16. David G. Stork, "Optics and realism in Renaissance art," Scientific American, December 2004
  17. David G. Stork, "Computer vision, image analysis and master art, Part I: Perspective and homographies," IEEE MultiMedia (2006)
  18. David G. Stork and M. Kimo Johnson, "Computer vision, image analysis and master art, Part II: Finding the illuminant in realist paintings," IEEE MultiMedia (2006)
  19. Christopher W. Tyler, “Rosetta Stoned? Hockney, Falco and the sources of ‘opticality’ in Renaissance art,” Leonardo 2005
  20. Christopher W. Tyler and David G. Stork, "Did Lorenzo Lotto use optical projections when painting Husband and wife?" Optical Society of America Annual Meeting 2004 (abstract)

 

About Lecturer:

Dr. David G. Stork is Distinguished Research Scientist and Research Director at Rambus Labs and a Fellow of SPIE and of the International Association for Pattern Recognition (IAPR) "…for the application of computer vision to the study of art." In nearly four dozen scholarly publications, he and his colleagues have pioneered the application of computer vision and pattern recognition algorithms to the study of fine art. He co-chaired the world’s first conference symposia on the subject and co-edited its first three proceedings volumes, Computer image analysis in the study of art (SPIE 2008), Computer vision and image analysis of art (SPIE 2010), and Computer vision and image analysis of art II (SPIE, 2011); he is completing Pixels and paintings: Computer vision in the study of art (Wiley). He has delivered the world’s first short courses on the subject (at SPIE, ICIP, CVPR, and ICIAP) and has delivered over 250 talks on the subject in 19 countries, including major museums such as the Louvre, National Gallery London, Metropolitan Museum of Art, Museum of Modern Art, Art Institute of Chicago, Venice Biennale. He offered the world’s first university courses on this subject at Stanford University in both the Department of Art and Art History as well as in Computer Science. His other books include Seeing the Light: Optics in nature, photography, color, vision and holography (Wiley) the leading textbook on optics in the arts (now in its 21st printing) and Pattern Classification (2nd ed.), the world's all-time best-selling textbook in the field, widely used in computer vision courses. A graduate in physics of the Massachusetts Institute of Technology and the University of Maryland at College Park, he also studied art history at Wellesley College and was Artist-in-Residence through the New York State Council of the Arts. His anamorphic photographs and graphics (based on late Renaissance methods) have appeared in small art journals as well as Optics and Photonics News and Scientific American magazine. He has taught courses such as "Light, color and visual phenomena," "The physics of aesthetics and perception," and "Optics, perspective and Renaissance painting" over the last quarter century variously at leading liberal arts and research universities such as Wellesley College, Swarthmore College, Clark University and Stanford University. He has published over a hundred technical papers on human and machine learning and perception of patterns, physiological optics, image understanding, concurrency theory, theoretical mechanics, and five books, including He sits on the editorial boards of four international journals and has delivered nearly 46 plenary lectures at major international conferences. He created the PBS television documentary "2001: HAL's Legacy," based on his book HAL's Legacy: 2001's computer as dream and reality (MIT).

 

 

> Back to List of Tutorials

 

Archive: Call for Tutorial Proposals

 

 

(C) 2010-2012 ICPR2012 Organizing Committee. All rights reserved. http://www.icpr2012.org