WHY DOES THE WORLD APPEAR THE WAY IT DOES?
WHY DOES THE WORLD APPEAR THE WAY IT DOES? INTRODUCTION • Perceiving provides a sense of what’s real and what exists • Perceiving has two functional roles (thought to localize to two separate parts/streams within the brain): 1. Control of behavior/action § E.g. optical flow (overrides vestibular system in balance & coordination) 2. Recognition & awareness of the world (focus of this course) • Realist view: our experience captures the way the world is o Problem: our brain has to guess what’s in the world based on the light that reaches the eye – but these images are in 2D and there’s a lot of different factors that could have generated this image • Many different factors create the structure in light: 1. Reflectance & transmittance properties (e.g. color, lightness, transparency, translucency, sub-‐ surface scattering, gloss) 2. 3-‐D shape (change in surface pose relative to light source, or ‘shading’) 3. Illumination (intensity, spectral content, shadow) 4. Occlusion • We can usually distinguish different sources of image structure even when they’re locally identical o The reflectance image tells us what the intrinsic pigments are (light grey & dark grey) o The illumnance image tells us which sides are being illuminated by the light source (change in surface pose relative to the light source à shading) o In the picture, the shading edge is the exact same as the reflection edge, however our brain is able to tell that these edges mean different things à has to somehow take context into account
REFLECTANCE • When we see the surface of objects, all of the structure and light (except for translucency and the transparency) is created by the way the light strikes the object and is reflected back into our eyes o Objects omit light and is referred to as a secondary light source • Surface normal: Generalization of a perpendicular, or 90 degree angle • Two types/models sufficient to capture most things in the world o Diffuse, or ‘matte’, or ‘Lambertian reflectance’: when we talk about the color or lightness (subset of color) of an object, we are referring to our perception of what we think it’s diffuse reflectance is § Light hits the surface and is scattered uniformly in all directions o Specular: angle of incidence = angle of reflectance relative to the surface normal § Mirror is a perfectly specular surface • Objects can have a combination of diffuse and specular, in which some light is scattered uniformly and some light is reflected according to the angle of incidence o The specular model can also have a blur component = how spread out or scattered the specular reflectance is, or ‘micro roughness’ • Example one: What would happen in the diagram if the teapot changed color from white to black? o Answer: the arrows (representing light reflectance) would be smaller/thinner • Example two: What would happen in the diagram if the light source becomes weaker? o Answer: the same thing -‐ the arrows (representing light reflectance) would be smaller/thinner o i.e. we can create the exact same effect by changing the color of and object vs. turning the lights down
COLOR • Color: not just the amount/proportion of light that is reflected (lightness), but its spectral content (proportion of each wave length) • Lightness (albedo): the proportion of light a surface reflects (it’s about the object – we talk about things having ‘lightness’, not ‘brightness’) • Brightness: amount of total light (luminance) projected to the eye from something (surface or light source) o A world of more than one luminance gives rise to luminance values.
• Contrast is a measure of luminance differences divided by some measure of overall light level o Example: turning the ceiling lights on when using a projector – the projector is Ifstill oing pot the or same thisdtea statue was view thing, yet it is harder to see the screen. The difference between the light and dother ark parts o n t he position, the brightness o screen are still the same, except it is harder to distinguish these differences when the lights are on. would not change This is because the brain doesn’t just look at differences – it looks at differences divided by the overall light in the context. o This is known as ‘divisive normalization’, i.e. standardization
LAMBERTIAN REFLECTANCE (“MATTE”)
Lambertian reflectanc
• Lambert’s law: The brightness of a surface patch is independent of viewing direction; it only depends on the position(s) of the light source(s) – the brightness will look the (“matte”) same independent of where you view it from o Example: If this tea pot was viewed from some other position, the brightness of visible patches would not change o But why doesn’t the brightness change? The arrows are all different sizes, so if they are viewed from different directions, it suggests that the brightness should change o The total amount of reflected light (lumination) depends on both the amount of light striking the surface (illumination) and the proportion of light the surface reflects (its lightness) o Brightness level (illumination) changes at the exact same rate as the size of that surface patch area But why doesn’t the brightness change? you are viewing – so the relative amount of light (lumination) per area stays the same
are all different sizes, so if they are viewe different directions, it suggest that the b LUMINANCE should change • If we hold surface orientation fixed, then the following simple equation holds: o Luminance (what reached the eye) = reflectance (or lightness) x illumination o Fundamental ambiguity • What about luminance ratios? Imagine having a set of papers and you vary the illumination. As you change the amount of illumination, all of the papers are multiplied by a new number. Thus the ratios of two different luminances coming from each paper will remain constant, i.e. the illumination term cancels o But… for example, there’s an infinite number of ways to get a 2:1 ratio – at most, this could only explain how it is that you could make judgments like “surface x is twice as light as surface y”, not “surface x is light grey and surface y is very dark grey” • Gilchrist’s Dome Experiments – looked at the simplest percept of a surface (i.e. the dome covering participants’ eyes) to see how this would be perceived o FOUND: edge ratios can only give you relative lightness measures • Staircase Gelb – putting down progressively lighter squares o FOUND: the highest luminance at any time appears to be white
PERCEPTION OF AMBIGUOUS RATIOS • Inference: An anchoring principle is used to disambiguate luminance ratios o Transforms the relative values of edge ratios into an absolute scale of perceived lightness values (i.e., perceived shades of black, grey, and white) o Claim: Highest luminance assumed to be a white surface (~90% reflectance) • However, most scenes don’t have every shade, i.e. they are a subset of the full scale of luminance values o The anchoring problem: where to locate the range of luminance values on the scale of perceived gray shades? (a similar problem exists for color – we can select a “target white point” to adjust the overall color tint on a computer display, thereby changing what we call ‘white’) • ‘Anchoring’ shown to work with Mondrians in a simulated illumination experiment, such that the square with the highest luminance at any time was perceived to have a much higher reflectance than its true value • This “highest luminance” rule failed in a 3D world composed of a single reflectance – in a room, you have some sense of illumination or “light field” o And it makes sense given the reflectance distributions of natural scenes à things are hardly ever white, so it would be strange to just always call the lightest thing white o Note the importance of contrast in defining lightness à 3D world gives us a lot more information
• A black room with black objects only looked mid-‐grey in the room of one reflectance/color o This is due to shadow strength as a cue to lightness/reflectance o Variations in shadow strength is minimal in a white room with white objects à 90% of the light is reflected and bounces around the room, filling any shadows that would usually be there o On the other hand, a black room with black objects have higher variations à 95% of the light it absorbed and only 5% of light is reflected, leaving shadows in place o The anchoring rule would suggest that every shade is perceived as white in the experiment, but this is not true (black was perceived as mid-‐grey) • Checker shadow illusion: If the ratio remains the same but becomes globally deeper, this is a cue that the darkness is due to a shadow • Contrast: the difference divided by the average, i.e. the differences are scaled by the standard
Sub-surface scattering
SUB-‐SURFACE SCATTERING • Changes in color can also tell us something about the thickness of the object • Subsurface scattering of skin Transparency preserves image structure, translucency o Layered objects that have different properties on each layer blurs the structure of underlying surfaces o Notice the decreased variation in shadows (increased softening) due to sub-‐surface scattering • Transparency preserves image structure, translucency blurs the structure of underlying surfaces (i.e. light scattered) • Two types of constraints on transparency 1. There must be geometric continuity of the contours 2. Photometric (relative intensities): the contrast polarity (sign) of an underlying contour must be preserved • Metelli’s generative model of transparency (pre-‐computer graphics) – used a wooden disc with a gap in which light was let through o Alpha = angle of the open sector (gap) à the higher aplpha is, the more transparent the disc will appear when spun very fast, i.e. the more light let through o Geometric constraints are not captured in these equations, they are purely photometric constraints o Predicts that alpha should be independent of the color of the disc, however, this is not true à if the disc is a lighter color (e.g. white), more light is reflected § i.e., p is a weighted sum of the light that gets through from the background and the light that (1-α) gets reflected from the disc t • Ordinal constraints on transparency: polarity and magnitude
Metelli’s generative model of transparency
nal constraints on transparency:
(1) If polarity is preserved, it is consistent with the
The magnitude of contrast change provid the opacity of a transparent surface (its “
transparency or of there being a transparent surface – occurring possibility illumination in a region where the contrast goes down
o Note Non-‐reversing junctions, as opposed to snot ingle that geometric constraints are captured in t reversing j unctions, a re a mbiguous a s t o w hich equations, they are purely photometric constraints layer is on top and which is on bottom o Note the non-‐reversing junctions of shadows
transparency (2) The m agnitude of contrast change provides information about the opacity of a transparent surface (its “hiding p ower”; how opaque it is indicated b y lower contrast) neither
REFLECTANCE • When we see the surface of objects, all of the structure and light (except for translucency and the transparency) is created by the way the light strikes the object and is reflected back into our eyes o Objects omit light and is referred to as a secondary light source • Surface normal: Generalization of a perpendicular, or 90 degree angle • Two types/models sufficient to capture most things in the world o Diffuse, or ‘matte’, or ‘Lambertian reflectance’: when we talk about the color or lightness (subset of color) of an object, we are referring to our perception of what we think it’s diffuse reflectance is § Light hits the surface and is scattered uniformly in all directions o Specular: angle of incidence = angle of reflectance relative to the surface normal § Mirror is a perfectly specular surface • Objects can have a combination of diffuse and specular, in which some light is scattered uniformly and some light is reflected according to the angle of incidence o The specular model can also have a blur component = how spread out or scattered the specular reflectance is, or ‘micro roughness’ • Example one: What would happen in the diagram if the teapot changed color from white to black? o Answer: the arrows (representing light reflectance) would be smaller/thinner • Example two: What would happen in the diagram if the light source becomes weaker? o Answer: the same thing -‐ the arrows (representing light reflectance) would be smaller/thinner o i.e. we can create the exact same effect by changing the color of and object vs. turning the lights down
COLOR • Color: not just the amount/proportion of light that is reflected (lightness), but its spectral content (proportion of each wave length) • Lightness (albedo): the proportion of light a surface reflects (it’s about the object – we talk about things having ‘lightness’, not ‘brightness’) • Brightness: amount of total light (luminance) projected to the eye from something (surface or light source) o A world of more than one luminance gives rise to luminance values.
• Contrast is a measure of luminance differences divided by some measure of overall light level o Example: turning the ceiling lights on when using a projector – the projector is Ifstill oing pot the or same thisdtea statue was view thing, yet it is harder to see the screen. The difference between the light and dother ark parts o n t he position, the brightness o screen are still the same, except it is harder to distinguish these differences when the lights are on. would not change This is because the brain doesn’t just look at differences – it looks at differences divided by the overall light in the context. o This is known as ‘divisive normalization’, i.e. standardization
LAMBERTIAN REFLECTANCE (“MATTE”)
Lambertian reflectanc
• Lambert’s law: The brightness of a surface patch is independent of viewing direction; it only depends on the position(s) of the light source(s) – the brightness will look the (“matte”) same independent of where you view it from o Example: If this tea pot was viewed from some other position, the brightness of visible patches would not change o But why doesn’t the brightness change? The arrows are all different sizes, so if they are viewed from different directions, it suggests that the brightness should change o The total amount of reflected light (lumination) depends on both the amount of light striking the surface (illumination) and the proportion of light the surface reflects (its lightness) o Brightness level (illumination) changes at the exact same rate as the size of that surface patch area But why doesn’t the brightness change? you are viewing – so the relative amount of light (lumination) per area stays the same
are all different sizes, so if they are viewe different directions, it suggest that the b LUMINANCE should change • If we hold surface orientation fixed, then the following simple equation holds: o Luminance (what reached the eye) = reflectance (or lightness) x illumination o Fundamental ambiguity • What about luminance ratios? Imagine having a set of papers and you vary the illumination. As you change the amount of illumination, all of the papers are multiplied by a new number. Thus the ratios of two different luminances coming from each paper will remain constant, i.e. the illumination term cancels o But… for example, there’s an infinite number of ways to get a 2:1 ratio – at most, this could only explain how it is that you could make judgments like “surface x is twice as light as surface y”, not “surface x is light grey and surface y is very dark grey” • Gilchrist’s Dome Experiments – looked at the simplest percept of a surface (i.e. the dome covering participants’ eyes) to see how this would be perceived o FOUND: edge ratios can only give you relative lightness measures • Staircase Gelb – putting down progressively lighter squares o FOUND: the highest luminance at any time appears to be white
PERCEPTION OF AMBIGUOUS RATIOS • Inference: An anchoring principle is used to disambiguate luminance ratios o Transforms the relative values of edge ratios into an absolute scale of perceived lightness values (i.e., perceived shades of black, grey, and white) o Claim: Highest luminance assumed to be a white surface (~90% reflectance) • However, most scenes don’t have every shade, i.e. they are a subset of the full scale of luminance values o The anchoring problem: where to locate the range of luminance values on the scale of perceived gray shades? (a similar problem exists for color – we can select a “target white point” to adjust the overall color tint on a computer display, thereby changing what we call ‘white’) • ‘Anchoring’ shown to work with Mondrians in a simulated illumination experiment, such that the square with the highest luminance at any time was perceived to have a much higher reflectance than its true value • This “highest luminance” rule failed in a 3D world composed of a single reflectance – in a room, you have some sense of illumination or “light field” o And it makes sense given the reflectance distributions of natural scenes à things are hardly ever white, so it would be strange to just always call the lightest thing white o Note the importance of contrast in defining lightness à 3D world gives us a lot more information
• A black room with black objects only looked mid-‐grey in the room of one reflectance/color o This is due to shadow strength as a cue to lightness/reflectance o Variations in shadow strength is minimal in a white room with white objects à 90% of the light is reflected and bounces around the room, filling any shadows that would usually be there o On the other hand, a black room with black objects have higher variations à 95% of the light it absorbed and only 5% of light is reflected, leaving shadows in place o The anchoring rule would suggest that every shade is perceived as white in the experiment, but this is not true (black was perceived as mid-‐grey) • Checker shadow illusion: If the ratio remains the same but becomes globally deeper, this is a cue that the darkness is due to a shadow • Contrast: the difference divided by the average, i.e. the differences are scaled by the standard
Sub-surface scattering
SUB-‐SURFACE SCATTERING • Changes in color can also tell us something about the thickness of the object • Subsurface scattering of skin Transparency preserves image structure, translucency o Layered objects that have different properties on each layer blurs the structure of underlying surfaces o Notice the decreased variation in shadows (increased softening) due to sub-‐surface scattering • Transparency preserves image structure, translucency blurs the structure of underlying surfaces (i.e. light scattered) • Two types of constraints on transparency 1. There must be geometric continuity of the contours 2. Photometric (relative intensities): the contrast polarity (sign) of an underlying contour must be preserved • Metelli’s generative model of transparency (pre-‐computer graphics) – used a wooden disc with a gap in which light was let through o Alpha = angle of the open sector (gap) à the higher aplpha is, the more transparent the disc will appear when spun very fast, i.e. the more light let through o Geometric constraints are not captured in these equations, they are purely photometric constraints o Predicts that alpha should be independent of the color of the disc, however, this is not true à if the disc is a lighter color (e.g. white), more light is reflected § i.e., p is a weighted sum of the light that gets through from the background and the light that (1-α) gets reflected from the disc t • Ordinal constraints on transparency: polarity and magnitude
Metelli’s generative model of transparency
nal constraints on transparency:
(1) If polarity is preserved, it is consistent with the
The magnitude of contrast change provid the opacity of a transparent surface (its “
transparency or of there being a transparent surface – occurring possibility illumination in a region where the contrast goes down
o Note Non-‐reversing junctions, as opposed to snot ingle that geometric constraints are captured in t reversing j unctions, a re a mbiguous a s t o w hich equations, they are purely photometric constraints layer is on top and which is on bottom o Note the non-‐reversing junctions of shadows
transparency (2) The m agnitude of contrast change provides information about the opacity of a transparent surface (its “hiding p ower”; how opaque it is indicated b y lower contrast) neither
