Pergamon
PII:
VisionRes., Vol. 36, No. 23, pp. 3815-3819, 1996 Copyright@)1996 Elsevier Science Ltd. All rights reserved Printed in Great Britain S0042-6989(96)00099-5 0042-6989196$15.00+ 0.00
Short Communication Depth Asymmetry in da Vinci Stereopsis JUKKA HAKKINEN,*~ GOTE NYMAN* Received 3 February 1995; in revisedform 20 September 1995; in jinal form 25 March 1996 We investigated processes that determine the depth localization of monocular points which have no unambiguous depth. It is known that horizontally adjacent binocular objects are used in depth localization and for a distance of 2540 min arc monocular points localize to the leading edge of a depth constraint zone, which is an area defined by the visibility lines between which the points in the real world must be. We demonstrate that this rule is not valid in complex depth scenes. Adding other disparate objects to the scene changes the localization of the monocular point in a way that cannot be explained by the da Vinci explanation of monocuiar-binocular integration. The effect of additional disparate objects is asymmetric in depth: a crossed object does not affect the da Vinci effect but an uncrossed object biases the depth localization of monocular objects to uncrossed direction. We conclude that a horizontally adjacent binocular plane does not completely determine the depth localization of a monocular point and that depth spreading from other binocular elements biases the localization process. Copyright @ 1996 Elsevier Science Ltd
interaction Depthperception QVinci stereopsis Binocular-monocular
INTRODUCTION
visible to one eye and without adjacent areas from which Because near objects occlude distant objects to different to inherit depth information can theoretically lie anyextents, some areas visible to one eye have no matching where in the correspondingvisual direction. The visual areas in the other eye. However, our visual system system can utilize the probability of an ecologically combines the binocular and monocular areas to an typical occlusion relation between monocular and accurate three-dimensional percept. The process has binocular areas to reduce the number of theoretically been named da Vinci stereopsis(Nakayama & Shimojo, possible depth locations. This can be expressed in the 1990) and has been shown to affect the temporal aspects form of a constraint, the depth constraint zone, which of stereo fusion (Gillam & Borsting, 1988), binocular suggeststhat in the vicinityof a binocularplane there can rivalry (Shimojo & Nakayama, 1990) and three-dimen- be only a limited number of ecologically valid depth sional localization of monocular points (Nakayama & locations “for a monocular dot”. Thus a monocular Shimojo, 1990). Perceived depth of a monocular object becomes localized within a particular section of occluded area is often determined by a depth spreading the visual axis which is invisible to the opposite eye. from an adjacent unambiguous area (Anderson & Figure 1 demonstratesthat the monocularpoint m which Nakayama, 1994; Gillam & Borsting, 1988; Collet, is visibleto the right eye is most probablylocalized in the 1985). Sometimes there is no such depth cue and other leading edge of the depth constraintzone. The results of Nakayama and Shimojo (1990) indicate heuristics based on typical occlusion relations in the that the localization of a valid monocular point consists visualworld must be used (Nakayama& Shimojo,1990). of two parts. Firstly, in the immediatehorizontalvicinity In this article we investigate what kind of processes of the binocularplane a monocularpoint localizes to the determinethe depth positioningof a monocularobject in leading edge of the depth constraint zone. The lateral the latter, more ambiguous case. width of this area is 25-40 min arc. Secondly,beyond the Da Vinci stereopsiscan be considered as an intelligent initial occlusion area the perceived depth of the method that our brain uses to solve the basic localization monocular point gradually returns back to the zero ambiguity of monocular points. A monocular point disparity. The monocular points which are on the ecologically invalid side of the occluding rectangle, i.e. *Department ofPsychology, University ofHelsinki, GeneralPsychol- on the side which does not support an occlusion ogyDivision, P.O.B.13 00014, University of Helsinki, Helsinki, interpretation, are always seen as equidistant to the Finland. ~To whom all correspondence should be addressed IEmailjhakki occludingplane. It seems that the equidistancetendency
[email protected]]. suggested by Gogel (1956) determines the depth 3815
SHORT COMMUNICATION
3816 LE only
RE
-... ..
:. : .. :,.
: :
00 LE
only
RE
FIGURE1. Da Vinci stereopsis.If an opaquebinocularplane is located in front of a perceiver, a small monocular area remains in the immediate horizontal vicinity of the binocular plane. The size of the monoculararea can be calculated from the visibility lines from the left and right eye. According to the results of Nakayama and Shimojo (1990)the depth of the monocularpoint m is determinedby the leading edge of the depth constraint zone.
localization in all ambiguous areas except for the one which allows an occlusion interpretation. Although the experiments of ‘Nakayamaand Shimojo (1990) convincingly demonstrate the da Vinci phenomenon,we think that there exist depth configurationswhere their theory is insufficient.It is known that many threedimensional configurations are depth asymmetric, i.e. there are qualitative changes in the appearance of the scene when disparity configuration is changed from crossed to uncrossed disparity. For example neon color spreading (Nakayama & Shimojo, 1992), depth capture (Ramachandran & Cavanagh, 1985) and depth spread (Collet, 1985; Takeichi et al., 1992) display depth asymmetry. These qualitative differences are probably related to experimental results that suggest different neural systems for crossed and uncrossed depths (Richards, 1970, 1971; Mustillo, 1985). Consequently, if disparity spread from the elements of the scene affects the da Vinci stereopsis, it should be possible to change the depth localization of the monocular points without changing the occlusion configuration,simply by adding disparate elements to the scene. The effects of additional objects can be predicted by noticing that besides occlusion constraints,there are also other rules that determine the depth localization of monocular objects. It is known, for example, that monocular points drop to the level of the background of the scene (Julesz, 1964; Collet, 1985). In those experiments background can be defined as the most uncrossed object of the scene. Because it has been shown that uncrossed disparity information spreads more than crossed disparity information (Takeichi et al., 1992) we hypothesizedthat any uncrossedobjectwould capturethe monocular point that is localized to the leading edge of the depth constraint zone.
The depth localization of objects is constrained by a disparity gradient limit of stereopsis. Burt and Julesz (1980a,b) investigated the concept of Panum’s fusional area and found that the disparity gradient, not the magnitude of disparity itself, is the major determinant of fusion and diplopia.They demonstratedthat a disparity gradient (disparitydifference/angularseparation)of >1.0 leads to diplopia. When this gradient is less than the critical gradient value, the stereogram can be fused. In a sense a binocular element creates a warped forbidden area where other objects cannot be fused. We think that this constraint affects the da Vinci stereopsis too. Because the visual system presumably tries to avoid diplopia, the monocular points which drop towards the background stop when they reach the critical gradient area. The critical value 1.0 of disparity gradient and a typical depth constraint zone (viewing distance 75– 100 cm) do not differ very significantly,so the change from one process to another may not be visible in some depth configurations [see for example Nakayama & Shimojo (1990) where depth constraint zone and disparity gradient 1.0 are equal]. To summarize, we, suggest that uncrossed objects affect monocular points that are adjacent to a binocular plane. Thus, the depth localization curve of monocular points shouldbe biased to uncrosseddisparitywhen there are additional uncrossed objects in the scene. Furthermore, the capture is constrainedbecause of the disparity gradient limit of stereopsis, which prevents the monocular points from falling too steeply to uncrossed disparity. Because of this, the uncrossed bias of monocular point localization should be smooth, i.e. the localizationcurve shouldbe anchored to the horizontally adjacent binocular plane. To test this hypothesis, we conducted an experiment in which an additional binocular element was added to the stimulus and the disparityof the occludingplane was changed.According to our hypothesis,the depth localizationof the monocular points should change when relative depth between the planes change. METHODS
The stimuli were presented on a 20” Eizo Flexscan 9500-screen. The stereoscopic effect was created by dividingthe screen into two areas which were seen by the left and right eye separately. The two views were separated by a cardboard perpendicular to the screen and a prism stereoscopewas attached to the other side of the cardboard.The stinpuliwere located on the screen in such a way that the vergence was consistent with the viewing distance. Stimuli The experimental stimuli consisted of a fixation and target stimulus. The fixation stimulus was a binocular cross whose parts were a 26.1.x 1.7 min arc horizontal and a 1.3 x 10.2 min arc vertical line. Two 1.3 x 17 min arc dichoptic nonius lines were placed in the upper and lower side of the cross. The presentation time of the
SHORT COMMUNICATION
m
A
,
(!
m
(B) m
❑ A
.
El
UA
c FIGURE 2. Experimental stimuli. The stereograms are designed for divergent fusing. (A) The occluding plane A is nearer the additional plane B. The task of the subjects was to determine the depth of a small monocular dot that was in some lateral position relative to the occludingplane by movingthe depthprobe C, (B) The occludingplane A is farther than the additional plane B. The actual experiment included five different relative depths: 39.3 min arc crossed, 13.1min arc crossed, 13.1min arc uncrossed, 39.3 min arc uncrossed and 0 min arc, i.e. a configurationwhere planes A and B were equidistant.
fixation stimulus was unlimited and the minimum viewing time was 2 sec. The end of the fixation was signalled to the experimental subject by enhancing the brightness of the binocular cross slightly.The task of the subjectwas to fixateat the binocularcross and at the same time to monitor the lateral movement of the nonius lines. When the lines appeared aligned the subject pressed a button to view the test stimulus. The target stimulus (Fig. 2) consisted of four objects: an occludingplane (A) (52.2x 34 min arc), an additional plane (B) (52.2 x 78.2 min arc), a movable depth probe (C) (3.9x 17 min arc) and a monocular dot (1.3 x 1.7 min arc). The viewing distance was 100 cm and mean luminance of the display was 14.27 cd/m2. The measurement of perceived depth was accomplishedby using a binocular probe (Foley & Richards, 1972; Harris & Gregory, 1973). The task of the subject was to position the probe to the apparent depth of a randomly chosen monocular dot which was located in one of ten different positions ca 1.3–58.5min arc laterally from the occludingrectangle. Dots could appear in either eye and on the left or right side of the rectangle. Lateral position of the depth probe changed randomly during the experiment so that monocular vernier cues would not bias the result. At the beginning of each stimulusthe probe was at the same depth as the additional plane and was vertically located under the occluding plane in order to minimize the interactions between the
3817
depth probe and the monocular dot. These interactions can be a source of problems because the apparent depth of the monocular dot may appear to chrmgc with the movementof the binocularprobe (Nakayama& Shimojo, 1990). To test the effects of disparity, the depth of the occluding rectangle was changed randomly in each stimulus.The occludingrectangle could be at the fixation plane or in depth nearer or further from the additional plane (13.1 or 39.2 min arc). Because the retinal size of the occluding rectangle remained constant its apparent size changed slightly. This ensured that the depth constraint zone remained actually constant throughout the experiment. Each sessionconsistedof 240 depth measurements(six configurationsx two eyes x two sides x ten lateral positions). The experiment was repeated four times which resulted in a total of 960 depth measurements. The precision of the method may suffer from the unlimitedviewing time and the simultaneousvisibilityof the depth probe and monoculardot. This could affect the result because the depth scene changed dynamically all the time. The visual system may seek some intermediate disparity through vergence movements in order to avoid diplopia. However, it is known that in briefly flashed displays the interpolationof ambiguous dots is stronger (Mitchison & McKee, 1987) and thus a depth discontinuity interpretation might be more difficult to achieve. The experimental method with free eye movements as used here is not a less precise method as a studyby Foley and Richards (1972) demonstrates.Because of this result and the fact that similar procedures have been used successfullyin other studies(Lodge & Wist, 1968;Frisby & Mayhew, 1978; Collet, 1985; Mitchison & McKee, 1987; Nakayama & Shimojo, 1990) we think that the continuousvisibilityis not a problemin our experimental setup. Subjects
Four psychology students and the first author (JH) acted as subjects. The students were naive about the purpose of the experiment but three of them had extensive experience of stereo experiments. Before the experiment the subjects viewed freely some anaglyph stereograms and performed a short stereo acuity test. Initially seven subjects attended the test but two of them were excluded from the main experiment because of stereoblindness. After the initial test the subjects performedone trainingsessionconsistingof 240 stimulus presentations. RESULTS
The results are shown in Fig. 3 where the apparent depth of a monocular dot is plotted as a function of its distancefrom the binocularedge. Each curve is a mean of five subjects and represents a disparity configurationin which the occluding plane is located in front or behind the additionalplane. The valid monoculardots are in Fig. 3(A) and invalid dots in Fig. 3(B). The valid and invalid dots of Omin arc configuration,which is equivalentto the
SHORTCOMMUNICATION
3818
-50
(A)
1
-40 -u- 39.3 min arc uncrossed
-30
I I
-20
I
-10
I
n
0
+
13.1 min arc uncrossed
+
Omin arc
+
13.1 min arc crossed
+39.3.
min arc crossed
I I
I I
10 20 30 40 50 60 Distance from binocular edge (rein arc)
-50
(B)
-40 +
39.3 min arc uncrossed
+
13.1 min arc uncrossed
-20
+
o min arc
-lo
+
13.1 min arc crossed
+
39.3. min arccrossed
-30
u
al .-> al Cl %
a
0 10
o
10 20 30 40 50 60 Distance from binocular edge (rein arc)
FIGURE 3. Mean depth settings as a function of horizontal distance from the edge of the occluding plane. Each graph plots averaged data for one disparity condition (i.e. the disparity of the occludingplane) and five subjects. Ecologically valid and invalid points are plotted separately in (A) and (B). The continuousthick line is the depth constraint zone.
experiment of Nakayama and Shimojo (1990), appear similar to their experimentalthoughthe magnitudeof the discontinuityeffect is smaller.In the Odisparitycondition the monoculardots follow the depth constraintzone for a distance of l@15 min arc. If the occluding plane is farther than the additional plane, the depth localization curve is almost similar to the zero disparitycondition.On the other hand, if the occluding plane is nearer than the additionalplane the curve bends clearly to the uncrossed direction.Surprisingly,also invalid dots were affectedby the additionalplane. When the occludingplane is farther than the additionalplane the matchingcurvesremain near the zero disparity level, but when the occluding plane is nearer than the additional plane the curves are clearly biased towards uncrossed disparity. An analysis of variance with repeated measures was performed, with the mean of each disparityconfiguration as a dependent variable. The main effect for disparity of the occluding rectangle was significant both in valid [F(4,16) = 20.54, PC 0.001] and invalid situations [F(4,16) = 9.90, P< 0.001]. Our hypothesis that more uncrossed objects bias the depth localization of monocular points to uncrossed direction was confirmed.
Dk3CUSSION
The depth localization of a monocular dot is affected by uncrossedbinocularobjectsof the scene.Althoughthe depth localizationcurve of monoculardots is anchoredto the horizontally adjacent binocular edge, the curve is biased to uncrossed dinectionwhen there are uncrossed objects in the scene. Interestingly, also the invalid monocular dots show similar bias. The reason for this remains to be researched. It has been speculated that the ocular dominance columnsof V1 which preserve eye-of-origininformation, could form the neural basis of the process (Nakayama & Shimojo, 1990; Anderson & Nakayama, 1994). We suggest that the process consists of disparity inhibition and excitation,where monoculardots in the valid side of an occluding rectangle are inhibited in such a way that they fall furtherto the uncrosseddirection.The inhibition is constrained by the disparity gradient limit which preventsthe dots from falling too steeplyto the uncrossed direction. After the initial inhibition zone the visual system interpolates the ambiguous dots according to a depth map of the scene. If the map is very ambiguous,as in our experiment,the activationof the neural systemthat
SHORT COMMUNICATION
3819
is processing uncrossed disparity information biases the Julesz, B. (1964).Binoculardepth perceptionwithoutfamiliarity cues. Science, 145,356362. depth localization of monocular points to the uncrossed Lodge,H. & Wist, E. R. (1968).The growth of equidistancetendency direction.The fact that crossed elementsdo not introduce over time. Perception and Psychophysics,.?,97-103. such a bias is well in accordancewith other resultswhich Mitchison,G. J. & MCKCC,S. P. (1987).The resolution of ambiguous show that crossed disparity spread is not as wide as stereoscopic matches by interpolation. Vision Research, 27, 285– 294. uncrossed disparity spread (Takeichi et al., 1992). REFERENCES
Mustillo, P. (1985). Binocular mechanisms mediating crossed and uncrossed stereopsis. PsychologicalBulletin, 97, 187-201. Nakayama, K. & Shimojo,S. (1990). Da Vinci srereopsis: Depth and subjective occluding contours from unpaired image points. Vision Research, 11, 1811-1825. Nakayama, K. & Shimojo, S. (1992). Experiencing and perceiving visual surfaces. Science, 257, 1357-1363. Ramachandran, V. S. & Cavanagh, P. (1985). Subjective contours capture stereopsis.Nature, 317, 527–530. Richards, W. (1970). Stereopsis and stercoblindness. Experimental Brain Research, IO, 380-388. Richards, W. (1971). Anomalous stereoscopic depth perception. Journal of the Optical Society of America, 61, 410414. Shimojo,S. & Nakayama,K. (1990).Real world occlusionconstraints and binocular rivalry. VisionResearch, 30, 69–80. Takeichi, H., Watanabe, T. & Shimojo, S. (1992). Illusory occluding contours and surface formation by depth propagation.Perception, 21, 177–184.
Anderson, B. L. & Nakayama, K. (1994). Toward a general therrryof stercopsis: Binocular matching, occluding contours, andfusion. psychological Review, 101, 414445. Burt, P. & Julesz, B. (1980a). A disparity gradient limit for binocular vision. Science, 208, 615–617. Burt, P. & Jrrlesz,B. (1980b). Modificationsof the classical notion of Panum’s fusirmal area. Perception, 9, 671+82. Collet, T. S. (1985). Extrapolatingand interpolatingsurfaces in depth. Proceedings of the Royal Society of London,B, 224, 43-56. Foley, J. M. & Richards, W. (1972). Effects of voluntary eye movement and convergenceon the binocular appreciationof depth. Perception and Psychophysics, 11, 423-427. Frisby, J. P. & Mayhew, J. E. (1978). The relationship between apparent depth and disparity in rivalrous-texture stereograms. Perception, 78, 661-678. Gillam, B. & Borsting, E. (1988). The role of monocular regions in stereoscopic displays.Perception, 19, 603–608. Gogel,W. C. (1956). The tendencyto see objects as equidistantand its Acknowledgements—Thisresearch was supportedby a grant from the inverse relation to lateral separation. Psychological Monographs, University of Helsinki. We thank Samuel Kaski, Jari Laarni, Pentti Laurinen, Tarja Peromaa, and Janne Sinkkonen for their valuable 70, 1-17. Harris, J. P. & Gregory, R. L. (1973). Fusion and rivalry of illusory commentson an earlier draft of this manuscriptand Miika Silfverberg for his technical assistance. contours.Perception, 2, 235–247.