Extremal edges: Evidence in natural images

In complex natural scenes, information about figure ground organization is obtained from a combination of global and local features. In this study, we searched for local features that predict figure-ground assignment in natural images. A large number of image patches were extracted from natural images along the boundary of perceptual objects and Principal Component (PC) Analysis was applied to these patches. Two sets of experiments were carried out. In the first (E1), only patches were included that did not contain any occlusion features (T-junctions) in the background. In the second (E2), both patches with and without occlusions in the background were included. As expected, the first principal component in both datasets is a step edge, and the second PC in E2 is a T-junction in the background. The next principal component (PC2 in E1 and PC3 in E2) in both datasets has uniform intensity on the background side, and a large contrast gradient on the figure side. This is the signature of an extremal edge [1]; the gradient is a local feature indicative of an object occluding a background. To test if these components are predictive of figure-ground segregation, statistical analyses of distributions obtained by the projection of patches on the respective PCs were performed. Both datasets, E1 and E2, show significant differences on Students t-test (p ≤ 10^-22) and the Bayesian t-test (JZSbf ≤ 10^-19) between figure and background distributions. We conclude that: (1) next to the T-junction the second strongest local feature in natural images predictive of figure ground organization is a luminance gradient of the figure side; (2) figure-side gradients are prevalent irrespective of the presence or absence of occlusion features.