Some kind of segmentation
Attempt an efficient cuda segmentation which divides 8x8 blocks into one or two segments by thresholding edges and colours until done. A second step then iteratively combines segments with neighbors and generates a unique ID or colour for the segment. After this a median depth and normal can be estimated for each segment.
Edited by Nicolas Pope