True borders of certain skin cancers are hard to detect by the human eye. For this reason, techniques such as polarized light imaging have been used to enhance skin cancer contrast before Mohs surgery procedures. In standard polarized light imaging the effect of the exposed rough surface is minimized using an index-matched boundary, such as a glass slide with gel. Moreover, surface glare is eliminated using indirect illumination. We studied the effect of surface roughness on the polarized light backscattered from skin demonstrating that rough surface effects can be minimized using out-of-plane polarized illumination in conjunction with polarized viewing, without the need for an index-matched boundary.