Tissue abnormality characterization is a generalized segmentation problem which aims at determining a continuous score that can be assigned to the tissue which characterizes the extent of tissue deterioration, with completely healthy tissue being one end of the spectrum and fully abnormal tissue such as lesions, being on the other end. Our method is based on the assumptions that there is some tissue that is neither fully healthy or nor completely abnormal but lies in between the two in terms of abnormality; and that the voxel-wise score of tissue abnormality lies on a spatially and temporally smooth manifold of abnormality. Unlike in a pure classification problem which associates an independent label with each voxel without considering correlation with neighbors, or an absolute clustering problem which does not consider a priori knowledge of tissue type, we assume that diseased and healthy tissue lie on a manifold that encompasses the healthy tissue and diseased tissue, stretching from one to the other. We propose a semi-supervised method for determining such as abnormality manifold, using multi-parametric features incorporated into a support vector machine framework in combination with manifold regularization. We apply the framework towards the characterization of tissue abnormality to brains of multiple sclerosis patients.