Capsule endoscopy (CE) is aimed at diagnosing disease in areas of the gastrointestinal (GI) tract beyond the reach of conventional endoscopy. Recent work has addressed various methods for reducing the complexity of CE diagnosis and the time needed for analyzing the data. This includes detection of lumen and its contractions, fluids such as blood and intestinal juices, as well as extraneous matter such as food and bubbles. This paper outlines our ongoing work to segment lesions (in particular Crohn's disease) and other abnormalities in CE images. In particular, here we describe the data collection and clinical analysis for our project and preliminary results for segmenting abnormal and extraneous images from a set of 10 CE studies.