In the recent years, tactile sensing has attracted great interest of the robotics community, specially in high end applications such as prosthesis and exploration of unstructured environments. Tactile sensing in both humans and robots can help to estimate surface properties like temperature, texture and hardness. These properties are important to perceive the environment as well as for object recognition and robotic manipulation of objects. In this paper, we focus on the classification of textures using a neuromorphic tactile sensor. The sensor uses a piezoresistive fabric sandwiched between conductive traces and encodes only changes in pressure intensity in the form of spike events. Textures are classified based on their geometry and the spatial frequencies. To do so, high-contrast tactile images are created by combining event data obtained across a palpation sequence and rotation-invariant features are used for classification using support vector machines (SVM). Our best results show average classification accuracy of 98% across different textures and palpation done in multiple directions.