This paper presents an approach for recognition of offline handwritten isolated Urdu character based on Invariant Moments. Handwritten Urdu character recognition is lagging behind due to segmentation dilemma and complexity of Urdu letter writing. An attempt is made to apply Moment Invariant technique followed by Primary and secondary component separation. The Urdu letters were grouped into single component and multi-component characters. If letter is multi-component then Secondary component were separated from primary component. SVM is adopted for classification and position of secondary component (Above, Below and middle) is considered for recognition. For each of 46 characters 200 image samples were used for training and 600 for testing respectively. In this manner overall 36800 handwritten characters were used to apply the technique. Overall performance rate is found to be 93.59% for all offline handwritten isolated Urdu characters. It is possible to enhance the accuracy of system by combining more structural and statistical features.