In the field of security automated surveillance systems are very useful. Surveillance system can beused to detect and track the moving objects. First phase of the system is to detect the moving objects in thevideo. Second phase of the system will track the detected object. In this paper, detection of the moving objecthas been done using simple background subtraction and tracking of single moving object has been doneusing Kalman filter. The algorithm has been applied successfully on standard surveillance video datasets ofCAVIAR  and PETS . The videos used for testing have been taken using still cameras, which arelocated in indoor as well as outdoor environment having moderate to complex environments.