Self-driving car has the capability to navigate autonomously with high awareness of the surrounding environment. They have an intelligent method to avoid collision and predict movements of other objects. One of the key technology in order to achieve this level of safety is object tracking. Accurate and quick vision-based method compete to increase the reliability. Several tracking methods utilized visible spectrum camera to gather data. However, there are challenges that arise, especially in minimal lighting such as cloudy and rainy conditions or at night. To overcome this problem, we use thermal image. We propose a newly enhanced Smoothing Stochastic Approximate Monte Carlo (SSAMC) based tracker with unique preprocessing Gamma Normalization and Median Filter. We tested our tracker in self-driving car theme data from Linköping Thermal InfraRed (LTIR). This data is captured from both a moving and a static sensor, both have different difficulty level. The experiment results show that the tracker achieved a better result compared to other methods. We achieved an accuracy of 0.8786 with a higher frame per second computation time of 4.6405.