A major challenge in many modern superresolution fluorescence microscopy techniques at the nanoscale lies in the correct alignment of long sequences of sparse but spatially and temporally highly resolved images. This is caused by the temporal drift of the protein structure, e.g. due to temporal thermal inhomogeneity of the object of interest or its supporting area during the observation process. We develop a simple semiparametric model for drift correction in SMS microscopy. Then we propose an M-estimator for the drift and show its asymptotic normality. This is used to correct the final image and it is shown that this purely statistical method is competitive with state of the art calibration techniques which require to incorporate fiducial markers into the specimen. Moreover, a simple bootstrap algorithm allows to quantify the precision of the drift estimate and its effect on the final image estimation. We argue that purely statistical drift correction is even more robust than fiducial tracking rendering the latter superfluous in many applications. The practicability of our method is demonstrated by a simulation study and by an SMS application. This serves as a prototype for many other typical imaging techniques where sparse observations with highly temporal resolution are blurred by motion of the object to be reconstructed.