Trading Statistical Efficiency for Speed in Parameter Estimation Problems

Conventional parameter estimation approaches struggle when confronted with the vast quantities of data present in modern applications. We show that restricting this data to a low dimensional subspace trades a decreased run time for an increased error in the parameter estimates. Using ideas from experimental design we deterministically pick this low dimensional subspace to minimize the loss of accuracy guided by an asymptotic covariance estimate. The largest gains come from tailoring a subspace to the structure of a specific problem. We demonstrate one such family of subspaces for the exponential fitting problem that obtains parameter estimates with near-optimal accuracy and whose dimension is the number of free parameters. By solving the nonlinear least squares problem restricted to a similarly well designed subspace, we obtain 30--100 times speedups while sacrificing a negligible amount of accuracy. We hope the exponential fitting problem provides a template for applying this experimental design guided dimension reduction technique to other large scale parameter estimation problems.