Early identification of persons at risk for cognitive decline in aging is critical to optimizing treatment to delay or avoid a clinical diagnosis of mild cognitive impairment (MCI) or dementia due to Alzheimer's disease (AD). To accomplish early identification, it is essential that trajectories of cognitive change be characterized and associations with established biomarkers of MCI and AD be examined during the phase in which older persons are considered cognitively healthy. Here we examined the association of rate of cognitive decline in the years leading up to structural magnetic resonance imaging with an established biomarker, hippocampal volume. The sample comprised 211 participants of the Rush Memory and Aging Project who had an average of 5.5 years of cognitive data prior to structural scanning. Results showed that there was significant variability in the trajectories of cognitive change prior to imaging and that faster cognitive decline was associated with smaller hippocampal volumes. Domain-specific analyses suggested that this association was primarily driven by decline in working memory. The results emphasize the importance of closely examining cognitive change and its association with brain structure during the years in which older persons are considered cognitively healthy.