With SIRTF and NGST in the offing, it is interesting to examine what the stellar populations of z~3 galaxies models imply for the existence and nature of Lyman-break galaxies at higher redshift. To this end, we ``turn back the clock'' on the stellar population models that have been fit to optical and infrared data of Lyman-break galaxies at z~3. The generally young ages (typically 10^8 +- 0.5 yr) of these galaxies imply that their stars were not present much beyond z=4. For smooth star-formation histories SFR(t) and Salpeter IMFs, the ionizing radiation from early star-formation in these galaxies would be insufficient to reionize the intergalactic medium at z~6, and the luminosity density at z~4 would be significantly lower than observed. We examine possible ways to increase the global star-formation rate at higher redshift without violating the stellar-population constraints at z~3.