In recent years, a number of alternative theories of gravity have been proposed as possible resolutions of certain cosmological problems or as toy models for possible but heretofore unobserved effects. However, the implications of such theories for the stability of structures such as stars have not been fully investigated. We use our "generalized variational principle", described in a previous work, to analyze the stability of static spherically symmetric solutions to spherically symmetric perturbations in three such alternative theories: Carroll et al.'s f(R) gravity, Jacobson & Mattingly's "Einstein-aether theory", and Bekenstein's TeVeS. We find that in the presence of matter, f(R) gravity is highly unstable; that the stability conditions for spherically symmetric curved vacuum Einstein-aether backgrounds are the same as those for linearized stability about flat spacetime, with one exceptional case; and that the "kinetic terms" of vacuum TeVeS are indefinite in a curved background, leading to an instability.