A Bethe-Peierls treatment to dilution in frustrated magnets and spin liquids is given. A spin glass phase is present at low temperatures and close to the percolation point as soon as frustration takes a finite value in the dilute magnet model; the spin glass phase is reentrant inside the ferromagnetic phase. An extension of the model is given, in which the spin glass / ferromagnet phase boundary is shown not to reenter inside the ferromagnetic phase asymptotically close to the tricritical point whereas it has a turning point at lower temperatures. We conjecture similar phase diagrams to exist in finite dimensional models not constraint by a Nishimori's line. We increase frustration to study the effect of dilution in a spin liquid state. This provides a ``minimal'' ordering by disorder from an Ising paramagnet to an Ising spin glass.