YAG is not only one of the most important solid laser crystals, but also a promising scintillation material. Since it comes out, the studies of defects in YAG crystal have played key roles on the improvement and development of crystal properties. As a special kind of defect in the YAG and LuAG crystals, anti-site defects exert crucial influence on the carrier transportation and other properties. In this paper, the type and the origin of the anti-site defects in the YAG and LuAG crystals, as well as the mechanism of their effects on the crystal properties are described. Also, how to detect and inhibit the anti-site defects in the YAG and LuAG crystals are discussed.