Systems biology approaches combining theoretical modeling with experiments have been singularly successful in uncovering novel features of cellular phenomena. One such feature is that of binary gene expression in which the expression level is either low or high, i.e., digital in nature. This gives rise to two distinct subpopulations in a population of genetically identical cells. The fraction of cells in the high expression state is raised as the strength of the inducing signal is increased indicating that the response is not graded. In this review, we discuss the possible origins of binary gene expression with emphasis on three principal mechanisms: purely stochastic, positive feedback-based and emergent bistability. In the latter case, two stable expression states are obtained due to an autoregulatory positive feedback loop in protein synthesis along with cell growth retardation by the proteins synthesized. The theoretical foundations of the observed phenomena are described in each case.