We introduce some new algebraic structures arising naturally in the geometry of Calabi-Yau manifolds and mirror symmetry. We give a universal construction of Calabi-Yau algebras in terms of a noncommutative symplectic DG algebra resolution. In dimension 3, the resolution is determined by a noncommutative potential. Representation varieties of the Calabi-Yau algebra are intimately related to the set of critical points, and to the sheaf of vanishing cycles of the potential. Numerical invariants, like ranks of cyclic homology groups, are expected to be given by `matrix integrals' over representation varieties. We discuss examples of Calabi-Yau algebras involving quivers, 3-dimensional McKay correspondence, crepant resolutions, Sklyanin algebras, hyperbolic 3-manifolds and Chern-Simons. Examples related to quantum Del Pezzo surfaces will be discussed in [EtGi].