We study a particular class of open manifolds. In the category of Riemannian manifolds these are complete manifolds with cylindrical ends. We give a natural setting for the conformal geometry on such manifolds including an appropriate notion of the cylindrical Yamabe constant/invariant. This leads to a corresponding version of the Yamabe problem on cylindrical manifolds. We affirmatively solve this Yamabe problem: we prove the existence of minimizing metrics and analyze their singularities near infinity. These singularities turn out to be of very particular type: either almost conical or almost cusp singularities. We describe the supremum case, i.e. when the cylindrical Yamabe constant is equal to the Yamabe invariant of the sphere. We prove that in this case such a cylindrical manifold coincides conformally with the standard sphere punctured at a finite number of points. In the course of studying the supremum case, we establish a Positive Mass Theorem for specific asymptotically flat manifolds with two almost conical singularities. As a by-product, we revisit known results on surgery and the Yamabe invariant. Key words: manifolds with cylindrical ends, Yamabe constant/invariant, Yamabe problem, conical metric singularities, cusp metric singularities, Positive Mass Theorem, surgery and Yamabe invariant.