We investigated the relationship between the X-ray variability amplitude and X-ray luminosity for a sample of 14 bright Ultra-luminous X-ray sources (ULXs) with XMM-Newton/EPIC data, and compare it with the well established similar relationship for Active Galactic Nuclei (AGN). We computed the normalised excess variance in the 2-10 keV light curves of these objects and their 2-10 keV band intrinsic luminosity. We also determined model "variability-luminosity" relationships for AGN, under several assumptions regarding their power-spectral shape. We compared these model predictions at low luminosities with the ULX data. The variability amplitude of the ULXs is significantly smaller than that expected from a simple extrapolation of the AGN "variability-luminosity" relationship at low luminosities. We also find evidence for an anti-correlation between the variability amplitude and L(2-10 keV) for ULXs. The shape of this relationship is consistent with the AGN data but only if the ULXs data are shifted by four orders of magnitudes in luminosity. Most (but not all) of the ULXs could be "scaled-down" version of AGN if we assume that: i) their black hole mass and accretion rate are of the order of ~(2.5-30)x 10E+03 Msolar and ~ 1-80 % of the Eddington limit, and ii) their Power Spectral Density has a doubly broken power-law shape. This PDS shape and accretion rate is consistent with Galactic black hole systems operating in their so-called "low-hard" and "very-high" states.