We investigate whether stellar-mass black holes have to receive natal kicks in order to explain the observed distribution of low-mass X-ray binaries containing black holes within our Galaxy. Such binaries are the product of binary evolution, where the massive primary has exploded forming a stellar-mass black hole, probably after a common envelope phase where the system contracted down to separations of order 10-30 Rsun. We perform population synthesis calculations of these binaries, applying both kicks due to supernova mass-loss and natal kicks to the newly-formed black hole. We then integrate the trajectories of the binary systems within the Galactic potential. We find that natal kicks are in fact necessary to reach the large distances above the Galactic plane achieved by some binaries. Further, we find that the distribution of natal kicks would seem to be similar to that of neutron stars, rather than one where the kick velocities are reduced by the ratio of black hole to neutron-star mass (i.e. where the kicks have the same momentum). This result is somewhat surprising; in many pictures of stellar-mass black-hole formation, one might have expected black holes to receive kicks having the same momentum (rather than the same speed) as those given to neutron stars.