Motivated by recent experiments, in which the Kondo effect has been observed for the first time in a double quantum-dot structure, we study electron transport through a system consisting of two ultrasmall, capacitively-coupled dots with large level spacing and charging energy. Due to strong interdot Coulomb correlations, the Kondo effect has two possible sources, the spin and orbital degeneracies, and it is maximized when both occur simultaneously. The large number of tunable parameters allows a range of manipulations of the Kondo physics -- in particular, the Kondo effect in each dot is sensitive to changes in the state of the other dot. For a thorough account of the system dynamics, the linear and nonlinear conductance is calculated in perturbative and non-perturbative approaches. In addition, the temperature dependence of the resonant peak heights is evaluated in the framework of a renormalization group analysis.