Many cities around the World have established the development objective of becoming a low-carbon city. Evaluation of such a city is important for its progress. A new evaluation framework of urban low-carbon development level is proposed in this paper, which integrates synthetic evaluation based on a bottom-up idea and analytical diagnosis based on a top-down idea. Further, set pair analysis is combined for synthetic evaluation and analytical diagnosis by comparing urban low-carbon development levels of different cities, through which the comprehensive state of urban low-carbon development level can be obtained and limiting factors identified. Based on the proposed framework and set pair analysis, low-carbon development levels of 12 Chinese cities are compared. Some suggestions are provided, based on results of overall situations of urban low-carbon development level and concrete performances of various factors and specific indicators. We conclude that both synthetic evaluation and analytical diagnosis are important for evaluation of urban low-carbon development level. The proposed framework and method can be widely applied in the evaluation of different cities over a long-term period.