A high quality dataset on the photo-oxidation of benzene, toluene, p-xylene and 1,3,5-trimethylbenzene has been obtained from experiments in the European Photoreactor (EUPHORE), a large outdoor environmental reaction chamber. The experiments were designed to test sensitive features of detailed aromatic mechanisms, and the dataset has been used to evaluate the performance of the Master Chemical Mechanism Version 3 (MCMv3). An updated version (MCMv3.1) was constructed based on recent experimental data, and details of its development are described in a companion paper. The MCMv3.1 aromatic mechanisms have also been evaluated using the EUPHORE dataset. Significant deficiencies have been identified in the mechanisms, in particular: 1) an over-estimation of the ozone concentration, 2) an under-estimation of the NO oxidation rate, 3) an under-estimation of OH. The use of MCMv3.1 improves the model-measurement agreement in some areas but significant discrepancies remain.