Human-computer intelligent interaction (HCII) is usually based on facial expression recognition. A dynamic facial expression recognition method based on video sequence is proposed in this paper, which uses Gaussian of Mixture Hidden Markov Model. Firstly, we get some special facial expression regions, in which the motion features are extracted and described as phase form and then constituted to eigen-sequences. Secondly we use Gaussian of Mixture Hidden Markov Model to learn and test these eigen-sequences, and recognize six universal facial expressions: angry, disgust, fear, happy, sad and surprise. And we developed an experimental system based on our algorithm. The experimental results show that the computing time and the error of vector quantization is reduced, while the classification efficiency is improved.