The limit set of a topological transformation group on generated by two generators is proved to be totally disconnected (or thin) and perfect if the conditions (i–v) are satisfied. A concrete application to a Doubly Periodic Riccati equation is given. 1. Introduction The conception limit set of a transformation group plays an important role in both theory and application of modern mathematics. Assume that is a transformation group (or semigroup) formed with some continuous self-mapping on a Hausdorff space . For any , the set is called an orbit through under the action of . For any subset of , let A subset of is called a -invariant set if A subset of is called the least invariant set of if it is a nonempty closed invariant set, in which there is not any nonempty closed proper subset which is -invariant. Based on the continuity of the elements in , it is easy to obtain the following proposition. Proposition 1.1. Let be a least invariant set of . For any , if there is such a point that then For any , a least invariant set of is called a limit set of the orbit if It is easy to prove the following proposition. Proposition 1.2. If the topology space is compact, then under the action of the transformation group (or semigroup) , any least invariant set is perfect if it is not finite. Therefore, it is quite possible that a limit set under the action of may be totally disconnected and perfect (a Cantor set), and that may be with fractal structure when some measure is attached. It is an important subject in the modern nonlinear science to study the exact structure of the limit set for a given nonlinear system, especially, to determine if the limit set is a totally disconnected and perfect set (for simple, called a Cantor set). However, it is usually not an easy task to do so, because of the strong nonlinearity and nonintegrability of the system. So it is necessary to explore the conditions for the existence of Cantor-type limit set. As an example, a traditional dynamical system is a continuous (or discrete) group or a semigroup (or ), acting on a manifold . The related group or semigroup is usually generated by a single generator. Both the -limit set and -limit set of an orbit through a point are limit sets of the corresponding group by the present definition [1]. If a least invariant set or a limit set of a dynamical system is Cantor type, then the related complicated motion is described as deterministic chaos. This kind of complicated motion has been considered widely. And some methods for determining if the least invariant set of a dynamical system is Cantor
References
[1]
J. Guckenheimer and P. Holmes, Nonlinear Oscillations, Dynamical Systems, and Bifurcations of Vector Fields, vol. 42 of Applied Mathematical Sciences, Springer, New York, NY, USA, 1983.
[2]
B. A. Dubrovin, A. T. Fomenko, and S. P. Novikov, Modern Geometry—Methods and Application II, Springer, New York, NY, USA, 1985.
[3]
V. I. Arnold and Yu. S. Il’yashenko, Dynamical Systems. I, vol. 1 of Encyclopaedia of Mathematical Sciences, Springer, Berlin, Germany, 1988.
[4]
H. M. Farkas and I. Kra, Riemann Surfaces, vol. 71 of Graduate Texts in Mathematics, Springer, New York, NY, USA, 1980.
[5]
V. V. Golubev, Lectures on the Analytic Theory of Differential Equations, Gosudarstv. Izdat. Tehn.-Teor. Lit., Moscow-Leningrad, Russia, 2nd edition, 1950.
[6]
K. Guan and J. Lei, “Integrability of second order autonomous system,” Annals of Differential Equations, vol. 18, no. 2, pp. 117–135, 2002.
[7]
K. Guan and R. Cheng, “Global first integrals and admitted Lie groups of second-order polynomial systems in the complex domain,” Journal of Nanjing University, Mathematical Biquarterly, pp. 229–235, 1993.
[8]
K. Guan, “The module structure of the infinite-dimensional lie algebra attached to a vector field,” in Lie Groups: New Research, pp. 139–167, Nova Science, New York, NY, USA, 2009.
[9]
I. Kaplansky, An Introduction to Differential Algebra, Actualités Sci. Ind., No. 1251 = Publ. Inst. Math. Univ. Nancago, No. 5, Hermann, Paris, France, 1957.
[10]
E. R. Kolchin, “Algebraic matric groups and the Picard-Vessiot theory of homogeneous linear ordinary differential equations,” Annals of Mathematics, vol. 49, pp. 1–42, 1948.
[11]
K. Guan and J. Lei, “An attempt to differential Galois theory of second order polynomial system and solvable subgroup of M？bius transformations,” Science in China. Series A, vol. 50, no. 5, pp. 748–760, 2007.
[12]
E. Hille, Ordinary Differential Equations in the Complex Domain, Dover Publications, Mineola, NY, USA, 1997.
[13]
Z. Chen and K. Guan, “The complexity of the limit set of solution space of a doubly periodic riccati equation on the complex domain,” Science in China, vol. 24, no. 8, pp. 800–807, 1994.
[14]
L. Ma and K. Guan, “Integrability of second-order Fuchsian equation on torus T2,” Chinese Science Bulletin, vol. 40, no. 12, pp. 1064–1067, 1996.
[15]
K. Guan and S. Zhang, “Structure of solvable subgroup of and integrability of Fuchsian equations on torus T2,” Science in China. Series A, vol. 39, no. 5, pp. 501–508, 1996.
[16]
L. Ma, C. Wang, and K. Guan, “Properties of solutions and integrability of Fuchsian equations on the torus T2,” Journal of Beijing University of Aeronautics and Astronautics, vol. 22, no. 1, pp. 71–77, 1996.
[17]
L. Ma and K. Guan, “Integrability of doubly-periodic Riccati equation,” International Journal of Mathematics and Mathematical Sciences, vol. 21, no. 4, pp. 785–790, 1998.
[18]
K. Guan, “The fractal structure of limit set of solution space of a doubly periodic Riccati equation,” International Journal of Mathematics and Mathematical Sciences, vol. 20, no. 4, pp. 707–712, 1997.
[19]
K. Guan, L. Ma, and C. Wang, “Global analysis of monodromy group of a Riccati equation on torus T2,” Annals of Differential Equations, vol. 15, no. 1, pp. 1–13, 1999.
[20]
A. F. Beardon, The Geometry of Discrete Groups, vol. 91 of Graduate Texts in Mathematics, Springer, New York, NY, USA, 1983.
