The effect of the strange quark in the finite temperature phase transition of QCD is studied on the lattice. Using the one-plaquette gauge action and the Wilson quark action, the transition in the chiral limit is shown to be continuous for the case of degenerate two flavors, $N_F=2$, while it is of first order for $N_F \geq 3$. For a more realistic case of massless up and down quarks and a light strange quark, $N_F=2+1$, clear two state signals are observed both for $m_s \simeq 150$ and 400 MeV. In contrast to a previous result with staggered quarks, this suggests a first order transition in the real world. In order to see the implication of these results to the continuum limit, we started to study these issues using improved actions. First results using a RG improved gauge action combined with the standard Wilson quark is presented for the case of $N_F=2$: With this action the finite temperature transition is shown to be continuous in the chiral limit confirming the result of the standard action. Furthermore, not like the case of the standard action where lattice artifacts make the transition once very strong at intermediate values of the hopping parameter $K$ on $N_t=4$ and 6 lattices, a smooth crossover is found for the improved action when we increase $1/K-1/K_c$, in accord with a naive expectation about the fate of second order chiral transition at finite $m_q$.