Performance analysis for optimum transmission and comparison with maximal ratio transmission for MIMO systems with cochannel interference

This article presents the performance analysis of multiple-input/multiple-output (MIMO) systems with quadrature amplitude modulation (QAM) transmission in the presence of cochannel interference (CCI) in nonfading and flat Rayleigh fading environments. The use of optimum transmission (OT) and maximum ratio transmission (MRT) is considered and compared. In addition to determining precise results for the performance of QAM in the presence of CCI, it is our another aim in this article to examine the validity of the Gaussian interference model in the MRT-based systems. Nyquist pulse shaping and the effects of cross-channel intersymbol interference produced by CCI due to random symbol of the interfering signals are considered in the precise interference model. The error probability for each fading channel is estimated fast and accurately using Gauss quadrature rules which can approximate the probability density function (pdf) of the output residual interference. The results of this article indicate that Gaussian interference model may overestimate the effects of interference, particularly, for high-order MRT-based MIMO systems over fading channels. In addition, OT cannot always outperform MRT due to the significant noise enhancement when OT intends to cancel CCI, depending on the combination of the antennas at the transmitter and the receiver, number of interference and the statistical characteristics of the channel.