Receiver design for MIMO relay stations in multi-cell downlink system

In multi-cell downlink system, quality of service (QoS) of multiuser MIMO is an important issue, in particular, for cell-edge users. For QoS of cell-edge users in the system, the utilization of multiple-input multiple-output (MIMO) relay stations (RSs) is a promising solution that improves the signal-to-interference plus noise ratio (SINR) from neighboring base stations (BSs) to each RS. Since the capacity between the BS and the RS relies heavily on the receive performance, it is necessary to reflect interference channels in the design of the receive weight vector. In this paper, we use a geometric approach to derive the effective channel gain of the RS according to the receive weight vector. The geometric relationship between the desired and interference channels is also described. Using the description, we propose a receiver, called the maximum lower bound of expected SINR receiver (MLESR). Through a performance analysis of the MLESR over the interference-limited regime, we derive closed terms for the performance bounds in terms of the number of RS antennas, the interference channel rank, and the BS transmit power according to the rate gap of the MLESR with respect to an ideal case, i.e., a no interference case.