Transmitter and Receiver Window Designs for Orthogonal Time-Frequency Space Modulation

In this paper, we investigate the impacts of transmitter and receiver windows on the performance of orthogonal time-frequency space (OTFS) modulation and propose window designs to improve the OTFS channel estimation and data detection performance. In particular, assuming ideal pulse shaping filters at the transceiver, we derive the impacts of windowing on the effective channel and its estimation performance in the delay-Doppler (DD) domain, the total average transmit power, and the effective noise covariance matrix. When the channel state information (CSI) is available at the transceiver, we analyze the minimum squared error (MSE) of data detection and propose an optimal transmitter window to minimize the detection MSE. The proposed optimal transmitter window can be interpreted as a mercury/water-filling power allocation scheme, where the mercury is firstly filled before pouring water to pre-equalize the time-frequency (TF) domain channels. When the CSI is not available at the transmitter but can be estimated at the receiver, we propose to apply a Dolph-Chebyshev (DC) window at either the transmitter or the receiver, which can effectively enhance the sparsity of the effective channel in the DD domain. Thanks to the enhanced DD domain channel sparsity, the channel spread due to the fractional Doppler is significantly reduced, which leads to a lower error floor in both channel estimation and data detection compared with that of rectangular window. Simulation results verify the accuracy of the obtained analytical results and confirm the superiority of the proposed window designs in improving the channel estimation and data detection performance over the conventional rectangular window design.