Hybrid Beamforming Design with Overlapped Subarrays for Massive MIMO-ISAC Systems

Integrated sensing and communications (ISAC), supported by massive multiple-input multiple-output (MIMO), can provide simultaneously improvement of sensing capability and communication capacity. However, employing the conventional fully digital beamforming architecture with a large-scale antenna array will incur the prohibitively high hardware cost and power consumption. In this paper, we propose a hybrid beamforming design with the overlapped subarrays (OSA)-based hybrid architecture for massive MIMO-ISAC systems. We design the analog and digital beamformers by jointly optimizing the spectral efficiency of communication and beampattern mean squared error of sensing under the specific constraints of OSA structures, power budget, and constant modulus. To tackle the resulting non-convex problem, we relax it as a weighted summation minimization problem, where the Euclidean distance between the designed hybrid beamformers and the optimal communication/desired sensing beamformers is minimized. We further decompose the formulated problem into three subproblems and develop an effective alternating minimization algorithm. Numerical simulations demonstrate the effectiveness and flexibility of the proposed OSA-based hybrid beamforming design in terms of spectral efficiency and sensing beampattern performance.