Improving spectral efficiency in distributed massive MIMO in multi-user downlink millimeter wave

Abstract

Analog and digital precoding are used in distributed massive multiple-input multiple-output (MIMO) at millimeter wave (mmWave) frequencies to efficiently manage data transfer across several antennas and base stations (BSs) situated at different locations. This method enhances spectral efficiency(SE) in spite of having a smaller amount complexity and cost compared fully digital systems. This paper presents a fully connected hybrid precoding design for a downlink mmWave dispensed or distributed massive multi-user MIMO. The objective function for the optimization problem is the SE of the proposed system, subject to constraints on analog radio frequency (RF) precoding and power budget. The main aim is to maximize SE. Due to the nonconvex nature of the problem, a two-stage iterative algorithm is proposed to conclude the optimal analog and digital beamforming matrices and sum rate. The 1st stage obtains the optimal digital matrix assuming the analog RF precoder matrix is known, followed by acquiring the optimal analog RF precoder matrix in the next step. The Karush–Kuhn–Tucker (KKT) condition for each maximization problem are compute and examine to derive the solving algorithms for each stage. The simulation results display that the proposed design outperforms current methods in sum rate and approaches the performance of fully digital systems with reduced complexity compared to other alternatives.

Introduction

The Massive MIMO (Multiple Input Multiple Output) has emerged as anessential technology to address these demands, enabling simultaneous communication with multiple users through the installation of a high quantity of antennas at the base station (BS)¹. This technology not only increases spectral efficiency but also enhances energy efficiency, making it a cornerstone of future wireless networks.MmWave technology, also known as millimeter wave. The wide bandwidth offered by mmWave frequencies enables the transmission of large volumes of data at high speeds^2,3,4. This characteristic is crucial in a massive MIMO system as it supports many antennas. Additionally, mmWave signals have shorter wavelengths, allowing for more antennas to be accommodated in a smaller space. Consequently, this enables enhanced spatial multiplexing, facilitating the simultaneous transmission of multiple data streams across different channels. MmWave signals possess strong directional properties, making them well-suited for beamforming techniques in massive MIMO systems. Since the strong directional nature of mmWave signals, interference between adjacent cells or users can be minimized, thereby improving network capacity and reliability.

The use of hybrid precoding has gained attention as a promising solution to address these issues, allowing for efficient resource allocation in mmWave systems while reducing hardware⁵. This technique combines analog and digital precoding methods, allowing for efficient resource allocation while minimizing hardware complexity by leveraging both domains.hybrid precoding can effectively manage the trade-offs between performance and cost, making it particularly suitable for mmWave distributed multi-user (MU) massive MIMO systems.In mmWave systems, the use of hybrid precoding can significantly enhance the system’s ability to serve multiple users simultaneously. Analog precoding utilizes phase shifters to manipulate signals before transmission, while digital precoding optimizes signal processing at the baseband level⁶. This dual approach is crucial in maximizing spatial diversity and improving overall system performance.

In traditional massive MIMO systems, a single BS is equipped with a large number of antennas. In distributed massive MIMO, multiple BSs are deployed over a larger area, each hasa less quantity of antennas. This distribution allows for better coverage and capacity.The base stations in a distributed system can coordinate their operations to manage interference and optimize performance. This coordination can be achieved through techniques like joint processing, where signals from different base stations are processed collectively.Each base station can focus on serving users in its vicinity, leading to improved signal quality and reduced latency. This is particularly beneficial in densely populated areas where user demand is high.By having base stations spread out, the system can exploit spatial diversity more effectively⁷. This helps mitigate fading and improves overall reliability.Coordinated transmission among distributed BSs can minimize inter-cell interference, leading to higher data rates and improved Quality of Service (QoS)⁸.Distributed BSs can assist in load balancing by dynamically allocating resources based on user demand and channel conditions, ensuring efficient use of network resources.The base stations need reliable backhaul connections to communicate with a central controller or network core. This connectivity is crucial for coordinating operations and managing data traffic and usually fiber optical is used.

This is an important concern in practical deployments. While our current model assumes ideal and high-capacity backhaul links between SBSs and a central controller, real-world backhaul constraints. Synchronization and coordination, and scalability.

The integration of hybrid precoding in distributed systems introduces unique challenges that must be addressed. The design of effective precoding algorithms must consider channel state information (CSI) and hardware constraints⁹. Additionally, optimizing the hybrid precoding matrix requires balancing between analog and digital components to achieve maximum performance.Performance metrics play a crucial role in evaluating hybrid precoding in mmWave distributed MU massive MIMO systems. Metrics includingenergy efficiency, spectral efficiency, and user fairness are essential for assessing overall system effectiveness. Understanding the interactions among these metrics will provide deeper insights into system design and optimization strategies.

Recent studies have focused on developing robust algorithms for hybrid precoding that can adopt to varying channel conditions and user distributions. Techniques such as alternating optimization and deep learning (DL)¹⁰ have shown promise in improving algorithm efficiency¹¹. presents effective alternating minimization (AltMin) techniques for two distinct hybrid precoding configurations: fullyconnected and partiallyconnected. To emulate the performance of fully digital precoding, an AltMin algorithm utilizing manifold optimization is recommended for the fully connected framework. AltMin algorithm is subsequentlyapplied. Additionally, for partially connected, the mentioned method with the aid of semidefinite relaxation is applied. These AltMin algorithms are then adopted for practical implementation in wideband scenarios.The phased zero forcing (PZF) is a hybrid precoding method presented in¹², which aims to achieve nearly optimal performance in massive MIMO systems. PZF manages only the phase at the radio frequency domain. Since the power limitation and cost, fully digital beamforming approaches face challenges in large-scale antenna arrays, as explored in¹³. To address this, a hybrid beamforming (HBF) architecture that combines digital and analog components is proposed to achieve similar performance with fewer RF chains. The study shows that with twice the number of RF chains as data streams, the hybrid structure can achieve the same level of performance as a fully digital beamformer. Furthermore, the mentioned paper discusses design challenges for specific scenarios and suggests heuristic solutions. Lastly, the proposed algorithms are adjusted to enable practical implementation¹⁴. examines the application of mean square error (MSE) as a metric for assessing transmission reliability, specifically in the context of hybrid beamforming design for wideband mmWave transmissions. To optimize transmit and receive beamformers, it utilizes the alternating minimization method, placing special importance on minimizing computational complexity and enhancing convergence speed. Additionally, the research applies sum MSE to incorporate weighted sum-MSE and explores its relationship with spectral efficiency based design¹⁵. introduces a hybrid beamforming approach for distributed small-cell BSs (SBSs)systems. In this suggested approach, the users and SBSs equipped with multiple antennas. In this scheme, each SBS simultaneously transmit data streams to different users, making use of analog beamformers to reduce interference between user channels and make the most of additional spatial degrees of freedom for data multiplexing¹⁶. focuses on developing hybrid beamforming for a downlink mmWave massive multi-user MIMO system. The goal is maximizing the total data transmission rate. A linear algorithm with closed-form solutions is suggested for analog precoder and combing. Another algorithm for designing the digital beamformer is suggested, leading to reduced computational complexity and a simple implementation. The research paper¹⁷ introduces HBF algorithms designed to maximize spectral efficiency in wideband scenarios in mmWave massive MIMO using a partiallyconnected scheme. The introduced system makes tradeoff between the problem of maximizing spectral efficiency and a Weighted MMSE problem approach. The paper breaks down the equivalent WMMSE problem, suggests solutions for optimal digital precoding and combining, and puts forward algorithms for the more complex analog precoder and combiner. The HBF algorithms have been demonstrated to enhance spectral efficiency and exhibit convergence. Furthermore, the paper also introduces modified algorithms with reduced complexity and proposes the use of finite-resolution phase shifters¹⁸. addresses the difficulties associated with channel estimation and beamforming within a MU massive MIMO framework, presenting a DL-based Hybrid Beamformer as a solution tailored for 5G communication systems. This innovative design emphasizes precise channel estimation through machine learning techniques and the design of a hybrid beamformer using Neural networks. It tackles the challenges posed by low signal-to-noise ratio (SNR) conditions via the Improved Proximal Policy Optimization (IPPO) algorithm.The MSE is the key focus of^14,19 when evaluating transmission reliability in broadband mmWave transmissions. It delves into using the minimum sum-MSE criterion for designing hybrid beamforming (HBF). Integrated Sensing and Communication (ISAC), which is a key emerging application for mmWave technology. While our paper focuses purely on spectral efficiency maximization for communication²⁰, –²¹ addresses the same fundamental optimization problem (maximizing spectral efficiency) but within the more complex framework of ISAC and under practical constraints, such as constant-modulus constraints on phase shifters. It tackles the issue of computational complexity by suggesting various algorithms for narrowband and broadband scenarios. This manuscript proposes a hybrid precoding scheme for a distributed millimeter-wave multi-user massive MIMO system from the transmitter’s perspective.This study is highly relevant as it addresses hybrid transceiver design and optimal power allocation in the downlink of mmWave MIMO systems to all central themes in our paper. Furthermore, it considers a Cognitive Radio (CR) system, introducing an additional layer of complexity related to interference management, a challenge also prominent in distributed systems.The methodology of optimizing the hybrid transceiver and power allocation in²² provides a valuable benchmark and complementary approach to our proposed two-stage iterative algorithm, which also targets the joint optimization of digital/analog precoders and adheres to a total power budget P_T. Our focus is on a fully structured and distributed BSsetup, where the BS is partitioned into multiple small-cell BSs (SBSs), each equipped with various antennas. We employ a two-stage iterative algorithm to maximize spectral efficiency. In the first stage, we obtain the optimal digital precoder matrix under the assumption of knowing the radio frequency (RF) precoder matrix. Subsequently, we derive the optimal RF precoder matrix using the obtained optimal digital precoder matrix. In conclusion, the primary contributions of this study are outlined as follows:

• We are examining a distributed system with mmWave multi-user massive MIMO, as a distributed structure has shown better performance compared to a collocated one¹⁵. Our focus is on a distributed BS scheme, comprising several SBSs, serving multiple users with numerous antennas in a fullyconnected configuration. To our knowledge, only a few hybrid precoding efforts have been undertaken for distributed schemes.

• We form a uniform linear array (ULA) for the configuration of antennas and develop the corresponding system model within the context of mmWave wireless communication channels. Additionally, we demonstrate that a notable rise in the quantity of transmitter antennas leads to the complete elimination of inter-user interference. Consequently, in our calculations of spectral efficiency, we exclude the term associated with inter-user interference.

• We develop a two-stage iterative algorithm to address the optimization problem. The main objective is to enhance spectral efficiency while adhering to constraints imposed by the RF precoder matrix and total power limits. In the initial phase, we assume the RF precoder is predetermined, allowing us to derive the optimal digital precoder matrix through the optimization framework. In the subsequent phase, we determine the optimal RF precoder. To adopt the iterative algorithm in both phases, we establish the Lagrange multipliers for each optimization scenario and apply the Karush-Kuhn-Tucker (KKT) conditions, subsequently calculating the desired optimal matrix. Finally, we execute the iterative algorithm for each phase.

• In the simulation section, we initially compare the spectral efficiency performance of distributed massive MIMO with the collocated system. As expected, the distributed scheme demonstrates superior performance. We also evaluate the proposed algorithm across various scenarios, including different SNRs, numbers of data streams, users, and SBS antennas.Our findings indicate that the proposed algorithm exhibits better spectral efficiency performance compared to other existing hybrid beamforming algorithms such as analog beamforming, orthogonal matching pursuit (OMP), and PE- AltMin¹¹. It is important to note that the proposed algorithm performs exceptionally well, particularly when the number of RF chains is double the number of data streams. Furthermore, we conduct tests to confirm the convergence of the proposed algorithm, which demonstrates that the algorithm converges after a few iterations. When compared to other algorithms, the proposed scheme has lower complexity, making it a suitable low-complexity algorithm.

Section “System model” introduce an overview ofthe distributed mmWave massive multi-user MIMO system, along with the presentation of the proposed fullyconnected hybrid precoding at the transmitter side. Section “Channel model” discuss the mmWave channel model utilized in this study and includes the demonstration of the orthogonality of different user channels. The spectral efficiency of the proposed system is computed in Sect. “Spectral efficiency”. The optimization problem aimed at identifying the optimal analog and digital matrices is outlined in Sect. “Spectral efficiency optimizationproblem definition”. Sect. “Solving the problem” addresses the solution of the optimization and proposes a two-stage iterative algorithm. The findings from the simulation are elaborated upon in Sect. “Simulation results”, and the final section delineates the conclusions.

System model

Take into account the downlink distributed massive multi-user (MU) MIMO system as shown in Fig. 1. Where the BS is divided into $\:N$ SBSs. Each SBS has $\:M$ antennas and $\:{N}_{RF}$ RF chains. These SBSs simultaneously serve$\:K$ users, each of which has $\:P$ antennas. Furthermore, it is assumed that all SBSs are connected to a central processing unit.

This paper aims to develop a hybrid precoding for the specified system model at the transmitter side. The symbol $\:{\varvec{s}}_{k}$ represents a $\:q\times\:1$ column-vector of symbols intended for the $\:k$th user, where $\:k$ ranges from 1 to $\:K$. The vector $\:\varvec{s}\triangleq\:{\left[{\varvec{s}}_{1}^{T},{\varvec{s}}_{2}^{T},\dots\:.,{\varvec{s}}_{K}^{T}\right]}^{T}$ denotes the transmitted symbols of length $\:{N}_{sym}\triangleq\:Kq$. In the illustration in Fig. 2, each SBS uses an $\:{N}_{RF}\times\:{N}_{sym}$ digital baseband precoding matrix, denoted by $\:{\varvec{F}}_{B{B}_{i}}=\left[{\varvec{F}}_{B{B}_{i,1}},{\varvec{F}}_{B{B}_{i,2}},\dots\:,{\varvec{F}}_{B{B}_{i,K}}\right],\:0\le\:i\le\:N$, to precode the symbols vector. The resulting streams are then upconverted to the radio frequency in the $\:{N}_{RF}$ RF chains. These analog signals are subsequently phase-shifted using an $\:M\times\:{N}_{RF}$analog RF precoder denoted by $\:{\varvec{F}}_{R{F}_{i}}$ before being transmitted to the users using the SBS’s $\:M$ antennas. Figure 1 shows the distributed massive MU-MIMO system model.

Fig. 1

The considered downlink distributed massive MU-MIMO system model.

The signal transmitted in each SBS can be represented as follows:

$$\:{\mathbf{x}}_{i}={{\varvec{F}}_{RF}}_{i}{\varvec{F}}_{B{B}_{i}}\varvec{s}$$

(1)

In the above equation, it is assumed that $\:{\varvec{s}}_{1},{\varvec{s}}_{2},\dots\:,{\varvec{s}}_{K}$ are orthogonal, and $\:i$ shows the number of SBSs. Additionally, the signal received by user $\:k$ can be represented as:

$$\:{\varvec{y}}_{i,k}={\varvec{H}}_{i,k}{\varvec{F}}_{R{F}_{i}}{\varvec{F}}_{B{B}_{i,k}}{\varvec{s}}_{k}+\sum\limits_{l\ne\:k}{\varvec{H}}_{i,k}{\varvec{F}}_{R{F}_{i}}{\varvec{F}}_{B{B}_{i,l}}{\varvec{s}}_{l}+{\varvec{z}}_{k}$$

(2)

$$\:{\varvec{y}}_{k}=\sum\limits_{i=1}^{N}({\varvec{H}}_{i,k}{\varvec{F}}_{R{F}_{i}}{\varvec{F}}_{B{B}_{i,k}}{\varvec{s}}_{k}+\sum\limits_{l\ne\:k}{\varvec{H}}_{i,k}{\varvec{F}}_{R{F}_{i}}{\varvec{F}}_{B{B}_{i,l}}{\varvec{s}}_{l})+{\varvec{z}}_{k}$$

(3)

Where $\:{\varvec{H}}_{i,k}$is the $\:P\times\:M$is channel matrix from the transmit antennas of each SBS to the $\:k$th user and $\:{\varvec{z}}_{k}\sim CN(0,{\sigma\:}^{2}{\mathbf{I}}_{\text{P}})$ represents the additive white Gaussian noise.

Channel model

The channel between the $\:i$-th SBS and $\:k$-th user is characterized by $\:{\mathbf{H}}_{i,k}=\sqrt{{\beta\:}_{i,k}}{\dot{\mathbf{H}}}_{i,k}$. The term $\:\sqrt{{\beta\:}_{i,k}}$ denotes the large-scale fading, while $\:{\dot{\mathbf{H}}}_{i,k}$ represents the fast fading. In this context, we are considering the mmWave channel with limited scattering and a finite number of propagation paths, denoted as$\:{L}_{k}$, where $\:k=\{\text{1,2},\dots\:,K\}$. This is defined⁵:

$$\:{\dot{\mathbf{H}}}_{i,k}=\sqrt{\frac{PM}{{L}_{k}}}\sum\limits_{l=1}^{{L}_{k}}{\alpha\:}_{i,k,l}{\varvec{a}}_{R}\left({\theta\:}_{i,k,l}\right){\varvec{a}}_{T}^{H}\left({\phi\:}_{i,k,l}\right)$$

(4)

The notation $\:{\alpha\:}_{i,k,l}$denotes the path fading coefficient associated with the $\:l$ -th path for the $\:k$-th user within the $\:i$-th SBS. This coefficient is characterized by a complex Gaussian distribution, exhibiting a mean of zero and a variance of one. Within the context of the $\:i$-th SBS, the angles of arrival and departure (AoA/AoD) for the $\:l$-th path corresponding to the $\:k$-th user are represented by $\:{\theta\:}_{i,k,l}$and $\:{\phi\:}_{i,k,l}$, respectively. Additionally, the receive and transmit array response vectors at these specified azimuth angles are denoted as$\:{\varvec{a}}_{R}\left({\theta\:}_{i,k,l}\right)$and $\:{\varvec{a}}_{T}\left({\phi\:}_{i,k,l}\right)$. It is assumed that all the values of $\:{\theta\:}_{i,k,l}$ and $\:{\phi\:}_{i,k,l}$ are uniformly distributed on the interval [0,2π), and the antennas in each user have a uniform linear array (ULA) arrangement. The formulations for $\:{\varvec{a}}_{R}\left({\theta\:}_{i,k,l}\right)$and $\:{\varvec{a}}_{T}\left({\phi\:}_{i,k,l}\right)$ are as follows:

$$\:{\varvec{a}}_{R}\left({\theta\:}_{i,k,l}\right)\:=\frac{1}{\sqrt{P}}{\left[1,{e}^{-\frac{j2\pi\:d}{\lambda\:}\text{s}\text{i}\text{n}\left({\theta\:}_{i,k,l}\right)},\dots\:,{e}^{-\frac{j2\pi\:\left(P-1\right)d}{\lambda\:}\text{s}\text{i}\text{n}\left({\theta\:}_{i,k,l}\right)}\right]}^{T}$$

(5)

$$\:{\varvec{a}}_{T}\left({\phi\:}_{i,k,l}\right)\:={\frac{1}{\sqrt{M}}\left[1,{e}^{-\frac{j2\pi\:d}{\lambda\:}\text{s}\text{i}\text{n}\left({\phi\:}_{i,k,l}\right)},\dots\:,{e}^{-\frac{j2\pi\:\left(M-1\right)d}{\lambda\:}\text{s}\text{i}\text{n}\left({\phi\:}_{i,k,l}\right)}\right]}^{T}$$

(6)

Where $\:=\sqrt{-1}$, $\:\lambda\:$ is the carrier wavelength and $\:d$ denotes the spacing between two neighbor antennas. In this paper, we assume $\:\lambda\:/2$for $\:d$.

It now explicitly connects ULA properties with channel orthogonality, referencing the theoretical foundation leading to zero interference. Here, λ represents the carrier wavelength, and d denotes the inter-element spacing between adjacent antennas in the array. In our model, we assume a uniform linear array (ULA) with half-wavelength spacing, i.e., d = λ/2, which facilitates simplified analysis and efficient beamforming in mmWave channels.

Proposition

In the context of multi-user MIMO in a Massive MIMO system with a large number of transmitter antennas in a ULA configuration, the different user channels exhibit orthogonality, leading to the elimination of interference. i.e. :

$$\:\underset{M\to\:\infty\:}{\text{lim}}\frac{1}{M}{\varvec{h}}_{a}\left(u\right){\varvec{h}}_{b}^{H}\left(v\right)=0fora\ne\:b\:,\:u\ne\:v$$

(7)

Where, $\:a$ and $\:b$ shows the different users.

we assume the transmitted symbols across users are orthogonal to simplify system modeling. This assumption holds true due to the use of different precoding vectors and time/frequency resource allocation per user. Furthermore, we prove mathematically (see Eqs. 7–17) that inter-user interference is negligible under large antenna regimes and uniform linear array configurations, which supports the assumption of orthogonality in practical distributed MIMO systems.

Proof

In this section, we are examining interference between users and aiming to demonstrate the independence of users from each other. As a result, for simplicity and without losing any generality, we will omit the index $\:i$ in Eq. (8) and focus on the channel from a specific SBS to all users. This approach can be extended to cover all SBSs.

If $\:{h}_{k}\left(w\right)$ denotes the channel vector of $\:w$-th antenna in $\:k$-th user for mmWave channel, the channel can be demonstrated as:

$$\:{\varvec{h}}_{k}\left(w\right)=\sqrt{\frac{M}{{L}_{k}}}\sum\limits_{l=1}^{{L}_{k}}\sqrt{{\beta\:}_{k}}{\alpha\:}_{k,l}{e}^{-j\left(w-1\right)\frac{2\pi\:d}{\lambda\:}\text{s}\text{i}\text{n}\left({\theta\:}_{k,l}\right)\:}{\varvec{a}}_{T}^{H}\left({\phi\:}_{k,l}\right)$$

(8)

By introducing$\:{F}_{{l}_{u},u}=\sqrt{{\beta\:}_{a}}{\alpha\:}_{a,{l}_{u}}{e}^{-j\left(u-1\right)\frac{2\pi\:d}{\lambda\:}\text{s}\text{i}\text{n}\left({\theta\:}_{a,{l}_{u}}\right)\:}$and $\:{F}_{{l}_{v},v}=\sqrt{{\beta\:}_{b}}{\alpha\:}_{b,{l}_{v}}{e}^{-j\left(v-1\right)\frac{2\pi\:d}{\lambda\:}\text{s}\text{i}\text{n}\left({\theta\:}_{b,{l}_{v}}\right)\:}$, we can express:

$$\begin{aligned}\:\frac{1}{M}{\varvec{h}}_{a}\left(u\right){\varvec{h}}_{b}^{H}\left(v\right)&=\frac{1}{\sqrt{{L}_{u}}}\sum\limits_{{l}_{u}=1}^{{L}_{u}}{F}_{{l}_{u},u}{\varvec{a}}_{T}^{H}\left({\phi\:}_{a,{l}_{u}}\right)\times\:\frac{1}{\sqrt{{L}_{v}}}\sum\limits_{{l}_{v}=1}^{{L}_{v}}{F}_{{l}_{v},v}^{*}{\varvec{a}}_{T}\left({\phi\:}_{b,{l}_{v}}\right)\\&=\sqrt{\frac{1}{{L}_{u}{L}_{v}}}\sum\limits_{{l}_{u}=1}^{{L}_{u}}\sum\limits_{lv=1}^{{L}_{v}}({F}_{{l}_{u},u}{F}_{{l}_{v},v}^{*}\times\:\frac{1}{M}\sum\:_{m=1}^{M}{e}^{-j\left(m-1\right)\left(\frac{2\pi\:d}{\lambda\:}\right)(\text{sin}\left({\phi\:}_{a,{l}_{u}}\right)-\text{sin}\left({\phi\:}_{b,{l}_{v}}\right))})\end{aligned}$$

(9)

To prove (7), we just need to demonstrate that the mean and variance of $\:\underset{M\to\:\infty\:}{\text{lim}}\frac{1}{M}{\varvec{h}}_{a}\left(u\right){\varvec{h}}_{b}^{H}\left(v\right)$ are both equal to zero.

Before computing the first and second moments of $\:\underset{M\to\:\infty\:}{\text{lim}}\frac{1}{M}{\varvec{h}}_{a}\left(u\right){\varvec{h}}_{b}^{H}\left(v\right)$, it is important to highlight that the expression $\:\frac{1}{M}{\varvec{h}}_{a}\left(u\right){\varvec{h}}_{b}^{H}\left(v\right)$ takes the form of $\:{e}^{\pm\:jxsin\gamma\:}$ where $\:\gamma\:$ is a random variable. If $\:\gamma\:$ is uniformly distributed between 0 and 2π, then:

$$\:{\mathbb{E}}_{\gamma\:}\left\{{e}^{\pm\:jxsin\gamma\:}\right\}=\frac{1}{2\pi\:}\int_{0}^{2\pi\:}{e}^{\pm\:jxsin\gamma\:}d\gamma\:={J}_{0}\left(x\right)$$

(10)

Where $\:{J}_{0}(.)$ in (10) shows the zero-order Bessel function of the first kind.

We know that all AoAs and AoDs are uniformly distributed between 0 and $\:2\pi\:$ and they are independent, therefore:

$$\begin{aligned}\:\mathbb{E}\left\{\frac{1}{M}{\varvec{h}}_{a}\left(u\right){\varvec{h}}_{b}^{H}\left(v\right)\right\}\\&=\sqrt{\frac{1}{{L}_{u}{L}_{v}}}\sum\limits_{{l}_{u}=1}^{{L}_{u}}\sum\limits_{lv=1}^{{L}_{v}}\mathbb{E}\left\{\sqrt{{\beta\:}_{a}{\beta\:}_{b}^{*}}{\alpha\:}_{a,{l}_{u}}{\alpha\:}_{b,{l}_{v}}^{*}\right\}\\&\times\:{J}_{0}\left({x}_{u}\right){J}_{0}\left({x}_{v}\right)\frac{1}{M}\sum\limits_{m=1}^{M}{J}_{0}{\left({x}_{m}\right)}^{2}\end{aligned}$$

(11)

$$\:\text{W}\text{h}\text{e}\text{r}\text{e}\:{x}_{u}=\frac{2\pi\:d}{\lambda\:}(u-1),\:{x}_{v}=\frac{2\pi\:d}{\lambda\:}(v-1),\:\text{a}\text{n}\text{d}\:{x}_{m}=\frac{2\pi\:d}{\lambda\:}(m-1).$$

.

Since the magnitude of $\:{J}_{0}\left(z\right)$ decays proportional to $\:\frac{1}{\sqrt{z}}$, therefore:

$$\:\underset{M\to\:\infty\:}{\text{lim}}\frac{1}{M}\sum\limits_{m=1}^{M}{J}_{0}{\left({x}_{m}\right)}^{2}=0$$

(12)

In result, it can be obtained:

$$\:{\mathbb{E}}_{\gamma\:}\left\{\underset{M\to\:\infty\:}{\text{lim}}\frac{1}{M}{\varvec{h}}_{a}\left(u\right){\varvec{h}}_{b}^{H}\left(v\right)\right\}=0$$

(13)

We know that the variance of variable $\:X$is $\:ar\left(X\right)={\mathbb{E}}_{X}\left\{{\left(X-\mathbb{E}\left(X\right)\right)}^{2}\right\}$, therefore according to (7):

$$\:Var\left\{\frac{1}{M}{\varvec{h}}_{a}\left(u\right){\varvec{h}}_{b}^{H}\left(v\right)\right\}=\mathbb{E}\left\{\frac{1}{{M}^{2}}{\left|{\varvec{h}}_{a}\left(u\right){\varvec{h}}_{b}^{H}\left(v\right)\right|}^{2}\right\}\:\:\:\text{f}\text{o}\text{r}\:M\to\:\infty\:$$

(14)

In additions,

$$\begin{aligned}\:\mathbb{E}\left\{\frac{1}{{M}^{2}}{\left|{\varvec{h}}_{a}\left(u\right){\varvec{h}}_{b}^{H}\left(v\right)\right|}^{2}\right\}\\&=\sqrt{\frac{1}{{L}_{u}{L}_{v}}}\sum\:_{{l}_{u}=1}^{{L}_{u}}\sum\:_{lv=1}^{{L}_{v}}\mathbb{E}\left\{{\left|\sqrt{{\beta\:}_{a}{\beta\:}_{b}^{*}}{\alpha\:}_{a,{l}_{u}}{\alpha\:}_{b,{l}_{v}}^{*}\right|}^{2}\right\}\times\:\frac{1}{{M}^{2}}\sum\:_{m=1}^{M}\sum\:_{m{\prime\:}=1}^{M}{J}_{0}{\left({x}_{m,m{\prime\:}}\right)}^{2}\end{aligned}$$

(15)

Where $\:{x}_{m,{m}^{{\prime\:}}}=\frac{2\pi\:d}{\lambda\:}(m-{m}^{{\prime\:}})$ and $\:C\to\:0$if $\:M\to\:\infty\:$. So, by substituting (15) into (14):

$$\:Var\left\{\frac{1}{M}{\varvec{h}}_{a}\left(u\right){\varvec{h}}_{b}^{H}\left(v\right)\right\}=O\left(\frac{1}{M}\right)$$

(16)

Where $\:O\left(x\right)$ shows that the value is proportional to $\:x$. Therefore:

$$\:Var\left\{\underset{M\to\:\infty\:}{\text{lim}}\frac{1}{M}{\varvec{h}}_{a}\left(u\right){\varvec{h}}_{b}^{H}\left(v\right)\right\}=0$$

(17)

Spectral efficiency

This section assesses the spectral efficiency of the proposed framework. As demonstrated in the previous part, the interference among users is proven to be zero. Therefore, the total spectral efficiency of the user $\:k$ in the distributed system proposed is given by:

$$\:{R}_{k}={\text{log}}_{2}\left|{\varvec{I}}_{P}+\frac{\sum\:_{i=1}^{N}{\varvec{H}}_{i,k}{\varvec{F}}_{R{F}_{i}}{\varvec{F}}_{B{B}_{i}}{\varvec{F}}_{B{B}_{i}}^{H}{{\varvec{F}}_{R{F}_{i}}^{H}\varvec{H}}_{i,k}^{H}}{{\sigma\:}^{2}}\right|$$

(18)

The total spectral efficiency for all users from all SBSs in the suggested system, denoted as $\:{R}_{total}$, is calculated by adding up the spectral efficiencies of individual users from all SBSs in the system.

$$\:{R}_{total}=\sum\limits_{k=1}^{K}{\text{log}}_{2}\left|{\varvec{I}}_{P}+\frac{\sum\:_{i=1}^{N}{\varvec{H}}_{i,k}{\varvec{F}}_{R{F}_{i}}{\varvec{F}}_{B{B}_{i}}{\varvec{F}}_{B{B}_{i}}^{H}{{\varvec{F}}_{R{F}_{i}}^{H}\varvec{H}}_{i,k}^{H}}{{\sigma\:}^{2}}\right|$$

(19)

The noise variance, denoted as $\:{\sigma\:}^{2}$, is assumed to be constant in all paths, and the entity matrix is represented by $\:{I}_{P}$.

Spectral efficiency optimizationproblem definition

This study seeks to enhance the total spectral efficiencywhile adhering to analog RF precoding and total power constraints. Our goal is to derive the best hybrid precoding (analog RF and digital) matrices at SBSs by solving the specified optimization problem:

$$\:\underset{{\varvec{F}}_{B{B}_{i}},{\varvec{F}}_{R{F}_{i}}}{\text{maximize}}{R}_{total}$$

(20a)

$$\:s.t\::\:\:\:\:\:\:Tr\left({\varvec{F}}_{R{F}_{i}}{\varvec{F}}_{B{B}_{i}}{\varvec{F}}_{B{B}_{i}}^{H}{\varvec{F}}_{R{F}_{i}}^{H}\right)\le\:{P}_{T}$$

(20b)

$$\:{\varvec{F}}_{R{F}_{i}}{\varvec{F}}_{R{F}_{i}}^{H}={\varvec{I}}_{M}$$

(20c)

Where, $\:{P}_{T}$is the total power budget and $\:{\varvec{I}}_{M}$and $\:{\varvec{I}}_{P}$ are entity matrices.The system transmit power limitations are represented by the first constraint, while the second constraint is satisfied through the precoding sections.

Solving the problem

The optimization problem in the previous Sect. (20) deals with hybrid precoders at the transmitter. However, addressing this design problem for analog/digital matrices is extremely challenging⁴. Furthermore, the nonconvex constraint (20b) complicates the problem by making it non-convex. As a result, we propose a new two-stage approach to tackle this issue. First, we consider the analog RF precoding matrix to be identifiedand concentrate on finding the optimal digital precoding matrix. Subsequently, we use the optimal digital precoding matrix to obtain the optimal RF precoding matrix. This procedure is illustrated in Fig. 3.

Solving the digital precoder problem

We assume that the analog RF precoder,$\:{\varvec{F}}_{R{F}_{i}}$, is known and then compute the digital precoder,$\:{\varvec{F}}_{B{B}_{i}}$. By considering this assumption, we can say that $\:{\varvec{H}}_{i,k}^{e}={\varvec{H}}_{i,k}{\varvec{F}}_{R{F}_{i}}$and also the problem (20) is changed as:

$$\:\underset{{\varvec{F}}_{B{B}_{i}}}{\text{m}\text{a}\text{x}\text{i}\text{m}\text{i}\text{z}\text{e}}\sum\limits_{k=1}^{K}{\text{log}}_{2}\left|{\varvec{I}}_{P}+\frac{\sum\:_{i=1}^{N}{\varvec{H}}_{i,k}^{e}{\varvec{F}}_{B{B}_{i}}{\varvec{F}}_{B{B}_{i}}^{H}{\left({\varvec{H}}_{i,k}^{e}\right)}^{H}}{{\sigma\:}^{2}}\right|$$

(21a)

$$\:subjectto:\:\:\:\:\:\:Tr\left({\varvec{W}}_{i}{\varvec{F}}_{B{B}_{i}}{\varvec{F}}_{B{B}_{i}}^{H}\right)\le\:{P}_{T}$$

(21b)

In above equations, we define $\:{\varvec{W}}_{i}={{\varvec{F}}_{R{F}_{i}}^{H}\varvec{F}}_{R{F}_{i}}$ to simplify the problem (20).

For solving the above problem, we utilize the Lagrange multiplier method.

$$\:\mathcal{L}\left({{\varvec{F}}_{BB}}_{i},\lambda\:\right)=\sum\limits_{k=1}^{K}{\text{log}}_{2}\left|{\varvec{I}}_{P}+\frac{\sum\:_{i=1}^{N}{\varvec{H}}_{i,k}^{e}{\varvec{F}}_{B{B}_{i}}{{\varvec{F}}_{B{B}_{i}}^{H}\left({\varvec{H}}_{i,k}^{e}\right)}^{H}}{{\sigma\:}^{2}}\right|+\lambda\:(\:Tr\left({\varvec{W}}_{i}{\varvec{F}}_{B{B}_{i}}{\varvec{F}}_{B{B}_{i}}^{H}\right)-{P}_{T})$$

(22)

According KKT conditions, we have:

$$\:\frac{\partial\:\mathcal{L}\left({{\varvec{F}}_{BB}}_{i},\lambda\:\right)}{\partial\:{{\varvec{F}}_{BB}}_{i}}=0$$

(23)

$$\:\frac{\partial\:\mathcal{L}\left({{\varvec{F}}_{BB}}_{i},\lambda\:\right)}{\partial\:\lambda\:}=0$$

(24)

Therefore, the (23) can be calculated as:

$$\begin{aligned}\:\frac{\partial\:\mathcal{L}\left({{\varvec{F}}_{BB}}_{i},\lambda\:\right)}{\partial\:{{\varvec{F}}_{BB}}_{i}}=&\stackrel{\text{I}}{\overbrace{\frac{\partial\:\left(\sum\:_{k=1}^{K}{\text{log}}_{2}\left|{\varvec{I}}_{P}+\frac{\sum\:_{i=1}^{N}{\varvec{H}}_{i,k}^{e}{\varvec{F}}_{B{B}_{i}}{{\varvec{F}}_{B{B}_{i}}^{H}\left({\varvec{H}}_{i,k}^{e}\right)}^{H}}{{\sigma\:}^{2}}\right|\right)}{\partial\:{{\varvec{F}}_{BB}}_{i}}}{}}\\ & +\underset{\text{I}\text{I}}{\underbrace{\frac{\partial\:\lambda\:(\:Tr\left({\varvec{W}}_{i}{\varvec{F}}_{B{B}_{i}}{\varvec{F}}_{B{B}_{i}}^{H}\right)-{P}_{T}))}{\partial\:{{\varvec{F}}_{BB}}_{i}}}}=0\end{aligned}$$

(25)

Therefore, we can compute (I) in equation (25) as:

$$\begin{aligned}\:\text{I}=\sum\limits_{k=1}^{K}\frac{\partial\:}{\partial\:{{\varvec{F}}_{BB}}_{i}}{\text{log}}_{2}&\left|{\varvec{I}}_{P}+\frac{\sum\limits_{i=1}^{N}{\varvec{H}}_{i,k}^{e}{\varvec{F}}_{B{B}_{i}}{{\varvec{F}}_{B{B}_{i}}^{H}\left({\varvec{H}}_{i,k}^{e}\right)}^{H}}{{\sigma\:}^{2}}\right|\\&=\sum\limits_{k=1}^{K}\frac{\partial\:}{\partial\:{{\varvec{F}}_{BB}}_{i}}{\text{log}}_{2}\left(\frac{1}{{\sigma\:}^{2}}\right)+\:\frac{\partial\:}{\partial\:{{\varvec{F}}_{BB}}_{i}}{\text{log}}_{2}\left|{\sigma\:}^{2}{\varvec{I}}_{P}+\sum\:_{i=1}^{N}{\varvec{H}}_{i,k}^{e}{\varvec{F}}_{B{B}_{i}}{{\varvec{F}}_{B{B}_{i}}^{H}\left({\varvec{H}}_{i,k}^{e}\right)}^{H}\right|\\&=\:\sum\limits_{k=1}^{K}\frac{\partial\:}{\partial\:{{\varvec{F}}_{BB}}_{i}}{\text{log}}_{2}\left|{\sigma\:}^{2}{\varvec{I}}_{P}+\sum\limits_{i=1}^{N}{\varvec{H}}_{i,k}^{e}{\varvec{F}}_{B{B}_{i}}{{\varvec{F}}_{B{B}_{i}}^{H}\left({\varvec{H}}_{i,k}^{e}\right)}^{H}\right|\end{aligned}$$

(26)

By knowing $\:\text{L}\text{o}{\text{g}}_{2}\left(x\right)=\frac{ln\left(x\right)}{ln2}$, the equation (26) is changed as:

$$\begin{aligned}\:\frac{\partial\:}{\partial\:{\varvec{F}}_{B{B}_{i}}}\sum\limits_{k=1}^{K}{\text{log}}_{2}|{\sigma\:}^{2}{\mathbf{I}}_{\text{P}}&+\sum\limits_{i=1}^{N}{\varvec{H}}_{i,k}^{e}{\varvec{F}}_{B{B}_{i}}{{\varvec{F}}_{B{B}_{i}}^{H}\left({\varvec{H}}_{i,k}^{e}\right)}^{H}|\\&=\frac{1}{ln2}\sum\limits_{k=1}^{K}\frac{\partial\:}{\partial\:{\varvec{F}}_{B{B}_{i}}}({ln}|{\sigma\:}^{2}{\mathbf{I}}_{\text{P}}+\sum\limits_{i=1}^{N}{\varvec{H}}_{i,k}^{e}{\varvec{F}}_{B{B}_{i}}{{\varvec{F}}_{B{B}_{i}}^{H}\left({\varvec{H}}_{i,k}^{e}\right)}^{H}|)\end{aligned}$$

(27)

We know that $\:({\text{ln}\left(f\left(x\right)\right))}^{{\prime\:}}=\frac{{f}^{{\prime\:}}\left(x\right)}{f\left(x\right)}$ :

$$\begin{array}{lll}\:\frac{\partial\:}{\partial\:{\varvec{F}}_{B{B}_{i}}}({ln}|{\sigma\:}^{2}{\mathbf{I}}_{\text{P}}+\sum\:_{i=1}^{N}{\varvec{H}}_{i,k}^{e}{\varvec{F}}_{B{B}_{i}}{{\varvec{F}}_{B{B}_{i}}^{H}\left({\varvec{H}}_{i,k}^{e}\right)}^{H}|)\\=\frac{\frac{\partial\:}{\partial\:{\varvec{F}}_{B{B}_{i}}}|{\sigma\:}^{2}{\mathbf{I}}_{\text{P}}+\sum\:_{i=1}^{N}{\varvec{H}}_{i,k}^{e}{\varvec{F}}_{B{B}_{i}}{{\varvec{F}}_{B{B}_{i}}^{H}\left({\varvec{H}}_{i,k}^{e}\right)}^{H}|}{|{\sigma\:}^{2}{\mathbf{I}}_{\text{P}}+\sum\:_{i=1}^{N}{\varvec{H}}_{i,k}^{e}{\varvec{F}}_{B{B}_{i}}{{\varvec{F}}_{B{B}_{i}}^{H}\left({\varvec{H}}_{i,k}^{e}\right)}^{H}|}\\=\frac{\left|{\sigma\:}^{2}{\mathbf{I}}_{\text{P}}+\sum\:_{i=1}^{N}{\varvec{H}}_{i,k}^{e}{\varvec{F}}_{B{B}_{i}}{{\varvec{F}}_{B{B}_{i}}^{H}\left({\varvec{H}}_{i,k}^{e}\right)}^{H}\right|\:2\sum\:_{i=1}^{N}{\left({\mathbf{H}}_{i,k}^{e}\right)}^{H}{\mathbf{H}}_{i,k}^{e}{\varvec{F}}_{B{B}_{i}}}{|{\sigma\:}^{2}{\mathbf{I}}_{\text{P}}+\sum\:_{i=1}^{N}{\varvec{H}}_{i,k}^{e}{\varvec{F}}_{B{B}_{i}}{{\varvec{F}}_{B{B}_{i}}^{H}\left({\varvec{H}}_{i,k}^{e}\right)}^{H}|}=2\sum\limits_{i=1}^{N}{\left({\mathbf{H}}_{i,k}^{e}\right)}^{H}{\mathbf{H}}_{i,k}^{e}{\varvec{F}}_{B{B}_{i}}\end{array}$$

(28)

Therefore:

$$\begin{array}{ll}\:\frac{1}{ln2}\sum\limits_{k=1}^{K}\frac{\partial\:}{\partial\:{\varvec{F}}_{B{B}_{i}}}({ln}|{\sigma\:}^{2}{\mathbf{I}}_{\text{P}}+\sum\limits_{i=1}^{N}{\varvec{H}}_{i,k}^{e}{\varvec{F}}_{B{B}_{i}}{{\varvec{F}}_{B{B}_{i}}^{H}\left({\varvec{H}}_{i,k}^{e}\right)}^{H}|)\\=\frac{2}{ln2}\sum\limits_{k=1}^{K}\sum\limits_{i=1}^{N}{\left({\mathbf{H}}_{i,k}^{e}\right)}^{H}{\mathbf{H}}_{i,k}^{e}{\varvec{F}}_{B{B}_{i}}\end{array}$$

(29)

According to $\:\frac{\partial\:}{\partial\:\varvec{X}}Tr\left(\varvec{B}\varvec{X}{\varvec{X}}^{T}\right)=\left(\varvec{B}+{\varvec{B}}^{T}\right)\varvec{X}$, the second part in equation (25), II, is calculated as:

$$\:\text{I}\text{I}=\frac{\partial\:\lambda\:(\:Tr\left({\varvec{W}}_{i}{\varvec{F}}_{B{B}_{i}}{\varvec{F}}_{B{B}_{i}}^{H}\right)-{P}_{T}))}{\partial\:{{\varvec{F}}_{BB}}_{i}}=\lambda\:\left({\varvec{W}}_{i}+{\varvec{W}}_{i}^{H}\right){\varvec{F}}_{B{B}_{i}}$$

(30)

So, according to KKT conditions and equations (29) and (30):

$$\begin{array}{ll}\:\frac{\partial\:\mathcal{L}({\varvec{F}}_{B{B}_{i}},\lambda\:)}{\partial\:{\varvec{F}}_{B{B}_{i}}}=0\Rightarrow\:\\\frac{\partial\:\mathcal{L}({\varvec{F}}_{B{B}_{i}},\lambda\:)}{\partial\:{\varvec{F}}_{B{B}_{i}}}=\frac{2}{ln2}\sum\:_{k=1}^{K}\sum\:_{i=1}^{N}{\left({\mathbf{H}}_{i,k}^{e}\right)}^{H}{\mathbf{H}}_{i,k}^{e}{\varvec{F}}_{B{B}_{i}}+\lambda\:\left({\varvec{W}}_{i}+{\varvec{W}}_{i}^{H}\right){\varvec{F}}_{B{B}_{i}}=0\end{array}$$

(31)

According to (30), we can conclude that :

$$\:{\varvec{F}}_{B{B}_{i}}^{*}=\frac{-2}{\lambda\:ln2}{\left({\varvec{W}}_{i}+{\varvec{W}}_{i}^{H}\right)}^{-1}(\sum\limits_{k=1}^{K}\sum\limits_{i=1}^{N}{\left({\mathbf{H}}_{i,k}^{e}\right)}^{H}{\mathbf{H}}_{i,k}^{e}{\varvec{F}}_{B{B}_{i}})$$

(32)

And also, based on KKT conditions:

$$\begin{array}{ll}\:\frac{\partial\:\mathcal{L}({\varvec{F}}_{B{B}_{i}},\lambda\:)}{\partial\:\lambda\:}=0\Rightarrow\:\\\frac{\partial\:\mathcal{L}({\varvec{F}}_{B{B}_{i}},\lambda\:)}{\partial\:\lambda\:}=Tr\left({\varvec{W}}_{i}{\varvec{F}}_{B{B}_{i}}{\varvec{F}}_{B{B}_{i}}^{H}\right)-{P}_{T}=0\Rightarrow\:Tr\left({\varvec{W}}_{i}{\varvec{F}}_{B{B}_{i}}{\varvec{F}}_{B{B}_{i}}^{H}\right)={P}_{T}\end{array}$$

(33)

Now, we should investigate the conditions and choose the suitable case for the parameter:

$$\:if\lambda\:\left(Tr\left({\varvec{W}}_{i}{\varvec{F}}_{B{B}_{i}}{\varvec{F}}_{B{B}_{i}}^{H}\right)-{P}_{T}\right)=0\Rightarrow\:\left\{\begin{array}{c}\lambda\:=0\:\:\:\:\:\:\:Tr\left({\varvec{W}}_{i}{\varvec{F}}_{B{B}_{i}}{\varvec{F}}_{B{B}_{i}}^{H}\right)<{P}_{T}\:\:\:\:\:\left(A\right)\\\:\lambda\:>0\:\:\:\:\:\:\:Tr\left({\varvec{W}}_{i}{\varvec{F}}_{B{B}_{i}}{\varvec{F}}_{B{B}_{i}}^{H}\right)={P}_{T}\:\:\:\left(B\right)\end{array}\right.$$

(34)

In equation (32), the $\:\lambda\:$ could not be zero, because if $\:\lambda\:=0\:$, $\:{\varvec{F}}_{B{B}_{i}}^{*}=\infty\:$. So, the condition (B) in equation (34), $\:>0\:\:\:\:Tr\left({\varvec{W}}_{i}{\varvec{F}}_{B{B}_{i}}{\varvec{F}}_{B{B}_{i}}^{H}\right)={P}_{T}$, is satisfied.

In result, our optimization problem is changed as:

$$\:\left\{\begin{array}{c}{\varvec{F}}_{B{B}_{i}}^{*}=\frac{-2}{\lambda\:ln2}{\left({\varvec{W}}_{i}+{\varvec{W}}_{i}^{H}\right)}^{-1}\left(\sum\limits_{k=1}^{K}\sum\limits_{i=1}^{N}{\left({\mathbf{H}}_{i,k}^{e}\right)}^{H}{\mathbf{H}}_{i,k}^{e}{\varvec{F}}_{B{B}_{i}}\right)\\\:Tr\left({\varvec{W}}_{i}{\varvec{F}}_{B{B}_{i}}{\varvec{F}}_{B{B}_{i}}^{H}\right)={P}_{T}\end{array}\right.$$

(35)

For solving the problem equation (35), we use the proposed iterative algorithm 1 as follows. Note that $\:{\varvec{F}}_{B{B}_{i}}^{*}$ shows the optimal $\:{\varvec{F}}_{B{B}_{i}}$.

In the first step of Algorithm 1, we start by setting $\:{\lambda\:}_{min}$and $\:{\lambda\:}_{max}$, and then calculate $\:\lambda\:$ as $\:\lambda\:=\frac{{\lambda\:}_{max}+{\lambda\:}_{min}}{2}$. Using this initial $\:\lambda\:$, we compute $\:{\varvec{F}}_{B{B}_{i}}^{*}$ for each SBS and user. If $\:Tr\left({\varvec{W}}_{i}{\varvec{F}}_{B{B}_{i}}{\varvec{F}}_{B{B}_{i}}^{H}\right)={P}_{T}$ holds true for $\:{\varvec{F}}_{B{B}_{i}}^{*}$, we update $\:{\lambda\:}_{min}$ to $\:\lambda\:$ and keep $\:{\lambda\:}_{max}$ as is, then calculate the new $\:\lambda\:$. If the stopping condition abs ($\:{\lambda\:}_{min}-{\lambda\:}_{max})>{\upepsilon\:}$ is met, the algorithm stops and we obtain the optimal digital precoding matrix. If not, we repeat the algorithm until the optimal $\:{\varvec{F}}_{B{B}_{i}}$ is achieved.

The proposed scheme relies on the joint optimization of both digital and analog precoders to approach the performance of fully digital systems. Removing either component has a significant performance impact.Removing either component negatively impacts performance. Analog-only schemes lack flexibility and precision, while digital-only schemes are hardware intensive at mmWave. Figure 6 demonstrates that analog beamforming performs worse than hybrid schemes, confirming that the joint use of analog and digital components is essential for achieving high spectral efficiency at reduced complexity.

Algorithm 1 The pseudo-code of the proposed iterative algorithm for finding $\:{\varvec{F}}_{\varvec{B}{\varvec{B}}_{\varvec{i}}}$.

When determining the complexity of the proposed algorithm. It should be emphasized thatin the distributed case, the complexity is $\:\frac{1}{N}$ of the collocated case, where $\:N$ represents the number of SBS. This is because the antennas are divided among $\:N$ separate SBSs, making our proposed scheme less complex than other collocated schemes. However, when comparing with other algorithms without considering the BS configurations, we can calculate the complexity for one SBS and compare it with other algorithms. In this particular scenario, the complexity is $\:{\rm\:O}\:(M\times\:{N}_{Sym}\times\:P\times\:K)$. Thus, if we set$\:K=1$, the complexity becomes $\:{\rm\:O}\:(M\times\:{N}_{Sym}\times\:P)$, which is similar to the complexity in¹³. Similarly, if we consider a multi-user scenario with $\:P=1$, the complexity becomes $\:{\rm\:O}\:(M\times\:{N}_{Sym}\times\:K)$, which is also similar to¹³. Thus, in all the investigated cases, the proposed iterative algorithm exhibits low complexity.

we provide a high-level analysis of the computational complexity of the proposed two-stage iterative algorithm. To elaborate further, for one SBS, the complexity is shown to be $\:O\left({K}^{2}{N}_{RF}^{2}\right)$, which aligns with the order of complexity found in comparable schemes like the hybrid digital-analog precoding proposed in¹³. Unlike collocated systems, the distributed setup reduces the per-SBS computational load, as the complexity scales with the number of SBSs M as 1/M of the collocated case. In future revisions, we will include runtime (in seconds) comparisons with PE-AltMin, OMP, and analog beamforming across the same simulation setup to provide a clearer benchmark.

Simulation results

We will assess the performance of the proposed hybrid precoding algorithm under various conditions. We will focus on a ULA configuration with antennas spaced at half the wavelength distance. Our evaluations will be conducted for a distributed massive multi-user MIMO system operating in the mmWave wireless communication channel. All simulations are executed by MATLAB software, and the results will be based on an average of 100 channel realizations. For the mmWave channel model, we will set $\:{L}_{k}=3$. The AoAsand AoDs will follow a uniform distribution within the range [0,2π]. We will use $\:\epsilon\:={10}^{-6}$for our proposed algorithm. In most of our simulations, we will set $\:N=4$, $\:M=64$, $\:K=6$, $\:P=16$, and $\:N\_RF={N}_{sym}or\:2*{N}_{sym}$. However, we will adjust certain parameters for each figure based on the conditions and provide details in the figure’s description. We will assume a noise variance of $\:{\sigma\:}^{2}=1$ for all paths, and SNR will be calculated as $\:\frac{{P}_{T}}{{\sigma\:}^{2}}$.

While our current study focuses on simulation-based evaluation using MATLAB under various system configurations and channel realizations, we recognize the importance of real-world validation. At this stage, the proposed scheme has not been tested on a physical mmWave testbed.

A comparison of spectral efficiency at various SNRs in the distributed and collocated massive MIMO setup is presented in Fig. 2. As indicated in¹⁵, it is evident that the distributed massive MIMO exhibits superior spectral efficiency, particularly when there are high quantities of antennas. In other words, astransmitter and user antenna counts grow, the spectral efficiency in a distributed scheme increases at a steeper rate.

Fig. 2

Comparing the spectral efficiency of collocated and distributed massive multi-user MIMO.

In Fig. 3, the spectral efficiency is illustrated for various SNR values, with different$\:{N}_{sym}$values being considered. The results demonstrate that the spectral efficiency increases as the SNR rises in every cases, indicating that the proposed scheme closely approaches the performance of a fullydigital structure. This supports the assertion that the proposed hybrid precoding methoddemonstrates superior performance while maintaining reduced complexity incomparison to a fully digital system.Furthermore, by increasing the $\:{N}_{sym}$ values, the system’s performance is enhanced. We consider the 2,4, and 8 for $\:{N}_{sym}$ in this scenario. Wesee that the $\:{N}_{sym}=8$ has better performance than others. Note that in this figure, we assume $\:{N}_{RF}=2*{N}_{sym}$, which in¹³ shows the hybrid precoding scheme can have close performance to fully digital.

Fig. 3

The spectral efficiency of the proposed hybrid precoding scheme in different SNRs with $\:{N}_{RF}=2*Nsym$.

Figure 4 analyzes how the quantity of $\:{N}_{RF}$ influences the proposed system. The findings indicate that in our proposed structure, the spectral efficiency is closefor both $\:{N}_{RF}=2*{N}_{sym}$and $\:{N}_{RF}={N}_{sym}$, with both approaching the performance of the fully digital case. This shows that our proposed hybrid precoding design can achieve optimal results with reduced complexity. These results hold for various $\:{N}_{sym}$ values, consistently yielding favorable outcomes.

Fig. 4

The comparison of spectral efficiency in the proposed hybrid precoding scheme in two cases:$\:{N}_{RF}=2*Nsym$and $\:{N}_{RF}={N}_{sym}$.

Fig. 5

The comparison of spectral efficiency in the proposed hybrid precoding scheme in different $\:{N}_{RF}$.

In Fig. 5, we demonstrated that the spectral performance for both $\:{N}_{RF}={N}_{sym}$ and $\:{N}_{RF}=2*{N}_{sym}$ are close, However, Fig. 5 shows that the performance slightly improves when $\:{N}_{RF}$ is increased from $\:{N}_{sym}$to $\:2*{N}_{sym}$, and it becomes closer to a fully digital structure at $\:2*{N}_{sym}$. It should be noted that in this figure, $\:{N}_{sym}$ is set to 4 and SNR=0dB, and we are comparing thespectral efficiency of the suggested hybrid precoding frameworkfor $\:{N}_{RF}=5\:$to $\:{N}_{RF}=8$.

Fig. 6

The comparison of spectral efficiency in the proposed hybrid precoding algorithm with some other existing algorithms.

Figure 6 shows a comparison between the newly proposed hybrid precoding algorithm and other algorithms currently in use.We compare it with PE-Altmin¹¹, OMP²¹, analog beamforming, and fully-digital algorithms. The figure illustrates that the proposed algorithm exhibits superior spectral efficiency performancethan others although the PE-Altmin and OMP have close performance and are better than analog beamforming. In addition, the performance of the proposed algorithm is very near to a fully-digital structure. This figure also depicted thatthe proposed distributed case outperforms the collocated scheme.

Fig. 7

The spectral efficiency in the proposed hybrid precoding algorithm in different quantities of users.

Figure 7 examines how the spectral efficiency of the proposed hybrid precoding is affected by the quantity of users. This shows that the performance is improved by increasing the number of users. This increment is higher if more transmit symbols are sent. In other words, we examine the proposed scheme in different $\:{N}_{sym}$, $\:{N}_{sym}=\text{1,2},and\:4$, and in SNR=0dB, and we see that the spectral efficiency performance in $\:{N}_{sym}=4$ is higher than others, especially when the number of users is increasing. This proves that in the high-scale system, the interference between users is eliminated and the performance is increased. The Table 1 shows the comparison between proposed and existing methods.

Table 1 Comparison table of the proposed method relative to existing techniques.

Conclusion

We conducted a study to evaluate the effectiveness of a two-stage iterative fullyconnected hybrid precoding technique in distributed massive MU MIMO systems with a focus on enhancing the spectral efficiency in modern wireless communication networks. The incorporation of hybrid precoding methods, which leverage analog and digital processing, offers a potential remedyto address the challenges posed by high-dimensional data and limited hardware resources. Our study focuses on a distributed massive multi-user MIMO system model with the BS divided into multiple SBSs to enhance spatial diversity and signal quality, especially in urban environments with potential multipath propagation issues. Each SBS is capable of serving users within its proximity to minimize interference and improve throughput. We specifically apply this system model in the mmWave channel and demonstrate that with a substantial increase in the number of antennas in SBS, interference among users can be eradicated. Our primary objective is to maximize spectral efficiency in this system model, leading us to formulate an optimization problem with the total spectral efficiency as the objective function and two constraints on the analog RFprecoding matrix and the total transmit power budget. Due to the nonconvex nature of this problem, we propose a two-stage iterative algorithm to solve it.