Continuous-Time Channel Prediction Based on Tensor Neural Ordinary Differential Equation

Mingyao Cui ,Hao Jiang ,Yuhao Chen ,Yang Du ,Linglong Dai,*

1 The Department of Electronic Engineering,Tsinghua University,Beijing 100084,China

2 Huawei Technology Co.,Ltd.Chengdu Research Institute,Chengdu 611730,China

Abstract: Channel prediction is critical to address the channel aging issue in mobile scenarios.Existing channel prediction techniques are mainly designed for discrete channel prediction,which can only predict the future channel in a fixed time slot per frame,while the other intra-frame channels are usually recovered by interpolation.However,these approaches suffer from a serious interpolation loss,especially for mobile millimeter-wave communications.To solve this challenging problem,we propose a tensor neural ordinary differential equation (TN-ODE)based continuous-time channel prediction scheme to realize the direct prediction of intra-frame channels.Specifically,inspired by the recently developed continuous mapping model named neural ODE in the field of machine learning,we first utilize the neural ODE model to predict future continuous-time channels.To improve the channel prediction accuracy and reduce computational complexity,we then propose the TN-ODE scheme to learn the structural characteristics of the high-dimensional channel by low-dimensional learnable transform.Simulation results show that the proposed scheme is able to achieve higher intra-frame channel prediction accuracy than existing schemes.

Keywords: channel prediction;massive multipleinput-multiple-output;millimeter-wave communications;ordinary differential equation

I.INTRODUCTION

Millimeter-wave (mmWave) massive multiple-input multiple-output (MIMO) has been a critical technology for boosting data transmission speed in 5G communication networks [1].By deploying a large number of antennas at the base station (BS),massive MIMO can achieve several orders of magnitude improvements in beamforming gain[2].To fully realize this potential,accurate channel state information(CSI)is required at the BS for the efficient design of precoding.According to the current 5G standard [3],each frame in 5G wireless communication systems contains multiple time slots,while only the first time slot of each frame is used to estimate the CSI through the predefined sounding reference signal(SRS).Then,the subsequent time slots within the same frame perform precoding design according to the CSI estimated in the first slot.

However,since the channel is time varying in mobile scenarios,the CSI in the first time slot may significantly differ from the actual channels in the subsequent time slots.This is called channel aging in the literature [4,5].Specifically,the channel coherence time is inversely proportional to the carrier frequency and user mobile speed,which could be shorter than the channel estimation period or SRS period in mobile scenarios.For example,for the case of 28 GHz carrier frequency and 60 km/h user mobile speed,the channel coherence time is about 0.32 ms,which is smaller than the shortest SRS period of 0.625 ms defined by the 5G standard[3].In this case,the estimated CSI in the first time slot becomes outdated,which could cause a serious spectral efficiency loss of about 30% [6].Therefore,the channel aging problem has to be carefully addressed to enable fast user mobility in mmWave massive MIMO systems.

1.1 Prior Works

To address the channel aging problem,channel prediction techniques have been widely studied to predict the future channels by exploring the channel correlation in the time domain [6–13].There are two typical categories of channel prediction techniques,i.e.,model-based and data-based channel prediction.For the first category [6–8],some classical models are utilized to characterize the time-varying channels,such as the linear extrapolation model [6],the autoregressive (AR) model [7],and the spatio-temporal auto-regressive (ST-AR) model [8].However,since the actual mobile channels simultaneously suffer from the multi-path effect and the Doppler effect,the timevarying characteristics of actual channels are complicated.Thus,for this category of channel prediction techniques,the fossilized models are difficult to match the time-varying channels,resulting in the unreliable performance in mobile scenarios.

To deal with this problem,data-based channel prediction techniques have been recently proposed to match the time-varying channels in the data-driven way [9–13].Since the neural network models are able to learn the intrinsic complicated feature from data,which could be exploited to improve the channel prediction accuracy.Specifically,in [9],a fullyconnected(FC)network was utilized to predict future channels according to the input of high-dimensional historical channels.Then,to decrease the training complexity caused by high-dimensional historical inputs,the recurrent neural network (RNN) like architectures,such as RNN,gate recurrent unit(GRU),and long-short term memory (LSTM),were trained to iteratively process historical channels[10–12].Furthermore,to avoid the prediction error propagation problem of the sequential prediction of future channels,the transformer model was used to predict future channels in parallel in[13].

However,the existing channel prediction techniques[6–13]were designed for discrete channel prediction,while they fail to directly predict the channels in all time slots of each frame.To be more specific,as we discussed before,the channels can only be estimated in the first time slot of each frame through the transmission of SRS.Based on these discretely estimated historical channels,the future channels with the same time interval are predicted by existing channel prediction techniques.Then,the channels in other time slots between two adjacent SRS could be recovered by using interpolation methods.Unfortunately,there exists a serious interpolation loss for these discrete channel prediction techniques in mobile scenarios.One possible solution is continuous-time channel prediction for all time slots of each frame.Unfortunately,to the best of our knowledge,none of the existing methods can achieve continuous-time channel prediction.

1.2 Our Contributions

To fill in this gap,we propose a tensor neural ordinary differential equation (TN-ODE) based continuoustime channel prediction scheme in this paper.Specifically,inspired by the recently developed continuoustime signal processing technology named neural ODE in the field of machine learning[14],we adopt the neural ODE architecture proposed in [14] to model the continuous-time channel prediction problem.In the above architecture,a GRU-based encoder is used to preprocess the discretely sampled historical channels,then a neural ODE-based decoder is used to predict future channels in consecutive time slots.Furthermore,to improve the channel prediction accuracy and reduce the computational complexity of the neural ODE,we propose the TN-ODE to exploit the structural characteristics of channels in multiple domains by a series of low-dimensional learnable transforms.To be more specific,in the antenna domain,the channel model is described by different angles of arrival(AoAs)and angles of departure (AoDs),while in the frequency domain,the channel model is mainly determined by multiple times of arrival(ToAs).Thanks to these structural characteristics,the proposed TN-ODE allows us to decouple the complicated high-dimensional channel prediction into efficient low-dimensional channel prediction in multiple domains.Simulation results show that the proposed TN-ODE based continuous-time channel prediction technique can effectively mitigate the interpolation loss and improve the channel prediction performance in all time slots of each frame.

1.3 Organization and Notation

The remainder of this paper is organized as follows.In Section II,the system model of the mmWave massive MIMO is introduced,and the continuous-time channel prediction problem in this system is then formulated.After that,we elaborate on the proposed TNODE based continuous-time channel prediction model in Section III.Section IV illustrates the simulation results.Finally,conclusions are drawn in Section V.

Notation:We denote the column vectoraand matrixAby boldface lower-case and upper-case letters,respectively;AT,AH,andA−1are the transpose,conjugate transpose,and inverse of the matrixA,respectively;A⊗Bis the Kronecker product of the matrixAand matrixB;A◦Bis the Hadamard product ofAandB;INdenotes an N ×N identity matrix.CN（µ,σ2）is the probability density function of the circularly symmetric complex Gaussian distribution with mean µ and variance σ2.E{·} denotes the statistical expectation.We use vec(A) to denote the vectorization of matrixA.σ(x)=and tanh(x)=represent the Sigmoid function and hyperbolic tangent function,respectively.We denote h[n],n ∈Z as a discrete-time sequence and h(t),t ∈R as a continuous-time sequence.

II.SYSTEM MODEL

In this section,we will first introduce the signal model of the mmWave massive MIMO system.Then,the continuous-time channel prediction is formulated to avoid the interpolation loss problem in existing discrete channel prediction schemes.

2.1 Signal Model

In this paper,we consider an uplink time division duplexing (TDD) based mmWave massive MIMO system with orthogonal frequency division multiplexing(OFDM).The base station(BS)equipped with an NTantenna uniform linear array (ULA) [15] serves an NR-antenna user with M subcarriers.To reduce energy consumption,hybrid precoding structure is employed in the BS[16],where the number of radio frequency (RF) chains is NRF,as indicated in Figure 1.According to the 5G standard [3],the time resources for communication are divided into frames and each frame consists of Q time slots.As shown in Figure 2,the Q time slots could be further divided into three parts,i.e.,uplink pilots,uplink data transmission,and downlink data transmission.For the q-th time slot,Nqsymbols are included and the channel remains unchanged during the Nqsymbols based on the blockfading assumption [17],where the channel remains time-invariant within each time slot and changes between different time slots.

Figure 1. Hybrid precoding for mmWave massive MIMO.

Figure 2. The 5G frame structure defined by 3GPP[3].

LetHm(t)∈denote the channel at the time t.Due to the limited number of scattering clusters in the mmWave propagation environment,we adopt the widely used geometric Saleh-Valenzuela multipath channel model[16]to characterize the mmWave channel.Under this model,Hm(t)can be denoted as

where L is the number of the paths,αl,vl,τl,ϕl,T,and ϕl,Rare the complex path gain,Doppler shift,ToA,AoA,and AoD of the l-th path,respectively.For m ∈{1,2,···,M},fm=f+denotes the m-th subcarrier frequency,with f,B,and M being the carrier frequency,bandwidth,and the number of subcarriers.Since the ULA is considered in this paper,the array steering vectoraT(ϕl,T)andaR(ϕl,R)could be represented by[18]

wherenT=[0,1,···,NT−1]TandnR=[0,1,···,NR−1]T,λ is the carrier wavelength,and d is the antenna spacing usually set as d=λ/2.

We denote Tfand Tsas the duration time of one frame and one time slot,where Tf=QTs.Accordingly,we can use=Hm(tp,q) to denote the channel at the q-th time slot of the p-th frame and the m-th subcarrier,where tp,q=pTf+qTs.Then,the received signalat the q-th time slot of the p-th frame and the m-th subcarrier in BS can be expressed by

We utilize the discrete Fourier transmission (DFT)codebook to design the analog combiningA(p,q)[19].In the DFT codebook,each codeword points to a specific azimuth AoA and all codewords will cover the entire beam space.By traversing all codewords,the strongest NRFcodewords could be selected to constructA(p,q).Benefiting from the fact that the timevarying channel is mainly caused by the Doppler effect,while the AoA and AoD are time-invariant in several frames during tens of milliseconds [20],the optimal combining matrix stays unchanged in several frames.In this case,we supposeA(p,q)=A,∀p∈{0,1,···,P−1},∀q∈{0,1,···,Q−1},where P is the number of frames in the order of tens of milliseconds.

In particular,when q=0,the effective channelof the first time slot in the p-th frame is estimated according to the predefined pilot sequenceand received signal.Generally,we use the least square (LS) channel estimation method to recover the effective channel,which could be represented by

We utilize the classical zero-forcing method[2]to design the digital precodingin the q-th time slot of the p-th frame and the m-th subcarrier as:

2.2 Problem Formulation

To mitigate the performance loss caused by channel aging,some channel prediction techniques[6–13]have been proposed to deal with the channel aging issue by exploring the temporal correlation of the timevarying channel.Specifically,the existing channel prediction schemes could predict future channels in discrete frames,i.e.,,based on the historical channelswith the same time interval.Since these channel prediction methods are designed for discrete channel prediction,which only predict the channel in the first time slot of each frame,they can not realize the direct prediction of the channels for all time slots in future frames.Thus,the interpolation method has to be utilized to recover the channelswith q ＞0 as

where k=0,1,···,K −1.However,due to the complicated change of the channel,simple interpolation is difficult to describe the actual change of the channel.Therefore,there is an interpolation loss for the existing discrete channel prediction schemes.

Unlike the existing discrete channel prediction schemes,we reformulate the channel prediction problem as a continuous-time channel mapping problem to avoid interpolation loss.Specifically,we utilize the historical discrete channels from the past J frames to predict the future continuous-time channels in the next K frames,which could be formulated as

where f(·) is the proposed continuous-time channel prediction model and θ is the parameters of the model.Since the normalized mean square error(NMSE)is not affected by the amplitude of the channel,we adopt the NMSE as the minimization target to realize stable convergence.It is worth noting that the estimated historical channels are discretely sampled at the first time slot of each frame.Correspondingly,the predicted channels are continuously distributed at any time slot of each future frame.By contrast,the existing discrete channel prediction schemes only predict the channel at the first time slot of the future frames.Thus,the proposed continuous-time channel prediction scheme realizes the direct prediction of the future channel in any time slot so that the interpolation loss can be avoided.

III.PROPOSED METHOD

In this section,we first introduce the background of neural ODE and elaborate on the framework of neural ODE based channel prediction.Then,we propose the TN-ODE to explore the mmWave channel structure to improve the channel prediction performance.

3.1 Background of Neural ODE

To achieve continuous-time channel prediction,it is crucial to find an appropriate technique to process continuous-time signals.Recently,with the rapid advancement in the field of dynamical systems,neural ODE becomes an attractive technology for modeling continuous-time sequences [14,21,22].Neural ODEs use first-order differential equations to fit the hidden state of time sequences,so it is capable of handling continuous-time signals.To make this paper self-contained,we provide a brief background of neural ODE.Specifically,classical RNN-like architectures,including RNN,GRU,and LSTM,build complicated networks to encode time sequences into a series of hidden states:

Here,h[n]represents the hidden state at the n-th discrete time,g(·)denotes the state transition function realized by neural networks,and θ is the network parameters.The transition in(10)is built on a discrete difference equation,which is awkward to deal with signals not belonging to discrete time samples.On the contrary,neural ODEs define a continuous-time hidden stateh(t),which can be formulated as a time-invariant differential equation:

Besides,(11) is equivalent to the following integral form:

Here,h(t0) is the initial hidden state,and function f(h(t),θ)describes the dynamic of hidden stateh(t).One can acquire the hidden stateh(t) at an arbitrary time t by solving problem(12)through an ODE solver:

As indicated in [21],such an ODE solver can be implemented by various numerical schemes,including the forward and backward Euler methods,the Runge-Kutta method,and the linear multi-step method.As a consequence,applying neural ODE models (12) and solvers (13) allows us to deal with continuous-time sequences,so as to achieve continuous-time channel prediction.

3.2 Framework of Neural ODE Based Channel Prediction

Based on the above background,the framework of neural ODE-based channel prediction is presented in this subsection.Our aim is to predict the channels for all time slots of the future K frames by processing those historical J channels.The Latent ODE architecture introduced in [22] is adopted to model this process.For expression clarity,we denoteand.

As shown in Figure 3,the neural ODE-based channel prediction is composed of two modules,i.e.,an encoder and a decoder [22].Generally speaking,the encoder is responsible for extracting features from the historical channelsfor n={0,−1,···,−J+1}.The output of the encoder serves as the initial state of the decoder.Correspondingly,the decoder exploits a neural ODE to infer future continuous-time channelsfor t ＞0.

Figure 3. The framework of neural ODE-based channel prediction.

Specifically,the encoder’s role is to extract the features from historical channels.Since the SRS signals are transmitted and received with equally-sized time interval Tf,RNN-like architectures are enough to deal with these sequences.We denote the hidden state ofasR[n].Then,based on the Markov property of RNN models,the map fromR[n−1]toR[n]can be written as

where EncoderCell(·)is the transition function of the RNN-like network with θEbeing the learnable parameters.

For the decoder,a neural ODE model is deployed to specific the dynamic of the future channel’s hidden state.This hidden state is defined asO(t).Besides,the final outputR[0]of the encoder is regarded as the initial stateO(0)of decoder.Therefore,for any times t ＞0,the hidden stateO(t)can be presented as

where DecoderCell(·) denotes the transition function of the neural ODE network with θDbeing its learnable parameters.Note that (15) can be solved by the ODESolver as presented in(13).After that,one layer neural network Pred(·)is built to output the predicted channelfrom the hidden stateO(t):

where θPdenotes its parameters.All in all,following this neural ODE framework,we are capable of extracting features from the previous channels and then predicting future continuous-time channels for any t.

3.3 TN-ODE Based Channel Prediction

In this subsection,we elaborate on the idea of tensor neural ODE for designing the three crucial transition functions: EncoderCell(·),DecoderCell(·),and Pred(·).

We commence our discussion by briefly introducing the transition functions widely used in classical neural ODE framework[22].The authors in[22]deployed a GRU model as its encoder transition function and modified the GRU model to act as the decoder transition function.To fit in our channel prediction framework,the inputs,hidden states,and outputs should be first vectorized as the following column vectors:,r[n]=vec(R[n]),o(t)=vec(O(t)),and.Then,according to the GRU architecture[22],EncoderCell(·)consists of the following modules:

where matrices{U,W}are the learnable parameters.As for the decoder,it is different from the encoder which can receive external stimulusto update its states.The hidden state transition of DecoderCell(·)is an auto-regressive process without external stimulus.Thereby,to fit GRU model in this decoder,we can carry out the steps below to modify GRU: removefrom (17a)-(17d);replacer[n −1] andr[n] witho(t) and,respectively.Finally,function Pred(·)can be realized by a fully connected layer,i.e.=Who(t).As a result,the entire neural ODE-based channel prediction is successfully established based on the classical GRU model.

There is no denying that the above transition functions have the ability to process continuous-time sequences.However,they will suffer from two serious problems when applied to channel prediction.First,these transition functions fail to exploit the underlying channel structure.As shown in(1),mmWave channels exhibit obvious correlations in multiple domains.For example,the antenna-domain channel is constructed by the superposition of multiple array steering vectors with different AoAs and AoDs.Besides,in the frequency domain,the channel structure can be captured by several ToAs.However,simply vectorizing channels to fit in the GRU model will undermine such regular structures.Second,the computational complexity of these transition functions is also unaffordable.Take the function Pred(·) as an example,we supposeWhis a square matrix.As the dimension ofis NRFNRM×1,then matrixWhwill containfloating points.If NRF=NR=4 and M=256,then the number of floating points ofWhis=33,554,432,which costs unacceptable computational resources.

As a consequence,our proposed TD-ODE takes advantage of the continuous-time signal processing capability of ODE and the multi-domain structure of mmWave channels,so it is promising to achieve efficient continuous-time channel prediction,which will be demonstrated in the simulation section.

3.4 Training and Testing Details

Based on this loss function,the Adam optimizer is adopted to update the network parameters using their gradients.Notice that adjoint sensitivities proposed in[14] are used to efficiently compute the ODE’s gradients.The above procedure is carried out batch by batch until convergence.

The data size in the testing stage is Ntest,where each channel sample is still divided into two periods.The first period is the same as that in the training stage.Regarding the second period,our target is to predict channels for future KQ time slots (or K frames).Therefore,we sample KQ slots with an equal time interval of Ts,which are denoted by ti=iTs,i=1,···,KQ.Then,we use the well-trained TNODE model to predict,i=1,···,KQ.Finally,these predicted channels are used for precoding.

3.5 Computational Complexity

In this subsection,we provide a detailed computational complexity analysis of the proposed scheme in the testing stage.Here,we mainly count the number of complex-valued multiplications.

For a sequence of historical channels,the total J channels are processed by the EncoderCell (18a)-(18d) sequentially.Steps (18a)-(18c) have a complexity in the order of O(FlNRFNRM +FlMFr),and step(18d)has a complexity of O(FlFr).Therefore,taking into account the J channels,the computational complexity of the encoder is O(JFlNRFNRM+JFlMFr)+O(JFlFr)=O(JFlNRFNRM +JFlMFr).

As for the decoder,we can similarly derive that the computational complexities of calculating the functions DecoderCell(·)and Pred(·)areand O(NRFNRFlFr+NRFNRFrM),respectively.Moreover,the ODESolver(·)in(13)needs to calculate the DecoderCell(·) for G times,where G is proportional to KQ.Therefore,the computational complexity of the decoder is.Finally,as KQ future channels are predicted,the overall number of complex-valued multiplications of the function Pred(·)is O(KQNRFNRFlFr+KQNRFNRFrM).

As a consequence,the computional complexity of the proposed TN-ODE model is

IV.SIMULATION RESULTS

In this section,simulation results are provided to demonstrate the superiority of our scheme.The CDLB channel model in the Matlab 5G toolbox[13]is utilized to generate the data set.For each channel sample,the velocity of user is randomly generated from the uniform distribution U(30 km/h,60 km/h)and the delay spread is randomly chosen from the uniform distribution U(50 ns,200 ns).The simulation configurations are presented in Table 1.The compared benchmarks are as follows: 1) the perfect CSI;2) the classical AI-based algorithms,including the GRU-based channel prediction[11]and the FC network based algorithm [9];3) the classical model-based techniques,including the prony-based angular-delay domain channel prediction(PAD)[6]and ST-AR[8]algorithms;4)utilizing the outdated channels without prediction.

Table 1. Simulation configurations.

In Figure 4,the average rate performance is evaluated.We follow the 5G standard and set Tfas 0.625 ms and Tsas 0.125 ms.Therefore,the classical GRU,FC,ST-AR,and PAD algorithms predict the channels at the 5-th and 10-th time slots,and then recover the channels at other time slots through linear interpolation.It is clear from Figure 4 that the average rate performance of classical algorithms degrades at the interpolated channels.Fortunately,our proposed scheme is able to avoid interpolation loss by predicting the future channels at all time slots with the assistance of TN-ODE.Additionally,the proposed TN-ODE exploits the multi-domain channel structure,so it can even achieve higher average rate than classical algorithms at the 5-th and 10-th time slots.

Figure 4. Average rate performance against time slots.

In Figure 5,the real part of the true future channels and the predicted channels for an arbitrary antenna index and subcarrier are presented.We can observe from this figure that the existing discrete-time channel prediction techniques can only accurately predict the future channels at SRS positions,while the interpolated channels considerably deviate from the true channels.On the contrary,the proposed TN-ODE scheme well captures the dynamic of continuous-time channels.The simulation result in Figure 6 further supports our discussion,where the NMSE performance against time slots is illustrated.It is obvious that the achieved NMSE of classical algorithms intensively fluctuates with respect to time slots,which is induced by the error of interpolation.On the contrary,the NMSE performance of our scheme smoothly deteriorates over time,and it is always lower than -10 dB.As a result,we can conclude that our TN-ODE based approach accomplishes accurate continuous-time channel prediction.

Figure 5. The real part of the true future channels and the predicted channels.

Figure 6. NMSE performance against time slots.

V.CONCLUSION

In this paper,we have investigated the essential problem of continuous-time channel prediction in mobile mmWave massive MIMO systems.At first,we adopted the neural ODE to model the temporal correlation of mmWave channels,and then we introduced the neural ODE based channel prediction framework.This framework deployed a GRU-based encoder to extract features from historical channels and used a neural ODE based decoder to predict future continuoustime channels.After that,a TN-ODE model was proposed to improve this framework,which makes full use of the multi-domain channel structure.Simulations demonstrated that our scheme accomplished accurate channel prediction in all time slots of several future frames.The proposed TN-ODE model can be potentially extended to various continuous-time channel prediction scenarios,such as cell-free communication scenarios and RIS-aided communication scenarios.In the future,we will investigate the multi-user continuous-time channel prediction.

ACKNOWLEDGEMENT

This work was supported in part by the National Key Research and Development Program of China(Grant No.2020YFB1805005),in part by the National Natural Science Foundation of China(Grant No.62031019),and in part by the European Commission through the H2020-MSCA-ITN META WIRELESS Research Project under Grant 956256.