Flexibility Prediction of Aggregated Electric Vehicles and Domestic Hot Water Systems in Smart Grids

Junjie Hu,Huynrn Zhou,Yihong Zhou,Hijing Zhng,Lrs Nordströmd,Gungy Yng

a State Key Laboratory of Alternate Electrical Power System with Renewable Energy Sources(North China Electric Power University),Beijing 102206,China

b Division of Electric Power and Energy Systems,School of Electrical Engineering and Computer Science,KTH Royal Institute of Technology,Stockholm 10044,Sweden

c Center for Electric Power and Energy Department of Electrical Engineering,Technical University of Denmark,Kgs Lyngby 2800,Denmark

ABSTRACT With the growth of intermittent renewable energy generation in power grids,there is an increasing demand for controllable resources to be deployed to guarantee power quality and frequency stability.The flexibility of demand response(DR)resources has become a valuable solution to this problem.However,existing research indicates that problems on flexibility prediction of DR resources have not been investigated.This study applied the temporal convolution network(TCN)-combined transformer,a deep learning technique to predict the aggregated flexibility of two types of DR resources,that is,electric vehicles(EVs)and domestic hot water system(DHWS).The prediction uses historical power consumption data of these DR resources and DR signals(DSs)to facilitate prediction.The prediction can generate the size and maintenance time of the aggregated flexibility.The accuracy of the flexibility prediction results was verified through simulations of case studies.The simulation results show that under different maintenance times,the size of the flexibility changed.The proposed DR resource flexibility prediction method demonstrates its application in unlocking the demand-side flexibility to provide a reserve to grids.

Keywords:Load flexibility Electric vehicles Domestic hot water system Temporal convolution network-combined transformer Deep learning

1.Introduction

Environmental concerns have largely promoted the use of renewable energy in recent decades[1].This transition brings challenges to power system operation owing to the intrinsic uncertainty of renewable generation and distributed energy resources.The main challenge is the increasing power imbalance between demand and supply,which leads to an increasing demand for controllable resources to be deployed in the system,while the magnitudes of conventional ancillary service resources are decreasing[2].In addition,the increasing number of connected distributed energy resources at medium and low voltage levels may cause congestion issues[3].

With the development of smart grids and electricity markets,demand response(DR)resources can be an integral part of the system operation.Thus,a promising solution to preserve the ancillary service provision resource is to aggregate the flexibility of DR resources[4].Note that,according to the definition of the International Energy Agency(IEA),flexibility in the power system is the ability to maintain reliability by adjusting the generation or load during large disturbances[5].Therefore,this study defined flexibility as the ability of DR resources to increase and decrease electricity demand.The flexibility of DR resources can be employed by the aggregator through price compensation or economic incentives,which allows the DR resources to modify their own electricity consumption behavior according to the needs of the system operation[6,7].The aggregator can then trade the flexibility of DR resources in the ancillary service market to provide controllable resources for power system operation.

To date,many studies have been conducted on how to make full use of the flexibility of DR resources in providing ancillary services to grids.To achieve the peak load shifting effect,Li et al.[8]established a model for aggregated electric vehicles(EVs)to participate in price-based DR.Sanandaji et al.[9]employed the electricity consumption flexibility of aggregated residential heating,ventilation,and air conditioning loads to regulate reserve services with a certain ramping rate.Hu et al.[10]proposed the idea of using DR resources to achieve a power balance between supply and demand for a multi-area power system.The approach presented in Ref.[11]established reward rules for DR participants who provide flexibility and use the DR flexibility to realize load shifting as well as voltage improvement in supply feeder.

As can be seen in Refs.[8–11],the flexibility of aggregated distributed energy resources determines the effect of the implementation of the DR program[12].However,considering the stochastic feature of end-user behaviors and the complex physical characteristics of DR resources,the aggregated flexibility of DR resources is usually uncertain before the implementation of the DR program.Therefore,several risk control strategies are widely adopted in DR programs to handle the uncertainties of the flexibility of DR resources.Ref.[13]introduced a conditional value-at-risk into EVs problems,thereby providing reserve in the ancillary service market to deal with the uncertainty of EV flexibility.Ref.[14]established a robust optimization problem for the optimal scheduling problem of aggregated EVs to obtain a conservative frequency regulation market participation strategy.To deal with the deviation between the actual and pre-scheduled flexibility of grid-connected EVs in the DR grogram,Ref.[15]formulated an optimization model based on the predictive control-based rolling horizon method.Han et al.[16]used a multiple-scenario-based stochastic programming method to handle the uncertainties of EV flexibility in the DR program.Although the risk control method used in Refs.[13–16]can help the aggregator consider the uncertainties in the day-ahead DR planning,the results may be highly conservative,which may lead to loss of economic benefits of the DR users and even prevent the full flexibility of DR resources in grid operations.

An accurate flexibility prediction model for aggregators is essential for dealing with uncertainty when participating in grid operations.However,only a few studies have been conducted on the flexibility prediction problem.Many studies have mainly focused on electricity demand prediction rather than directly predicting flexibility.Wang et al.[17]and Chen et al.[18]proposed a probabilistic load forecast method that is used in system planning and dispatch.The authors in Ref.[19]reviewed the current energy forecast method and provided future directions.However,they failed to discuss the direct representation and forecast of the flexibility method.At present,most of the research on flexibility is limited to the definition and evaluation of flexibility.Ref.[20]provided a representation method for EV flexibility,considering the participation of EVs in frequency regulation,and evaluated the flexibility of EVs in frequency regulation using three-year data.For residential DR load,the authors in Ref.[21]established a binomial probability distribution model of demand change and used maximum likelihood estimation to evaluate the flexibility of demand increase,demand decrease,or demand immobility.

In Ref.[22],a recurrent neural network(RNN)based flexibility prediction method was proposed to acquire the flexibility of aggregated domestic hot water system(DHWS).Using the flexibility prediction results of DHWS,the load shift scheduling problem in day-ahead and real-time can be carried out with certain accuracy.However,the proposed RNN-based flexibility prediction method can only predict the size of the aggregated flexibility,but cannot present the maintenance time of flexibility.Furthermore,normal RNN-based methods may show poor performance in complex problems.For example,they may suffer from gradient exploration,and the gradient vanishes when the input sequence is sufficiently long;thus,they cannot be applied to general cases[23,24].

Building on the previously developed RNN-based flexibility prediction method in Ref.[22],an advanced deep learning network,the temporal convolution network(TCN)-combined transformer,was adopted in this study to predict the real-time aggregated flexibility of two typical DR resources,EV and DHWS multi-stepahead.This method considers the physical characteristics as well as the different power consumption strategies.The main contributions of this study are twofold.First,the study provided a general multi-step-ahead flexibility prediction method for DR resources.Through the proposed flexibility prediction method,the size of the aggregation flexibility and the maintenance time of flexibility can be obtained.Second,we applied the TCN-combined transformer model to address the flexibility prediction of aggregated EVs and DHWSs.The TCN-combined transformer-based prediction model is one of the most advanced models for modeling long-timedependency problems.The accuracy of the flexibility prediction with a long predicted slot was verified through simulation of a case study.The DR planning and operation problem in,for example,offering reserve service to power grids,can be supported by the proposed flexibility prediction method.

The remainder of this paper is organized as follows:Section 2 provides the preliminaries of our work,including the description of the problem,system framework,and some important definitions.Subsequently,the TCN-combined transformer-based prediction method is described in detail in Section 3.Section 4 presents the simulation results and discussion.Finally,the conclusions are presented in Section 5.

2.Preliminaries

2.1.Definition of DR resources flexibility

In a broad sense,flexibility refers to the ability of the system to respond to internal and external uncertainties,that is,the response ability of the system when internal or external variables change[25].However,in power systems,there is no uniform definition of flexibility.Currently,a widely accepted definition comes from IEA.It defines power system flexibility as the ability to maintain reliability by adjusting the generation or load during large disturbances[5].According to IEA’s definition of flexibility,for the DR resources on the load side,flexibility is reflected in the ability to increase or decrease electricity demand.

Fig.1 illustrates the flexibility of DR resources.In terms of the base power demand(Pbase)of DR resources,that is,the power consumption in normal use,the power demand can be increased toPmaxor reduced toPmin,wherePmin＜0 indicates that the DR resources can provide power to the grid.The values ofPmaxandPminare constrained to the physical characteristics and user requirements of the DR resource,which are discussed in detail in Section 2.3.The distance betweenPbaseandPmin/Pmaxis the capacity to reduce or increase the electricity demand,that is,flexibility.

Fig.1.The flexibility of DR resources.

This flexibility can be used as a reserve for power systems.In the traditional sense,reserves have an additional generation capacity above the expected load[16].The reserve capacity setting protects the power system against the uncertain occurrence of future operating events,including loss of energy or load forecasting errors.When such operating events occur,the DR resources can also cover the imbalance between the supply and demand by increasing or decreasing its electricity demand.Accordingly,the capacity of DR to reduce/increase electricity demand can be called the up/down reserve capacity,as shown in Fig.1.

A reasonable way for DR resources to provide reserves is to allow them to declare the capacity of up and down reserves asymmetrically.However,the uncertainties and coupling in energy and power make the description of DR flexibility a complex problem.To better use the flexibility of DR resources in power systems,the flexibility prediction method needs to be explored.

2.2.Problem formulation and DR system framework

The main aim of our study was to predict the flexibility of two typical DR resources,that is,EV and DHWS,in multi-step-ahead by processing realistic electricity consumption data using the recorded historical end-users’ behavior database.

In our prediction,we studied the DR flexibility in the form of aggregation as it can offset the uncertainty of internally distributed DR resources,and the existing electricity market rules do not allow the participation of individual customers.Considering the different electricity consumption characteristics of EVs and DHWSs,an EV aggregator(EVA)and a DHWS aggregator(DHWSA)were used to aggregate EVs and DHWSs,respectively.

The flexibility of EVA and DHWSA at time steptcan be denoted asFEVA(t)andFDHWSA(t),respectively,which can be derived from Eq.(1).PEVA(t)andPDHWSA(t)represent the base power demand of EVs and DHWSs at time stept,respectively,which are equivalent toPbasein Section 2.1.^PEVA(t)and ^PDHWSA(t)represent the aggregated power of EVs and DHWSs at time steptwhen engaging in the DR program,respectively,which are further discussed in detail in the following section.

Fig.2 shows the DR system framework.Note that we defined the power direction from the grid to the demand side as the positive direction.The EVA and DHWSA can participate in the electricity energy market and ancillary services market for reserve bidding.To assist the prediction and application of flexibility,the EVA and DHWSA can coordinate and schedule the power of all the distributed EVs and DHWSs,respectively,by sending a DR signal(DS,discussed in detail in Section 2.4)as an instruction.They can offer attractive discounts in the DR users’ electricity bill when they change the electricity consumption behavior according to the DR signal instructions.To carry out the flexibility prediction of distributed EVs and DHWSs,both EVA and DHWSA use TCN-combined transformer-based technology.The TCN-combined transformer model is described in detail in Section 3.

2.3.Base power demand of EV and DHWS

As mentioned in Section 2.1,the flexibility of aggregated DR resources is closely related to the base power demand,and the base power demand of aggregated DR resources is influenced by the physical characteristics of DR resources and end-users’electricity consumption strategies.

2.3.1.Physical characteristics of each DR resource

Our study considered three operational modes of each EV,that is,fast-charging with rated fast-charging powerPfast,slowcharging with rated slow-charging powerPslow,and discharging with rated discharging powerPdis.We formulated the following constraints to represent the characteristics of a single EV,as shown in Eqs.(2)–(8):

wherePEV(t)represents the power of EV at time stept.PEVA(t)=∑

Fig.2.System framework.Pb:the power consumption of the boiler in the DHWS;Pdis:discharging with rated discharging power;Pfast:fast-charging with rated fast-charging power; Pslow:slow-charging with rated slow-charging power.

EV∈EVAPEVA(t)represents the relationship betweenPEV(t)andPEVA(t).The state of charge(SOC)of EV at time steptis denoted asE(t).The energy transfer efficiency between the charging pile(grid)and the EV battery is indicated as η.We used Δtto represent the sampling timescale.The initial SOC of a single EV when EV is plugged in at the charging start timetstartis represented asEstart.The expected SOC at the time of departuretdepis denoted asEexp.The maximum limitation of SOC is represented asEmax.The EV must be charged to a certain SOC to meet the driving demand,which is described in Eq.(6).The EV charging demand can be ensured whenEmsis set,which means that the EV must be in charging state when its SOC is lower thanEms.

The DHWS user’s requirement is to keep the water tank temperatureTtankclose to a predefined temperatureTref,with a maximum allowed deviation equal to the deadbandTdb,as described in Eq.(9).The power consumption of the boiler in the DHWS is represented asPbin this study,and the DHWS operation model is formulated as follows:

whereTtank(t)is the temperature of the tank in the DHWS at time stept.The DHWS keeps theTtank(t)close to the set valueTrefby controlling the heat emittedQheat(t)from the thermostatically controlled boiler.Qdrain(t)andQloss(t)are the heat losses from the hot water usage of the end users and the insulated tank to the surroundings at time stept,respectively.CpandVtankare the specific heat capacity of water and volume of the water tank,respectively.Pbrefers to the power consumption of the boiler in the DHWS,and Δtrepresents the sampling time scale.The power of DHWS at time steptcan be denoted asPDHWS(t),andPDHWSA(t)=∑DHWS∈DHWSAPDHWS(t).TinletandToutletare the temperatures of the inlet and outlet water,respectively.Vflow(t)is the total hot water flow used to serve the current DHWS activity of the household member.Λtankis the insulation coefficient of the water tank,andTamb(t)is the ambient air temperature at time stept.

2.3.2.Power consumption strategies for DR resources

In addition to the characteristics of DR resources,the user’s preferred power consumption strategies also have a significant impact on the base power demand.In this study,two typical power consumption strategies are considered to investigate the impact on flexibility:Strategy 1 is the most primitive electricity consumption method,which is completely dependent on the users’ electricity demand and strategy 2 is the most economical way to consume electricity.The user in this way would fully take the electricity tariff into consideration,and tend to consume more electricity when the tariff is low.In the period when the time of use(ToU)tariff is not favorable,strategy 1 is the main power consumption strategy of DR users.However,with the common use of ToU tariffs,strategy 2 becomes increasingly common among DR users.

In strategy 1,the EV starts charging as soon as it parks,and the water heater is only heated when the tank temperature reaches the lower limit.

However,as ToU tariff has been widely used in EV parking lots as well as in residential areas,the power consumption of EVs and DHWSs may be guided by the ToU tariff.Thus,in strategy 2,we consider an optimization power-consumption mode under the ToU tariff.The optimization objectives in strategy 2 for aggregated EVs and DHWSs are shown in Eq.(15).

wherecToU(t)represents the ToU tariff.

Compared with strategy 1,some DR flexibility is activated by ToU at some time steps under strategy 2;however,the DR resources still have additional flexibility based on strategy 2.If the power grid has emergency reserve demand,predicting the DR flexibility in advance using strategy 2 would be helpful for DR resources to provide reserve for the power grid.This type of demand responds process can be motivated by additional subsidies.

Both power consumption strategies are constrained by the physical characteristics of each EV(Eqs.(2)–(8))and DHWS(Eqs.(9)–(14)),and we assume that the fast-charging mode of the EV is only enabled when engaging in the DR program.

2.4.Demand response signal

In this section,we used the DS to predict DR flexibility and help in the implementation of DR management.As per the instructions in DR management,the DS is sent from the EVA or DHWSA to the distributed EVs or DHWSs.The different values of DS represent the different DR requirements,and the distributed EVs and DHWSs need to switch their electricity consumption according to the different values of DS they received.We defined three DS values to represent the corresponding responses of the EV and DHWS,as listed in Table 1.To better clarify the electric power and state changes of a single EV and DHWS corresponding to different DS values,a schematic diagram is shown in Fig.3.Note that the same DS was sent from the EVA or DHWSA to all the distributed EVs and DHWSs in this study.

Table 1 DS value and the corresponding reaction of single EV and DHWS.

2.5.DS-based flexibility formulation

Fig.3.The power consumption and state of a single EV and DHWS under different values of DS for two time steps:(a)DS = 1;(b)DS = -1;and(c)DS = 0.

DR programs usually require the flexibility of DR resources to provide a certain value that can be maintained for a period.Thus,a DS-based flexibility calculation method was proposed to derive flexibility under different maintenance times.First,we should fully activate the flexibility value of all the distributed EVs and DHWS forktime steps.Thus,the EVA and DHSWA can send DS(t)=1,DS(t+1)=1,...,DS(t+k)=1(or DS(t)=-1,DS(t+1)=-1,...,DS(t+k)=-1)to all the distributed EVs and DHWS,which means increasing(or decreasing)electricity consumption at minimumremainktimesteps.Then,theflexibilityF(t)，F(t+1)，...，F(t+k)can be derived based on the DS signal.Based on the value ofF(t)，F(t+1)，...，F(t+k),the flexibility remaining at minimumktime stepsFkcan be obtained by finding the minimum value ofas illustrated in Eq.(17),where the subscript indicates whether it is applied to EVA or DHWSA.Note that the physical meaning of finding the minimum value is to ensure that the flexibility can be presented in a block form with a constant value over several consecutive time steps.From Eq.(17),we can infer that,in addition to the base schedule,the maintenance time is also causes variation in flexibility as the energy of the DR resources is constrained.If the power causes the SOC reach the boundary of the EV battery capacity or the tank temperature of the DHWS to reach the boundary of the reference temperature,the DR resources cannot flow power to and from the grid.Thus,the flexibility decreases accordingly.

Thus,the corresponding flexibility of the aggregated EVs and DHWSs can be obtained based on the above analysis and modeling method.To explain the proposed DS-based flexibility calculation process more intuitively,we draw an illustration in Fig.3,which gives an example of a single EV and DHWS under different values of DS lasting for two time steps.The flexibility calculation results for the example shown in Fig.3 can be found in Table 2.

Table 2 Flexibility calculation results corresponding to Fig.3.

As defined in Eq.(16),when DS =1,the flexibility of the example EV in Fig.3 isPfast-Pslow,and the flexibility can maintain two time steps.When DS = -1,the scheduling flexibility isPdis-Pslow;however,considering the energy demand of EV users,the flexibility can only maintain one time step.Thus,if we calculate the flexibility of the example EV for two time steps,then its flexibility is 0 according to Eq.(16).

As for the example DHWS in Fig.3,when DS=1,it can provide a scheduling flexibility ofPband maintenance time of two time steps.Owing to the tank temperature constraint,the DHWS cannot provide any flexibility when DS=-1.Considering that both EV and DHWS have time-coupling constraints,as shown in Eqs.(3)and(10),the flexibility of the next time step is affected by the flexibility of the previous time steps.Thus,if the state of the tank is heating water rather than losing heat when DS = -1,the DHWS can interrupt the heating to provide scheduling flexibility.

3.Flexibility prediction method and algorithm

3.1.Sequence to sequence(Seq2Seq)-based flexibility prediction method

3.2.Transformer

‘‘Transformer”was first introduced in 2017[29].Prior to that,to achieve a Seq2Seq model,various types of RNNs and convolutional neural networks(CNNs)were the major choices.However,these RNN/CNN-based models are associated with many problems,particularly RNN-based models.One of the most severe problems in RNN-based models is the long-time dependency.When proceeding with long-sequence input,RNN-based models could suffer gradient exploration and gradient vanish[23,24],which results in poor performance.Although some updated RNN structures such as long short-term memory(LSTM)[30]and gate recurrent unit[31]have been proposed,these updated RNNs also fail as the sequence becomes longer.

The transformer shows its great ability in proceeding with longtime dependency due to its multi-head attention and self-attention mechanisms,which are the core of the model.These mechanisms enable the model to determine the inputs that are more important and those that are less dynamic.It is easy for the transformer to learn the time-coupling relationship through the training dataset.In our study,one day was divided into 96 time steps,which made the input sequence longer;thus,there is no doubt that the transformer would be a suitable model to solve this problem.In addition,the multi-head attention and self-attention mechanisms also make the transformer applicable to more general cases.As we need to predict the flexibility of two DR resources,we used this stronger tool.

The structure of the transformer is relatively complex;therefore,we will not explain the detailed mechanism.More details about the transformer model can be found in Ref.[29].

3.3.TCN-based input embedding modul e

In ordinary transformer,the input needs to pass through an‘‘input embedding layer” before it is sent to the internal structure.In NLP,the function of the input embedding layer is to use a much lower-dimensional vector to represent the relatively highdimensional one-hot encoding of words.More generally,this can be regarded as a type of feature extraction method.Based on this,we found a way to achieve feature extraction of the input in our study.

CNN may be a good choice for our study,as the convolution operation is a strong tool for extracting information from the original input.Considering that the inputs in this study are time-series,it is natural that a structure that can capture sequence information may work.Among different kinds of CNNbased models,the model called ‘‘TCN” proposed in 2018[32]is one of the most advanced CNN-based models to handle temporal sequence.In the later section of this paper,we show how to use a type of TCN that could achieve such feature extractions in the original transformer model.

The core structures of TCN are shown in Fig.4.The left figure,which is called‘‘dilated causal conv,”represents the integration of causal convolution and dilated convolution,which enables a shorter sequence to represent a longer sequence.The number of blue lines connected to a square is ‘‘kernel size”(in this figure,the kernel size = 3).‘‘d” refers to dilation factors,representing the distance between two adjacent blue lines connected to a square.As we can see,when the kernel size is 3,a network of depth 4(the number of layers)makes the final output(the blue square on the top layer)contains all the information of the input.The right figure,which is called ‘‘residual block in TCN,” represents a designed structure to make the network deeper without suffering degradation[33].This structure is called ‘‘residual connection,” which is proposed in Ref.[33].Weight norm[34]scales the layer parameters,and thus improves the model’s performance.

Fig.4.The core structures of TCN.ReLU:rectified linear unit.

Finally,by connecting several residual blocks in series,we got the overall model of TCN,namely ‘‘TCN-based input embedding module.”

It should be noted that in the TCN model,the lengths of the input and output are identical;however,we could only choose the outputs of the last 96 time steps on the top of the TCN structure as the input of the transformer.There are two reasons for this.On the one hand,the last 96 outputs contained all the information of the input sequence by reasonable design;on the other hand,a shorter sequence could decrease the computational burden of the transformer.We call the overall structure the TCN-combined transformer model,which is plotted in Fig.5.It can be seen that this model structure exactly achieves the form of Eqs.(25)and(26).

3.4.Training details

3.4.1.Model inputs and outputs

3.4.2.Model inputs and outputs for the EVA

3.4.3.Model inputs and outputs for DHWSA

The characteristics of DHWSA were similar to those of EVA;thus,for strategy 1,the input and output were the same as those of EVA,as shown in Eqs.(32)and(33).

3.4.4.Hyperparameter settings

To train the model,the hyperparameters must be specified.The hyperparameters used in this study are listed in Table 3.The Adam algorithm performs well in network training,and can be easily applied to many machine learning tasks[35].We chose the Adam optimizer to train the proposed network.

Table 3 Hyperparameter for our TCN-combined transformer model.

3.4.5.Training dataset

We needed to specify the training dataset.We created the training dataset according to the required variables,namely,the inputs and outputs in Eqs.(27)–(36).The training dataset for the flexibility prediction of EVA and DHWSA is introduced as follows.Please note that we needed to train several different models as the characteristics are different for different scenarios(EVA or DHWSA under different strategies).

Fig.5.TCN-combined transformer.

3.4.6.Training dataset for EVA

To train the TCN-combined transformer model,we assumed that the historical information of all the distributed EVs,including the arrival time,departure time,battery capacity,and state of charge,can be collected by EVA.Then,based on Eqs.(2)–(8),we generated the base electricity consumption power under strategy 1.Based on Eqs.(2)–(8)and(15),we obtained the base electricity consumption power under strategy 2.Thus,we obtained the base schedule powerPEVAwith no DS in strategies 1 and 2.Based on the base power,according to Table 1,EVA can derive the EV charging/discharging powerunder DS = 1 and -1 on each day.

3.4.7.Training dataset for DHWSA

The basic information of DHWS,including physical parameters,water temperature,and switching information,was needed for the training of the DHWSA network.After collecting the historical data of the DHWS,based on Eqs.(9)–(14),we produced the base electricity consumption power under strategy 1.Based on Eqs.(9)–(15),we produced the base electricity consumption power under strategy 2.Thus,we can obtain the base schedule powerPDHWSAwith no DS in strategies 1 and 2.Based on the base power,according to Table 1,EVA can derive the EV charging/discharging powerunder DS = 1 and -1 on each day.

Please note that for strategy 2,the ToU tariff on each day needed to be collected for both EVA and DHWSA.

3.5.Application

After training,our TCN-combined transformer model could provide accurate predictions.In real time,once we want to predict the flexibility multi-step-ahead,a new set of EV/DHWS data is used as input to the training process of EVA/DHWSA under different electricity consumption strategies.The corresponding power of EVA/DHWSA activated by different values of DS(including DS = 0 and DS ≠0)can be generated for one-step-ahead.For example,if we need to predict the up-regulation flexibility at time stept+k,we can make DS(t+k)=-1 as the decoder inputs,and obtain the network output of the prediction power att+kunder DS(t+k)= -1.Next,we make DS(t+k)= 0 as the decoder inputs,and obtain the network output of the prediction power att+kunder DS(t+k)=0.Using Eq.(16),we obtain the up-regulation flexibility using the prediction power under DS(t+k)=-1 and by subtracting the prediction power under DS(t+k)= 0.Then,using the output as the next input,the corresponding power multi-step-ahead can be derived.The multi-step-ahead flexibility can be easily obtained using Eqs.(16)and(17).

3.6.Summary of proposed algorithm

In this section,we introduce Algorithm 1 as a summary of the overall procedure of the proposed TCN-combined transformerbased flexibility prediction.Note that in application,the EVA/DHWSA only needs to repeat step 6 after the models are trained.

Algorithm 1.Algorithm for the TCN-combined transformerbased DR flexibility prediction.1.Prepare 4 TCN-combined transformer models to predict the flexibility of EVA and DHWS under strategies 1 and 2,respectively.2.Get historical dataset of EVA and DHWSA considering their characteristics for strategy 2;ToU tariff should also be collected.3.Specify the inputs and outputs of the model for EVA and DHWSA under different strategies based on Eqs.(27)–(36).4.Set the model hyperparameters for EVA and DHWSA under different strategies.5.Train the 4 models.6.Input the real-time data of EV/DHWS to the network and achieve a one-step-ahead prediction.Then,repeatedly use the output as the next input to achieve the multistep-ahead prediction;thus,the multi-step-ahead flexibility can be predicted using Eqs.(16)and(17).

4.Case study

To demonstrate the effectiveness of the proposed TCNcombined transformer-based method in the flexibility prediction of EVs and DHWs,simulations of case studies were carried out,and results are presented in this section.We used a computer with an Intel®CoreTMi7-7500U CPU@2.70–2.90 GHz,8 GB of RAM,and RTX2060,as well as a cloud server with 6 × Intel®Xeon®CPU E5-2678 v3 @ 2.50 GHz,11 GB of RAM,and RTX2080 Ti to run all the simulations.

4.1.Basic dat a

The simulation was based on a residential area with 2000 DHWSs and 1000 EVs.The data and parameters of the parking behavior of the EVs were obtained from Ref.[36].The data and parameters of the usage behavior of the DHWSs were obtained from Ref.[37].The key parameters mentioned above are listed in Table 4.The two strategies data came from 256 weekdays per year.We selected the first 196 days to form the training dataset of the transformer network.To avoid over-fitting of the network,the remaining 197–226 days were selected as the validation data set.Finally,the 30 days from 227th to 256th days were randomly selected to test the performance of the proposed algorithm.The ToU tariffs for EV charging stations and residential areas are shown in Fig.6.

Table 4 Some Parameters of EVs and DHWSs.

Owing to the differences between EVs and DHWSs in power consumption time,the studied horizon for DHWSs in one day was from 00:00 to 24:00,while the studied horizon for EVs was from 12:00 to 12:00 on the next day.In this study,one time step was 15 min;thus,one day can be divided into 96 time steps.

4.2.Convergence performance of the transformer network

In this study,the mean square error between the target values and the network outputs in the training dataset was used as the loss function.Fig.7 illustrates the network learning performances under the two strategies of the two loads.As shown in Fig.7,all loss functions experienced a steep drop over the previous period of epochs,and as epochs increased,they decreased marginally and steadily at approximately 10-5.Despite the slight fluctuations,the validation loss functions maintained a downward trend without any abnormal surge,which indicates that the learning processes avoided overfitting.Overall,considering the total training time of 4 h(600 epochs)and 6 h(1000 epochs)as well as a high-grade convergence performance,it was proven that the proposed network structure achieved a balance between complexity and performance.

4.3.Flexibility prediction results analysis

To evaluate the prediction performance of EVA and DHWSA flexibility under different maintenance times,we compared the results under strategy 1(without considering optimization power consumption)and strategy 2(considering optimization power consumption)in six scenarios:①DS = -1 maintained for 15 min;②DS = -1 maintained for 30 min;③DS = -1 maintained for 60 min;④DS = 1 maintained for 15 min;⑤DS = 1 maintained for 30 min;⑥DS=1 maintained for 60 min.Moreover,to validate if our proposed multi-step-ahead flexibility prediction method can predict a long-range slot,we selected a 1-day predicted slot,which means we needed to test the accuracy of the 96-time-step-ahead prediction;hence,k= 96.Thus,our multi-step-ahead prediction was achieved by repeatedly making a one-step-ahead prediction.In real time,it can be used to predict any-step-ahead flexibility at any time,depending on the situation.

Fig.6.ToU tariff for EV charging station and residential area.

To illustrate the superiority of the proposed TCN-combined transformer-based algorithm,we also used the LSTM method(a type of updated RNN structure,as introduced in Section 3.2)to predict the flexibility described in this section.The trained LSTM contained one input layer,two hidden layers,and 128 nodes in each hidden layer.

4.3.1.Flexibility prediction results in one day

Figs.8–11 demonstrate the flexibility prediction results of a randomly selected day in the test dataset.It can be seen from the figures that the peak values of flexibility are slightly more difficult to predict than with other values by both the TCN-combined transformer and LSTM.However,in general,compared with the real value,the predictions of the TCN-combined transformer were more accurate than those of the LSTM.Note that the real value was derived by sending DS to EVs and DHWSs in real time,as discussed in Sections 2.4 and 2.5.

As shown in Figs.8–11,different maintenance times had different flexibility values.Flexibility size decreased with an increase in maintenance time.As the DR resources are decentralized,and with large uncertainty,it is difficult for them to provide aggregated stable flexibility over a long period of time.Thus,the maintenance time of DR resources in the DR program should be set at a smaller value,which is conducive to activate more potential DR flexibility.

In addition,the flexibilities using strategies 1 and 2 were different between periods.For EVA,the flexibility mainly appeared between 16:00 and 8:00.The positive flexibility(absorbing power from the grid)under strategy 2 was greater than that under strategy 1,while the negative flexibility(releasing power to the grid)under strategy 2 was less than that under strategy 1.For DHWSA,the flexibility appeared all day.The period between 8:00 and 24:00 for strategy 2 provided more positive flexibility and less negative flexibility,while the period between 0:00 and 8:00 for strategy 2 provided less positive flexibility and more negative flexibility.Furthermore,the lower the maintenance time,the more significant is the difference.

Moreover,under the same strategy,DR resources exhibited different characteristics in different scenarios.For EVA,an EV has the potential to discharge once it is charged higher than the minimum SOC.However,when it is close to the departure time,discharging would hardly exist to meet the EV energy demand.Thus,when DS = -1,there was a negative flexibility between 16:00 and 8:00,with a maximum of -4.5 MW/15 min.When DS = 1,the EV SOC increased rapidly in the fast-charging mode;however,the EV did not continue to provide flexibility owing to the limitation of the maximum battery capacity.Therefore,the EVA provided a maximum flexibility of approximately 3 MW/15 min from 16:00 to 24:00 when DS = 1.For DHWSA,the positive and negative flexibilities were distributed relatively uniformly during the predicted slot.Due to the changes in the use of the DHWS(e.g.,bathing in the evening,showering in the morning,barely no usage between 13:00 and 16:00,and delay of temperature after usage),which were described in Ref.[37],the negative flexibility reached low points of approximately -4.5 to-4.8 MW/15 min at around 17:00,while the positive flexibility reached a peak point of approximately -5 MW/15 min between 00:00 and 8:00.

Fig.7.Loss function of EVA and DHWSA under two scenarios.

Fig.8.The flexibility prediction results of EVA under strategy 1.

Fig.9.The flexibility prediction results of EVA under strategy 2.

Fig.10.The flexibility prediction results of DHWSA under strategy 1.

4.3.2.Prediction accuracy for 30 consecutive days

Fig.11.The flexibility prediction results of DHWSA under strategy 2.

Tables 5–8 provide information on the mean absolute error(MAE,the average absolute value of the error between the predicted and real values)of the 30-day prediction under different strategies and scenarios.In Figs.8–11 and Tables 5–8,it can be seen that the predicted flexibility of the proposed TCN-combined transformer was closer to the real value than that of the LSTM.In the simpler scenario,that is,EVA prediction under strategy 1,the TCN-combined transformer performed slightly better than the LSTM.As the scenario became more complex,such as EVA prediction under strategy 2 and DHWSA prediction under both strategies 1 and 2(the DHWSA experienced greater complexity owing to the large number of DHWS users),the prediction accuracy of the TCNcombined transformer was relatively better than that of the LSTM.As described in Section 3.2,LSTM is an updated RNN structure,which can significantly improve the long-time dependency problem of the RNN.With the increase in influencing factors,such as the ToU tariff,number of users,and randomness of user behavior,the LSTM still experienced difficulty of fully capturing the information in the input sequence.In general cases,with the multi-head attention and self-attention mechanisms,the TCN-combined transformer can easily learn time-series information through the input sequence.It was found that the predicted flexibility of the TCNcombined transformer was close to the real value with high accuracy,and that the TCN-combined transformer is more suitable for flexibility prediction of DR resources in different electricity consumption scenarios.

Table 5 MAE for 30 d of the EVA prediction results under strategy 1.

Table 6 MAE for 30 d of the EVA prediction results under strategy 2.

Table 7 MAE for 30 d of DHWSA prediction results under strategy 1.

Table 8 MAE for 30 d of DHWSA prediction results under strategy 2.

4.4.Future application prospect for the proposed flexibility prediction method

As discussed in Section 2.1,the negative flexibility could provide up reserve,and the positive flexibility could provide down reserve.From the prediction,the flexibility of aggregated DR resources is obtained through multi-step-ahead,and the schedulable flexibility of the aggregator’s owned resources can be predicted at the day-ahead and real-time levels.The obtained prediction results can provide a reference for the decision-making of aggregator or superior scheduling organizations,thus achieving further cooperation between the aggregator and operators of different levels of electric power systems,such as transmission system operators(TSOs)and distribution system operators(DSOs).

At the day-ahead level,by inputting different DS signals,the amount and maintenance time of the flexibility can be obtained and used to grasp the overall distribution of flexibility and make a general assessment of resource flexibility.The aggregator can trade the flexibility of DR resources in the electrical energy and ancillary service markets based on the day-ahead forecast results.For DR users,electricity cost savings can be achieved.For the power system operator,the power consumption of DR resources and the reserve capacity of the power system can be ensured in advance,which is conducive for reducing the impact of random electricity consumption behavior of DR resources and uncertain renewable energy production on the power system,thereby promoting load balance and ensuring part of the emergency reserve.For example,according to Fig.11,under strategy 2,there was adequate downward reserve flexibility of EVs in the period of 20:00 and 24:00.The upward reserve flexibility was adequate between 4:00 and 8:00 in short maintenance time(15–30 min),distributing uniformly and decreasing between 24:00 and 8:00 when the maintenance time reached 60 min.Thus,in the day ahead,based on this projected result,aggregators are inclined to offer downward reserve between 20:00 and 24:00 and upward reserve between 4:00 and 8:00 in the reserve market,as a more robust decision.Thus,the reserve capacity can be used for system operation,such as frequency regulation at the TSO level or congestion alleviation at the DSO level.

At the real-time level,the flexibility and maintenance time of DR resources defined in the day ahead can be updated by continuously rolling the prediction.As real-time prediction is an ultrashort-term prediction,it can achieve a more accurate prediction than that at the day-ahead stage.The updated flexibility is reported to the system operator accordingly;thus,the system operator is aware of the available reserve in a timely manner.In addition,the aggregator may need to participate in intra-day market,balancing market,and real-time operation;thus,the updated flexibility is necessary in the aggregator optimization problem,which defines the upper and lower boundaries of power of distributed energy resources.

5.Conclusions

A TCN-combined transformer-based algorithm was developed in this study to predict both the size and maintenance time of EV and DHWS flexibility in the DR program.The prediction is based on the network training of historical power consumption behaviors of EVs and DHWS,as well as the DSs.The accuracy of the flexibility prediction was verified through the case study of a group of residential EV and DHWS end-users under different power consumption strategies.We can infer that the flexibility size would decrease with an increase in maintenance time.In general,the proposed TCN-combined transformer-based flexibility prediction method can support DR scheduling in the daily grid operation.

The main challenge in the application of deep learning technology in the field of power grids is the lack of sufficient data as there is usually not enough data in engineering for such a learning-based method.Most of the measured data were of low quality.To further apply our proposed flexibility method,we should take advantage of the rapid development of smart meters and actively consider and carry out beneficial attempts to obtain real-world data.In addition,a further development in data generation technology is highly important to supplement a large amount of data when obtaining accurate data is difficult.As some real-world data can be lost in the transmission process,corresponding data repair technology can also be helpful in improving the quality of data.

Acknowledgements

This work was supported by the National Natural Science Foundation of China(51877078 and 52061635102)and the Beijing Nova Program(Z201100006820106).

Compliance with ethics guidelines

Junjie Hu,Huayanran Zhou,Yihong Zhou,Haijing Zhang,Lars Nordströmd,and Guangya Yang declare that they have no conflicts of interest or financial conflicts to disclose.