Distributed demand response market model for facilitating wind power integration

To cope with wind power uncertainty, balancing authorities are required to procure adequate ancillary services (ASs) with the aim of maintaining the security of the power system operation. The transmission system operator (TSO) is responsible for maintaining the balance between supply and demand in delivery hours. Besides the generating units, demand response (DR) has the potential capabilities to be considered as a source of AS. The demand-side AS can be used both locally (by the local entities in distribution networks) and system-wide (by the TSO). However, the optimal coordination between the local and global beneficiaries is a challenging task. This study proposes a distributed DR market model, in which the DR is traded as a public good among the providers and beneficiaries through the local DR markets. The local DR markets can be run in each load bus to trade the DR provided by retail customers connected to that bus with the buyers. To include the interactions between the energy/reserve market and the local DR markets, a bi-level programming model is proposed. The bi-level problem is translated into a single-level mixed-integer linear programming problem using the duality theorem. The proposed model is verified by simple and realistic case studies.


Nomenclature
A. Indices and numbers n index of system buses, running from 1 to N B i index of generating units, running from 1 to N U j index of load buses, running from 1 to N L d index of customers, running from 1 to N D t index of time periods, running from 1 to N T k index of DR supply blocks offered by DRPs, running from 1 to N k h index of DR demand blocks bid by local DR buyers, running from 1 to N h g index of customer groups, running from 1 to N Gj b index of local DR buyers, running from 1 to N Bj l index of DRPs, running from 1 to N Aj w index of wind power scenarios, running from 1 to N w

Motivation
Due to environmental and economic factors, the penetration level of wind power in the electricity generation sector is increasing worldwide. Across the global market in 2018, more than 590 GW of wind power was installed, which now comprises more than 90 countries. In some countries such as Denmark, about 44% of the total consumed energy is produced by wind [1]. Large-scale wind turbine installations introduce new challenges to operation and planning of power systems. The intermittent nature of the wind power makes its prediction very difficult. Therefore, the power system faces with an extra source of uncertainty. Moreover, the variation of the wind power output is much higher than load variance. Thus, this high generation variance in power systems, which are mostly designed to follow the fluctuations in the demand, poses operational challenges that need to be addressed. To cope with these problems, the power system should become more flexible. In a power system with high wind power integration, the balancing authorities are needed to procure more ancillary services (ASs) for ensuring the security of the system operation in real-time conditions.
In traditional power systems, the AS are provided by resources connecting to the transmission network, such as thermal and hydro power plants. Distributed energy resources (DERs), such as photovoltaic and electrical vehicles are growing rapidly in distribution networks. Technical advances of DER and utilisation of automation and monitoring technologies make possible the provision of AS by these resources. However, as the AS provided by DER can be used both locally and system-wide, there are obstacles towards integration of these resources into power systems in terms of optimal coordination between its local and global beneficiaries and market design. Researches are carried out in multiple projects, such as evolvDSO [2] and ENTSO-e [3], to address these obstacles. In the European context, the SmartNet project [4] aims at providing market design and coordination schemes for optimised interaction between transmission system operator (TSO), as the global beneficiary, and distribution system operators (DSOs), as one of the local beneficiaries, in managing the exchange of AS provided by the DER.
Demand-side resources may also be considered as distributed AS providers. Demand response (DR), due to its low cost and potential capabilities, is recently taken into consideration as an efficient flexibility resource to cope with wind power uncertainty in power system operation [5][6][7][8]. The FERC Order 745 encourages demand-side incorporation by allowing DR to participate in wholesale energy markets [9].

Literature review
There are some researches dealing with managing the variable generation of wind power producers in power systems using both the generation-and demand-side resources. The utilisation of the demand-side resources in a power system with wind generating units is investigated in [10] via a deterministic approach. Due to the stochastic nature of the wind power, deterministic models for managing wind power deployment in power systems may not be efficient. A two-stage stochastic programming for energy/reserve market clearing is proposed in [11], in which DR is used to provide reserve requirements for the wind power and load variations as well as contingencies. In order to deal with large-scale scenario representation of uncertainty, a robust scheduling model is proposed in [12] to manage the wind power variability by coordinating DR and energy storages. A stochastic multi-objective multi-criteria decision-making problem has been proposed in [13], which incorporates incentive-based DR to cope with uncertain and variable characteristics of wind energy. In [7], the authors introduced a joint energy and reserve scheduling and dispatch tool incorporating demand-side resources. They used load serving entities (LSEs) and industrial loads, as demand-side resources, to provide load-following reserves required in a power system with high wind power penetration. The DR and energy storage systems, besides the traditional flexibility resources, are utilised in [8] to manage the uncertainty of wind power generation. A multi-stage robust unit-commitment formulation is applied in [14] to maximise social welfare under the joint worst-case wind power output and price-based DR uncertainty. In [15], a risk-averse energy/reserve market clearing procedure incorporating demand-side resource is proposed. The main aim of [15] is to investigate the behaviour of energy and reserve scheduling for a risk-averse system operator in a power system with significant penetration of wind power. The use of DR programs in promoting wind power integration in power systems is investigated in [16]. To make the best use of price-based DR, the optimal time-of-use price level is determined. Moreover, incentive-based DR is scheduled in a novel two-stage unit commitment. An optimal residential management strategy based on price-based DR and combined heat and power (CHP) units with considering wind generation is investigated in [17]. The authors of [18] proposed a chance constrained day-ahead generation scheduling model for variable energy resources, which considers hourly forecast errors of wind generation and price-based DR. A systematic approach for the joint dispatch of energy and reserve incorporating price-based DR is proposed in [19], in which a dynamic scenario generation is utilised to model the wind generation uncertainty. In [20], the authors proposed a novel incentive-based DR with the aim of encouraging residential customers to participate in DR programs. They showed that if the proposed strategy is utilised in a day-ahead planning context, the total operation cost will be reduced by 10%. The authors of [21] investigated the effects of allowing large demand-side resources to provide reserves in a power system with high wind power penetration. They studied the potential of the large industrial customers participating in DR programs in exercising market power in an electricity market with wind power.
In all of the above-mentioned papers, the DR is scheduled and allocated from the viewpoint of one DR beneficiary, i.e. the TSO, while it may have other beneficiaries at the same time. Nguyen and co-authors [22] showed that the DR provided by a customer can be considered as a 'public good', which is a special type of resource with each single quantity jointly used by multiple players and its use by one player does not reduce the benefit of other players. One MW of energy can be consumed by just one customer, while one MW load reduction may provide benefit to several beneficiaries (TSO, DSOs, retailers etc.) at the same time. For example, onemegawatt load reduction by a customer at the peak hour can provide reserve for the TSO to deal with uncertainties. At the same time, this load reduction by the customer connecting to the distribution network can fix the reliability problems of the DSO who operates that grid. Also, besides the mentioned uses the DR can increase the profit of the retailer, who is in contract with the customer providing the DR, by less energy procurement at the high price of the peak hour. Hence, this one-megawatt DR is shared between three players, i.e. the TSO, DSO and retailer, and its use by each of these players does not reduce its availability to others. Moreover, all the DR beneficiaries will be faced with the load reduction and one of them cannot be excluded from its use. Since the DR as a virtual resource is conceptually different from the electricity, a DR exchange market, which clears separately from the energy market, is introduced in [23] with the aim of exchanging the DR between its providers and beneficiaries. While the provision of the most of the existing public goods (e.g. national defence, street lighting etc.) are usually under government control through the use of taxation, the DR as a form of public goods is different. The DR provision is not subject to government intervention. The DR can be provided by any customer whenever they are paid to do so. Also, the DR can be supplied to any buyers (i.e. retailers, TSO, DSOs) whenever they are willing to pay. In this context, developing a competitive market for trading the DR as public goods is both feasible and necessary [23].
There is no comprehensive approach in the literature to consider this characteristic of the DR in the operation of the power system. Few studies such as [24][25][26] are carried out to consider the concept of the DR exchange in power system operation. A DR market clearing framework is proposed in [24], in which the DR market is cleared after the clearance of the energy/reserve market. The objective function and constraints of the DR market clearing model are developed based on the outputs of the energy/reserve market. However, the sequential clearance of these coupled markets, i.e. the energy/reserve and DR markets, may not be efficient. In [25], a model for joint clearing of the energy/reserve and DR markets is proposed. The DR market is utilised to satisfy the system requirement for reserve capacity, which is determined based on deterministic criteria. However, in [25] scheduling of the DR in real-time conditions is not addressed. Moreover, the proposed model in [25] is a challenging optimisation problem due to its complementarity constraints. The utilisation of the DR market in the stochastic day-ahead energy/reserve market with variable renewable generation is investigated in [26]. A two-step sequential market framework is introduced to clear the energy/reserve and DR market, separately.
The DR is a type of AS that is provided by the customers connecting to the distribution networks which can be used locally (by the local entities in distribution networks) and system-wide (by the TSO). Recently, some researchers have tried to address the coordination of TSO-DSO to achieve the full benefit of the AS provided by distributed generations. In [27], towards coordination of TSO and DSOs, an active-reactive power chart, which characterises the flexibility capability of distribution grids in order to provide AS to TSO. The authors in [28] showed that significant cost reduction can be achieved if comprehensive coordination between TSO and DSOs for trading AS is used. A distributed economic dispatch model for optimal cooperation of TSO-DSO is proposed in [29] through a hierarchical coordination mechanism. However, the linkage between TSO and DSO is done by a generalised bid function that comprises a set of parameters by which the marginal cost of DER is approximated by the TSO. In [30], active and reactive power flexibility areas at the TSO-DSO interface are estimated. A review on different coordination schemes between TSO and DSO is provided in [31]. In [32], the TSO-DSO interactions are analysed through a game-theoretic framework for three coordination schemes, including cooptimisation of transmission/distribution resources as a benchmark, non-cooperative game between TSO and DSOs and Stackelberg game. It was shown that the highest efficiency can be achieved from the first scheme and the Stackelberg game leads to more social welfare that the non-cooperative game. To the best of our knowledge, there is no work in the literature regarding the coordination of the demand-side resources for providing AS at transmission level considering the specific characteristic of the DR, which it can be considered as a public good.

Contributions
The main aim of this paper is to address the problem of optimal coordination of the resources connected to the distribution and transmission networks to manage wind power uncertainty in power system operation. Our focus in this paper is on the DR provided by retail customers connecting to distribution grids, considering its difference with the energy. For this purpose, first, a distributed DR market model for local trading of the DR is proposed, in which the DR is treated as a public good. The clearing rule of the DR markets is different from the energy markets. The proposed local DR market provides a competitive environment for DR transactions at each load bus which faces the aggregators, as DR providers, with DSOs, retailers, etc., as local DR buyers. On the other hand, the TSO may also benefit from the AS provided by the demand-side resources at each load bus. Beside the AS provided by the resources connecting to the transmission network, the TSO can utilise the demand-side AS provided from the distribution grids. For optimal coordination of these resources, a bi-level model is proposed, by which the TSO's problem (energy/reserve market clearing) and the DR markets clearing problems are solved, jointly. Through the upper-level of the proposed bi-level model, the DR demand of the global DR buyer, i.e. the TSO, at each load bus is determined, precisely. Then, considering the DR demands of the global and local buyers, the DR markets at each load bus are cleared in the lower-level problems. The aim of the proposed framework is to manage the wind power uncertainty in system operation. The uncertainty of the wind power and deployment of DR in the plausible wind power scenarios are considered. The duality theorem is used to translate the bi-level model into a singlelevel mixed-integer linear programming (MILP) problem that can be solved by commercially available solvers. The main contributions of this paper are summarised below: • Proposing a multi-period stochastic DR market clearing model to manage the transactions of the DR provided by retail customers at a load bus considering the DR as a public good. • Optimal cooperation of the demand-side resources, which are connected to the distribution network, and the resources in the transmission grid through a bi-level programming model. • Managing wind power uncertainty using the distributed DR market model.

Organisation
The rest of this paper is organised as follows. Section 2 describes the proposed local DR market model as well as wind power uncertainty model. The formulations of the energy/reserve and DR markets and the bi-level problem of joint clearance of these markets are presented in Section 3. In Section 4, the results of the implementation of the proposed model on an illustrative example are reported. Section 5 provides the results from a realistic case study. Finally, some relevant conclusions are provided in Section 6.

Model description
The proposed scheme for managing the resources in transmission and distribution levels is depicted in Fig. 1. In this paper, we focus on DR as one of the resources connected to distribution networks considering its specific characteristic which can be considered as a public good. For this purpose, it is suggested that the DR is traded through local DR markets which are run in each load bus, separately from the energy/reserve market. To consider DR as a public good in DR transactions, the settlement rules in DR markets are designed differently from the energy markets. The motivations for introducing the local DR markets are summarised below: • The DR is conceptually different from the energy and can be considered as a public good. To consider this characteristic of the DR in the DR transactions, a separate market with specific settlement rules is utilised. This approach leads to fair allocation of DR costs across all of its beneficiaries [23]. • The local DR markets can provide an appropriate opportunity for retail customers to participate in the electricity markets, effectively. This leads to more demand-side participation. • Since the DR transactions at different load buses are independent of each other, a distributed DR market model is utilised.
Moreover, for the clearance of the local DR markets, besides the energy/reserve market, a bi-level programming model is proposed. Through the proposed bi-level model, the TSO' demand for the DR is determined precisely according to the technical and economic constraints.

Participants:
• DR sellers: DR providers (DRPs) are the DR sellers who enrol customers to participate in DR programs and offer the aggregated responses into the DR market. It should be noted that only the event-driven DR [33] is considered in our model. The response of the customers is assumed perfectly reliable. Both the upward DR (load reduction) and downward DR (load increment) are considered. It is assumed that the DRP l at bus j bids a step-wise cost function (price steps denoted by a jld in Fig. 2) for providing the DR. In practice, the upward DR is requested during peak hours while the downward DR is needed in off-peak periods. Therefore, it is practically assumed that DRPs offer for providing up-and downward DR in separate periods. • Local DR buyers: The local DR buyers of load bus j are those market entities who are physically or financially under contract with the customers providing the DR at that bus. Some market participants, like retailers and DSOs, who act as mediums between the retail customers and the wholesale market operator may be the local DR buyers. Retailers are market entities who purchase electricity from the wholesale market at spot prices and resell it to retail customers at generally flat rates. A retailer can reduce its financial risks during the horizons that the energy spot prices are volatile by procuring DR from its contracted customers. DSOs, who are the operators of distribution systems, can benefit from DR by using it to enhance the security of their networks, relieve voltage deviations, and defer new network investments at the distribution level. In this paper, it is assumed that the local DR buyer b of bus j offers for purchasing the DR provided by the group g of customers connecting to bus j, which the local DR buyer is under contract with them, via a step-wise benefit function (price steps denoted by α jbg in Fig. 2). • Global DR buyer: The global DR buyer is a player who may request the DR at each load bus of the system. In this paper, the energy/reserve market operator, called TSO, who may be the beneficiary of the aggregated DR provided by the retail customers located at a specific load bus is considered as global DR buyer. The TSO is responsible for the secure operation of the power system. The TSO's demand for the DR, which is denoted by R j in Fig. 2, is determined in the energy/reserve market clearing problem.

Market operator:
The trading of the DR between sellers and buyers at each load bus j is coordinated and settled by a new system operator, which is called DR market operator (DRMO) [34]. It is assumed that all DR transactions are cleared at each load bus by a dedicated computer playing the role of being a DRMO [34]. An Internet-based communication system can be utilised to collect the demand and supply from the participants, automatically. Therefore, there is no need for creating a physical marketplace for the local DR markets. The DRMO gathers the demand-side bids for providing up-and downward DR as well as purchasing offers of the local DR buyers and runs the market clearing problem to determine the clearing prices. The DR clearing prices for the TSO and local DR buyers are shown in Fig. 2 by γ j and λ jbg , respectively. It should be noted that the TSO and local DR buyers are charged for their purchased DR at their corresponding prices. The income gathered from the DR buyers must be paid to the DRPs. Hence, the DRPs are paid for providing DR at a price that is equal to the sum of the DR prices for the TSO and local DR buyers [23]. Please note that the DR is a minor resource beside the electricity, as the major resource. Therefore, if the DR at a load bus is not requested by a buyer, the local DR market at that bus will not be run.

Time horizon:
The scheduling horizon for the local DR markets is considered one day on an hourly basis. In this paper, we have assumed that the DR transactions between DRPs and local DR buyers at different load buses are independent of each other. However, the global DR buyer, i.e. TSO, may request the DR from multiple load buses. Hence, for optimal coordination of the local and global DR buyers, the local DR markets and the energy/reserve market (TSO's problem) are cleared, simultaneously.

Wind power uncertainty
In order to assess specifically the stochastic nature of wind generation, only the wind power uncertainty is considered in our model. Hence, uncertainties associated with demand and equipment failures are not considered. For the sake of simplicity, it is assumed that wind power is generated by a wind farm that is located at a single bus of the considered power network [35]. To model wind generation uncertainty, a set of scenarios representing all plausible realisations of wind power is created. The hourly wind power generation depends on hourly wind speed and wind turbine power curve. Wind energy is a variable and uncertain resource that depends on the effects of the natural and meteorological conditions. Moreover, wind speed at a specific hour is related to previous hours. Short-term forecast of the wind speed can be done using time series models, such as ARMA [36,37]. To generate wind speed scenarios representing the future realisations of this stochastic process, a normal distribution is applied to the wind speed forecast error [38]. Then, the wind speed scenarios are transformed into power by using the power curve of wind turbines. Finally, the corresponding scenario sets of the wind farm power output for each period of the scheduling horizon, i.e. p tw WP , t = 1,…, N T ; w = 1,…, N w , are obtained by aggregating the generation of the available turbines in the wind farm.

Formulation
In this section, the mathematical formulation of the local DR market clearing problem is presented. Since the TSO's demand for the DR, which is used in the DR market clearing problem, is determined in the energy/reserve market clearing problem, this problem is first described. Moreover, the bi-level model for joint clearance of the energy/reserve and local DR markets is presented in Section 3.3.

Energy/reserve market with high wind power penetration
The TSO's demand for the DR is determined by running the TSO's problem with the aim of managing the wind power uncertainty in system operation. The stochastic multi-period energy/reserve market clearing model for a wind integrated power system introduced in [35] is used in this paper. The model proposed in [35] makes use of a two-stage stochastic programming framework and considers a network-constrained market-clearing procedure. In the first-stage, the market is cleared before the realisation of any one of the wind power scenarios. The first-stage variables that represent the scheduled quantities in the market clearing process are shown in uppercase letters. In the second-stage, the uncertainty associated with wind power is realised. The second-stage variables that realise the actual operation of the power system for the wind power scenarios, i.e. p tw WP , t = 1,…, N T ; w = 1,…, N w , are shown with lowercase letters. The compact representation of the market clearing model is as follows [35]: subject to: The objective function (1) to be minimised is the expected cost of scheduling and deploying energy and reserves as well as the costs of load shedding and wind power spillage. The objective function is subjected to the set of equality and inequality constraints (2)-(7). The power balance and generating units constraints for the scheduled variables of the first-stage are expressed by (2) and (3), respectively. The power balance constraints, line flow limits, generation-and demand-side constraints in the second-stage that realise the actual operation of the power system for the wind power scenarios are enforced by (4)- (7), respectively. Please note that the bold symbols denote the vector of all their corresponding variables. The above energy/reserve market clearing problem is a MILP problem. The detailed presentation of the above equations can be found in [35]. The optimal values of the variables R jt and r jtw , obtained by running (1)- (7), are the TSO's demand for the DR at load bus j during period t.

Local DR market
The multi-period stochastic clearing of the DR market j is formulated below. Please note that to avoid complexity, the formulation is stated only for the upward DR, whereas the downward DR is also included in this model.
subject to: P jldt DR, min ≤ P jldt DR, U ≤ P jldt DR, max : μ jldt U , μ jldt U , ∀l, d, t d jbgwt DR, U (h) ≥ 0, ∀b, g, w, t, h p jldwt DR, U (k) ≥ 0, ∀l, d, w, t, k The objective function (8) maximises the social welfare. In the first line of (8), the DRPs' bid for the DR capacity is subtracted from the local DR buyers' offer for purchasing the DR capacity. Flat rates are assumed for the DR capacity selling and purchasing offers. The second line of (8) presents the expected local DR buyers' benefit minus the expected DRPs' cost. It is assumed that each local DR buyer submits step-wise decreasing benefit functions for the DR to the DR market. Each step of the step-wise benefit function is identified by a quantity d jbgt DR, max (h) and price α jbgt dU (h) . On the other hand, each DRP offers step-wise increasing cost functions for providing the DR. Similarly, each step of the step-wise cost function is identified by a quantity p jldt DR, max (k) and price a jldt dU (k) . Determination of the DR benefit and cost functions for the DR buyers and sellers is based on their profit maximisation problems and is beyond the scope of this paper. The DR demandsupply balance of the TSO for the DR capacity and deployment are modelled through constraints (9) and (11), respectively. Please note that the scheduled quantities are shown with uppercase letters and the variables representing the realisation of the actual operation for the considered scenarios are shown with lowercase letters. The TSO's demand for the DR in (9) and (11), i.e. R jt U and r jwt U , are determined in the upper-level problem. Similarly, (10) and (12) present the DR demand-supply balance of the local DR buyers for the DR capacity and deployment, respectively. The binary coefficients c ld bg in (10) and (12) represent a relational status of the customer d of DRP l to the group g of the DR buyer b. The local DR buyer b who is physically or financially under contract with group g of customers can offer to local DR market j to purchase DR provided by its contracted customers. For example, retailer b may be under contract with group g of customers to sell electricity to them at flat rates. These customers may also be under contract with different DRPs to be able to offer their DR into the local DR market. Clearly, just the DR provided by the group g of the customers would be useful for retailer b (to help him manage his financial risks caused by spot price volatility of energy). Therefore, to show this relational status between the DR buyers and the customers of each DRP, a binary coefficient is used. Constraints (13) and (14) represent the DR provision and ramp rate limits of the customers, respectively. Constraint (15) ensures that the scheduled DR capacity of the customer d of DRP l at period t is higher than or equal to its deployed DR in all of the scenarios. Similarly, the relation between the purchased DR capacity and deployment by the local DR buyers is modelled in (16). The upper limits of each block of the offered step-wise cost function of DRPs and the local DR buyers' demand function are enforced by constraints (17) and (18), respectively. Please note that the dual variables associated with the lower-level problem constraints (9)-(18) are provided after the corresponding constraints separated by a colon.
In the DR market clearing formulation, constraints (9)-(12) are the balancing equations that settle the DR as a public good [23]. Since customer d of DRP l supplies a common DR to the TSO, retailers, DSOs and so on, the customer is included in different balancing constraints associated with these DR buyers, respectively. Therefore, its quantity P jldt DR, U appears in the corresponding balancing equations, constraint (9) for the TSO and constraint (10) for other DR buyers (DSOs, retailers etc.). This means the provided DR must be equal to the requested DR by the TSO R jt U as well as other DR buyers D jbgt DR, U . This repetition shows that DR from a customer can be considered public good, which is a special type of resource with each single quantity jointly used by multiple players [23].

Local/central markets cooperation
In this paper, the cooperation of the TSO's problem and the local DR markets is modelled through a bi-level programming problem (Fig. 3). In the proposed local DR market clearing model, it is assumed that the local DR buyers offer for purchasing DR via their submitted benefit functions. The TSO's demand for the DR is not included in (8). The TSO's demand for the DR at each load bus depends on the DR prices and technical/physical conditions of the power system. Knowing the DR prices, the TSO's demand for the DR can be determined by running the energy/reserve market clearing problem (the optimal values of the variables R jt and r jtw ). The price of DR capacity and deployment for the TSO at load bus j, i.e. γ jt c and γ jtw d , respectively, are determined in the jth local DR market clearing problem (the dual variables associated with constraints (9) and (11), respectively). The clearance of the local DR market j depends on the TSO's demand for the DR at the load bus j (the TSO's demand for the DR, R jt and r jtw , is included in constraints (9) and (11), respectively). The interactions between the energy/reserve market and the local DR markets are depicted in Fig. 3. To model these interactions, a bi-level programming approach is proposed. In the bi-level problem, the upper-level belongs to the energy/reserve market (1)-(7) and the local DR markets (8)- (22), ∀ j considered as the lower-level problems. The TSO's demand for the DR capacity and deployment is used in the supply-demand equilibrium of the DR market (9) and (11), respectively. The dual variables of constraints (9) and (11), which are the clearing prices of the DR for the TSO, are used in the objective function of the energy/reserve market model (1). The terms related to the DR reserve cost in (1) can be rewritten as Please note that γ jt c is the dual variable of the DR demand-supply balance constraint of the TSO for DR capacity (9). The optimal value of this dual variable is equivalent to the clearing price of the DR capacity for the TSO (see Appendix 1). Also, the optimal value of γ jtw d , i.e. dual variable of (11), is the price of the DR deployed at scenario w with the probability of π w . Therefore, the DR deployed cost in (24) is calculated by multiplying the probability-removed price of the DR, i.e. γ jtw d /π w , and the deployed DR r jtw . The bi-level problem for joint clearing of the energy/reserve and local DR markets can be formed as where γ jt cU , γ jt cD ∀ j, t; γ jtw dD , γ jtw dD , ∀ j, t, w ∈ arg subjectto: (9) − (22) ∀ j (28) Please note that in the objective function of the bi-level problem (25), the terms defining DR reserve cost, i.e. RC D c and RC D d , in (1) are replaced by (23) and (24), respectively. The solution approach for the bi-level model (25)- (28) is provided in the Appendix 2.

Illustrative example
To validate the proposed bi-level model, the three-bus test system (Fig. 4) is used [35]. The data for generating units, load during the scheduling horizon (4 h) and the assumed wind power scenarios for wind power producer located at Bus 3 are extracted from [35]. Please note that three scenarios are considered for wind power production: as predicted (Pre), highest production (High) and lowest production (Low) with the probabilities 0.6, 0.2 and 0.2, respectively. A DRP at Bus 3 bids for providing DR into the DR market up to 10% of the hourly load. The bid price for DR capacity is $10/MWh. A single-block supply function for DR deployment at a price of $50/MWh is assumed. The capacity of the line connecting Bus 2 to Bus 3 is assumed 25 MW while the capacity of other lines is equal to 55 MW.
Three cases were analysed. In Case I, DR market is not included. The DR market without involving local DR buyers is considered in Case II. Finally, in Case III, two local DR buyers are included. The local DR buyers in Case III submit capacity price of $2/MWh and a deployment price of $10/MWh into the DR market. Table 1 reports the results of the market clearing in Cases I and II. The results obtained in Case II are shown in square brackets. Congestion on the line connecting Bus 2 to Bus 3 causes that all of the wind power production cannot be delivered to the load. Therefore, wind power spillage is occurring (up to 35 MW in scenario High of Case I). However, if the potential of the DR market is used in the energy/reserve market (Case II), the expected wind power spillage will be decreased from 22.4 MW down to 12.4 MW (about 45% decrement). As a result, the scheduled wind power in the energy/reserve market increased from 15 up to 30 MW. Please note that due to utilising more wind power in Case II, the scheduled generation of G1 is reduced down to 25 MW (compared to 45 MW without DR). Table 2 compares the scheduled reserves in the considered cases. As can be seen in Table 2, the scheduled upward reserves (including synchronous and non-synchronous) in Cases I and II are the same. However, in Case II, 10 MWh of DR reserve is scheduled. The scheduled DR reserve in Case II is used in realtime to facilitate wind power integration by reducing the expected wind power spillage by 45%. Incorporation of local DR buyers in the DR market in Case III results in more scheduled DR up to 17 MWh. More utilisation of the DR leads to decrement of the upward reserve by 26% and expected wind power spillage by 6.7% in comparison to Case II. It should be noted that in Case III, the DR    cost will be shared between the local and global DR buyers. Hence, more DR is utilised in the energy/reserve market. The energy, reserve and DR costs from the TSO point of view are reported in Table 3. While the DR is utilised in Case II and caused the energy cost reduction, but the total cost is about the same as in Case I. The reason is that the DRP's bid prices are assumed the highest amounts that could be scheduled in the energy/ reserve market. In Case III, the deployment price of DR obtained $30/MWh, which is decreased $20/MWh in comparison to Case II. The reason for this reduction is that each of the two local DR buyers in Case III will pay $10 per each MW of deployed DR.

Case study
The IEEE reliability test system (RTS) over a 24-h horizon is used for simulations. The required data for simulations have been extracted from [39]. It is assumed that all the generating units offer to provide spinning reserves at the rate of 25% of their highest marginal cost of energy production [40]. The wind power producer with the installed capacity of 400 MW is located on Bus 3. The wind power production data are extracted from [41], which is corresponding to the wind farm located in the southwest zone of the IESO on 5 September 2017. The maximum variation of the wind power from the predicted values is assumed to be 20%. The probability of the predicted scenario is 50%. To simulate the high stressed condition, it is assumed that the capacity of the lines connecting Bus 3 to the network is limited to 80%.
It is assumed that at each load bus, a local DR market with two local DR buyers and one DRP is run (if the DR is requested). The DR capacity bid of all the DRPs (up-and downward DR) is assumed $11/MWh. The upward DR bid is considered a four-block incremental function with equal price/power blocks. The first price block is assumed higher than the most expensive energy bid price of the generating units, i.e. $45/MWh. The price steps are equal to $1/MWh. For downward DR, the price blocks start at $30/MWh with $1/MWh increasing steps. The capacity price offers of all the local DR buyers for purchasing up-and downward DR are assumed $3/MWh and $2/MWh, respectively. The DR price offers of the local DR buyers are assumed $11/MWh and $5/MWh for up-and downward DR, respectively. The value of lost load is considered $3000/MWh. The maximum participation of DR at each bus is assumed 10% of its connected load. Simulations are conducted for the following cases: Case I: Energy/reserve market without DR market, Case II: Bi-level energy/reserve and DR market and Case III: Bi-level energy/reserve and DR market with considering $100/MWh wind spillage cost.
The problem is modelled in GAMS and is solved using CPLEX 12.5.1 [42] with a duality gap of 0.01%. The CPU solution time was 819 s on a Windows-based computer with a 2.4-GHz, Intel (R) Core (TM) i5-6300U CPU and 8-GB of RAM. In order to increase the convergence speed without much deviation from the optimal solution, the non-spinning reserve can be excluded from the energy/reserve market clearing problem, as suggested in [35]. By excluding non-spinning reserve, the CPU solution time is reduced to 21.8 s. In general, some mathematical algorithms, such as benders' decomposition [43], can be used to reduce the computational requirement.
The scheduled reserves, DR and wind power spillage in three considered cases are reported in Table 4. As can be seen, while the assumed DR bid prices are higher than the generating unit cost offers, in Case II, 8.48 MWh of DR is scheduled. In Case II, the load flexibility leads to a reduction of the scheduled reserves for the generating units in comparison to Case I. However, in Cases I and II, a high amount of wind power spillage is scheduled due to the poor correlation between the wind power and load profile (see Fig. 5), and the high cost of reserve scheduling. It should be noted that in Case III, the TSO offers for purchasing the DR at all the load buses except for Bus 16 while in Case II, only the DR is requested from buses 2, 3 and 14.
High reserve scheduling in Case III reduces the amount of wind power spillage by 83%. As can be seen in Table 4, the demand increment as DR, i.e. downward DR, is called in Case III. Utilising the downward DR, besides the downward reserve of the generating units, during high-wind periods leads to the reduction in wind power spillage. Fig. 5 illustrates the load and wind power profile during the scheduling horizon as well as scheduled DRs in Case III. Since during off-peak hours, the wind power production is high, the downward DR is scheduled to reduce the wind power spillage.   Utilisation of the DR in Case II results in reduction of the energy and reserve costs. Because of the high cost of wind power spillage in Case III, more wind power is utilised and consequently more reserves are scheduled. Therefore, the operation cost is increased up to 1.5% in comparison to Case II. Table 6 compares the proposed model that utilises DR exchange concept with those in the literature that are partial approaches for the DR scheduling problem, such as [44], in which DR is scheduled from the TSO's point of view without considering other buyers.
The comparison is carried out for Case III. As can be seen, due to more DR scheduling, the total operation cost is decreased in the proposed model. The cost of reserve required for covering wind power uncertainty in the proposed model is obtained 8.9% lower in comparison to partial approaches.
Figs. 6 and 7 illustrate the impact of DR cost and wind power penetration level on the energy/reserve market objective function and the amount of scheduled DR, respectively. The DR cost is varied from 0 to 200% of the assumed quantities in the case study. As can be seen, increasing the DR cost leads to decrement in scheduled DR in the energy/reserve market (Fig. 7) and hence, the effectiveness of the DR in the energy/reserve market cost reduction is decreased (Fig. 6). Moreover, by increasing the wind power in the system more DR is utilised in the energy/reserve market. However, due to more reserve requirements the energy/reserve cost is increased from about 17 to 30% wind penetration level.
Finally, the impacts of wind power prediction and variation on scheduled reserve and DR are investigated. For this purpose, the proposed bi-level model is run for different probabilities of wind power prediction, varying from 100 to 0%. In this case, a 50% variation of wind power from the predicted values is considered. To show the impact of wind power variation, the probability of the predicted wind power is assumed to be 50%. Wind power generation in different scenarios is varied from 0 to 100%. The results of this investigation are illustrated in Fig. 8. As can be seen in Fig. 8, (i) more reserve is needed for managing the variation of the wind power in comparison to the prediction error, (ii) the amount of the scheduled reserve for managing wind power prediction error is almost constant for prediction accuracy under 90%. If the wind power is predicted with the accuracy above 90%, which may not be achieved easily, the reserve requirements will be decreased; significantly, (iii) the proposed model has a significant impact on decreasing the scheduled reserves for the prediction accuracies up to 40%. For example, in wind power prediction accuracy of 65%, the proposed model leads to a decrease in the scheduled reserve by 24.5%. The reason is that trading the DR via the local DR markets makes it possible to share the high cost of the DR between all the DR buyers. Therefore, from the viewpoint of the TSO, the DR is considered as an efficient resource to be used for covering the wind power uncertainty. The high utilisation of the DR increases the system flexibility and consequently decreases the system requirements for reserves, (iv) increasing the wind power variation level from the predicted value proportionally increases the scheduled reserves and (v) by increasing the wind power variation, the impact of the proposed model on the decrement of reserves increases. For 100% variation from the predicted value, the scheduled reserves will decrease down to 7.85% in comparison to the case without DR.  In this paper, we incorporate local DR market model in the context of stochastic energy/reserve market with high wind power. The local DR markets are run in load buses to face local DR buyers with retail DR providers. For optimal coordination of the demandside resources with the resources in transmission level, a stochastic multi-period bi-level programming problem is utilised. In the proposed bi-level model, the upper-level problem belongs to the energy/reserve market clearing in a wind-integrated power system. The clearance of the local DR markets is considered as lower-level problems.
In order to verify the proposed model and show its applicability, it was tested on simple and real test systems. Results showed the effectiveness of the proposed model in reducing the reserve costs and wind power spillage. Furthermore, it was shown that the model had a significant impact on decreasing the required reserve for managing the wind power prediction error. The proposed model for trading the DR was compared by the partial approaches in the literature, in which DR is managed from the viewpoint of only one beneficiary. The comparison showed that more utilisation of the DR in the energy/reserve market and lower reserve cost for managing wind power uncertainty can be achieved from the proposed approach.
Since for some customers, the demand curtailment may be subject to load recovery, the proposed DR market model can be extended to include the load recovery effects. Furthermore, implementation of efficient methods for solving the resulting largescale MILP problem makes the object of future research. (10) and (12), represent the shadow prices of these constraints [45].
In the DR market model, the optimal values of these dual variables, i.e. λ jbgt cU * and λ jbgwt dU * , which reflect the incremental cost of supplying the next megawatt of the DR demand (capacity and deployment), are known as DR clearing prices for the local DR buyers.
The TSO' demand for the DR, as the global DR buyer, is not included in (8) and is determined in the upper-level problem (1)- (7). Here, we show that in the proposed DR market clearing model the optimal values of the dual variables associated with the DR demand-supply balance constraints of the TSO, i.e. constraints (9) and (11), can be considered as DR clearing prices for the TSO. For the sake of simplicity, suppose that in the optimal point of the lower-level optimisation problem (8)- (22), none of the inequality constraints is active. Hence, based on the KKT optimality equations [45], the dual variables associated with constraints (13)- (18) are equal to zero. From the KKT conditions, we have (see (29)) where L j is the Lagrangian function for the optimisation problem related to the local DR market j [45]. At the optimal point γ jt Based on (30), the sum of the DR clearing prices for the local DR buyers and the optimal value of the dual variable associated with constraint (9) is equal to the cost of one-megawatt DR capacity provided by customer d of DRP l, i.e. a jldt cU . Therefore, γ jt cU * is obtained by subtracting the paid money by the local DR buyers for one-megawatt DR capacity from the cost of providing onemegawatt DR capacity.
Therefore, the optimal value of the dual variable γ jt cU represents the share of the TSO, as a DR buyer, in compensation of the cost of one-megawatt DR capacity. Hence, we can conclude that the optimal value of the dual variable associated with (9) is equivalent to the price of the DR capacity for the TSO. Please note that this dual variable is included in the objective function (1) of the upperlevel problem, which is minimised. Considering the inequality constraints of the lower-level problem (8)- (22), the clearing prices for the DR capacity must satisfy the following (KKT conditions) equations: The clearing prices for the DR deployment, i.e. γ jwt dU and λ jbgwt dU are determined, similarly.

Appendix 2: Solution approach
To solve the bi-level problem (25)- (28), first, it is converted to a single-level programming problem. Then, the resulted problem is linearised to be solved using commercial solvers as well.
The DR market clearing problem at bus j (8)-(22) is a linear programming problem. Hence it presents a continues and convex problem. Based on the duality theory, each lower-level problem can be represented by its constraints, its dual problem constraints and the strong duality condition [45]. By incorporating the constraints of the primal and dual problems of the lower-level problems as well as the strong duality condition into the upperlevel problem, the bi-level problem can be converted to a singlelevel problem. The dual problem of the lower-level problem j is (see (34) On the other hand, the strong duality theorem [45] states that at the optimal point, the objective functions of the primal and dual problems have the same value.
Finally, the equivalent single-level mixed-integer non-linear programming problem of the proposed bi-level model can be expressed as follows: Minimise (1) (39) subject to: (2) − (7) (40) Because of the products of the upper-level variables (R jt and r jtw ) and the lower-level variables (γ jt c and γ jtw d ) in (23) and (24) that appear in the objective function (1), the bi-level problem is nonlinear. Based on the strong duality condition (34), the sum of the bilinear products γ jt c R jt and γ jtw d r jtw can be replaced by its equivalent linear terms. Hence, the bi-level problem can be represented by a MILP problem.