Energy pricing and demand scheduling in retail market: how microgrids’ integration affects the market

: This study proposes a single-leader-multi follower game to model a bilevel retail market among an aggregator and multiple microgrids to determine the optimal demand scheduling of the consumer, as well as price-power bidding strategies of microgrids in an interactive scheme. In the lower level, microgrids which include several distributed energy resources and energy storage units, compete with each other and offer the optimal energy-price bids such that their individual profit is maximised, while energy dispatch among their energy resources is also determined. Then, in the upper-level problem, the aggregator leads the competition taking advantages of demand-side management including interruptible and shiftable loads to minimise its energy payment for real-time pricing of generation units. By means of Karush–Kuhn–Tucker optimality condition, the bilevel optimisation of Stackelberg game is reduced to a single-level mixed-integer linear programming problem. Moreover, impact of microgrids’ integration on the retail market clearance mechanism, as well as required incentives for such integration has been discussed in a separate scenario.


Introduction
Development of smart grid technologies and emergence of distributed energy resources (DERs) have widely affected the electricity market [1,2]. Small power generation and energy storage units such as wind generators, thermal microturbines, photovoltaic (PV) cells, and batteries are installed at the consumers' side and their integration in an adjacent area along with advanced control and communication facilities has formed microgrids [3]. Microgrids are expected to supply their local demand. However, they are capable to take part in the retail markets to offer the surplus generated power to the consumers similar to huge power plants participation in the power and ancillary service wholesale markets [4]. According to smaller scale and lower power generation capacity of microgrids, their share in market is less than other entities. On the other hand, taking the most advantages of smart grids, consumers are not further a passive agent in the competition. Smart grid has equipped the consumer with an appropriate incentive, i.e. demand-side management (DSM), which enables it to act consciously among the other agents, and change its strategy in response to other players actions to maximise its utility or equally minimise its energy payment [5][6][7].
In general, generation units are regarded as active players and the market is organised among them to find the optimal bidding strategies neglecting the presence of active consumers [8][9][10]. However, recently, consumers have been studied separately to determine the best load scheduling that maximises their utility function [11][12][13]. To have a more accurate study, models should consider the effective roles of both consumers and producers, as well as their interactions in a market competition [14][15][16]. An appropriate method for modelling interactive competitions is game theory [17]. Game theory provides us with a variety of approaches to deal with multi-agent optimisation problems from different aspects. Currently, applications of game theory in power systems have been increased impressively. Mohsenian-Rad et al. [18] have proposed an autonomous DSM based on the game theory, wherein all consumers of a common energy resource participate in a competition and Nash equilibrium of game determines the optimal consumption scheduling. Several types of potential games are introduced in [19][20][21] to design efficient mechanisms for interactions among energy hubs to enhance the operation of system via Nash equilibrium.
Competitions are not always expected to be fair and complete. Sometimes the system structure and players heterogeneous capabilities cause the market to shift from a perfect competition toward an oligopoly or monopoly, in which the more powerful player takes the chance to lead the competition. Thus, the simultaneous decision making should be replaced by a hierarchical type. A decent approach in game theory to model such imperfect competitions is the Stackelberg game [16]. The problems of electrical vehicles and grid energy exchange have been studied in [22] using a non-cooperative Stackelberg game. The grid manager is considered to be the leader who sets the real-time prices (RTPs), and the electric vehicles as the followers schedule their charging strategy to maximise their utility. Liu et al. [23] have utilised a bilevel optimisation to model an effective energy sharing management for microgrids with PV generation. In [24], a bilevel game between energy retailers and consumers with firm loads is modelled and a discretisation technique is proposed to simplify the problem. A modified auction-based Stackelberg game is proposed in [25], which models the interaction among the shared facility controllers and residential units to determine the auction price and the allocation of energy storage shared.
Although there is a rich literature on game theory and, in particular, Stackelberg game applications in power system studies, but few of them have modelled generation and demand side with exact details. Besides, most of these papers which deployed Stackelberg game among generation units and consumers have simply neglected the effects of variable generation costs on the solution.
In this paper, a type of multi-microgrid competition is studied, wherein an aggregator has planned to supply its demand from the microgrids through a retail market. The competition among aggregator and microgrids is looked as an imperfect type, where aggregator has been considered as leader and microgrids as followers. According to the reviewed studies and analysis, the major contributions of this paper are as listed: (i) Modelling an interactive retail market among an aggregator and multiple microgrids in details through a single-leader-multi follower game. The lower-level problem results in the optimal energy-price bidding, as well as energy dispatch among DERs for each microgrid. While, the upper-level optimisation leads to optimal demand scheduling for aggregator, in addition to determining the amount of energy purchased from each microgrid. (ii) Investigating the impact of microgrids' integration on the profit of microgrids in the same retail market, as well as its impact on the energy payment of aggregator. Through this stage, it has also been discussed whether sufficient incentives exist for microgrids to form an integration. Indeed, in this stage, we aim to investigate how different combinations of microgrids (i.e. various market layouts) in the lower-level problem might affect the retail market, which has not been studied as our approach in any prior works up to best of our knowledge.
The rest of this paper is organised as follows. Section 2 describes the retail market setting, mathematical equations, and various approaches which are utilised in this paper. Formulation of problem and its solution is explained in Section 3. Integration of microgrids in the retail market is investigated in Section 4. Numerical results and case studies are discussed in Section 5, and finally, conclusion is drawn in Section 6.

Retail market setting
In our model, several microgrids and an aggregator participate in an interactive retail market as shown in Fig. 1. Each of microgrids possesses several types of DERs and offers the amount of energy and corresponding price to the aggregator. Aggregator receives the RTP of energy from all microgrids and schedules its demand such that energy payment is minimised. Aggregator is willing to buy the required amount of energy by a lower price from the microgrids. Otherwise, required energy can be afforded from the wholesale market. Aggregator has three types of loads: first type is neither interruptible nor shiftable load. Second type is shiftable load whose time of use might be shifted to other time periods without forcing any extra costs. The third type is interruptible load that consumer might stop using it, though it causes loss of utility. In this paper, the competition has been considered to be imperfect among small generation units and aggregator. Game theory is an appropriate approach to model the competitions among multi-players, since makes it possible to model the effect of one player decision making on the objective function of others. While Nash games are suitable for investigating competitions with simultaneous decision makings, wherein no player have any priority over others; in this work, we investigate an imperfect game, where a hierarchical decision making exists. To do so, the Stackelberg game has been applied to the model.

Microgrid utility function
The decision variables of microgrids include amount of output power for each DER, and price of energy. In other words, microgrids compete with each other to decide on the level of generation and corresponding price to maximise their utility function for demand. While setting high prices encourages the aggregator to abandon the competition and attend the wholesale market, low prices would also lead to revenue shrinkage of microgrids. Assume that I denotes the set of microgrids and decision variables vector for microgrid i at time slot t is shown by wherein P i t denotes for the RTP offered by microgrid i at time slot t, P m, i t is the output power of microturbines, P w, i t is the wind generation, P v, i t is the PV generation, P ch, i t , P dch, i t are, respectively, charged and discharged powers of batteries, and E s, i t denotes the state of charge (SoC) of battery. So, for each microgrid, the total amount of power offered at each time period t is Generation cost for microturbines is a quadratic cost function as Cost function for wind generators is as below: The first term in (3) corresponds to the penalty cost for wind generators when all of available output powers are not utilised (i.e. the cost for inserting dummy resistors to maintain the power balance for wind generator), wherein ζ w, i is the penalty price and P w, i avl, t is the available wind output for time slot t. The second term is the expected cost for not serving the energy, wherein γ t is the penalty price for not serving energy at time slot t. Without loss of generality, we assume that probability of not serving wind energy due to uncertainty, i.e. ℙ w ξ; t has a uniform distribution as σ w, i t . So, the expected cost for not serving energy would be equal to We have considered the uniform distribution for wind generation because of its simple form. Indeed, for the uniform distribution, all values of uncertain parameter do have the same probability, which helps us to linearise the model. However, any other probability distribution functions can be linearised through a piecewise linear approximation as defined in [26] to be replaced for ℙ w ξ; t in (4). The cost function for PV only includes the expected cost for not serving the energy (7). Again, considering a uniform distribution for probability of PV generation due to uncertainty of sun irradiance, the cost of PV would be similar to (5) as Finally, the cost of operation for battery storages for each microgrid i is where ρ s, i is the constant operation cost and ϵ ch, i and ϵ dch, i are the variable costs of storage charge and discharge. The total cost of generation and operation for each microgrid is the summation of mentioned costs as Let us assume D i t denotes the amount of power that aggregator buys from microgrid i at time slot t. Thus, for each microgrid, the utility function which is considered to be the microgrid profit is According to (10), maximising the utility function for microgrids, i.e. the optimal bid not only depends on the price, but also on generation cost. So, microgrids need to decide on both power levels of each DER and RTP.

Aggregator cost function
For the aggregator, the decision variable is the amount of demand at each time period. In fact, aggregator should determine its demand such that cost function is minimised. Demand side is equipped with DSM. Therefore, it can shift or interrupt a share of its load as a response to the RTP. The vector of decision variables for the aggregator at time slot t is denoted by wherein D IL t denotes for the interruptible load and d up t , d dn t corresponds to the increasing and decreasing time-shiftable loads. The cost function for aggregator is First term in (11) refers to the energy payment, while the second term refers to the utility loss of aggregator caused by load interruption, which is a quadratic cost function and α IL t and β IL t are selected such that the function is non-decreasing.

Stackelberg game model
To model the multi-microgird retail market, Stackelberg game is applied. In this paper, aggregator is considered as the leader and microgrids are the followers. To formulate a Stackelberg game, a bilevel optimisation needs to be constructed. On the lower level, the intersection of the best responses of microgrids results in the Nash equilibrium point of the non-cooperative game among followers, which maximises the utility function of each microgrid for strategies of other players. In the upper-level optimisation, aggregator schedules its demand such that cost function is minimised for the optimal biddings of microgrids.

Followers competition: lower-level optimisation
As mentioned in Section 2.1, the objective function for each microgrid is to maximise the utility function (10). Constraints of this optimisation include the following. Since optimisation of Stackelberg game is bilevel, we have used Karush-Kuhn-Tucker (KKT) optimality condition to change it into a single-level problem. So, for each constraint, corresponding dual variables are shown in parentheses next to it. Each microturbine output power is bounded as Energy prices offered by microgrids are bounded due to the rules of retail market constructed between aggregator and suppliers. Prices should be among a certain range as (19). Also, average of RTP offered by each microgrid over a day should be less than average of forecasted price for the wholesale market, i.e. P avg, f as (20). Otherwise, aggregator would leave the retail market and buy its electricity from wholesale market

Aggregator competition: higher-level optimisation
Objective function for aggregator to be minimised is (11). There are constraints over this minimisation, which include as follows. Equation (21) shows that summation of purchased power from microgrids should be equal to the load of customer for each time slot Inequality (22) states that purchased power from each microgrid should be less or equal than the offered power by microgrid Expression (23) shows that demand at each time slot is equal to scheduled load (D sch t ) minus the decreasing shiftable load and interruptible loads plus the increasing shiftable load Inequalities (24) and (25) denote that upper-level bound for shiftable load is a factor of the scheduled load (τ up t , τ dn t ), which varies for different time slots Inequalities (26) and (27) show the relationship between customer elasticity toward the prices and its shiftable loads, wherein ε up and ε dn are the increasing and decreasing load elasticities and P avg, mg t is the average of RTP offered by microgrids for time slot t (28). In fact, the average RTP of microgrids is less than the average forecasted price for wholesale market, then aggregator increases its shiftable loads regarding its elasticity. However, if forecasted price for wholesale market is less than the average RTP of microgrids, then it decreases time-shiftable loads for elasticity Equality (29) shows that summation of increased and decreased time-shiftable loads over the time should be equal, i.e. there is no reduce or increase in the amount of shiftable load. It is only shifted from one time slot to another Inequality (30) expresses that interruptible load for each time slot is upper bounded to a maximum value

Stackelberg game: bilevel optimisation
Regarding the derived relations in previous sections, we can model the bilevel optimisation of Stackelberg game as follows: This is a bilevel optimisation. One common method to solve the bilevel problem is to use the KKT optimality condition to transform the lower-level optimisation into equality and inequality constraints for the upper-level problem. Using the defined dual variables for the constraints of lower-level problem, we can obtain the KKT condition as follows, wherein g j y i t are the inequality constraints for lower-level problem including (12)- (14), (16)-(20), and h j y i t are the equality constraints including (15). Since, lowerlevel problem is a maximisation, we simply multiply it by a negative sign, to turn it into standard form. Since the allowable bounds for the renewables' generation, as well as the probability distribution function for their uncertain parameter are convex sets, using KKT optimality condition is legitimate So, we need to take the derivatives of Lagrangian function in (32), for the decision variables of microgrids, which leads to Here, (34) and (35) are KKT complementary slackness conditions, where notation x⊥y denotes that at least one of x or y variable is zero, since x ⋅ y = 0. Taking derivative of Lagrangian for wind generator output power, we get Gradient of Lagrangian for PV output power gives us Taking derivative of Lagrangian for decision variables associated with batteries, leads to It should be noted that derivative of Lagrangian for dual variable λ s, i t leads to constraint (15). Finally, by taking derivative of Lagrangian for RTP, we get KKT complementary slackness conditions are non-linear, since they include product of dual variables and primal variable. These constraints can be linearised through introducing auxiliary binary variables. For instance, we can linearise the non-linear constraint (34) as wherein M is a sufficiently large constant and q m, i t is the auxiliary binary variable corresponding to lower bound of microturbine output power. A same approach is used to linearise the rest of complementary slackness condition constraints. So, we can transform the bilevel optimisation (31) into a single-level optimisation as Now, the only non-linearity in optimisation (56) is in the objective function. We replace the quadratic loss function for interruptible load by a piecewise linear approximation to resolve it, though quadratic non-linearity is not very challenging in optimisation problems. The other non-linearity term is the product of P i t D i t . Using strong duality theorem [27] regarding the convexity of defined problem, we can replace this term by So, we can replace (57) for the bilinear term in (56) to finally derive the mixed-integer linear form of the proposed bilevel Stackelberg competition. All the complementary slackness conditions are also replaced by their corresponding linearised form as (55).

Integration of microgrids
As another scenario, assume that we want to investigate the effect of microgrid integration on the profit of each player to find out whether integration helps microgrids to increase their profit in the retail market or not. In this regard, without loss of generality, we assume that physical infrastructure for any type of integration among microgrids is available without imposing any extra cost. So, the focus here is on the economic analysis of this scenario, but not the technical aspect. To investigate microgrids' integration, we need to develop a fair profit sharing mechanism among the microgrids participating in the integration. One naive approach is to simply share the profit based on the amount of contributed power for each microgrid. However, this approach may not be fair, because integration itself causes changes in both formation and parameters of bilevel optimisation problem of Stackelberg game. It should be noted that for an integration, we consider all involved microgrids in the integration as a unique player who aims to maximise the total utility of whole integration in the lower level of Stackelberg game in competition with any other microgrids (or any other integration of microgrids). One fair mechanism of profit sharing among the participants of an integration is to determine the value that the presence of each microgrid adds to the utility function when compared with the case that microgrid is absent in the integration. However, it should be kept in mind that the presence of a microgrid might even cause a decline in the integration utility. The main reason is the structure of bilevel competition. The game equilibrium points are all obtained based on the parameters of microgrids and formation of optimisation. One integration might increase the revenue for all members, while another may not. If an integration increases the share of profit for each of participant microgrids, then it provides enough incentives for them to take part in the integration. Otherwise, there would be no incentives for microgrids to form an integration.
In this regard, to determine the share of each individual microgrid from the total profit of integration, once market should be solved for a complete integration, wherein all microgrids are grouped as a unique player and find the revenue of the integration. Then, another problem should be solved for the case, wherein a particular microgrid is excluded from the integration. Then, it is investigated whether the added value due to the participation of microgrid is positive or negative. If added value is negative, it is obvious that there would be no incentives for players to form this integration. However, if it is positive, then a same procedure is applied for all involved microgrids, and revenue is shared among them in proportion to the added values. As soon as it is revealed that participation of one microgrid adds a negative value to the utility function, formation of that integration would be cancelled, because it does not bring up any incentives for players. The proposed mechanism for revenue sharing in an integration, wherein added values to the utility function are non-negative is as follows: wherein U S denotes for the utility of integration, U S − i denotes for the utility of integration without microgrid i, and U i denotes for the share of microgrid from the integration. It should be noted that considering all types of integrations is not possible due to the vast number of possibilities in forming the integrations and limits of this paper. So, we have only discussed the full integration in details in the case study section, wherein all microgrids are grouped together. However, a same approach can be taken for all other possible integrations. Also, discussed integration is in the lower-level problem of Stackelberg game. So, method of solution for the whole problem is similar to what has been discussed in Section 3. Indeed for the case of integration, problem formulation is as same as (31), but i index of microgrids is vanished and all microgrids offer a unique bid to the consumer as an individual player.

Simulation data
The studied multi-microgrids competition is established among five microgrids and an aggregator. The DERs for each microgrid are listed in Table 1. The quadratic cost function of microturbines is linearised with a line segment, whose slope is shown in Table 1 as b m, i ′ . Also, the value of rated power and rated energy are equal for batteries. Charge and discharge efficiencies of batteries are also same. Minimum generation of microturbines and minimum storage capacity is zero. Total available output power of wind generators and PVs are shown in Figs. 2 and 3. The probabilities of not serving the wind generation for t=1-6, 7-12, 13-18, 19-24 are, respectively, 0.01, 0.005, 0.02, 0.01. The probabilities of not serving the PV generation for t=5-12, 13-20 are 0.001 and 0.005, respectively. The amount of ζ w, i for all wind generators is equal to 0.1 ¢/kWh. Penalty factor for not serving energy is 5 ¢/kWh for t = 12 − 21 and 1 ¢/kWh for the rest of time periods. The average forecasted price of wholesale market during a day is 11 ¢/kWh. Data of aggregator is presented in Table 2. Increasing and decreasing demand elasticities are 3 and 2%, respectively.

Case 1: a retail market through Stackelberg competition
The bilevel Stackelberg optimisation is modelled in GAMS software and CPLEX solver is used to solve it. Each microgrid offers its bid to the aggregator without any information from other  Table 3. Results show that during the low-price periods (from 12 am to 11 am), optimal pricing strategy is similar for all microgrids and equal to the maximum allowable price. However, as it gets closer to the high-price periods, according to various types and generation levels of DERs, pricing strategies vary for microgrids. While microgrids which possess PV with high generation capacity and almost zero operation cost (e.g. microgrid 1) supply the demand with low price comparing the others at the daytime, during the evening and at time of highprices, first those microgrids with wind production or storage discharge clear the market. However, due to the high amount of demand, microgrids with microturbines and lower marginal cost (e.g. microgrid 2) provide the remaining required power with the maximum prices. The pricing strategy at these time periods is highly influenced by the demand elasticity of consumer, which causes various optimal offering prices. Referring (23)- (26), for the case, in which all microgrids set the maximum price, DSM shifts the demand to other time periods and shrink the revenue of microgrid. DSM, which is considered to have the leadership over the microgrids is sensitive to the high prices, causes the market to be cleared at lower prices between 12 pm to 8 pm despite of higher bounds of allowable price at this time interval.
On the basis of the parameters of DERs, linear formulation of problem might lead to multiple solutions, which is a rare case that two microgrids offer the same energy-price bid might happen. A typical approach in real markets to deal with this issue is to determine the share of each generation unit for average cost function of each unit, as well as their ratings in the databases, where the less costly unit is selected by the market operator.

Case 2: microgrids integration
In the second case study, we want to investigate the effect of microgrids' integration on the equilibrium of retail market competition, and to find out whether enough incentives exist for microgrids to be integrated or not. In this regard, first, we consider a complete integration, wherein all microgrids submit a common bid to the aggregator, i.e. the game is changed into a single-leader, single-follower competition. For this setting, we have to solve an optimisation similar to (56). However, index of microgrids no more exists. To make the comparison fair, we use the same data as case 1. For this case, total revenue and total cost of aggregator are shown in Table 4. Also, optimal RTP and share of each microgrid are shown in Figs. 7 and 8. As we see, for the integration, the total cost of aggregator is increased. Also, revenue of integration is greater than the total revenue of each microgrid. So, overall, integration has helped microgrids to increase the total revenue and caused the aggregator to pay more. It can be concluded that when number of players in a competition is decreased, competition moves toward an imperfect type rather than being a complete competition.
Although total revenue of integration is higher than the sum of individual revenue in case 1, but it should be investigated whether such incentive exists for microgrids to abandon their previous form of participation in market and form an integration. That incentive is brought in, whenever that share of each microgrid from the total share of integration is higher than its share from the previous case. On the basis of the defined sharing mechanism in Section 4, first, we should find out what is the added value to the utility function due to participation of each microgrid as defined by (58). To do so, we have to solve (55) for five different integrations, wherein four microgrids form an integration and the other one participates in the retail market individually. Added value for each of five integrations and its percentage is outlined in Table 5.
So, based on the defined fair revenue sharing mechanism (59), share of each microgrid from the complete integration is as shown in Table 6.  Tables 3 and 6 shows that share of revenue for microgrids 2 and 3 in the integration is less than case 1, wherein all microgrids participate as individual players in the retail market. So, this integration with current revenue sharing mechanism does not provide enough incentive for microgrids 2 and 3 to participate in this complete integration. Thus, complete integration may not be formed, unless revenue sharing mechanism is changed. The same study can be done for other types of integrations for which 4, 3, and 2 microgrids are integrated. Performing a same study on all possible combinations of microgrids for other integrations, it is revealed that integration does not bring up enough incentives for all the involved microgrids to form it. In general, integration increases the revenue share for some microgrids, and lowers it for the rest, even though the total revenue of whole integration has been increased. It might be due to the fact that all the microgrids are equipped with different types of DERs and they are capable to take part in the retail market as individuals to offer their own bid rather than forming a unique player. So, in the current setting, none of the microgrids has got the incentive to form an integration among others, which simultaneously increases the share of revenue for all participants. So, they continue their participation as individual agents in the retail market.

Conclusion
This paper represented a hierarchical decision making among microgrids and an aggregator in a retail market to derive the optimal bidding strategies for the producers, as well as optimal demand schedule for the aggregator. A Stackelberg game is applied to model the problem, in which the aggregator is considered as the leader, and microgrids as followers. Stackelberg game is modelled through a bilevel optimisation and transformed to a single-level linear optimisation via KKT optimality condition, which might further be utilised in solving real similar decision-making problems. As a separate case study, the problem of microgrid integration is investigated to find its effect on the retail market. Also, it is discussed whether an integration provides enough incentives for microgrids to get involved or not. The comprehensive and detailed model of generation side and DSM allows us to investigate the effect of various crucial parameters such as demand elasticity, storage capacity, interruptible load prices etc. on the behaviour of a multi-agent competition in the energy retail market. Case studies have revealed that in daytime, optimal RTPs are identical for all players, though their generation might differ due to types DERs. While, on time of high-demand, adequate DERs capacity may help an agent in the followers game to take the largest share of revenue. Besides, through the defined fair revenue sharing for the case of integration, it is concluded that integration for this setting and for the defined sharing mechanism does not bring up enough incentives for all players, though the total utility of integration is increased when compared with the other case.