Concession Strategies for Negotiating Bilateral ... - Semantic Scholar
Concession Strategies for Negotiating Bilateral Contracts in Multi-Agent Electricity Markets Fernando Lopes LNEG−National Research Institute Estrada do Pac¸o do Lumiar 22 1649-038 Lisbon, Portugal [email protected]
Abstract—Electricity markets are systems for effecting the purchase and sale of electricity using supply and demand to set energy prices. Market participants are commonly exposed to financial risks, particularly high price volatility. This article addresses the challenge of using software agents with negotiation competence to help manage the complexity of electricity markets. Specifically, it presents a model for software agents that handles two-party and multi-issue negotiation. Market participants equipped with the model are able to enter into forward contracts to protect themselves from volatility at times of peak demand and supply shortages. Keywords-electricity markets; bilateral contracts; intelligent software agents; automated negotiation.
I. I NTRODUCTION The electricity industry throughout the world, which has long been dominated by vertically integrated utilities, has experienced major changes. In particular, liberalization has led to the establishment of a wholesale market for electricity generation and a retail market for electricity retailing. Market forces drive now the price of electricity and reduce the net cost through increased competition. Electricity markets (EMs) differ from their more traditional counterparts because energy is difficult to store and has to be available on demand. Consequently, market participants are forced to work with consumption prognoses, which, in turn, create a number of risks. Firstly, the price of electricity mainly depends on the production cost and generating more than is consumed is not economical. Secondly, if suppliers cannot match the demand, the lack of energy can cause power cuts (brownouts) or, if prolonged, blackouts. Finally, there are non-negligible costs stemming from variations in the electricity production volume that most traditional types of energy generators (e.g., hydroelectric or thermoelectric) have to face. Clearly, financial risk management is a high priority for market participants. This work was performed under the project MAN-REM: Multi-agent Negotiation and Risk Management in Electricity Markets (FCOMP-010124-FEDER-020397), and supported by both FEDER and National funds through the program “COMPETE−Programa Operacional Tem´atico Factores de Competividade”.
Helder Coelho University of Lisbon Bloco C6, Piso 3, Campo Grande 1749-016 Lisbon, Portugal [email protected]
The two key objectives of EMs are ensuring a secure and efficient operation and decreasing the cost of electricity utilization. To achieve these goals, three major market models have been considered [16]: pools, bilateral contracts, and hybrid models. A pool, or power exchange, is a market place where electricity-generating companies submit production bids and corresponding market-prices, and consumer companies submit consumption bids. A market operator uses a market-clearing tool, typically a standard uniform auction, to set market prices. Bilateral contracts are negotiable agreements between two parties to exchange electric power under a set of specified conditions, such as price, MW amount, time of delivery, and duration. Market participants set the terms and conditions of agreements independent of the market operator. They often enter into bilateral contracts to hedge against pool price volatility. Furthermore, these contracts are very flexible since the negotiating parties can specify their own contract terms. The hybrid model combines several features of pools and bilateral contracts. In this model, a pool is not mandatory, and customers can either negotiate power supply agreements directly with suppliers or accept power at the pool market price. Thus, this model offers a true customer choice and an impetus for the creation of a wide variety of services and pricing options to best meet individual customer needs. Multi-agent systems are essentially loosely coupled networks of software agents that interact to solve problems that are beyond the individual capabilities of each agent. Agents are computer systems capable of flexible, autonomous action in order to meet their design objectives. Agent technology has been used to solve real-world problems in a range of industrial and commercial applications (see, e.g., [11]). Conceptually, a multi-agent approach is an ideal fit to the naturally distributed domain of a deregulated electricity market. Accordingly, this work looks at using software agents with negotiation competence to help manage the complexity of EMs, particularly the issues associated with the negotiation of bilateral contracts, towards protecting market participants from price risks.
Specifically, this paper presents a model for software agents that handles two-party and multi-issue negotiation. The model incorporates a bilateral negotiation protocol, a set of concession strategies, and a set of concession tactics. The protocol formalizes the set of possible tasks that the agents can perform during the course of negotiation. The strategies and tactics formalize the tasks that each agent should perform to negotiate effectively. This paper builds on our previous work in the areas of automated negotiation and electricity markets. In particular, it extends the work presented in [5], [6] and [7] by introducing precise definitions for concession strategies. It also extends the work presented in [8] and [9] by formalizing a set of energy dependent tactics for computing new values for each issue at stake. The remainder of the paper is structured as follows. Section II presents a negotiation model for software agents. Section III discusses related work and compares the negotiation model with other existing models. Finally, section IV presents concluding remarks and indicates future avenues of research. II. A N EGOTIATION M ODEL Let A = {a1 , a2 } be the set of autonomous agents (negotiating parties). Both the number of agents and their identity are fixed and known to all the participants. Let I = {x1 , . . . , xn } be the negotiating agenda— the set of issues to be deliberated during negotiation. Let D = {D1 , . . . , Dn } be the set of issue domains. For each issue xk , the range of acceptable values is represented by the interval Dk = [mink , maxk ]. A. Pre-Negotiation Pre-negotiation is the process of preparing and planning for negotiation and involves mainly the creation of a wellconceived plan specifying the activities that negotiators should attend to before actually starting to negotiate. In particular, effective pre-negotiation requires that negotiators prioritize the issues at stake, define the limits and targets, select an appropriate protocol, and specify the preferences. Prioritization involves deciding which issues are most important and which are least important. Target setting involves defining two key points for each issue: the resistance point or limit and the target point or level of aspiration. Definition 1 (Priority, Weight). The priority prtk of an agent ai ∈ A for an issue xk ∈ I is a number that represents the importance of xk . The weight wk is a number that represents the preference for xk . Definition 2 (Limit, Target Point). The limit limk of ai ∈ A for an issue xk ∈ I is the ultimate fallback position for xk , the point beyond which ai is unwilling to concede on xk . The target point trgk is the point at which ai is satisfied with the value of xk .
The negotiation protocol is an alternating offers protocol [10]. Two agents or players bargain over the division of the surplus of n ≥ 2 distinct issues. The players determine an allocation of the issues by alternately submitting proposals at times in T = {1, 2, . . .}. This means that one proposal is made per time period t ∈ T , with an agent, say ai ∈ A, offering in odd periods {1, 3, . . .}, and the other agent aj ∈ A offering in even periods {2, 4, . . .}. The agents have the ability to unilaterally opt out of the negotiation when responding to a proposal. The negotiation process starts with ai submitting a proposal p1i→j to aj in period t = 1. The agent aj receives p1i→j and can either accept the offer (Yes), reject it and opt out of the negotiation (Opt), or reject it and continue bargaining (No). In the first two cases the negotiation ends. Specifically, if p1i→j is accepted, negotiation ends successfully and the agreement is implemented. Conversely, if p1i→j is rejected and aj decides to opt out, negotiation terminates with no agreement. In the last case, negotiation proceeds to the next time period t = 2, in which aj makes a counter-proposal p2j→i . The tasks just described are then repeated. Once an agreement is reached, the agreed-upon allocations of the issues are implemented. Definition 3 (Proposal). Let A be the set of negotiating agents and I the set of issues at stake in negotiation. Let T be the set of time periods. A proposal pti→j submitted by an agent ai ∈ A to an agent aj ∈ A in period t ∈ T is a vector of issue values: pti→j = (v1 , . . . , vn ) where vk , k = 1, . . . , n, is a value of an issue xk ∈ I. Definition 4 (Agreement, Possible Agreements). An agreement is a proposal accepted by all the negotiating agents in A. The set of possible agreements is: S = {(v1 , . . . , vn ) ∈
Rn : vk ∈ Dk ,
for k = 1, . . . , n}
where vk is a value of an issue xk ∈ I. Negotiators should express their own preferences to rate and compare incoming offers and counteroffers. Let I = {x1 , . . . , xn } be the agenda and D = {D1 , . . . , Dn } the set of issue domains. We consider that each agent ai ∈ A has a continuous utility function: Ui : {D1×. . .×Dn } ∪ {Opt, Disagreement} → . The outcome Opt is interpreted as one of the agents opting out of the negotiation in a given period of time. Perpetual disagreement is denoted by Disagreement. Now, the additive model is probably the most widely used in multi-issue negotiation: the parties assign numerical values to the different levels on each issue and add them to get an entire offer evaluation [14]. This model is simple and intuitive, and therefore well suited to the purposes of this work.
R
Definition 5 (Multi-Issue Utility Function). Let A = {a1 , a2 } be the set of negotiating agents and I = {x1 , . . . , xn } the negotiating agenda. The utility function Ui of an agent ai ∈ A to rate offers and counter-offers takes the form: Ui (x1 , . . . , xn ) =
n X
wk Vk (xk )
k=1
where: (i) (ii)
wk is the weight of ai for an issue xk ∈ I; Vk (xk ) is the (marginal) utility function of ai for xk , i.e., the function that gives the score ai assigns to a value of an issue xk . Negotiation may end with either agreement or no agreement. Failure to agree can occur in two ways: (i) either party decides to opt out unilaterally, or (ii) the two do not agree to any proposal. The resistance points or limits play a key role in reaching agreement when the parties have the ability to unilaterally opt out of the negotiation—they define the worst agreement for a given party which is still better than opting out. For each agent ai ∈ A, we will denote this agreement by sˆi ∈ S. Hence, sˆi will be the least-acceptable agreement for ai , i.e., the worst (but still acceptable) agreement for ai . The set of all agreements that are preferred by ai to opting out will be denoted by Si .
The negotiation protocol defines the states (e.g., accepting a proposal), the valid actions of the agents in particular states (e.g., which messages can be sent by whom, to whom, at what stage), and the events that cause states to change (e.g., proposal accepted). It marks branching points at which agents have to make decisions according to their strategies. Thus, at each step of negotiation, agents often need to follow their strategies to choose among different possible actions to execute. 1) Concession Strategies: These strategies are computationally tractable functions that model typical patterns of concessions. For a given time period t > 1 of negotiation, they specify the concession tactics to be used in preparing counter-offers. The words “computationally tractable functions” presume that agents are able to compute concession strategies in a reasonable amount of time. A formal definition of a generic strategy follows. Definition 7 (Concession Strategy). Let A be the set of negotiating agents, I the negotiating agenda, T the set of time periods, and S the set of possible agreements. Let ai ∈ A be a negotiating agent and Ti its set of tactics. Let aj ∈ A be the other negotiating agent and pt−1 j→i the offer that aj has just proposed to ai in period t−1. A concession strategy Ci : T → S ∪ {Yes, No, Opt} for ai is a function with the following general form: apply Yi and prepare pti→j if 4Ui ≥ 0 accept pt−1 else reject, if aj ’s turn and Ui (pt−1 ) ≥ Ui (ˆsi )
Definition 6 (Least-acceptable Agreement, Acceptable Agreements). The least-acceptable agreement for an agent ai ∈ A is defined as: sˆi = (lim1 , . . . , limn ), where limk , j→i k = 1, . . . , n, is the limit of ai for an issue xk ∈ I. The Ci =reject pt−1 and quit, j→i set of acceptable agreements for ai is: t
offer compromise pi→j ,
Si = {s : s ∈ S, Ui (s) ≥ Ui (ˆ si )} where Ui (ˆ si ) is the utility of sˆi for ai . Perpetual disagreement is the least-preferred or worst outcome, i.e., disagreement is even worse than opting out. Formally, and more precisely, we state the following: (1) (Acceptable agreements versus opting out). For every agent ai ∈ A and acceptable agreement s ∈ Si , Ui (s) ≥ Ui (Opt). (2) (Opting out versus Disagreement). For every agent ai ∈ A, Ui (Opt) > Ui (Disagreement). B. Actual Negotiation Actual negotiation is the process of moving toward agreement and typically involves an iterative exchange of offers and counter-offers. In “good faith” negotiation, offers are made, and are either accepted or returned with counteroffers. There is an unstated assumption that the parties will show their commitment to the process of finding a solution by making concessions, and not simply by rejecting the offers of the others out of hand. To do so is often seen as “bad faith” bargaining [15].
j→i
if aj ’s turn and Ui (pt−1 si ) j→i ) < Ui (ˆ if ai ’s turn (time period t)
where: (i) (ii) (iii) (iv)
for each issue xk ∈ I, Yi is a concession tactic (see below); pti→j is the offer of ai for period t of negotiation; t 4Ui = Ui (pt−1 j→i ) − Ui (pi→j ); Ui (ˆ si ) is the utility of the least-acceptable agreement for ai , i.e., the worst (but still acceptable) agreement for ai .
Concession making can take several different forms and some representative examples are now presented. Negotiators frequently start with ambitious demands, well in excess of limits and aspirations, and concede slowly. High demands and slow concessions are often motivated by concern about position loss and image loss (or face-saving). Position loss is the abandonment of desirable alternatives, whereas image loss is the fear of appearing ready to make substantial concessions, i.e., the development in the opponent’s mind of an impression of lacking firmness [12]. Typically, high demands and slow concessions prevent position loss and guard against image loss. Furthermore, they often encourage the opponent to make concessions by the principle of reciprocity.
Bargainers generally view the world differently—they are not identical in their interests and preferences. In particular, they frequently have different strengths of preference for the issues at stake—they place greater emphasis on some key issues and make significant efforts to resolve them favourably. Hence, they concede more often on less important or low-priority issues. Low-priority concession making involves changes of proposals in which larger concessions are made on low-priority than on high-priority issues— bargainers concede and typically maintain high profits for themselves [12]. Interestingly, bargainers sometimes demonstrate good will and make one or more substantial concessions that seek reciprocal concessions (such moves are different from unilateral concessions that seek no quid pro quo). Yet overt concessions aimed at eliciting counterconcessions are risky and entail the possibility of position loss and image loss. Hence, they are more likely to be made when the other party is trusted, mainly because there is less concern about both forms of loss. When trust is low, bargainers turn to one of a host of less risky moves. These include fractionation of concessions—negotiators start with a relatively riskless action and move on toward increasing levels of risk. They make a small concession and wait to see if the opponent reciprocates. If so, they then may feel sufficiently confident to venture a larger concession [13]. The generic concession strategy presented in definition 7 allows agents to model these forms of concession making (and possibly other typical patterns of concession). For instance, consider the traditional behavior of starting with ambitious demands and conceding slowly, which often encourages concession making from the opponent. Conceding slowly is defined by considering the concession tactic “tough” (see the formal definition, below). 2) Concession Tactics: These tactics are functions that model the concessions to be made throughout negotiation. A formal definition of a generic concession tactic follows (in the interests of readability, and without loss of generality, we consider that a negotiating agent ai ∈ A wants to maximize an issue xk ∈ I). Being a “generic concession tactic” means that it can model different levels of concession magnitude (e.g., small and large). Definition 8 (Concession Tactic). Let A = {a1 , a2 } be the set of negotiating agents, I = {x1 , . . . , xn } the negotiating agenda, and D = {D1 , . . . , Dn } the set of issue domains. A concession tactic Yi : Dk ×[0, 1] → Dk of an agent ai ∈ A for an issue xk ∈ I is a function with the following general form: Yi (xk , fk ) = xk − fk (xk −limk ) where: (i) (ii)
fk ∈ [0, 1] is the concession factor of ai for xk ; limk is the limit of ai for xk .
Negotiators may consider strikingly different patterns of concessions as negotiation unfolds. However, the following three levels of concession magnitude are commonly discussed in the negotiation literature [3]: large, substantial, and small. To this we would add two other levels: null and complete. Accordingly, we consider the following five concession tactics: 1) stalemate: models a null concession on an issue xk at stake; 2) tough: models a small concession on xk ; 3) moderate: models a substantial concession on xk ; 4) soft: models a large concession on xk ; 5) accommodate: models a complete concession on xk . These and other similar tactics can be defined by considering specific values for the concession factor fk . In particular, the “stalemate” tactic is defined by fk = 0 and the “accommodate” tactic by fk = 1. The other three tactics are defined by considering values for fk in different ranges (e.g., the “tough” tactic by fk ∈ ]0.00, 0.05], the “moderate” tactic by fk ∈ ]0.05, 0.15], and the “soft” tactic by fk ∈ ]0.15, 0.20]). Now, concession tactics can generate new values for each issue at stake by considering specific criteria. Typical criteria include the time elapsed since the beginning of negotiation, the quantity of resources available, the previous behavior of the opponent, and the total concession made on each issue throughout negotiation (see, e.g., [1], [5]). In this work, we consider a new criterion—the amount or quantity of energy traded in a given period of a day. As stated earlier, deregulated electricity markets offer new opportunities but also create a number of challenges that need to be addressed to ensure long-term capacity sustainability. To this end, there have been a number of initiatives to distribute energy demand over time. In particular, many companies have already presented a two-rate tariff to smooth the daily demand profile (cheaper night tariff). This dual model can easily be refined if, instead of two rates, companies offer three rates or even an hour-wise tariff. For instance, consider that a company has set the hourly rates in accordance to the global demand, and suppose that from 7am to 9am and from 7pm to 9pm are peak hours, from 12am to 3pm and from 10pm to 11pm is a medium-load period and, finally, the rest is low-demand time (off peak). With this model, the company can improve its preference expression capability—not only it is clear that is better to consume at night rather than during the day, but also consuming at certain hours is favoured. This rating scheme can be seen as a communication tool between suppliers and customers. On the one hand, suppliers “advise” customers when to place consumption. From the point of view of customers, the increment of the price at certain hours constitutes an incentive to move consumption into cheaper hours.
IV. C ONCLUSION This paper has presented a model for software agents that handles two-party and multi-issue negotiation. The model incorporates a bilateral negotiation protocol, a set of concession strategies, and a set of concession tactics. Market participants equipped with the model are able to enter into bilateral contracts to protect themselves from volatility. Our aim for the future is to continue the development of the model and to perform a number of inter-related experiments to empirically evaluate its key components. R EFERENCES
Figure 1.
Exponential functions for the computation of f .
Consider a wise tariff involving m ∈ [1, 24] periods. Let ai ∈ A be a negotiating agent and E the amount of energy that ai is willing to trade in a specific period. We model the concession factor f of ai by the following family of exponential functions: f (E ) = exp where: (i) (ii)
−β
E ET
R
β ∈ + is a parameter; ET is the total amount of energy that ai is willing to trade in a day. This family of functions represents an infinite number of possible tactics, one for each value of β (see Fig. 1). III. R ELATED W ORK Multi-agent energy markets have received some attention lately and a number of prominent tools have been proposed in the literature, including: • EMCAS - Electricity Market Complex Adaptive System [2]: software agents with negotiation competence use strategies based on machine-learning and adaptation to simulate electricity markets; • AMES -Agent-based Modeling of Electricity Systems [4]: open-source computational laboratory for studying wholesale power markets, restructured in accordance with U.S. Federal Energy Regulatory Commission. Also, worthy to mention is the MASCEM system [17]. Nevertheless, despite the power and elegance of these and other existing EM simulators, they often lack generality and flexibility, mainly because they are limited to particular features of market players. Our work addresses the challenge of using software agents with negotiation competence to help manage the complexity of EMs. Our interest lies mainly in formalizing important strategies and tactics motivated by rules-of-thumb distilled from good behavioral practice in real-life negotiations.
[1] P. Faratin, C. Sierra and N. Jennings, Negotiation Decision Functions for Autonomous Agents, Robotics and Autonomous Systems, 24 (3-4), pp. 59-182, 1998. [2] V. Koritarov, Real-World Market Representation with Agents: Modeling the Electricity Market as a Complex Adaptive System with an Agent-Based Approach, IEEE Power & Energy magazine, pp 39–46, 2004. [3] R. Lewicki, B. Barry, D. Saunders and J. Minton, Negotiation, McGraw Hill, New York, 2003. [4] H. Li and L. Tesfatsion, Development of Open Source Software for Power Market Research: The AMES Test Bed, Journal of Energy Markets, 2:111–128, 2009. [5] F. Lopes, N. Mamede, A. Q. Novais and H. Coelho, A Negotiation Model for Autonomous Computational Agents: Formal Description and Empirical Evaluation, Journal of Intelligent & Fuzzy Systems, 12, pp. 195–212, 2002. [6] F. Lopes, N. Mamede, A. Q. Novais and H. Coelho, Negotiation Strategies for Autonomous Computational Agents, In: 16th European Conference on Artificial Intelligence (ECAI04), IOS Press, pp 38–42, 2004. [7] F. Lopes, M. Wooldridge and A. Q. Novais, Negotiation Among Autonomous Computational Agents: Principles, Analysis and Challenges, Artificial Intelligence Review, 29, pp. 1–44, 2008. [8] F. Lopes, A . Q. Novais and H. Coelho, Bilateral Negotiation in a Multi-Agent Energy Market, In: Emerging Intelligent Computing Technology and Applications, Springer Verlag, Heidelberg, pp. 655-664, 2009. [9] F. Lopes and H. Coelho, Multi-agent Negotiation in Electricity Markets, In: E-Commerce and Web Technologies (LNBIP 85), Springer Verlag, Heidelberg, pp. 114-123, 2011. [10] M. Osborne and A. Rubinstein, Bargaining and Markets, Academic Press, San Diego, 1990. [11] M. Pechoucek and V. Marik, Industrial Deployment of Multiagent Technologies: Review and Selected Case Studies, Autonomous Agents and Multi-Agent Systems, vol. 17, pp. 397– 431, 2008. [12] D. Pruitt, Negotiation Behavior, Academic Press, New York, 1981. [13] D. Pruitt and P. Carnevale, Negotiation in Social Conflict, Open University Press, Philadelphia, 1993. [14] H. Raiffa, The Art and Science of Negotiation, Harvard University Press, Cambridge, 1982. [15] H. Raiffa, J. Richardson and D. Metcalfe, Negotiation Analysis, Harvard University Press, Cambridge, 2002. [16] M. Shahidehpour, H. Yamin and Z. Li, Market Operations in Electric Power Systems, John Wiley & Sons, Chicester, 2002. [17] Z. Vale, T. Pinto, I. Prac¸a and H. Morais MASCEM - Electricity markets simulation with strategically acting players. IEEE Intelligent Systems, 26(2), 2011.
Abstract—Electricity markets are systems for effecting the purchase and sale of electricity using supply and demand to set energy prices. Market participants are commonly exposed to financial risks, particularly high price volatility. This article addresses the challenge of using software agents with negotiation competence to help manage the complexity of electricity markets. Specifically, it presents a model for software agents that handles two-party and multi-issue negotiation. Market participants equipped with the model are able to enter into forward contracts to protect themselves from volatility at times of peak demand and supply shortages. Keywords-electricity markets; bilateral contracts; intelligent software agents; automated negotiation.
I. I NTRODUCTION The electricity industry throughout the world, which has long been dominated by vertically integrated utilities, has experienced major changes. In particular, liberalization has led to the establishment of a wholesale market for electricity generation and a retail market for electricity retailing. Market forces drive now the price of electricity and reduce the net cost through increased competition. Electricity markets (EMs) differ from their more traditional counterparts because energy is difficult to store and has to be available on demand. Consequently, market participants are forced to work with consumption prognoses, which, in turn, create a number of risks. Firstly, the price of electricity mainly depends on the production cost and generating more than is consumed is not economical. Secondly, if suppliers cannot match the demand, the lack of energy can cause power cuts (brownouts) or, if prolonged, blackouts. Finally, there are non-negligible costs stemming from variations in the electricity production volume that most traditional types of energy generators (e.g., hydroelectric or thermoelectric) have to face. Clearly, financial risk management is a high priority for market participants. This work was performed under the project MAN-REM: Multi-agent Negotiation and Risk Management in Electricity Markets (FCOMP-010124-FEDER-020397), and supported by both FEDER and National funds through the program “COMPETE−Programa Operacional Tem´atico Factores de Competividade”.
Helder Coelho University of Lisbon Bloco C6, Piso 3, Campo Grande 1749-016 Lisbon, Portugal [email protected]
The two key objectives of EMs are ensuring a secure and efficient operation and decreasing the cost of electricity utilization. To achieve these goals, three major market models have been considered [16]: pools, bilateral contracts, and hybrid models. A pool, or power exchange, is a market place where electricity-generating companies submit production bids and corresponding market-prices, and consumer companies submit consumption bids. A market operator uses a market-clearing tool, typically a standard uniform auction, to set market prices. Bilateral contracts are negotiable agreements between two parties to exchange electric power under a set of specified conditions, such as price, MW amount, time of delivery, and duration. Market participants set the terms and conditions of agreements independent of the market operator. They often enter into bilateral contracts to hedge against pool price volatility. Furthermore, these contracts are very flexible since the negotiating parties can specify their own contract terms. The hybrid model combines several features of pools and bilateral contracts. In this model, a pool is not mandatory, and customers can either negotiate power supply agreements directly with suppliers or accept power at the pool market price. Thus, this model offers a true customer choice and an impetus for the creation of a wide variety of services and pricing options to best meet individual customer needs. Multi-agent systems are essentially loosely coupled networks of software agents that interact to solve problems that are beyond the individual capabilities of each agent. Agents are computer systems capable of flexible, autonomous action in order to meet their design objectives. Agent technology has been used to solve real-world problems in a range of industrial and commercial applications (see, e.g., [11]). Conceptually, a multi-agent approach is an ideal fit to the naturally distributed domain of a deregulated electricity market. Accordingly, this work looks at using software agents with negotiation competence to help manage the complexity of EMs, particularly the issues associated with the negotiation of bilateral contracts, towards protecting market participants from price risks.
Specifically, this paper presents a model for software agents that handles two-party and multi-issue negotiation. The model incorporates a bilateral negotiation protocol, a set of concession strategies, and a set of concession tactics. The protocol formalizes the set of possible tasks that the agents can perform during the course of negotiation. The strategies and tactics formalize the tasks that each agent should perform to negotiate effectively. This paper builds on our previous work in the areas of automated negotiation and electricity markets. In particular, it extends the work presented in [5], [6] and [7] by introducing precise definitions for concession strategies. It also extends the work presented in [8] and [9] by formalizing a set of energy dependent tactics for computing new values for each issue at stake. The remainder of the paper is structured as follows. Section II presents a negotiation model for software agents. Section III discusses related work and compares the negotiation model with other existing models. Finally, section IV presents concluding remarks and indicates future avenues of research. II. A N EGOTIATION M ODEL Let A = {a1 , a2 } be the set of autonomous agents (negotiating parties). Both the number of agents and their identity are fixed and known to all the participants. Let I = {x1 , . . . , xn } be the negotiating agenda— the set of issues to be deliberated during negotiation. Let D = {D1 , . . . , Dn } be the set of issue domains. For each issue xk , the range of acceptable values is represented by the interval Dk = [mink , maxk ]. A. Pre-Negotiation Pre-negotiation is the process of preparing and planning for negotiation and involves mainly the creation of a wellconceived plan specifying the activities that negotiators should attend to before actually starting to negotiate. In particular, effective pre-negotiation requires that negotiators prioritize the issues at stake, define the limits and targets, select an appropriate protocol, and specify the preferences. Prioritization involves deciding which issues are most important and which are least important. Target setting involves defining two key points for each issue: the resistance point or limit and the target point or level of aspiration. Definition 1 (Priority, Weight). The priority prtk of an agent ai ∈ A for an issue xk ∈ I is a number that represents the importance of xk . The weight wk is a number that represents the preference for xk . Definition 2 (Limit, Target Point). The limit limk of ai ∈ A for an issue xk ∈ I is the ultimate fallback position for xk , the point beyond which ai is unwilling to concede on xk . The target point trgk is the point at which ai is satisfied with the value of xk .
The negotiation protocol is an alternating offers protocol [10]. Two agents or players bargain over the division of the surplus of n ≥ 2 distinct issues. The players determine an allocation of the issues by alternately submitting proposals at times in T = {1, 2, . . .}. This means that one proposal is made per time period t ∈ T , with an agent, say ai ∈ A, offering in odd periods {1, 3, . . .}, and the other agent aj ∈ A offering in even periods {2, 4, . . .}. The agents have the ability to unilaterally opt out of the negotiation when responding to a proposal. The negotiation process starts with ai submitting a proposal p1i→j to aj in period t = 1. The agent aj receives p1i→j and can either accept the offer (Yes), reject it and opt out of the negotiation (Opt), or reject it and continue bargaining (No). In the first two cases the negotiation ends. Specifically, if p1i→j is accepted, negotiation ends successfully and the agreement is implemented. Conversely, if p1i→j is rejected and aj decides to opt out, negotiation terminates with no agreement. In the last case, negotiation proceeds to the next time period t = 2, in which aj makes a counter-proposal p2j→i . The tasks just described are then repeated. Once an agreement is reached, the agreed-upon allocations of the issues are implemented. Definition 3 (Proposal). Let A be the set of negotiating agents and I the set of issues at stake in negotiation. Let T be the set of time periods. A proposal pti→j submitted by an agent ai ∈ A to an agent aj ∈ A in period t ∈ T is a vector of issue values: pti→j = (v1 , . . . , vn ) where vk , k = 1, . . . , n, is a value of an issue xk ∈ I. Definition 4 (Agreement, Possible Agreements). An agreement is a proposal accepted by all the negotiating agents in A. The set of possible agreements is: S = {(v1 , . . . , vn ) ∈
Rn : vk ∈ Dk ,
for k = 1, . . . , n}
where vk is a value of an issue xk ∈ I. Negotiators should express their own preferences to rate and compare incoming offers and counteroffers. Let I = {x1 , . . . , xn } be the agenda and D = {D1 , . . . , Dn } the set of issue domains. We consider that each agent ai ∈ A has a continuous utility function: Ui : {D1×. . .×Dn } ∪ {Opt, Disagreement} → . The outcome Opt is interpreted as one of the agents opting out of the negotiation in a given period of time. Perpetual disagreement is denoted by Disagreement. Now, the additive model is probably the most widely used in multi-issue negotiation: the parties assign numerical values to the different levels on each issue and add them to get an entire offer evaluation [14]. This model is simple and intuitive, and therefore well suited to the purposes of this work.
R
Definition 5 (Multi-Issue Utility Function). Let A = {a1 , a2 } be the set of negotiating agents and I = {x1 , . . . , xn } the negotiating agenda. The utility function Ui of an agent ai ∈ A to rate offers and counter-offers takes the form: Ui (x1 , . . . , xn ) =
n X
wk Vk (xk )
k=1
where: (i) (ii)
wk is the weight of ai for an issue xk ∈ I; Vk (xk ) is the (marginal) utility function of ai for xk , i.e., the function that gives the score ai assigns to a value of an issue xk . Negotiation may end with either agreement or no agreement. Failure to agree can occur in two ways: (i) either party decides to opt out unilaterally, or (ii) the two do not agree to any proposal. The resistance points or limits play a key role in reaching agreement when the parties have the ability to unilaterally opt out of the negotiation—they define the worst agreement for a given party which is still better than opting out. For each agent ai ∈ A, we will denote this agreement by sˆi ∈ S. Hence, sˆi will be the least-acceptable agreement for ai , i.e., the worst (but still acceptable) agreement for ai . The set of all agreements that are preferred by ai to opting out will be denoted by Si .
The negotiation protocol defines the states (e.g., accepting a proposal), the valid actions of the agents in particular states (e.g., which messages can be sent by whom, to whom, at what stage), and the events that cause states to change (e.g., proposal accepted). It marks branching points at which agents have to make decisions according to their strategies. Thus, at each step of negotiation, agents often need to follow their strategies to choose among different possible actions to execute. 1) Concession Strategies: These strategies are computationally tractable functions that model typical patterns of concessions. For a given time period t > 1 of negotiation, they specify the concession tactics to be used in preparing counter-offers. The words “computationally tractable functions” presume that agents are able to compute concession strategies in a reasonable amount of time. A formal definition of a generic strategy follows. Definition 7 (Concession Strategy). Let A be the set of negotiating agents, I the negotiating agenda, T the set of time periods, and S the set of possible agreements. Let ai ∈ A be a negotiating agent and Ti its set of tactics. Let aj ∈ A be the other negotiating agent and pt−1 j→i the offer that aj has just proposed to ai in period t−1. A concession strategy Ci : T → S ∪ {Yes, No, Opt} for ai is a function with the following general form: apply Yi and prepare pti→j if 4Ui ≥ 0 accept pt−1 else reject, if aj ’s turn and Ui (pt−1 ) ≥ Ui (ˆsi )
Definition 6 (Least-acceptable Agreement, Acceptable Agreements). The least-acceptable agreement for an agent ai ∈ A is defined as: sˆi = (lim1 , . . . , limn ), where limk , j→i k = 1, . . . , n, is the limit of ai for an issue xk ∈ I. The Ci =reject pt−1 and quit, j→i set of acceptable agreements for ai is: t
offer compromise pi→j ,
Si = {s : s ∈ S, Ui (s) ≥ Ui (ˆ si )} where Ui (ˆ si ) is the utility of sˆi for ai . Perpetual disagreement is the least-preferred or worst outcome, i.e., disagreement is even worse than opting out. Formally, and more precisely, we state the following: (1) (Acceptable agreements versus opting out). For every agent ai ∈ A and acceptable agreement s ∈ Si , Ui (s) ≥ Ui (Opt). (2) (Opting out versus Disagreement). For every agent ai ∈ A, Ui (Opt) > Ui (Disagreement). B. Actual Negotiation Actual negotiation is the process of moving toward agreement and typically involves an iterative exchange of offers and counter-offers. In “good faith” negotiation, offers are made, and are either accepted or returned with counteroffers. There is an unstated assumption that the parties will show their commitment to the process of finding a solution by making concessions, and not simply by rejecting the offers of the others out of hand. To do so is often seen as “bad faith” bargaining [15].
j→i
if aj ’s turn and Ui (pt−1 si ) j→i ) < Ui (ˆ if ai ’s turn (time period t)
where: (i) (ii) (iii) (iv)
for each issue xk ∈ I, Yi is a concession tactic (see below); pti→j is the offer of ai for period t of negotiation; t 4Ui = Ui (pt−1 j→i ) − Ui (pi→j ); Ui (ˆ si ) is the utility of the least-acceptable agreement for ai , i.e., the worst (but still acceptable) agreement for ai .
Concession making can take several different forms and some representative examples are now presented. Negotiators frequently start with ambitious demands, well in excess of limits and aspirations, and concede slowly. High demands and slow concessions are often motivated by concern about position loss and image loss (or face-saving). Position loss is the abandonment of desirable alternatives, whereas image loss is the fear of appearing ready to make substantial concessions, i.e., the development in the opponent’s mind of an impression of lacking firmness [12]. Typically, high demands and slow concessions prevent position loss and guard against image loss. Furthermore, they often encourage the opponent to make concessions by the principle of reciprocity.
Bargainers generally view the world differently—they are not identical in their interests and preferences. In particular, they frequently have different strengths of preference for the issues at stake—they place greater emphasis on some key issues and make significant efforts to resolve them favourably. Hence, they concede more often on less important or low-priority issues. Low-priority concession making involves changes of proposals in which larger concessions are made on low-priority than on high-priority issues— bargainers concede and typically maintain high profits for themselves [12]. Interestingly, bargainers sometimes demonstrate good will and make one or more substantial concessions that seek reciprocal concessions (such moves are different from unilateral concessions that seek no quid pro quo). Yet overt concessions aimed at eliciting counterconcessions are risky and entail the possibility of position loss and image loss. Hence, they are more likely to be made when the other party is trusted, mainly because there is less concern about both forms of loss. When trust is low, bargainers turn to one of a host of less risky moves. These include fractionation of concessions—negotiators start with a relatively riskless action and move on toward increasing levels of risk. They make a small concession and wait to see if the opponent reciprocates. If so, they then may feel sufficiently confident to venture a larger concession [13]. The generic concession strategy presented in definition 7 allows agents to model these forms of concession making (and possibly other typical patterns of concession). For instance, consider the traditional behavior of starting with ambitious demands and conceding slowly, which often encourages concession making from the opponent. Conceding slowly is defined by considering the concession tactic “tough” (see the formal definition, below). 2) Concession Tactics: These tactics are functions that model the concessions to be made throughout negotiation. A formal definition of a generic concession tactic follows (in the interests of readability, and without loss of generality, we consider that a negotiating agent ai ∈ A wants to maximize an issue xk ∈ I). Being a “generic concession tactic” means that it can model different levels of concession magnitude (e.g., small and large). Definition 8 (Concession Tactic). Let A = {a1 , a2 } be the set of negotiating agents, I = {x1 , . . . , xn } the negotiating agenda, and D = {D1 , . . . , Dn } the set of issue domains. A concession tactic Yi : Dk ×[0, 1] → Dk of an agent ai ∈ A for an issue xk ∈ I is a function with the following general form: Yi (xk , fk ) = xk − fk (xk −limk ) where: (i) (ii)
fk ∈ [0, 1] is the concession factor of ai for xk ; limk is the limit of ai for xk .
Negotiators may consider strikingly different patterns of concessions as negotiation unfolds. However, the following three levels of concession magnitude are commonly discussed in the negotiation literature [3]: large, substantial, and small. To this we would add two other levels: null and complete. Accordingly, we consider the following five concession tactics: 1) stalemate: models a null concession on an issue xk at stake; 2) tough: models a small concession on xk ; 3) moderate: models a substantial concession on xk ; 4) soft: models a large concession on xk ; 5) accommodate: models a complete concession on xk . These and other similar tactics can be defined by considering specific values for the concession factor fk . In particular, the “stalemate” tactic is defined by fk = 0 and the “accommodate” tactic by fk = 1. The other three tactics are defined by considering values for fk in different ranges (e.g., the “tough” tactic by fk ∈ ]0.00, 0.05], the “moderate” tactic by fk ∈ ]0.05, 0.15], and the “soft” tactic by fk ∈ ]0.15, 0.20]). Now, concession tactics can generate new values for each issue at stake by considering specific criteria. Typical criteria include the time elapsed since the beginning of negotiation, the quantity of resources available, the previous behavior of the opponent, and the total concession made on each issue throughout negotiation (see, e.g., [1], [5]). In this work, we consider a new criterion—the amount or quantity of energy traded in a given period of a day. As stated earlier, deregulated electricity markets offer new opportunities but also create a number of challenges that need to be addressed to ensure long-term capacity sustainability. To this end, there have been a number of initiatives to distribute energy demand over time. In particular, many companies have already presented a two-rate tariff to smooth the daily demand profile (cheaper night tariff). This dual model can easily be refined if, instead of two rates, companies offer three rates or even an hour-wise tariff. For instance, consider that a company has set the hourly rates in accordance to the global demand, and suppose that from 7am to 9am and from 7pm to 9pm are peak hours, from 12am to 3pm and from 10pm to 11pm is a medium-load period and, finally, the rest is low-demand time (off peak). With this model, the company can improve its preference expression capability—not only it is clear that is better to consume at night rather than during the day, but also consuming at certain hours is favoured. This rating scheme can be seen as a communication tool between suppliers and customers. On the one hand, suppliers “advise” customers when to place consumption. From the point of view of customers, the increment of the price at certain hours constitutes an incentive to move consumption into cheaper hours.
IV. C ONCLUSION This paper has presented a model for software agents that handles two-party and multi-issue negotiation. The model incorporates a bilateral negotiation protocol, a set of concession strategies, and a set of concession tactics. Market participants equipped with the model are able to enter into bilateral contracts to protect themselves from volatility. Our aim for the future is to continue the development of the model and to perform a number of inter-related experiments to empirically evaluate its key components. R EFERENCES
Figure 1.
Exponential functions for the computation of f .
Consider a wise tariff involving m ∈ [1, 24] periods. Let ai ∈ A be a negotiating agent and E the amount of energy that ai is willing to trade in a specific period. We model the concession factor f of ai by the following family of exponential functions: f (E ) = exp where: (i) (ii)
−β
E ET
R
β ∈ + is a parameter; ET is the total amount of energy that ai is willing to trade in a day. This family of functions represents an infinite number of possible tactics, one for each value of β (see Fig. 1). III. R ELATED W ORK Multi-agent energy markets have received some attention lately and a number of prominent tools have been proposed in the literature, including: • EMCAS - Electricity Market Complex Adaptive System [2]: software agents with negotiation competence use strategies based on machine-learning and adaptation to simulate electricity markets; • AMES -Agent-based Modeling of Electricity Systems [4]: open-source computational laboratory for studying wholesale power markets, restructured in accordance with U.S. Federal Energy Regulatory Commission. Also, worthy to mention is the MASCEM system [17]. Nevertheless, despite the power and elegance of these and other existing EM simulators, they often lack generality and flexibility, mainly because they are limited to particular features of market players. Our work addresses the challenge of using software agents with negotiation competence to help manage the complexity of EMs. Our interest lies mainly in formalizing important strategies and tactics motivated by rules-of-thumb distilled from good behavioral practice in real-life negotiations.
[1] P. Faratin, C. Sierra and N. Jennings, Negotiation Decision Functions for Autonomous Agents, Robotics and Autonomous Systems, 24 (3-4), pp. 59-182, 1998. [2] V. Koritarov, Real-World Market Representation with Agents: Modeling the Electricity Market as a Complex Adaptive System with an Agent-Based Approach, IEEE Power & Energy magazine, pp 39–46, 2004. [3] R. Lewicki, B. Barry, D. Saunders and J. Minton, Negotiation, McGraw Hill, New York, 2003. [4] H. Li and L. Tesfatsion, Development of Open Source Software for Power Market Research: The AMES Test Bed, Journal of Energy Markets, 2:111–128, 2009. [5] F. Lopes, N. Mamede, A. Q. Novais and H. Coelho, A Negotiation Model for Autonomous Computational Agents: Formal Description and Empirical Evaluation, Journal of Intelligent & Fuzzy Systems, 12, pp. 195–212, 2002. [6] F. Lopes, N. Mamede, A. Q. Novais and H. Coelho, Negotiation Strategies for Autonomous Computational Agents, In: 16th European Conference on Artificial Intelligence (ECAI04), IOS Press, pp 38–42, 2004. [7] F. Lopes, M. Wooldridge and A. Q. Novais, Negotiation Among Autonomous Computational Agents: Principles, Analysis and Challenges, Artificial Intelligence Review, 29, pp. 1–44, 2008. [8] F. Lopes, A . Q. Novais and H. Coelho, Bilateral Negotiation in a Multi-Agent Energy Market, In: Emerging Intelligent Computing Technology and Applications, Springer Verlag, Heidelberg, pp. 655-664, 2009. [9] F. Lopes and H. Coelho, Multi-agent Negotiation in Electricity Markets, In: E-Commerce and Web Technologies (LNBIP 85), Springer Verlag, Heidelberg, pp. 114-123, 2011. [10] M. Osborne and A. Rubinstein, Bargaining and Markets, Academic Press, San Diego, 1990. [11] M. Pechoucek and V. Marik, Industrial Deployment of Multiagent Technologies: Review and Selected Case Studies, Autonomous Agents and Multi-Agent Systems, vol. 17, pp. 397– 431, 2008. [12] D. Pruitt, Negotiation Behavior, Academic Press, New York, 1981. [13] D. Pruitt and P. Carnevale, Negotiation in Social Conflict, Open University Press, Philadelphia, 1993. [14] H. Raiffa, The Art and Science of Negotiation, Harvard University Press, Cambridge, 1982. [15] H. Raiffa, J. Richardson and D. Metcalfe, Negotiation Analysis, Harvard University Press, Cambridge, 2002. [16] M. Shahidehpour, H. Yamin and Z. Li, Market Operations in Electric Power Systems, John Wiley & Sons, Chicester, 2002. [17] Z. Vale, T. Pinto, I. Prac¸a and H. Morais MASCEM - Electricity markets simulation with strategically acting players. IEEE Intelligent Systems, 26(2), 2011.
