AUCTIONS WITH ADAPTIVE ARTIFICIALLY INTELLIGENT AGENTS
Auctions with Adaptive Artificially Intelligent Agents James Andreoni John H. Miller
SFI WORKING PAPER: 1991-01-004
SFI Working Papers contain accounts of scientific work of the author(s) and do not necessarily represent the views of the Santa Fe Institute. We accept papers intended for publication in peer-reviewed journals or proceedings volumes, but not papers that have already appeared in print. Except for papers by our external faculty, papers must be based on work done at SFI, inspired by an invited visit to or collaboration at SFI, or funded by an SFI grant. ©NOTICE: This working paper is included by permission of the contributing author(s) as a means to ensure timely distribution of the scholarly and technical work on a non-commercial basis. Copyright and all rights therein are maintained by the author(s). It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may be reposted only with the explicit permission of the copyright holder. www.santafe.edu
SANTA FE INSTITUTE
AUCTIONS WITH ADAPTIVE ARTIFICIALLY INTELLIGENT AGENTS
J ames Andreoni University of Wisconsin
John H. Miller Carnegie Mellon University' and Santa Fe Institute
March, '1990 revised December 10, 1990
r
j ).
Abstract While there is an extremely active theoretical literature on auctions, little is known about the actual behavior of bidders in real auctions. This is partly due to the scientist's inability to observe the values or strategies of bidders. When values are induced using experiments, subjects are often unable to formulate Nash equilibrium bids. Rather than using human subjects, this paper examines the ability of agents to learn bidding strategies by using artificially intelligent agents who learn adaptively. We examine first-price common-value auctions, and first- and second-price auctions with both affiliated and independent private-values. We find that there are many striking parallels between the artificial agents and humans. Since we can observe both the values and the strategies of the artificial agents, we are able to conjecture about .the behavior of humans in the different auction environments. With this model of adaptive learning, we are able to understand many of the "bidder errors" 9bserved in experiments. We suggest several new hypotheses about bidding behavior, and about the various auction institutions.
Acknowledgments
This program is funded .by grants from Citicorp/Citibank and the Russell Sage Foundation and by grants to SFI from the John D. and Catherine T. MacArthur Foundation, the National Science Foundation (PHY8714918), and the U.S. Department of Energy (ER-FG05-88ER25054).
."
1. Introduction
In recent years there has been growing interest in the economic theory of auctions, and in the experimental examinations of these theories. 1 The experiments on auctions often indicate that subjects deviate systematically from risk-neutral Nash equilibrium behavior. These deviations are sometimes consistent with risk aversion, but at other times directly contradict it (see Kagel and Roth, 1990, for a discussion). Experimenters have attributed these deviations primarily to "bidder error." For instance, Kagel, Levin, and Harstad (1987) found· that in private-values auctions the Nash bidding model "falls far short of providing a complete characterization of the data: the bids... suggest a sizable portion of individuals whose
~ubjettive
expectations deviate from fully rational
expectations, and/or who fail to act on strategic implications of public information"(p. 1293). When discussing common-value auctions, Kagel and Levin (1986) indicate that "behavior fails to conform, in important ways, with the requirements of Nash equilibrium bidding strategies" (p. 10). In a summary and review of these results, Kagel and Roth (1990) conclude that subjects' behavior"must result, at least in part, from bidding errors relative to any well defined Nash equilibrium bidding model." In light of this', experimenters have begun to discuss the process by which subjects determine their bids. In several of their papers, Kagel and his co-authors have made observations about
sub~
jects like the following: "Learning .may be the result of trial and error which through information feedback results in the adoption of rules that 'work,' or it may be the result of the m'arket environment 'selecting for' those agents whose behavior enables them to survive" (Dyer, Kagel and Levin, 1989, p. 115), and that subjects "display clear patterns consistent with genuinely evolutionary
learning processes" (Kagel and Dyer, 1988, p. 196).2 Economists have recently been investigating models of learning, evolution, and· adaptation. Milgrom and Roberts (1990), Fudenberg. and, Kreps (1988), Fudenberg and Maskin (1990), Fudenberg and Levine (1990), Kalai and
L~hrer
(1990), Crawford (1990), van Da:qlme (1989, ch.
9), Selton (1989), Woo.dford (1990), Brock (1990) and others have studied theoretical models of
learning and adaptation in games.
St~ll
others have' focused on related models. of learning using
1 For theoretical work· on auctions, see Milgrom, and W~ber (1982a, 1982b), Maskin, and Riley (1984), Riley and Samueslon (1981), or for reviews see Milgrom (1985) and McAfe'e and Mc11illan (1987)~ For experiments, see Cox, Smith, and Walker (1983, 1985), Cox, Roberson, and Smith (1982), and Kagel, Harstad, and Levin (1987), Kagel ~nd Levin (1985, 1986), Kagel and Dyer (1988), Kagel, Levin, and Harstad (1988), and Dyer, Kagel, and Levin (1989). 2 See also Kagel and Levin (1986, p. 917), and Kagel, Levin and Harstad (1988).
1
adaptive artificially intelligent (AAI) agents, which are based on com-puter algorithms of adaptation, l~arnin-g, and search. These include Axelrod (1987), Miller (1989), Marimon, McGrattan, and Sargent (1990), Rust, Palmer, and Miller (1990), Boylon (1990), and Marimon and Miller (1990), and Binmore and Samuelson (1990). These authors have primarily used AAI techniques to identify potentially plausible solutions or equilibria in games, and to motivate further theoretical studies. These papers reflect the growing interest in using computer science models in economics. 3 Finally, several authors have used AAI techniques to study behavior in economic laboratory experiments. These studies include Crawford (1989) and Miller and Andreoni (1990). Using an AAI model of adaptive learning-, they make predictions about behavior in experiments. When the predictions match behavior, the AAI model can help generate hypotheses for why behavior mayor may not coincide with the Nash equilibrium predictions, and for how easily Nash equilibria can emerge among players who learn adaptively. These hypotheses can then help shape future theoretical and experimental studies. The purpose of this paper is to explore learning in auctions by using AAI bidders. We will consider an adaptive algorithm called a Genetic Learning Algorithm. Genetic algorithms are intended to simulate the kind of learning described by Kagel and co-authors, an-d have been used extens~vely
by computer scientists. We
conside~ :fi~st-price
common-value auctions, and first- and
second-price- independent-value~ and affiliated-values auctions. We findthat·patterns-of bidding in the AAI auctions parallel many of the .patterns of bidding observed in the auction experiments. Since we observe the strategies of-our AAI bidders as well as the bids, we are able to analyze bidding behavior and, as a result, can conjecture about both human behavior and auction institutions in general. We conclude, for instance, that Nash equilibrium bidding seems to be most- easily learned in affiliated-values- auctions. This is true for both first..: and second-price auctions. Surprisingly, both first- and second-price in-dependent-values auctions present the most difficult environments. Bidders in_ th-ese .auctions· have the most complete information on ·the distribution 'of the private values bfother bidders. Our results indicat-e that this information may serve mainly- to complicate the task of bidders-. This is because the info~mation given bidders in independent-values auctions is highly correlated, which makes adaptive learning esp'ecially difficult, even when the payoff space is relatively steep. We also find that risk aversion does not yield behavior more- consistent with Nash bidding in -first-price auctions, although it does improve perfot:mance in second-price auctions. 3_ See the Journ3.I of E-conomic Dynamics and Control, Special Issue on Computer Science and Economics, (1990). w-
2
Finally, by comparing small and large groups in common-value auctions, we can
conj~cture
that
the winner's curse observed in large groups may have more to do with the fact that a given error is more costly in a large group than with difficulties in learning or with meaningful deviations from rationality. In total, we are able to understand many of the "bidder errors" observed in experiments with this model of adaptive learning. The next section discusses the particular learning algorithm used in this. paper. Section, 3 describes the structures of the auctions studied. Sections 4 and 5 discuss the results, and section 6 provides a summary and conclusion. 2. Genetic Learning Algorithms
Genetic algorithms (GAs) were developed by Holland (1975)' as adaptive models of search, learning and optimization. GAs are based on natural models of selection and evolution. GAs are one of a the class of adaptive algorithms, such as replicator dynamics, classifier systems and neural networks. GAs are designed to work well in difficult environments, especially those large
se~rch
wit~
spaces, and are relatively robust to changes in the parameters of the algorithm. The
GA operates, when applied to game.theory, by evolving a population, of strategies. Strategies that perform- poorly are removed from the population, while strategies that ~perform well' are retained. New strategies are created by combining information-from existing strategies and by introducing new "genetic material" into the population. Hence, the GA represents learning through mimicking, experimentation, and innovation. A player's strategy is represented in the GA by a string of characters, typically binary. In our . application, a binary string is interpreted as parameters that form a bidding strategy. Each' strategy in the population interacts in an environment, ,usually consisting of the other strategies in the population, and receives a payoff.. A fraction of the strategies is then chosen to go directly into the new population, with the chance of "survival" depending on relative
payoff~
Hence, strategies that
perform better than others tend· to perpetuate., .This allows learning by the mimicking of successful strategies. Next, new strategies are created usin.g two "genetic operators" called crossover and mutation. Crossover recombines parts of two existing strategies at a randomly chosen point. For example, a crossover of the two 4-bit strings 1111 and" 0000 after the second bit results in the new strings, 1100 and 0011..Crossover represents experimentation with new combination of existing strategies in an effort to discover
improveme~ts.
3
The second genetic operator, mutation, alters a
single character of the string. If the string 0011 is mutated at the last location the result is 0010. Mutation allows innovation by introducing "new ideas" into the learning pro~ess.4 The algorithm used here has the following design. A population of 40 binary strings is randomly generated. Each string represents two parameters of a bidding function. 5 Depending on group size (either four or eight), each string participates in a round of auctions (either 100 or 200 auctions respectively6). The payoff to each string is the sum of its profits· for the auctions conducted during the round. A new population of 40 strings is then formed by first assigning each string a probability weight based on its relative payoffs in the last round, with higher payoffs yielding greater weight. 7 With these weights, 20 pairs of strings are randomly drawn from the population, with replacement.; With a 50% probability, a given pair of strings will go directly into the new population. Otherwise, the pair .will·undergo crossover.8 After the crossover, each newly formed string will also undergo mutation. Each bit of the new strings will be mutated with probability p, where p is initially 8% but exponentially decays in half every 250 generations. This completes a new generation. The strings are again introduced into the auction environment, and the proced·ure is repeated for 1000 generations.
In addition to the parameters· of the GA reported.here, ~e also considered a num~er of variations of these parameters. We found that. the results are robust to algorithmic changes that do not diminish the level of feedback to the
play~rs.
a~d
parametric
For instance, changing the muta-
tion rate has little effect. However, reducing the number of auctions in a round without increasing the number of generations does change the dynamics. In particular, since non-zero payoffs are only. received when an auction is won, reducing the number of auctions slows the convergence, although the· relative performance of the GA across auctions is not affected. For a detailed introduction· to GAs see Goldberg (1989). The· first parameter was restricted· to the range [0,2] and the second· one to [-2,.2]. Each parameter was coded by 10 bits. The 10 bits were interpreted as. a binary integer (if the parameter was signed·, then the initial bit determined the sign) normalized t.o·the appropriate range. 6 The different number of auctions was designed to give identical information flows ·under the two grollp sizes~ Recall that each simulation has· 40 total bidders. Hence, in the 4- bidder groups . there were (lOOauctions)/(group) X (10 groups) =1000 auctions per generation. Similarly, in the 8-bidder groups there were (200 auctions)j(group). x (5 groups)=1000 auctions per generation. 7 Strings with payoffs below 1.5 standard deviations from the mean were removed from. .the population. After these strings were removed the scores were normalized. See Miller (1989) for more detail. . 8. Th~ crossover was a circular crossover, that is, the strings were assumed to exist on a circle and a segment of the circle was exchanged. This technique prevents a bias towards preserving the end points that occurs in the linear operation illustrated earlier. 4
5
4
Three different environments are explored for each type of auction: coevolution, full-feedback, and' Nash. In the coevolutionary environment, strategies interact with other evolving strategies. Auctions are held in randomly formed groups, and payoffs equal the bidder's actual profit or los8. 9 In the full-feedback environment, strings do not participate in auctions but instead each strategy receives a payoff equal to minus its squared deviation from the Nash equilibrium bid. This environment is used to test the GA's ability to solve the function fitting problem. The payoffs in the full-feedback condition are independent of the choices of other agents, and do not depend on winning 'an auction. In addition, the payoff space in the full-feedback condition is the same across all auctions, that is, accuracy is rewarded identically. Finally, in the Nash environment, each strategy bids in auctions consisting entirely of opponents who use the Nash equilibrium bidding strategy. If the strategy wins the auction, then its payoff equals its profit, otherwise its payoff is zero. This differs from the coevolutionary condition in that each strategy evolves against a constant environment. Since the Nash equilibrium bid is the best response in an environment, consisting entirely of other Nash equilibrium bidders, this condition tests the system's ability to find a best-response function. 3. Theoretical and.. Experi~ental Background
In order to gain insights into the behavior of experimental subjects', our computer auctio~s
are deliberately tailored after existing auction experiments. All of the, auctions have common parameters and structures in order to facilitate comparisons am.ong them. In the rest of this section we will review these actions, and the experimental findings. 3.1 Common- Value Auctions In common-value auctions the bidders -compete for an item that is of unk,nown value at the time of the auction, such as an offshoreoilleas'e, but which will have the same value to the winner, regardless of which bidder wins. The .auction is by sealed bid, and the, highest bidder wins. The difficulty for bidders is that they must account for the fact that there is information in being the highest bidder. In particular, the highest bidder is most likely to have "over-estimated" the value of the, item. A bidder who fails to account for this information may suffer' a "winner's curse" since profits from winning will be unexpectedly low. 9 In this environment, two populations of 40 bidders are simultaneously evolved, where the two· populations share no "genetic material." By having two populations coevolve against one , another some endogeneity problems .inherent in a single population are avoided. For instance; purely stochastic forces can cause a single population to rapidly converge towards a common string. This genetic drift effect was first noticed in natural populations.
5
Experiments by Kagel and Levin (1986) found that, in general, subjects suffer a winner's ~urse, especially in large groups (5 to 7 bidders). However, they find that small groups
(4 bidders)
eventually learn to bid profitably on average, although they still earn less than the Nash equilibrium prediction. The large groups continue to lose money, even with experience. Dyer, Kagel and Levin (1989) found that these effects also exist among "professional" bidders. The structure of our simulated common-value auctions is the same as that used in the Kagel and Levin (1986) experiment. Let Xo
Xo
be the value of the item, and let n be the number of bidders.
is drawn randomly from a uniform distribution on the interval [1000,2000]. Each bidder is given
a signal of the item's value, interval [xo -
f, XO
Xi.
+ f], where f
The signals are drawn randomly from a uniform distribution on the is itself drawn randomly from a uniform distribution over (0, 500}.
Hence, each bidder's signal is an unbiased estimate of the value of the item. Kagel and Levin (1986) solve for the Nash equilibrium bid function for this model,
as~uming that
bidders know the process
by which values and signals are generated. They find .the Nash equilibrium bid function is 10
(1) The expected profits conditional on winning are l ! E1r
2£ n-l
=._-~
3.2 Affiliated Private- Values Auctions In private-values auctions, bidders. have their own values for the item. These privat.e values are affiliated if the bidders do not know the distribution from which all other private-values were drawn, but know only that the higher their own value, the higher other's values are likely to be. This will be true, for example, if an item is purchased for its speculative value. This type of informational 'assumption is discussed by Milgram and Weber (1982'a). Kagel, Harstad and Levin (1987).. found ,t4at subjects in first-price affiliated private-values auctions generally bid in excess of the risk-neutral Nash equilibrium prediction, and made profits By restricting' analysis to signals in the interval [1000+ f,2000 - €], they show that the Nash equilibrium bid ~unction is b(Xi) Xi - f + Y(Xi), 'Yhere Y(Xi) =. [2f/(n '+ 1)] exp[-(n/2f)(xi 1000 - f)]. However, because of its negative exponent, Y(Xi), moves negligibly close' to zero as Xi increases beyond 1000 + €. We do. not believe that this simplification of the bid function had any significant effect on our results or conclusions. This will be discussed in subsequent footnotes. 11 In Nas,h equilibrium the bidder with the highest signal will always be the highest bidder. Hence, the expected profits are the expected value of b(Xi) - x o , conditioning on the fact that Xi is the highest signal. 10
=
6
that were below the predicted levels'. Behavior often differed significantly from Nonetheless, they found that the
risk~neutral Nash
t~e
Nash prediction.
model did a better job of explaining the data,
than a model of (constant relative) risk aversion, or two ad hoc models. Our auctions derive their structure from the formulation of Kagel,'Harstad and Levin (1987). First, a value Xo is drawn from the uniform interval [1000,2000]. A value
€
is then drawn from
the uniform interval (0,500]. Then n private values, Xi, i = 1, ... , n, are drawn 'from the uniform interval [x o
- f, X o
+ f].
To make the auctions affiliated, bidders cannot be given full information
about the distribution of private values. Hence, as in Kagel, Harstad and Levin (1987), bidders are only allowed' to know
€,
in addition to their private value, Xi. For first-price auctions, Kagel,
Harstad and Levin (19~7) show that the Nash equilibrium bidding function in this model is 12
b(Xi)
2€ n
= Xi -
(2)
-.
Profits of the winning bidder are
2£
" -
",.--
Kagel, Harstad and Levin (1987)
~so
n
.
conducted
~econd-price
auctions (or Vickrey
auction~),
in which the object goes to the highest bidder at a price equal to the second high~st bid. As is well known, it is a dominant strategy for all bidders to bid their true values. The surprising result from the experiments is that subjects bid well in excess of the dominant strategy. In 80 percent of the auctions, the market clearing price exceeded the Nash level by significant amounts. 13 The inference made from these experiments is that subjects found the strategic interactions of second-price auctions too difficult to learn in the span of the experiment. We will also conduc~ second-price auctions. In this case, the predicted Nash equilibrium profits of the winning bidder are the expected difference between the nth and (n - l)th order statistic:' 2f E1r= - - . n+1
(3)
12 As before, over the range [1000+£, 2000-€], the exact bid function is b.(Xi) = Xi- 2f/ n +y(xi)/n, where Y( Xi) is the same as given above. Hence, this bid function is again the relevant portion. Again, we do not believe this simplification affects our results or conclusions. See footnote 13 below. 13 This result was in clear contrast to the ascending-clock auctions that Kagel, Harstad and Levin (1987) designeq.to test English open-outcry auction, which are, in theory, equivalent to second-price auctions. In this case, bidders almost always dropped out of.the auction when the price exceeded their value, and market clearing prices were very near the Nash prediction.
7
3.3 Independent Private- Values Auctions A striking result of auction theory, first noted by Vickrey (1961), is the revenue equivalence theorem. This states that when risk-neutral bidders have private values for an item, and when those values are completely independent, then the firs~-price and second-price auctions will yield the same expected revenue to the sellers. Private values are independent when all bidders know the distribution from ~hich all private values are drawn. In terms of ~he model presented in the last subsection, it means that bidders know
Xo
as well as
E
during each auction.
It was shown by Milgrom and Weber (1982a) that if people i"n a first-price affiliated-values auction are given information about the distribution of values, i.e., ·are told X o , then this infor.mation will increase the equilibrium price of the object. Kagel, Harstad and Levin (1987) tested this proposition by providing subjects with
Xo
after they bid in the affiliated-values auction, and thus
transforming the auction into an independent-values auction. In theory this should cause bidder's profits to fall. They found that average profits across all treatments did fall, but that they did not fall in every treatment. In three of the five treatments average profits were virtually unchanged, even though bidders reacted strongly to the new information, and in one of the treatments profits actually increased. Only in one treatment did the profits fall significantly.
I~
a study of individual
bidder behavior, Kagel and Levin (1985) reject all models they considered to explain the da~a,. including asymmetric risk aversion. Kagel, Harstad and Levin (1987) show that the Nash equilibrium bid· function for first-price independent-values auctions is14 .
b(Xi) =
n-l n
1
--Xi
+ -(xo n
f).
(4)
Expected profits of the winning bidder can be calculated as 15 2f E1r = - - ". n+1
(5)
14 In contrast to the bid functions (1) and (2) in the other first-price auctions, (4) is· the exact Nash bidding strategy. Hence, if the approximations given in (1) and (2) have any effect, ,ve would expect that, relative to the· independent values auctions, the. approximation should hamper convergence to the predicted parameters in the common- and affiliated-values auctions. As we will see, this does not appear to be the case. Hence, any effect of the approximation, if it is present, would tend to work agai~st the conclusions drawn in this paper. 15 Equilibrium expected profits can be found by substituting the expected value of the nth order statistic, x n , into (4), and evaluation x n - b(x n ).
8
We also conducted second-price auctions for independent-values. Again, the dominant strategy is for bidders to bid their values. Hence, the equilibrium profits are the same as (3). As predicted by the revenue equivalence theorem, (3) and (5) are identical.
3.4
Parameters for the AAI Auctions
We conducted all five auctions discussed above for both 4 and 8-bidder groups. The GA searched over 2-parameter linear functions in each auction. For common- an'd affiliated-values auctions, the function's variables are are
Xi
and
Xo
-
f.
Xi
and
€,
w.hile Jor independent-values auctions the variables
The predicted values of all the parameters on the bid functions, and of the
expected profits (based on the mean value of f of 250) are given in Table 1.
TABLE 1. Theoretical Predictions. Auction
Group Size
Nash Bidding Function Xo -
Common Value
Expected Profit
f
4
1.000
-1.000
100.0
- 8-
1.000
-1.000
55.5
4
1.000
- .500
125.0
8
1.000
- .250
62.5
4
1.000
.000
100.0
8
1.000
.000
55.5
4
.750'
.250
100.0
8
.875
, .125
55.5'
4
1.000
.000
100.0 '
8
1.000
.000
55.5
Affiliated' Values first-price
second~price
Independent Values first-price
second-price
Note that for each aaction there are two parameters that must be chosen ·by theAAI bidders. From this perspective, the parameter space over which the GA must search is identical for all auctions. This means that any difference in the ability of the GA to find Nash equilibria will
9
not be attributable to the dimensions of the search space, and instead must be attributed to the
inf~rmational differences
in the auctions. As indicated above, we conducted auctions under
three information conditions: coevolution, Nash opponents, and full-feedback. For each size and information condition we ran 20 trials, with each trial lasting 1000 generations.
4. Results of A uctions with AAI Bidders
4.1
Affiliated- Values Auctions The results of the first- and second-price auctions with affiliated-values are listed in Table
2. The numbers in this table, and in all subsequent tables, list the average values of the last 2'5 generations in each trial, averaged over all 20 trials. The averages, contain information on 500,000 auctions. 16 The standard deviations refer to the differences across trials. I7 We will refer to the parameters of the bid function as a pair ({31, (32) where /31 refers to the parameter on
Xi
in the
bidding function. Begin with first-price auctions under coevolution. For 4-bidder groups the predicted values of the bid function are (1, -.5), while the GA finds parameters of (.98982, -.46417). The standard errors on these parameters
ar~
small, indicating wide agreement on the bidding function across
"trials. Moreover, average profits are 113.42, which is 90%' of the Nash prediction of 125. There is similar success with the 8-bidder groups. The predicted parameters are (1, -.25), while th'e GA finds (.99540, -.24694). Again, the standard errors across trials are small. However,8-bidder groups earn average profits of 36.79, which is only 58% of the Nash prediction.
This indicates that the GA has largely been able to converge to the Nash equilibrium. The small errors that remain are only slightly costly to the 4-biddergronps, and more ,costly to the larger groups. This leads us to ask whether these remaining errors can be attributed to the algorithm per se, or whether they can be attributed to the' kind of informational environment present in a coevolutionary game. We can examine this question with the Nash and-full-feedback
conditions~
We see that in th,e Nash condition the GA performs about as well at' finding the bid function, but 16 25 generations/trial X 100 auctions/4-bidder group x 10 4-bidder groups/generation X 20 trials = 500,000 auctions. For 8-bidder groups we ran 200 auctions per group, but had 5 groups per generation, hence there were also 500,000 auctions. 17 The standard deviation~ within trials are very small by the 1000 th generation, usually on the order of .0001 to .05. This is due to the· fact that the probability of mutation has decayed from its initial 8% to about 1/2% by the last generation, coupled with the genetic drift phenomenon inherent in' small populations. Hence, variance across trials will be the most indicative of the algorithm's ability to find a Nash equilibrium.
10
TABLE 2. Results for Affiliated-Values Auctions. Mean Values (Standard Deviations)*
Information
Co-evolving first- price
Group Size
4 8
second price
second-price'
.98982 (.00725) .99540 (.00347)
-.46417 (.06139) -.24694 (.02816)
113.42 (8.77) 36.79 (10.99)
.99116 (.00755) .99303 (.00395)
-.46892 (.05013) -.21685
..99821 (.00860) .99471 (.00471)
.00881 (.06282) .03198 (.04163)
4
.99842 (.00776)
8
1.00130 (.00515)
-.49173 (.03447) -.25634 (.02009)
125.54 (4.55) 61.88 (4.45)
4
1.00107 (.00290) 1.00004 (.00280)
-.00495 (.01620) -.00391
100.11 (0.29) 55.50 (0.15)
4
4
8
Full-Feedback first-price
€
.00008 (.07861) .04897 (.04563)
8 second-price .
Xi
Average Profit t
.99955 (.01034) .99754 (.00475)
4 8
Nash Opponents first- price
Bidding Function
8
(.019~5)
(.01648)
96.66
(7.99) 48.16 (4.39) 122.85 (13.37) 42.07 (19.02) 94.92- , (~1.89)
53.23 (8.06)
• Means and standard deviations are over the last 25 gener~tions. t Profits reported for the Nash auctions are conditional on wifl~ing the auction. Profits for the Full-Feedback auctions are calculated for each simulation as expected values, using the average strategy o~er the last 25 generations. Means and standard deviations are calculated across simulations.
11
does better as measured in profits. This is true for both 4 and 8-bidder auctions. This
d~fference
can probably be ascribed to the fact that the variance of the bid functions across auctions is smaller. Turning to the full-feedback condition, we see that performance improves even more. In both 4 and 8-bidder auctions, the bid functions are almost exactly those predicted by the Nash equilibrium. These results indicate that; while the coevolutionary environment does not generate the exact Nash prediction, it is nonetheless quite successful at finding the Nash bidding s~rategies in affiliatedvalues auctions. This is consistent with the experimental finding that Nash equilibrium bidding organized the data better than other models, although errors still remain. The' fact that the full-feedback auction can make the small improvements necessary to find the exact Nash bidding strategy indicates that the failure of the GA to do even better in the coevolutionary environment cannot be blamed
~n th~
algorithm per se -
in ideal feedback conditions, the GA can find the
optimum. Instead, these results indicate that the small errors of the coevolutionary condition appear to be due to the fact that the feedback is less than ideal. Turning to the second-price auctions we find similar results. For both 4 and 8-bidder groups the coevolutionary strategi~s are extremely close to the Nash prediction of (1, 0). Moreover, average .profits are 96% of the Nasli prediction for the 4-bidder group, "and 87% of the Nash pred~ction"for the 8-bidder group. The. Nash' condition performs about as well as
th~
coevolution condition, with
slightly less variance on the bid functions, while the full-feedback condition performs the .best by attaining almost the exact Nash strategy. Figure 1 illustrates the evolution of bids in a 4;. and 8-bidder second-price coevolving auctions. The figure shows the average deviation of the winning bid from the dominant strategy bid over all 1000 generations (averaged over every 20 generations). The deviations converge to the dominant strategy from above.
We
can explain this by examining. the dynamics of adaptive learning.. A
strategy that bids beneath ·the dominant strategy will
inc~ease
both its absolute and relative
~er
formance by raising its bid to the dominant strategy level. However, if a bidder is bidding above the dominant strategy, then moving to the dominant strategy will increase the bidder's absolute performance, but' may reduce its relative performance. This is because the other strategies will now earn higher profits when this bidder is the second highest bidder. Hence, many situations exist in which a bidder will re~iIce its chance for survival by lowering its bid to the dominant strategy level. This indicates that
th~
evolutionary forces will quickly eliminate under-bidders (who win
no auctions) as well as the extreme over-bidders (who make losses), but may be slow to eliminate
12
moderate over-bidders. This implies that evolutionary learning is likely to lead. to convergence to the dominant strategy from above. Again, this is consistent with the experimental results.
4.2 Common- Value Auctions The results for common-value auctions are listed in Table 3. Looking first at the 4-bidder coevolutionary auctions, we see that the bid functions are in the neighborhood of the Nash equilibrium strategies, with small standard errors, but are much farther from the predicted levels than are the affiliated-values auctions. In terms of profits, the coevolutionary strategies earn 82% of the Nash prediction. In all 20 trials the AAI bidders earned positive profits, with the maxi,mum at 96.1 and the minimum at" 72.2. Turning to the 8-bidder coevolutionary auctions we see that the parameters of the bid function are closer to the Nash equilibrium values than in the 4-bidder auctions. Again, the standard deviations of these parameters are small. Average profits are 18.20, which is only 33% of the Nash equilibrium amount, and in two of the twenty trials the' AAI bidders actually made losses. Overall profits varied from 51.47 to -10.59. Hence, even though the bidders in 8-bidder auctions appear to make smaller errors, these errors are more costly. Recall that Kagel and Levin (1986), and Dyer, Kagel and Levin (1989) showed that, with suffi_cient experience.,
bidd~rs
.
in 4-bidder groups are able t
SFI WORKING PAPER: 1991-01-004
SFI Working Papers contain accounts of scientific work of the author(s) and do not necessarily represent the views of the Santa Fe Institute. We accept papers intended for publication in peer-reviewed journals or proceedings volumes, but not papers that have already appeared in print. Except for papers by our external faculty, papers must be based on work done at SFI, inspired by an invited visit to or collaboration at SFI, or funded by an SFI grant. ©NOTICE: This working paper is included by permission of the contributing author(s) as a means to ensure timely distribution of the scholarly and technical work on a non-commercial basis. Copyright and all rights therein are maintained by the author(s). It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may be reposted only with the explicit permission of the copyright holder. www.santafe.edu
SANTA FE INSTITUTE
AUCTIONS WITH ADAPTIVE ARTIFICIALLY INTELLIGENT AGENTS
J ames Andreoni University of Wisconsin
John H. Miller Carnegie Mellon University' and Santa Fe Institute
March, '1990 revised December 10, 1990
r
j ).
Abstract While there is an extremely active theoretical literature on auctions, little is known about the actual behavior of bidders in real auctions. This is partly due to the scientist's inability to observe the values or strategies of bidders. When values are induced using experiments, subjects are often unable to formulate Nash equilibrium bids. Rather than using human subjects, this paper examines the ability of agents to learn bidding strategies by using artificially intelligent agents who learn adaptively. We examine first-price common-value auctions, and first- and second-price auctions with both affiliated and independent private-values. We find that there are many striking parallels between the artificial agents and humans. Since we can observe both the values and the strategies of the artificial agents, we are able to conjecture about .the behavior of humans in the different auction environments. With this model of adaptive learning, we are able to understand many of the "bidder errors" 9bserved in experiments. We suggest several new hypotheses about bidding behavior, and about the various auction institutions.
Acknowledgments
This program is funded .by grants from Citicorp/Citibank and the Russell Sage Foundation and by grants to SFI from the John D. and Catherine T. MacArthur Foundation, the National Science Foundation (PHY8714918), and the U.S. Department of Energy (ER-FG05-88ER25054).
."
1. Introduction
In recent years there has been growing interest in the economic theory of auctions, and in the experimental examinations of these theories. 1 The experiments on auctions often indicate that subjects deviate systematically from risk-neutral Nash equilibrium behavior. These deviations are sometimes consistent with risk aversion, but at other times directly contradict it (see Kagel and Roth, 1990, for a discussion). Experimenters have attributed these deviations primarily to "bidder error." For instance, Kagel, Levin, and Harstad (1987) found· that in private-values auctions the Nash bidding model "falls far short of providing a complete characterization of the data: the bids... suggest a sizable portion of individuals whose
~ubjettive
expectations deviate from fully rational
expectations, and/or who fail to act on strategic implications of public information"(p. 1293). When discussing common-value auctions, Kagel and Levin (1986) indicate that "behavior fails to conform, in important ways, with the requirements of Nash equilibrium bidding strategies" (p. 10). In a summary and review of these results, Kagel and Roth (1990) conclude that subjects' behavior"must result, at least in part, from bidding errors relative to any well defined Nash equilibrium bidding model." In light of this', experimenters have begun to discuss the process by which subjects determine their bids. In several of their papers, Kagel and his co-authors have made observations about
sub~
jects like the following: "Learning .may be the result of trial and error which through information feedback results in the adoption of rules that 'work,' or it may be the result of the m'arket environment 'selecting for' those agents whose behavior enables them to survive" (Dyer, Kagel and Levin, 1989, p. 115), and that subjects "display clear patterns consistent with genuinely evolutionary
learning processes" (Kagel and Dyer, 1988, p. 196).2 Economists have recently been investigating models of learning, evolution, and· adaptation. Milgrom and Roberts (1990), Fudenberg. and, Kreps (1988), Fudenberg and Maskin (1990), Fudenberg and Levine (1990), Kalai and
L~hrer
(1990), Crawford (1990), van Da:qlme (1989, ch.
9), Selton (1989), Woo.dford (1990), Brock (1990) and others have studied theoretical models of
learning and adaptation in games.
St~ll
others have' focused on related models. of learning using
1 For theoretical work· on auctions, see Milgrom, and W~ber (1982a, 1982b), Maskin, and Riley (1984), Riley and Samueslon (1981), or for reviews see Milgrom (1985) and McAfe'e and Mc11illan (1987)~ For experiments, see Cox, Smith, and Walker (1983, 1985), Cox, Roberson, and Smith (1982), and Kagel, Harstad, and Levin (1987), Kagel ~nd Levin (1985, 1986), Kagel and Dyer (1988), Kagel, Levin, and Harstad (1988), and Dyer, Kagel, and Levin (1989). 2 See also Kagel and Levin (1986, p. 917), and Kagel, Levin and Harstad (1988).
1
adaptive artificially intelligent (AAI) agents, which are based on com-puter algorithms of adaptation, l~arnin-g, and search. These include Axelrod (1987), Miller (1989), Marimon, McGrattan, and Sargent (1990), Rust, Palmer, and Miller (1990), Boylon (1990), and Marimon and Miller (1990), and Binmore and Samuelson (1990). These authors have primarily used AAI techniques to identify potentially plausible solutions or equilibria in games, and to motivate further theoretical studies. These papers reflect the growing interest in using computer science models in economics. 3 Finally, several authors have used AAI techniques to study behavior in economic laboratory experiments. These studies include Crawford (1989) and Miller and Andreoni (1990). Using an AAI model of adaptive learning-, they make predictions about behavior in experiments. When the predictions match behavior, the AAI model can help generate hypotheses for why behavior mayor may not coincide with the Nash equilibrium predictions, and for how easily Nash equilibria can emerge among players who learn adaptively. These hypotheses can then help shape future theoretical and experimental studies. The purpose of this paper is to explore learning in auctions by using AAI bidders. We will consider an adaptive algorithm called a Genetic Learning Algorithm. Genetic algorithms are intended to simulate the kind of learning described by Kagel and co-authors, an-d have been used extens~vely
by computer scientists. We
conside~ :fi~st-price
common-value auctions, and first- and
second-price- independent-value~ and affiliated-values auctions. We findthat·patterns-of bidding in the AAI auctions parallel many of the .patterns of bidding observed in the auction experiments. Since we observe the strategies of-our AAI bidders as well as the bids, we are able to analyze bidding behavior and, as a result, can conjecture about both human behavior and auction institutions in general. We conclude, for instance, that Nash equilibrium bidding seems to be most- easily learned in affiliated-values- auctions. This is true for both first..: and second-price auctions. Surprisingly, both first- and second-price in-dependent-values auctions present the most difficult environments. Bidders in_ th-ese .auctions· have the most complete information on ·the distribution 'of the private values bfother bidders. Our results indicat-e that this information may serve mainly- to complicate the task of bidders-. This is because the info~mation given bidders in independent-values auctions is highly correlated, which makes adaptive learning esp'ecially difficult, even when the payoff space is relatively steep. We also find that risk aversion does not yield behavior more- consistent with Nash bidding in -first-price auctions, although it does improve perfot:mance in second-price auctions. 3_ See the Journ3.I of E-conomic Dynamics and Control, Special Issue on Computer Science and Economics, (1990). w-
2
Finally, by comparing small and large groups in common-value auctions, we can
conj~cture
that
the winner's curse observed in large groups may have more to do with the fact that a given error is more costly in a large group than with difficulties in learning or with meaningful deviations from rationality. In total, we are able to understand many of the "bidder errors" observed in experiments with this model of adaptive learning. The next section discusses the particular learning algorithm used in this. paper. Section, 3 describes the structures of the auctions studied. Sections 4 and 5 discuss the results, and section 6 provides a summary and conclusion. 2. Genetic Learning Algorithms
Genetic algorithms (GAs) were developed by Holland (1975)' as adaptive models of search, learning and optimization. GAs are based on natural models of selection and evolution. GAs are one of a the class of adaptive algorithms, such as replicator dynamics, classifier systems and neural networks. GAs are designed to work well in difficult environments, especially those large
se~rch
wit~
spaces, and are relatively robust to changes in the parameters of the algorithm. The
GA operates, when applied to game.theory, by evolving a population, of strategies. Strategies that perform- poorly are removed from the population, while strategies that ~perform well' are retained. New strategies are created by combining information-from existing strategies and by introducing new "genetic material" into the population. Hence, the GA represents learning through mimicking, experimentation, and innovation. A player's strategy is represented in the GA by a string of characters, typically binary. In our . application, a binary string is interpreted as parameters that form a bidding strategy. Each' strategy in the population interacts in an environment, ,usually consisting of the other strategies in the population, and receives a payoff.. A fraction of the strategies is then chosen to go directly into the new population, with the chance of "survival" depending on relative
payoff~
Hence, strategies that
perform better than others tend· to perpetuate., .This allows learning by the mimicking of successful strategies. Next, new strategies are created usin.g two "genetic operators" called crossover and mutation. Crossover recombines parts of two existing strategies at a randomly chosen point. For example, a crossover of the two 4-bit strings 1111 and" 0000 after the second bit results in the new strings, 1100 and 0011..Crossover represents experimentation with new combination of existing strategies in an effort to discover
improveme~ts.
3
The second genetic operator, mutation, alters a
single character of the string. If the string 0011 is mutated at the last location the result is 0010. Mutation allows innovation by introducing "new ideas" into the learning pro~ess.4 The algorithm used here has the following design. A population of 40 binary strings is randomly generated. Each string represents two parameters of a bidding function. 5 Depending on group size (either four or eight), each string participates in a round of auctions (either 100 or 200 auctions respectively6). The payoff to each string is the sum of its profits· for the auctions conducted during the round. A new population of 40 strings is then formed by first assigning each string a probability weight based on its relative payoffs in the last round, with higher payoffs yielding greater weight. 7 With these weights, 20 pairs of strings are randomly drawn from the population, with replacement.; With a 50% probability, a given pair of strings will go directly into the new population. Otherwise, the pair .will·undergo crossover.8 After the crossover, each newly formed string will also undergo mutation. Each bit of the new strings will be mutated with probability p, where p is initially 8% but exponentially decays in half every 250 generations. This completes a new generation. The strings are again introduced into the auction environment, and the proced·ure is repeated for 1000 generations.
In addition to the parameters· of the GA reported.here, ~e also considered a num~er of variations of these parameters. We found that. the results are robust to algorithmic changes that do not diminish the level of feedback to the
play~rs.
a~d
parametric
For instance, changing the muta-
tion rate has little effect. However, reducing the number of auctions in a round without increasing the number of generations does change the dynamics. In particular, since non-zero payoffs are only. received when an auction is won, reducing the number of auctions slows the convergence, although the· relative performance of the GA across auctions is not affected. For a detailed introduction· to GAs see Goldberg (1989). The· first parameter was restricted· to the range [0,2] and the second· one to [-2,.2]. Each parameter was coded by 10 bits. The 10 bits were interpreted as. a binary integer (if the parameter was signed·, then the initial bit determined the sign) normalized t.o·the appropriate range. 6 The different number of auctions was designed to give identical information flows ·under the two grollp sizes~ Recall that each simulation has· 40 total bidders. Hence, in the 4- bidder groups . there were (lOOauctions)/(group) X (10 groups) =1000 auctions per generation. Similarly, in the 8-bidder groups there were (200 auctions)j(group). x (5 groups)=1000 auctions per generation. 7 Strings with payoffs below 1.5 standard deviations from the mean were removed from. .the population. After these strings were removed the scores were normalized. See Miller (1989) for more detail. . 8. Th~ crossover was a circular crossover, that is, the strings were assumed to exist on a circle and a segment of the circle was exchanged. This technique prevents a bias towards preserving the end points that occurs in the linear operation illustrated earlier. 4
5
4
Three different environments are explored for each type of auction: coevolution, full-feedback, and' Nash. In the coevolutionary environment, strategies interact with other evolving strategies. Auctions are held in randomly formed groups, and payoffs equal the bidder's actual profit or los8. 9 In the full-feedback environment, strings do not participate in auctions but instead each strategy receives a payoff equal to minus its squared deviation from the Nash equilibrium bid. This environment is used to test the GA's ability to solve the function fitting problem. The payoffs in the full-feedback condition are independent of the choices of other agents, and do not depend on winning 'an auction. In addition, the payoff space in the full-feedback condition is the same across all auctions, that is, accuracy is rewarded identically. Finally, in the Nash environment, each strategy bids in auctions consisting entirely of opponents who use the Nash equilibrium bidding strategy. If the strategy wins the auction, then its payoff equals its profit, otherwise its payoff is zero. This differs from the coevolutionary condition in that each strategy evolves against a constant environment. Since the Nash equilibrium bid is the best response in an environment, consisting entirely of other Nash equilibrium bidders, this condition tests the system's ability to find a best-response function. 3. Theoretical and.. Experi~ental Background
In order to gain insights into the behavior of experimental subjects', our computer auctio~s
are deliberately tailored after existing auction experiments. All of the, auctions have common parameters and structures in order to facilitate comparisons am.ong them. In the rest of this section we will review these actions, and the experimental findings. 3.1 Common- Value Auctions In common-value auctions the bidders -compete for an item that is of unk,nown value at the time of the auction, such as an offshoreoilleas'e, but which will have the same value to the winner, regardless of which bidder wins. The .auction is by sealed bid, and the, highest bidder wins. The difficulty for bidders is that they must account for the fact that there is information in being the highest bidder. In particular, the highest bidder is most likely to have "over-estimated" the value of the, item. A bidder who fails to account for this information may suffer' a "winner's curse" since profits from winning will be unexpectedly low. 9 In this environment, two populations of 40 bidders are simultaneously evolved, where the two· populations share no "genetic material." By having two populations coevolve against one , another some endogeneity problems .inherent in a single population are avoided. For instance; purely stochastic forces can cause a single population to rapidly converge towards a common string. This genetic drift effect was first noticed in natural populations.
5
Experiments by Kagel and Levin (1986) found that, in general, subjects suffer a winner's ~urse, especially in large groups (5 to 7 bidders). However, they find that small groups
(4 bidders)
eventually learn to bid profitably on average, although they still earn less than the Nash equilibrium prediction. The large groups continue to lose money, even with experience. Dyer, Kagel and Levin (1989) found that these effects also exist among "professional" bidders. The structure of our simulated common-value auctions is the same as that used in the Kagel and Levin (1986) experiment. Let Xo
Xo
be the value of the item, and let n be the number of bidders.
is drawn randomly from a uniform distribution on the interval [1000,2000]. Each bidder is given
a signal of the item's value, interval [xo -
f, XO
Xi.
+ f], where f
The signals are drawn randomly from a uniform distribution on the is itself drawn randomly from a uniform distribution over (0, 500}.
Hence, each bidder's signal is an unbiased estimate of the value of the item. Kagel and Levin (1986) solve for the Nash equilibrium bid function for this model,
as~uming that
bidders know the process
by which values and signals are generated. They find .the Nash equilibrium bid function is 10
(1) The expected profits conditional on winning are l ! E1r
2£ n-l
=._-~
3.2 Affiliated Private- Values Auctions In private-values auctions, bidders. have their own values for the item. These privat.e values are affiliated if the bidders do not know the distribution from which all other private-values were drawn, but know only that the higher their own value, the higher other's values are likely to be. This will be true, for example, if an item is purchased for its speculative value. This type of informational 'assumption is discussed by Milgram and Weber (1982'a). Kagel, Harstad and Levin (1987).. found ,t4at subjects in first-price affiliated private-values auctions generally bid in excess of the risk-neutral Nash equilibrium prediction, and made profits By restricting' analysis to signals in the interval [1000+ f,2000 - €], they show that the Nash equilibrium bid ~unction is b(Xi) Xi - f + Y(Xi), 'Yhere Y(Xi) =. [2f/(n '+ 1)] exp[-(n/2f)(xi 1000 - f)]. However, because of its negative exponent, Y(Xi), moves negligibly close' to zero as Xi increases beyond 1000 + €. We do. not believe that this simplification of the bid function had any significant effect on our results or conclusions. This will be discussed in subsequent footnotes. 11 In Nas,h equilibrium the bidder with the highest signal will always be the highest bidder. Hence, the expected profits are the expected value of b(Xi) - x o , conditioning on the fact that Xi is the highest signal. 10
=
6
that were below the predicted levels'. Behavior often differed significantly from Nonetheless, they found that the
risk~neutral Nash
t~e
Nash prediction.
model did a better job of explaining the data,
than a model of (constant relative) risk aversion, or two ad hoc models. Our auctions derive their structure from the formulation of Kagel,'Harstad and Levin (1987). First, a value Xo is drawn from the uniform interval [1000,2000]. A value
€
is then drawn from
the uniform interval (0,500]. Then n private values, Xi, i = 1, ... , n, are drawn 'from the uniform interval [x o
- f, X o
+ f].
To make the auctions affiliated, bidders cannot be given full information
about the distribution of private values. Hence, as in Kagel, Harstad and Levin (1987), bidders are only allowed' to know
€,
in addition to their private value, Xi. For first-price auctions, Kagel,
Harstad and Levin (19~7) show that the Nash equilibrium bidding function in this model is 12
b(Xi)
2€ n
= Xi -
(2)
-.
Profits of the winning bidder are
2£
" -
",.--
Kagel, Harstad and Levin (1987)
~so
n
.
conducted
~econd-price
auctions (or Vickrey
auction~),
in which the object goes to the highest bidder at a price equal to the second high~st bid. As is well known, it is a dominant strategy for all bidders to bid their true values. The surprising result from the experiments is that subjects bid well in excess of the dominant strategy. In 80 percent of the auctions, the market clearing price exceeded the Nash level by significant amounts. 13 The inference made from these experiments is that subjects found the strategic interactions of second-price auctions too difficult to learn in the span of the experiment. We will also conduc~ second-price auctions. In this case, the predicted Nash equilibrium profits of the winning bidder are the expected difference between the nth and (n - l)th order statistic:' 2f E1r= - - . n+1
(3)
12 As before, over the range [1000+£, 2000-€], the exact bid function is b.(Xi) = Xi- 2f/ n +y(xi)/n, where Y( Xi) is the same as given above. Hence, this bid function is again the relevant portion. Again, we do not believe this simplification affects our results or conclusions. See footnote 13 below. 13 This result was in clear contrast to the ascending-clock auctions that Kagel, Harstad and Levin (1987) designeq.to test English open-outcry auction, which are, in theory, equivalent to second-price auctions. In this case, bidders almost always dropped out of.the auction when the price exceeded their value, and market clearing prices were very near the Nash prediction.
7
3.3 Independent Private- Values Auctions A striking result of auction theory, first noted by Vickrey (1961), is the revenue equivalence theorem. This states that when risk-neutral bidders have private values for an item, and when those values are completely independent, then the firs~-price and second-price auctions will yield the same expected revenue to the sellers. Private values are independent when all bidders know the distribution from ~hich all private values are drawn. In terms of ~he model presented in the last subsection, it means that bidders know
Xo
as well as
E
during each auction.
It was shown by Milgrom and Weber (1982a) that if people i"n a first-price affiliated-values auction are given information about the distribution of values, i.e., ·are told X o , then this infor.mation will increase the equilibrium price of the object. Kagel, Harstad and Levin (1987) tested this proposition by providing subjects with
Xo
after they bid in the affiliated-values auction, and thus
transforming the auction into an independent-values auction. In theory this should cause bidder's profits to fall. They found that average profits across all treatments did fall, but that they did not fall in every treatment. In three of the five treatments average profits were virtually unchanged, even though bidders reacted strongly to the new information, and in one of the treatments profits actually increased. Only in one treatment did the profits fall significantly.
I~
a study of individual
bidder behavior, Kagel and Levin (1985) reject all models they considered to explain the da~a,. including asymmetric risk aversion. Kagel, Harstad and Levin (1987) show that the Nash equilibrium bid· function for first-price independent-values auctions is14 .
b(Xi) =
n-l n
1
--Xi
+ -(xo n
f).
(4)
Expected profits of the winning bidder can be calculated as 15 2f E1r = - - ". n+1
(5)
14 In contrast to the bid functions (1) and (2) in the other first-price auctions, (4) is· the exact Nash bidding strategy. Hence, if the approximations given in (1) and (2) have any effect, ,ve would expect that, relative to the· independent values auctions, the. approximation should hamper convergence to the predicted parameters in the common- and affiliated-values auctions. As we will see, this does not appear to be the case. Hence, any effect of the approximation, if it is present, would tend to work agai~st the conclusions drawn in this paper. 15 Equilibrium expected profits can be found by substituting the expected value of the nth order statistic, x n , into (4), and evaluation x n - b(x n ).
8
We also conducted second-price auctions for independent-values. Again, the dominant strategy is for bidders to bid their values. Hence, the equilibrium profits are the same as (3). As predicted by the revenue equivalence theorem, (3) and (5) are identical.
3.4
Parameters for the AAI Auctions
We conducted all five auctions discussed above for both 4 and 8-bidder groups. The GA searched over 2-parameter linear functions in each auction. For common- an'd affiliated-values auctions, the function's variables are are
Xi
and
Xo
-
f.
Xi
and
€,
w.hile Jor independent-values auctions the variables
The predicted values of all the parameters on the bid functions, and of the
expected profits (based on the mean value of f of 250) are given in Table 1.
TABLE 1. Theoretical Predictions. Auction
Group Size
Nash Bidding Function Xo -
Common Value
Expected Profit
f
4
1.000
-1.000
100.0
- 8-
1.000
-1.000
55.5
4
1.000
- .500
125.0
8
1.000
- .250
62.5
4
1.000
.000
100.0
8
1.000
.000
55.5
4
.750'
.250
100.0
8
.875
, .125
55.5'
4
1.000
.000
100.0 '
8
1.000
.000
55.5
Affiliated' Values first-price
second~price
Independent Values first-price
second-price
Note that for each aaction there are two parameters that must be chosen ·by theAAI bidders. From this perspective, the parameter space over which the GA must search is identical for all auctions. This means that any difference in the ability of the GA to find Nash equilibria will
9
not be attributable to the dimensions of the search space, and instead must be attributed to the
inf~rmational differences
in the auctions. As indicated above, we conducted auctions under
three information conditions: coevolution, Nash opponents, and full-feedback. For each size and information condition we ran 20 trials, with each trial lasting 1000 generations.
4. Results of A uctions with AAI Bidders
4.1
Affiliated- Values Auctions The results of the first- and second-price auctions with affiliated-values are listed in Table
2. The numbers in this table, and in all subsequent tables, list the average values of the last 2'5 generations in each trial, averaged over all 20 trials. The averages, contain information on 500,000 auctions. 16 The standard deviations refer to the differences across trials. I7 We will refer to the parameters of the bid function as a pair ({31, (32) where /31 refers to the parameter on
Xi
in the
bidding function. Begin with first-price auctions under coevolution. For 4-bidder groups the predicted values of the bid function are (1, -.5), while the GA finds parameters of (.98982, -.46417). The standard errors on these parameters
ar~
small, indicating wide agreement on the bidding function across
"trials. Moreover, average profits are 113.42, which is 90%' of the Nash prediction of 125. There is similar success with the 8-bidder groups. The predicted parameters are (1, -.25), while th'e GA finds (.99540, -.24694). Again, the standard errors across trials are small. However,8-bidder groups earn average profits of 36.79, which is only 58% of the Nash prediction.
This indicates that the GA has largely been able to converge to the Nash equilibrium. The small errors that remain are only slightly costly to the 4-biddergronps, and more ,costly to the larger groups. This leads us to ask whether these remaining errors can be attributed to the algorithm per se, or whether they can be attributed to the' kind of informational environment present in a coevolutionary game. We can examine this question with the Nash and-full-feedback
conditions~
We see that in th,e Nash condition the GA performs about as well at' finding the bid function, but 16 25 generations/trial X 100 auctions/4-bidder group x 10 4-bidder groups/generation X 20 trials = 500,000 auctions. For 8-bidder groups we ran 200 auctions per group, but had 5 groups per generation, hence there were also 500,000 auctions. 17 The standard deviation~ within trials are very small by the 1000 th generation, usually on the order of .0001 to .05. This is due to the· fact that the probability of mutation has decayed from its initial 8% to about 1/2% by the last generation, coupled with the genetic drift phenomenon inherent in' small populations. Hence, variance across trials will be the most indicative of the algorithm's ability to find a Nash equilibrium.
10
TABLE 2. Results for Affiliated-Values Auctions. Mean Values (Standard Deviations)*
Information
Co-evolving first- price
Group Size
4 8
second price
second-price'
.98982 (.00725) .99540 (.00347)
-.46417 (.06139) -.24694 (.02816)
113.42 (8.77) 36.79 (10.99)
.99116 (.00755) .99303 (.00395)
-.46892 (.05013) -.21685
..99821 (.00860) .99471 (.00471)
.00881 (.06282) .03198 (.04163)
4
.99842 (.00776)
8
1.00130 (.00515)
-.49173 (.03447) -.25634 (.02009)
125.54 (4.55) 61.88 (4.45)
4
1.00107 (.00290) 1.00004 (.00280)
-.00495 (.01620) -.00391
100.11 (0.29) 55.50 (0.15)
4
4
8
Full-Feedback first-price
€
.00008 (.07861) .04897 (.04563)
8 second-price .
Xi
Average Profit t
.99955 (.01034) .99754 (.00475)
4 8
Nash Opponents first- price
Bidding Function
8
(.019~5)
(.01648)
96.66
(7.99) 48.16 (4.39) 122.85 (13.37) 42.07 (19.02) 94.92- , (~1.89)
53.23 (8.06)
• Means and standard deviations are over the last 25 gener~tions. t Profits reported for the Nash auctions are conditional on wifl~ing the auction. Profits for the Full-Feedback auctions are calculated for each simulation as expected values, using the average strategy o~er the last 25 generations. Means and standard deviations are calculated across simulations.
11
does better as measured in profits. This is true for both 4 and 8-bidder auctions. This
d~fference
can probably be ascribed to the fact that the variance of the bid functions across auctions is smaller. Turning to the full-feedback condition, we see that performance improves even more. In both 4 and 8-bidder auctions, the bid functions are almost exactly those predicted by the Nash equilibrium. These results indicate that; while the coevolutionary environment does not generate the exact Nash prediction, it is nonetheless quite successful at finding the Nash bidding s~rategies in affiliatedvalues auctions. This is consistent with the experimental finding that Nash equilibrium bidding organized the data better than other models, although errors still remain. The' fact that the full-feedback auction can make the small improvements necessary to find the exact Nash bidding strategy indicates that the failure of the GA to do even better in the coevolutionary environment cannot be blamed
~n th~
algorithm per se -
in ideal feedback conditions, the GA can find the
optimum. Instead, these results indicate that the small errors of the coevolutionary condition appear to be due to the fact that the feedback is less than ideal. Turning to the second-price auctions we find similar results. For both 4 and 8-bidder groups the coevolutionary strategi~s are extremely close to the Nash prediction of (1, 0). Moreover, average .profits are 96% of the Nasli prediction for the 4-bidder group, "and 87% of the Nash pred~ction"for the 8-bidder group. The. Nash' condition performs about as well as
th~
coevolution condition, with
slightly less variance on the bid functions, while the full-feedback condition performs the .best by attaining almost the exact Nash strategy. Figure 1 illustrates the evolution of bids in a 4;. and 8-bidder second-price coevolving auctions. The figure shows the average deviation of the winning bid from the dominant strategy bid over all 1000 generations (averaged over every 20 generations). The deviations converge to the dominant strategy from above.
We
can explain this by examining. the dynamics of adaptive learning.. A
strategy that bids beneath ·the dominant strategy will
inc~ease
both its absolute and relative
~er
formance by raising its bid to the dominant strategy level. However, if a bidder is bidding above the dominant strategy, then moving to the dominant strategy will increase the bidder's absolute performance, but' may reduce its relative performance. This is because the other strategies will now earn higher profits when this bidder is the second highest bidder. Hence, many situations exist in which a bidder will re~iIce its chance for survival by lowering its bid to the dominant strategy level. This indicates that
th~
evolutionary forces will quickly eliminate under-bidders (who win
no auctions) as well as the extreme over-bidders (who make losses), but may be slow to eliminate
12
moderate over-bidders. This implies that evolutionary learning is likely to lead. to convergence to the dominant strategy from above. Again, this is consistent with the experimental results.
4.2 Common- Value Auctions The results for common-value auctions are listed in Table 3. Looking first at the 4-bidder coevolutionary auctions, we see that the bid functions are in the neighborhood of the Nash equilibrium strategies, with small standard errors, but are much farther from the predicted levels than are the affiliated-values auctions. In terms of profits, the coevolutionary strategies earn 82% of the Nash prediction. In all 20 trials the AAI bidders earned positive profits, with the maxi,mum at 96.1 and the minimum at" 72.2. Turning to the 8-bidder coevolutionary auctions we see that the parameters of the bid function are closer to the Nash equilibrium values than in the 4-bidder auctions. Again, the standard deviations of these parameters are small. Average profits are 18.20, which is only 33% of the Nash equilibrium amount, and in two of the twenty trials the' AAI bidders actually made losses. Overall profits varied from 51.47 to -10.59. Hence, even though the bidders in 8-bidder auctions appear to make smaller errors, these errors are more costly. Recall that Kagel and Levin (1986), and Dyer, Kagel and Levin (1989) showed that, with suffi_cient experience.,
bidd~rs
.
in 4-bidder groups are able t
