• Home
• Research
• Tutorials

Structured populations:

Mixed strategies

by Christoph Hauert, Version 1.0, April 2004.

• Location:
• VirtualLabs
• » Cooperation
• » Structured populations - mixed strategies

In well-mixed populations the equilibrium configuration does not depend on whether the individuals adopt pure strategies (to cooperate or to defect) or mixed strategies (to cooperate with a certain probability). All that matters is the frequency of the two behavioral patterns when averaging over many random encounters. This ambiguity is resolved in spatially structured populations where individuals interact only within a limited neighborhood. Spatial structures are modeled by arranging individuals on a lattice. The lattice geometry can be either square with a neighborhood size of N = 4 or N = 8, triangular with N = 3 or hexagonal with N = 6. Whenever a site x gets updated, the focal individual and all its neighbors compete to re-populate the site. Their reproductive success depends on their performance, i.e. their average payoff, in interactions with their respective neighbors. If the individual in x cooperates with probability p and the neighboring y with probability q, then the payoff for x amounts to P_p = p q R + p (1 - q) S + (1 - p) q T + (1 - p) (1 - q) P and similarly for y to P_q = p q R + p (1 - q) S + (1 - p) q T + (1 - p)(1 - q) P. Note that this calculation of the payoffs assumes that individuals interact frequently such that payoffs from single interactions average out. In contrast, if the performance depends only on a single interaction, then an individual with strategy p would obtain against strategy q the payoffs R, S, T or P with probabilities p q, p (1 - q), (1 - p) q and (1 - p)(1 - q), respectively. In that case the fluctuations arising from the probabilistic payoffs destroy all spatial correlations and yield the same results as well-mixed populations.

The performance P_x of x is then determined by averaging the payoffs from interactions with all its neighbors. The neighbor y succeeds in populating the focal site x with a probability proportional to

where k denotes a noise term which introduces an interesting form of errors since worse performing individuals may still manage to reproduce with a small probability. The offspring inherits the parental strategy p but with a small probability a mutation occurs changing the offspring strategy to p + s where s is a Gaussian distributed random variable with mean zero and small standard deviation. For small mutations, the difference in payoffs P_y - P_x becomes small and the generalization of the replicator dynamics to spatial settings as introduced for structured populations with pure strategies no longer performs well. For this reason we chose the above, slighlty more complicated rule to determine the reproductive success for mixed strategies. With this rule, small differences in payoffs are amplified - at least for reasonably small k. The remaining part of the update procedure is identical to the pure strategy case: with probability p_x = (1 - w_y1)(1 - w_y2) … (1 - w_yN) all neighbors y_i fail to reproduce and the focal individual succeeds in placing its own offspring in site x. Otherwise, with probability 1 - p_x, one neighbor takes over the focal site. The relative probability for success of neighbors y is w_y/w where w = w₁ + w₂ + … + w_N.

Updating of the lattice can be synchronous referring to populations with discrete non-overlapping generations or asynchronous for populations with overlapping generations in continuous time. For synchronous updates, all individuals interact and accumulate payoffs and then everbody attempts to reproduce with successrates relative to each individuals payoff within its neighborhood. For asynchronous updates only a single randomly selected focal site gets updated at a time: first the payoffs of the selected individual and all its neighbors are determined and then they compete to re-populate the focal site as outlined above.

The following examples illustrate and highlight different relevant scenarios but at the same time they are meant as suggestions and starting points for further exploring and experimenting with the dynamics of the system. If your browser has JavaScript enabled, the following links open a new window containing a running lab that has all necessary parameters set as appropriate.

Legend	Time evolution of the propensity to cooperate in structured populations with individuals adopting mixed strategies and engaging in prisoner's dilemma and snowdrift interactions. Color code: Maximum Minimum Mean Strategy code: Defect               Cooperate
	Space promotes cooperation: Prisoner's Dilemma In spatially structured populations playing the Prisoner's Dilemma significant propensities to cooperate can evolve which contrasts with well-mixed populations where the levels of cooperation converge to zero. In spatial settings individuals with a higher readiness to cooperate can thrive by forming clusters and thereby reducing exploitation by less cooperative individuals. The snapshot on the left shows a typical lattice configuration with an average probability to cooperate of 20%. The population has not yet reached the equilibrium and the readiness to cooperate would further increase.
	Space inhibits cooperation: Snowdrift and Hawk-Dove games (synchronous) As in the case of pure strategies, for the Snowdrift and Hawk-Dove game spatial structure reduces the readiness to cooperate for mixed strategies if the population is updated synchronously, i.e. with discrete non-overlapping generations. The snapshot on the left show a typical configuration of a lattice population in equilibrium. The average probability to cooperate is roughly 10% for parameters where well-mixed populations sustain levels of 30% cooperation.
	Space promotes cooperation: Snowdrift and Hawk-Dove games (asynchronous) The situation changes considerably, if the Snowdrift or Hawk-Dove game is played in spatially structured populations that are updated asynchronously, i.e. overlapping generations in continuous time. In that case space again promotes cooperation and results in average propensities to cooperate that lie above those obtained for well-mixed populations. The snapshot on the left shows a typical lattice configuration with an average probability to cooperate of 20%. The population has not yet reached the equilibrium and the readiness to cooperate would further increase. The parameters are the same as in the synchronous case above where the equilibrium has already been reached at much lower levels of cooperation. This contrasts with the established view that any form of stochasticity (such as asynchronous updates) should favor defection. Comparing this snapshot to the one for the Prisoner's Dilemma above (with the same average probability to cooperate) shows that the variance of strategies is considerably smaller in the Snowdrift and Hawk-Dove game than in the Prisoner's Dilemma. In the simulations this can be clearly seen when switching to the histogram view depicting the frequency distribution of the strategies.

Along the bottom of the VirtualLab applet are several buttons arranged to control the execution and the speed of the simulations. Of particular importance are the Param button and the data views pop-up list on top. The former opens a panel that allows to set and change various parameters concerning the game as well as the population structure, while the latter displays the simulation data in different ways.

Note: In spatially structured populations the shade of grey indicates the individual's readiness to cooperate with black indicating defection and white full cooperation. Payoffs are similarly coded in shades of grey where black indicates low and white high payoffs.

The list below describes only the few parameters related to the Prisoner's Dilemma, Snowdrift and Hawk-Dove games. Follow the link for a complete list and detailed descriptions of all other parameters such as spatial arrangements or update rules on the player and population level.

Reward:

reward for mutual cooperation.

Temptation:

temptation to defect, i.e. payoff the defector gets when matched with a cooperator.

Sucker:

sucker's payoff which denotes the payoff the cooperator gets when matched with a defector.

Punishment:

punishment for mutual defection.

Init mean, init sdev:

initial fractions of cooperators and defectors. If they do not add up to 100%, the values will be scaled accordingly. Setting the fraction of cooperators to 100% and of defectors to zero, then the lattice is initialized with a symmetrical configuration suitable for observing evolutionary kaleidoscopes.

Mutation rate:

the probability for a mutation to occur whenever an individual reproduces. This parameter is found on the Misc parameter tab.

Mutation type, mutation sdev:

mutations can occur either uniformly distributed in the interval [0,1] or modify the parental strategy by a Gaussian distributed random variable with mean zero and the specified standard deviation. This parameter is found on the Misc parameter tab.