Results

  Detailed empirical testing indicates that the implementation detailed above is successful in the challenging communication environment of the Soccer Server. In this section, we report results reflecting the cost of communication, the agents' robustness to active interference, their ability to handle messages that require responses from multiple team members, and their ability to maintain a coordinated team strategy.

To test the cost of communication, we played a team using no communication (team A) against a team identical to the first in all regards except that its members say random strings periodically (team B). Thus team B gained no benefit from communication, but its action rate was reduced by the interleaving of random statements. With an average of 18% of its actions taken by these random communications, team B suffered a significant degradation in performance, losing to team A by an average score of 3.54 to 1.08 over 50 games. Clearly, communication in this domain involves a significant cost.

Relying on communication protocols also involves the danger that an opponent could actively interfere by mimicking an agent's obsolete messages. However, our <Endcoded-stamp> field guards against such an attempt. As a test, we played a communicating team (team C) against a team that periodically repeats past opponent messages (team D). Team C set the <Endcoded-stamp> field to <Uniform-number (send-time + 37). Thus teammates could determine send-time by inverting the same calculation (known to all through the locker-room agreement). Messages received more than a second after the send-time were disregarded. The one-second leeway accounts for the fact that teammates may have slightly different notions of the current global time.

In our experiment, agents from team D sent a total of 73 false messages over the course of a 5-minute game. Not knowing team C's locker-room agreement, they were unable to adjust the <Endcoded-stamp> field appropriately. The number of team C agents hearing a false message ranged from 0 to 11, averaging 3.6. In all cases, each of the team C agents hearing the false message correctly ignored it. Only one message truly from a team C player was incorrectly ignored by team C players, due to the sending agent's internal clock temporarily diverging from the correct value by more than a second. Although it didn't happen in the experiment, it is also theoretically possible that an agent from team D could mimic a message within a second of the time that it was originally sent, thus causing it to be indistinguishable from valid messages. However, in this case, the content of the message is presumably still appropriate and consequently unlikely to confuse team C.

Next we tested our proposed method of handling multiple simultaneous responses to a single message. Placing all 11 agents within hearing range, a single agent periodically sent a ``where are you'' message to the entire team and recorded the responses it received. In all cases, all 10 teammates heard the original message and responded. However, as shown in Table 3, the use of our proposed method dramatically increased the number of responses that got through to the sending agent. When the team used communicate-delay as specified in Section 4, message responses were staggered over the course of about 2.5 seconds, allowing most of the 10 responses to get through. When all agents responded at once (no delay), only one response (from a random teammate) was heard.

  
Table: When the team uses communicate-delay as specified in Section 4, an average of 7.1 more responses get through than when not using it. Average response time remains under one second. Both experiments were performed 50 times.

Finally, we tested the team's ability to maintain coordinated team strategies when changing formations via communication. One player was given the power to toggle the team's formation between a defensive and an offensive formation. Announcing the change only once, the rest of team had to either react to the original message, or get the news from another teammate via other communications. As described in Section 4, the <Formation-number> and <Formation-set-time> fields are used for this purpose. We ran two different experiments, each consisting of 50 formation changes. In the first, a midfielder made the changes, thus making it possible for most teammates to hear the original message. In the second experiment, fewer players heard the original message since it was sent by the goaltender from the far end of the field. Even so, the team was able to change formations in an average time of 3.4 seconds. Results are summarized in Table 4.

  
Table 4: The time it takes for the entire team to change team strategies even when a single agent makes the decision. Even when the decision-making agent is at the edge of the field (goaltender) so that fewer than half of teammates can hear the single message indicating the switch, the team is completely coordinated after an average of 3.4 seconds.

In addition to the above controlled experiments, we used our communication method in the CMUnited simulator team that competed in RoboCup'97. In a field of 29 teams, CMUnited made it to the semi-finals, indicating that the overall team construction, of which this communication model was a significant part, was successful.