Norms, Roles, and Simulated RoboCup

0 downloads 0 Views 49KB Size Report
Computer Science & Computer Engineering Department, Engineering Hall, Room 313,. Fayetteville, AR ... [12] points out a divide-and-conquer learning approach in designing agents for role ..... is to mark the opponents and to recover the ball.
1

Norms, Roles, and Simulated RoboCup Henry Hexmoor and Xin Zhang Computer Science & Computer Engineering Department, Engineering Hall, Room 313, Fayetteville, AR 72701 {hexmoor, xxz03}@uark.edu

Abstract. In this paper, we examine the concepts of norms and roles in multiagency. After a general treatment of relationships between goals, roles, and norms in a rational utilitarian agent, we turn to team-oriented decisionmaking, which is facilitated by notions of positional role hierarchies and group roles. We use simulated Robocup as a case study for team oriented decisionmaking and illustrated role exchange as an example of team-oriented reasoning.

1 Introduction Agent architectures are being augmented with roles and norms in multiagent systems [4][10][11]. These formulations help model complex interagent interaction. Issues of coordination and collaboration can be studied with models of roles and norms. The following is a simplified outline of roles and norms in the context of an agent’s toplevel decision-making loop. 1. Perceive the world and update beliefs in the world model 2. Consider options for individual goals, roles, and norms and adopt the most compatible ones 3. Consider plans to implement adopted goals, roles, norms that lead to options for action 4. Formulate intentions to act and perform actions. There are complex relationships among roles, goals, and norms. An agent may have several goals; each might provoke a set of norms and a set of roles. An agent may also have several roles or share a role with others, where each role can affect an agent’s goals and provoke a set of norms. We aim to provide a useful definition of roles and norms that helps to set up investigation of these relationships. Simulated Soccer is a good domain for the study of roles and norms. It is suggested that soccer players must change roles quickly [18][5]. [5] defines rolemaps as strategies for role adoption. The decision to change a role in their system is based on decisions of an agent in 3 dimensions: cooperation, authority, and specificity. [12] points out a divide-and-conquer learning approach in designing agents for role coordination and communication in teamwork.

2 In the Soccer team, players are assigned different roles such as forward, midfielder, goalie and etc. Each role requires a distinct set of abilities. Some of those abilities are specified offline, while other abilities maybe instantiated online [12]. Offline abilities are hard coded in the role while online abilities are based on learning and reasoning using the updated environment. Role adoption in simulated RoboCup is different than the general case in the real world. At the very beginning before the agent adopts any roles, all of the 11 players are the same, without any special abilities or goals. When they are instantiated with different roles, each agent will adopt the capabilities and goals of the role as its own. Roles are organized into formations, and players can fill any role in any formation [18]. Each role is assigned a home position according to the team formation. This style of role assignment ensures flexibility of role exchange, which benefited the FC Portugal team [16]. In [16], each player may exchange not only its position (place in the formation), but also its assigned role in the current formation. After the exchange, the agent will be instantiated with a new role and corresponding new abilities and goals, and completely losing properties of its old role. In the following two sections we will define a norm and a role.

2 Norms Norms are common in business rules, social goals, constraints and other structural aspects of the group. [17] defines norm from the AI perspective as “… a norm is operationalized as a behavior constraint on agents which interact with each others in a multi-agent society”. [19] summarized norms in multiagency as constraints, goals, and obligations. Treated as mere constraints, norms do not provide adaptive social reasoning. [4] argues for a deliberative normative agent with norm impacts on goal and plan generation and selection. Such norms can serve as filters to generate and select goals [2]. This approach to norms gives agents more flexibility. If norms are represented as goals, agents can reason about norms like any other goals. The drawback is that the agent will not be reliable. Norms as obligations combine the advantages of both constraints and goals. Agents are able to reason and at the same time, there is a measure of predictability. As opposed to passive or subconscious use of norms, we consider agents that are deliberative about their norms. We suggest the following minimal list of norm characteristics. - Involve two or more agents. Each agent understands and shares them. - Agents have power to not choose them. - There is no direct rational account of them available to the agents. - The bearer experiences an implicit or an explicit sanction or rewards for adoption. While strategies might fit the above requirements, norms dwell in pragmatics of interaction whereas strategies are means-end techniques. Norms are not defaults or

3 exceptions. Defaults and exceptions can be used for similar purposes as norms. e.g., in coordination. However, unlike defaults and exception, agents deliberate about norms. Obligations are strong norms that are usually very specific. We consider interdictions, bans, and other similar terms as kinds of norms. Rights (sometimes known as permission and privileges) are different than norms and gaining some attention in MAS [1]. They are like norms in that they have to be understood by others, but for instance rights do not carry sanctions or rewards and tend to be fairly specific. Norms might produce goals for an agent but the norm is not the goal itself but the surrounding social context. Some researchers have treated norms as constraints. Constraints are a knowledge encoding method regulating an agent’s actions. Beyond constraints, norms that are encoded like constraints will need to preserve the teleological foundations of such encoding. This is so the agent can perform reasoning over them. Norms are a subset of social laws and the agent’s pragmatics of using them. We believe norms to be broader than conventions. Conventions carry the connotation of being informal and carry a fairly low sanction or reward. Finally, we take policies to be kinds of norms that govern relationships among roles. The following succinctly defines a norm. This norm strictly belongs to a single agent. We are not modeling a norm that is shared by a coalition of agents. Definition: Norm = (O, R, G, U). O is the content of the norm set. These can be at least one goal to do or at least one state that it may avoid. R is the sanction that may result from not following the norm. G is the agent’s goal that invokes the norm set when the agent chooses to consider other agents. U is a utility function that considers the agent’s gains and losses in terms of G. The gains and losses might be rewards, penalties due to sanctions and otherwise, and issues related to reputation. If the agent follows the norm, the agent’s reputation is affected and this can be measured by a utility change. If the agent does not follow the norm it might produce a negative value or it might not have any effect. To compute this value, the agent must perceive a social standing and expectation of norm-following that it has established for itself. If the agent violates norm, it may sustain a penalty due as a result of the sanction. For example, if we do not follow the traffic “stop” sign, we will get a ticket and must pay. Not all norms produce sanctions. For example, violating the norm to “drive on the fast lane for faster speeds” may incur a time cost but it does not result in the penalty of traffic ticket. If the agent follows a norm, the agent’s utility gain or loss in terms of achieving the agent’s current goal G is affected. I.e., the change might be positive or negative. E.g., for a positive example, if we follow “drive on the fast lane for faster speeds”, we can maintain higher speed and get to our destination faster. For a negative example, if we follow the traffic “stop” sign we will slow down. In general, O differs from G. G is the agent’s own goal whereas O is something the agent may adopt due to social considerations. O and G might even be opposite, in which case the agent must consider possible revision of its decision to adopt G or the norm.

4 2.1 Norm Adoption Agents may adopt norms for a variety of reasons. Norm adoption for sociability is when an agent attempts to present a façade of following certain norms so as to gain a reputation. We will not catalogue all the motivations for norm adoption but focus on a general utilitarian norm adoption that is based on goal achievement utilities. Proposition: Norm selection for one goal. An agent considering goal G and norms N1…Nm will pick the norm Ni, which maximizes its goal achievement utilities. If all norms for G will produce a negative utility, no norm is selected. Proposition: Norm selection for multiple goals. An agent considering goals G1, G2, …, Gm and corresponding norm sets N11…N1m1, N11…N1m2, …, and N11…N1mm will pick one norm for each goal (Norms N1, …, Nm) but with the constraint that maximizes U1 + … + Um. At times, utilities due to norm selection can have such an effect as to influence the goal selection. The agent may not be able to select a norm since the utility losses outweigh any gains it might have. This agent must be prepared to pay the sanction penalties for non-adoption of a norm corresponding to the goal. If the sanction penalty is larger than the utility of the goal, the goal should be dropped. An example is burglary. The potential thief may think about the penalty of the sanction and it considers it to be higher than any gains, it should drop it. What happens if the agent has several goals and is faced with some sanctions that each outweighs the utility of the corresponding goal? Naturally, the utility maximizing agent will not drop the goals and will be prepared to pay the sanctions. Norms are also an essential part of an agent’s role [8]. In the next section we examine roles.

3 Roles Agents relate to one another primarily influenced by their roles [15]. Ferber defines a role as “an abstract representation of an agent function, service or identification” [7]. Role is a common sense notion for agents in communities. For instance in the area of modeling the game of soccer, “A role defines the part that an individual agent chooses to plays in a scenario…” [5]. [9] defines role as service + policy. In deliberative agents, specific agent roles provide specific services for the group according to the functionality of the role but subject to norms with which agents reason. We suggest the following minimal list of role characteristics.

5 - Several agents can adopt it individually, independently, and concurrently. One agent may adopt several simultaneously. Several agents may adopt it as a group. In general we will call this the adopter. - It is meaningful in the social context of other agents including (a) the adopter’s relationship to other agents and groups, (b) the agent’s mental attitudes about the social relationships, and (b) the available norms including obligations and responsibilities. - There are typical capabilities associated with the adopter. If the adopter is loses these abilities then the efficacy of the role is jeopardized. Although a role is typically an outgrowth of a plan or a strategy, it can exit apart from it. We refrain from the colloquial use of roles, which may ascribe roles to objects meaning their function or purpose. We reserve roles only for agents. As opposed to passive or subconscious use of roles, we consider agents that are deliberative about their roles. The following succinctly defines a role. Definition: Role = (A, C, O, G, Ab, U) A is the adopting entity. It can be a single agent or a group of agents. C is the social context of A. It includes the relationship between A and other agents and groups. It also includes A’s cognitive social state. It also includes the set of norms from which the agent A may select for reasoning about social welfare. O is the content of the role. These can be at least one goal to do or at least one state that it may avoid. O has a quality of persistence. G is the agent’s goal that invokes consideration of other agents. G exists prior to role adoption. Ab is the set of capabilities that are required (or typical) for A. U is a utility function that considers the agent’s gains and losses in terms of G. If the agent adopts the role, the agent may experience a utility gain or a utility loss in terms of performing its goal G. For instance, if the agent wants to have access to firearms (its G), by adopting a military position (i.e., a role), it will have better access. In general, O differs from G. G is the agent’s own goal whereas O is something the agent may adopt due to social considerations. O is persistent in that it lasts as long as the role is adopted. O and G might even be opposite, in which case the agent must consider possible revision of its decision to adopt G or the role.

3.1 Role Adoption There are many reasons an agent may select a role. Innate roles are by design or function and unalterable. The role of a paper-weight is by design. Another example is the role of hearer is a reflexive ability of a person or animal that hears sounds. A third

6 example is the role of a person being an inadvertent obstacle in the game of soccer. Agreement based role selection is when an agent somehow through negotiation or compromise assumes a role. The role of a master in a master-slave relationship is done by agreement. Another example is the role of an auto mechanic. In the game of soccer, selecting the role of sweeper is agreement based. These kinds of formation-related roles reflect the commonsense usage of the term. Role selection is performed prior to the start of the game or commonly determined by the coach. Goal-based role selection is when a problem-solving agent selects a rolemaximizing goal. For instance, an agent may purposefully intercept the ball as a defensive maneuver. Goal-based role selection is when an agent who knows about the general strategies of the soccer game, for example ISIS rules of the game [12], selects the role of intercept instead of the role of move-forward both in the context of attacking. Here, intercept and move-forward are both goals in the general game strategy. Since consideration of other agents lead to choosing one over another a role is generated that corresponds to the goal. The following proposition states the conditions for goals-based role selection. Proposition: Role selection for one goal. An agent considering goal G and roles R1…Rm will pick the role Ri which maximizes goal achievement utilities. If role utilities are negative, no role is selected. A reciprocal relationship can exist between goal-based role selection and rolebased goal selection. In the latter, a problem-solving agent selects the goal that maximizes fulfillment of its role. The following proposition states this. Proposition: Goal selection for one role. An agent considering a role R with content as goal G and other goals G1…Gm will pick the goal G iff the utility of the role (and its corresponding content G) is higher than utilities of all G1…Gm. The following proposition states conditions for selection of multiple roles. Conflicts among roles are implicitly accounted for by utility maximization. Proposition: Role selection for multiple goals. An agent considering goals G1, G2, …, Gm and corresponding norm sets R11…R1m1, R11…R1m2, …, and R11…R1mm will pick one role for each goal (Roles R1, …, Rm) but with the constraint that maximizes U1 + … + Um.

4 Roles and Norms in RoboCup In Soccer teams, individual roles are initially assigned. Reorganizations based on new formations are responsible for changes in roles [12]. This suggests a change to step 2 of our simplified agent’s top-level decision-making loop. Role hierarchy and role grouping are useful for selecting subsequent roles [13][14]. This implies yet another change in the agent’s decision-making. Role selection now takes into account other agent’s roles and that takes prominence over norms and individual goals. Let’s introduce some notions to help us refine the decision-making.

7

Definition: A positional role tree is a hierarchy of decisions where the tree leaves are specific positions that can be occupied by an agent. The arcs “specialize” from general to more specific. Positional role tree hierarchies differ from “isa”, “partof”, “boss-underling” hierarchies [13]. Although, similar to “boss-underling” it describes positions in an organization, the links do not describe authority relationships. Instead, our links describe functional/strategic specialization. In simulated RoboCup, there are several standard team formations such as 433, 442 and team formation can be changed during the game depending on different situations such as attack, defense. Since the team formation affects the role’s goal, each player is implemented with the most specialized role. The roles are specialized so that in a given role, the player should perform differently according to the team formation. For example, in the attack formation, midfielder’s goal is to pass the ball to the forward, while in defense formation, its goal is to mark the opponents and to recover the ball. In these 2 cases, the role of midfielder should be further specialized to 2 separate roles with different abilities. Therefore, roles in simulated RoboCup are formation dependent. We suggest roles in RoboCup be specialized with the following hierarchical tree, Figure 1. In the first level the decision about role is choice of “team formation”. Team formations will be used in different situations, such as “443 in attack formation”, “442 defense formation”, “corner kick attack formation”, “corner kick defense formation”, etc. There are typically a dozen or more formations. In the second level the decision about role specializes to “functionalities”. For certain team formation, 11 players can be assigned roles by different functionalities, such that some can be forwards, some can be midfielders, some can be defenders and one can be the goalie. In the third level the decision about role specializes to “abilities”. For the group of roles, which are in the same formation of the team and have the same functionality as “forward”, they still can be refined as different roles such as “left forward wing”, “central forward” and “right forward wing”. Next we will define grouping of agents with similar roles. Definition: A group role is a specialization shared by a group of agents that will also share a group norm. A group role is followed by actual individuals in a role and not similar position or functions that can be filled by agents. In the Robocup this is a group of players that share functionality like a group of forwards. In RoboCup, 11 roles play as a team instead of being independent. If each rational player maximizes its individual expected utility, the team utility is not maximized [3]. In order to maximize the utility of the whole team and realize the goal, role coordination is necessary. Therefore, there is a tradeoff between independence and coordination in the team organization [12]. Due to the real-time communication constraints, coordination is mainly obtained by tactics and strategies in 3 ways: observation of the general situation and knowledge, observation of indicative action of teammates, and explicit communication; whereas the decision trees can be implemented in the coordination

8 behavior [6]. Given a positional role tree and an agent in a group role, the following is the algorithm that the agent will follow for role exchange. 1. Search the role tree in breadth first order. - Find the roles that match the current situation. - Find already adopted roles in the group role. 2. If there is one role remaining adopt it, else pick the role that maximizes the utilities of the group role as a whole. If the utility computation is not possible at this time, don’t change roles. A feature of this algorithm is that the agent does not maximize individual utility but considers the group’s utility above its own. The other feature is that we have not allowed for negotiation of roles. We believe that the fast dynamics of the game calls for fast assignments. The speed of assignments helps alleviate conflicts in roles. The role trees among agent might be inconsistent but we rely on fast updates to remove insistencies. If further inconsistencies exist, agents might randomly reassign themselves to remove them. …… NoRole Team formations  Functionalities 

Attack (433) …





Midfielder

Forward …

Corner kick Corner kick Defense(442) Defense Attack Goalie

0 

Left wing

Central Right wing defender i Figure 1. A role hierarchy for player-agents in simulated RoboCup

Roles can have conflict with each other due to the insufficient coordination. For instance, when 2 players want to recover the ball in the same region, they may both rush to action or they both may think they should clear the way for the other, which results in neither of them being able to mark the opponent. [12] suggested assigning a non-overlapping region for each agent and the regions can be changed by formation. This solution can be effective for space conflict during the game. But there are also other kinds of conflicts, which may be caused by conflicts arising from incompatibilities of role properties. For example, the midfielder may pass the ball to the right forward wing and then want the right forward wing to pass the ball to the forward to shoot, because the forward may have a better chance to do that. But the right forward wing may want to shoot directly. At this time, the right wing interferes with the role of forward to shoot. Dividing regions cannot solve this kind of conflict.

9 An alternative might be to calculate the total utility of these 3 roles by applying different strategies and deciding which plan is the best.

5 Conclusion We examined roles and norms in general and then analyzed alterations to selfinterested reasoning that accounts for agents who adopt related roles in role hierarchy. We extended role hierarchies to a positional role hierarchy and then defined a group role as agents whose roles have a shared parent role. We looked at the game of simulated soccer and suggested an organization of roles and norms. Finally, we offered an algorithm for role exchange that uses the idea of group roles.

Acknowledgements This work is supported by AFOSR grant F49620-00-1-0302. We wish to thank Dr. Svet Brainov and Dr. Gordon Beavers for useful comments.

References 1. Eduardo Alonso. Rights and coordination in multi-agent systems. In Proceedings of the First Workshop of the UK Special Interest Group on Multi-Agent Systems (UKMAS-98), pages 18-25, Manchester, UK, 1998. 2. Magnus Boman. Norms in artificial decision making. Artificial Intelligence and Law, 7(1):17--35, March 1999 3. Magnus Boman, Johan Kummeneje, David Lyback, Hakan L. Younes, UBU Team. RoboCup-99 Team Descriptions Simulation League, pages 133-138. 4. Cristiano Castelfranchi, Frank Dignum, Catholijn M. Jonker, Jan Treur, Deliberate Normative Agents: Principles and Architecture (1999) pp. 364-378, ATAL-99. 5. Simon Ch’ng and Lin Padgham, From Roles to Teamwork: a framework and architecture, Applied Artificial Intelligence, special issue on Robocup, 1997. 6. Silvia Corideschi and Lars Karlsson, A Role-based Decision-Mechanism for Teams of Reactive and Coordinating agents. RoboCup-97: The First Robot World Cup Soccer Games and Conference, 1998. 7. Jacques Ferber, Olivier Gutknecht, Admission of agents in groups as a normative and organizational problem. Workshop on Norms and Institutions in Multi-agent systems at Normative agent, June 2000. 8. Joaquim Filipe, NORMAJINI: a norm-based multi-agent system architecture using JINI. Workshop on Norms and Institutions in Multi-agent systems at Normative agent, June 2000. 9. Joaquim Filipe, The EDA Model: An Organizational Semiotics Perspective to Norm-based Agent Design, Workshop on Norms and Institutions in Multi-agent systems at Normative agent, June 2000. 10. D. Fitoussi and M. Tennenholtz, 2000. Choosing social laws for multiagent systems: minimality and simplicity. Artificial Intelligence, 119(2000) 61-101.

10 11. Jurgen Lind, MASSIVE: Software Engineering for Multiagent Systems, PhD thesis, University of the Saarland, 2000 12.Stacey Marsella, Jafar Adibi, Yaser Al-Onaizau, Gal Kaminka, Ion Musela, and Milind Tambe, Experiences acquired in the design of Robocup teams: A comparison of two fielded teams, JAAMAS, special issue on "Best of Agents'99", 2000. 13.Jonathan Moffett and Emil Lupu, The Uses of Role Hierarchies in Access Control. In 4th ACM Workshop on Role Based Access Control (RBAC). 1999, George Mason University, Fairfax, VA. 14.SangYeob Na and SuhHyun Cheon, Role delegation in role-based access control Proceedings of the fifth ACM workshop on Role Based Access Control (RBAC), 2000, Berlin Germany. 15.P. Panzarasa, T. J. Norman, and N. R. Jennings. Modeling sociality in the bdi

framework. In J. Lui and N. Zhong, editors, Proceedings of the First Asia-Pacific Conference on Intelligent Agent Technology. World Scientific, 1999. 16. Luis Paulo Reis and Nuno Lau, FC Portugal Team Description: RoboCup 2000 Simulation League Champion, RoboCup 2000: Robot Soccer World Cup IV, Berlin, 2001. 17. Nicole J. Saam and Andreas Harrer., Simulating Norms, Social Inequality, and Functional Change in Artificial Societies, JASSS, 1999. 18. Peter Stone and Manuela Veloso, Task Decomposition, Dynamic Role Assignment, and Low-Bandwidth Communication for Real-Time Strategic Teamwork. Artificial Intelligence (AIJ), volume 100, number 2, June 1999. 19. Henricus Verhagen, Norm Autonomous Agents, PhD thesis, Department of Computer and Systems Sciences, The Royal Institute of Technology and Stockholm University, Sweden, 2000.