Handling multiple levels in agent-based models of complex socio-environmental systems: A comprehensive review

1. Introduction

1.1. Socio-environmental systems

We have been living in the so-called Anthropocene [1] for some time now, where human influence on the environment is expanding globally and driving the need for holistic and integrated approaches to understanding coupled natural and human systems, which includes monitoring, analyzing, and modeling the complex systems resulting from their interactions.

This is where the concept of a social-environmental system (or socio-ecological system), SES, comes in. It is defined as a complex dynamic system that includes people and nature and is continually changing in response to internal or external pressures [2]. Pressures can result from the behavior of their social (e.g., demographic changes) and ecological (e.g., climate fluctuations) components, or, more usually, from a combination of both (e.g., human-induced climate change).

However, as interesting as this concept is for describing human-nature complex systems, it significantly defies available modeling capabilities. On one hand, SES is not really different from “ordinary complex systems,” i.e., systems whose perceived complicated behaviors can be attributed to one or more of the following characteristics: a large number of possibly heterogeneous elements, numerous possibly non-linear and discontinuous relationships between these elements, a dynamic emergence of forms (phenomena, structures, aggregates, organisms, or problems), at different levels of abstraction or scales, to which the elements adapt [3].

This notion of “emergence,” and the related notions of “levels” or “scales” lie at the heart of the various complexity theories [4], where the spontaneous appearance of structured macroscopic patterns resulting from independent microscopic interactions between the elements of the system is considered as a distinctive signature that distinguishes simple systems from complex systems.

The “scales” used in complex systems usually refer to three possible quantifiable dimensions: space, time, and size [5], which are then used to define and position “levels” at which the system can be observed, described, or analyzed. It is common to distinguish between micro, meso, and macro levels, using the hierarchical view of systems and subsystems popularized by Herbert Simon in his famous article “The Architecture of Complexity” [6]. According to this vision, a large number of indivisible individual elements, engaged in local short-term interactions, compose the micro-level; as one progresses to the higher levels, the spatial and temporal scales become larger, and the number of elements smaller. Each level, except the macro-level, is nested into or spatially and temporally bounded by a higher level.

These levels may be explicit in the description of the system because they have a recognized existence in reality (e.g., an organism, a physical structure) or because they correspond to scales at which observers want to describe or understand the system. The passage between these levels, similarly, may be described explicitly in the description of the system (e.g., in the form of a count or aggregation or disaggregation functions between levels), or it may be considered emergent. Here, “emergence” refers to the way in which structural or functional properties can emerge spontaneously, in an unplanned manner, at a given level from the self-organization of elements identified at a finer level [7], this notion of self-organization refers to a bottom-up process in which a system modifies its internal organization to adapt to changes in its goals and the environment without explicit external or top-down control.

All these characteristics translate into a real difficulty in producing an analytical and deterministic description of the behaviors of complex systems, as models of these systems imply being able to represent non-linear, ill-posed, or chaotic behaviors operating simultaneously at multiple levels of abstraction. But there are also specific characteristics of SES, mainly related to the fact that they include human elements, which make this category of systems much more complicated to deal with using traditional modeling techniques.

1.2. Challenges of SES modeling

Socio-environmental systems modeling (SES) consists of developing models to study the complex problems that arise from interactions between human (i.e., social, economic) and natural (i.e., biophysical, ecological, and environmental) systems. While there are many ways to describe SEA (especially in the anthropological or sociological literature), dynamic models appear to be indispensable tools for understanding and managing them, as they allow us to study the behavior of these systems not only under past and present conditions but also in the future, through scenario analysis or the virtual exploration of possible paths. With the advent of new techniques and computational power on one hand and the growing sustainability challenges on the other, SES modeling is used to support multiple goals, such as informing decision-making and science or raising awareness, or promoting education and communication on sensible issues [8]. However, to be useful, these models must be able to handle, in an integrated way, the complexity of SES, often characterized by intricate feedback loops in human-nature and human-society interactions [9, 10], at different levels of abstraction and with different levels of details [11]. More generally, they need to address the eight major SES modeling challenges listed by Elsawah et al. [8]:

1. Bridging epistemologies across disciplines.

2. Integrated treatment of modeling uncertainty.

3. Combining qualitative and quantitative methods and data sources.

4. Dealing with scales and scaling.

5. Capturing Systemic changes in SES.

6. Integrating the human dimension.

7. Elevating the adoption of SES models.

8. Leveraging new data types and sources.

Reading the article and the arguments behind each of the “challenges,” it is easy to see that the first three, which concern interdisciplinary, uncertainty, and source diversity, are not at all unique to SES modeling, but are, of course, shared by any type of integrated modeling involving different domains. Similarly, the last two, which are concerned with the adoption and integration of models in decision-making, are largely shared by the whole field of modeling, whether integrated or not. This leaves challenges 4, 5, and 6 as specific to the modeling of SES, i.e., in short, those related to the representation of human behavior (individual and social) and to the representation of different levels of abstraction between which emergencies and “systemic” constraints can be identified.

Models of SES have been developed using diverse approaches, including system dynamics, Bayesian networks, agent-based models, dynamic stochastic equilibrium models, statistical micro-simulation models, and any hybridization of these methods [12], but only one so far, agent-based modeling (ABM), has proved its capacity to faithfully represent individual and social human behaviors.

1.3. ABM and SES modeling

Agent-based modeling effectively present clear opportunities for SES modeling and help to address some of the main weaknesses of the various categories of models described in Schulze et al. [13] and Giupponi et al. [2], particularly with respect to the simulation of individual and social behavior and their ability to provide a descriptive and generative representation of the simulated system along the four dimensions briefly described below (loosely based on Giupponi et al. [2]).

The first is heterogeneity

In general, ABMs consist of detailed dynamic simulations in which many heterogeneous human and natural agents interact: it avoids a coarse, average, and, therefore, unrealistic representation of the system components. Human agents may vary in their demographic characteristics, location, endowments, individual capabilities, worldview, attitudes, and behavior. Natural agents can also vary in spatial and temporal attributes.

The second is individual complexity

Compared to natural agents, human agents are more complex to simulate, as they perform deliberative processes and make autonomous individual decisions [14, 15]. Behavioral complexity arises from agents' mental models [16, 17] or more commonly their “architectures,” which include their cognition, reasoning, and learning capabilities, based on the abundance of social science theories about how human agents behave in various contexts. ABM has the potential to enable the exploration of this set of decision-making theories, including the ability of agents to learn from past experiences [18], which is extremely important for long-term simulations of SES evolution.

The third is Interactions and, in particular, social interactions

Not only are human agents deliberative, but they are also social: they communicate with other agents and their behavior arises from interactions in multiple contexts with other human and nonhuman agents and the environment [19]. This aspect is fundamental capturing dynamics such as clustering, imitation, learning, and diffusion processes. It is a crucial feature for modeling SES insofar as the interactions of agents, and in particular informal relationships and opinion dynamics, can shape collective patterns of behavior.

The fourth dimension deals with the representation of organizations and the emergence of organizational structures

Human agents are deliberative and social, but they are also organizational and can form themselves into various structures, hierarchical or not. At the same time, norms and institutions, whether fixed or emergent, may induce individuals to act differently from their individual choices, which may be crucial to understanding the appearance or disappearance of certain dynamics.

The potential of ABM is thus clear for representing the social, heterogeneity, and interaction dimensions of SES, and indeed is the approach of choice for the majority of researchers faced with this need [2], but considerable methodological challenges remain, in particular, as will be seen in this review, those related to the explicit representation of scales and levels of abstraction, absent from the classical meta-model of agent-based modeling [20].

1.4. Presentation of the review

This review will explore how multi-level agent-based modeling (ML-ABM) has evolved and what solution it proposes to address modelers' representation needs. The aforementioned propositions (either formalism, framework, or ad hoc implementations) will be organized along a continuum expressing the autonomy of the levels in terms of control.

Along this continuum will be described three milestones of interest. At one end of the continuum, this review starts with the pattern we name Section 2.2, the most restrictive in terms of the autonomy of levels, as it can only represent one active level at a time. It is followed, in the middle, by a pattern we name Section 2.3, which allows the levels to have a certain degree of autonomy in terms of behavior, but imposes to describe strict hierarchical coordination over these behaviors; then, at the last end of the continuum, the one we name Section 2.4 giving levels a high degree of autonomy and control, in exchange of some complexification in the description of their coordination.

In the course of the review, the strengths and limitations of these approaches will be highlighted in relation to the needs expressed by modelers of complex socio-environmental systems in terms of modeling and simulation, as well as from a software engineering perspective. Specifically, in terms of modeling, the review focuses on how each pattern supports the representation of levels and their interactions or feedback, the design of the overall architecture, and the ease of use for modelers. In terms of simulation, it focuses on the operational instantiation of processes such as the consideration of emergence phenomena between levels.

This review will initiate a discussion of the common limitations of the patterns. It will first point out that ABM lacks an explicit representation of time and space scales, which are necessary for ML-ABM. It will then outline the difficulty of escaping the “single viewpoint hierarchy” to describe and simulate phenomena where multiple hierarchies (social, environmental, etc.) may be involved at the same time.

This discussion will allow us to introduce the proposal of a more faithful representation of multiple scales in ABM (Section 3.2). Although ABM does not really provide a way to represent the entirety of the concepts used in complex systems science (notably simultaneous hierarchies of viewpoints), its software basis, multi-agent systems (MAS), had explored some interesting paradigms that could be used to enrich the patterns presented in this review. This will be illustrated, among others, with the AGR architecture and its main change being the use of roles to describe groups/levels and the aggregation of roles to describe agents. This would remove the obligation to have agents belonging to only one super-level but would also create more complexity for modelers to use and analyze emerging patterns.

Finally, it will conclude by summarizing the studies presented in the papers in relation to the necessities expressed by modelers trying to build models of complex socio-environmental systems. Then it lists their respective limitations and calls for renewed research in this area using the approach presented in the discussion.

2. Multi-level ABM in the literature

2.1. Introduction

We have seen in Section 1.3 that ABM has the potential to address some of the challenges raised by the modeling of SES, particularly the three points listed below:

4. Dealing with scales and scaling.

5. Capturing Systemic changes in SES.

6. Integrating the human dimension.

The way they are addressed in the ABM literature is of course highly dependent on the needs of modelers and the goals of the models themselves; not all the modeling questions require to represent simultaneously multiple levels or the emergence of structures and functions, which is why there exist different architectures for implementing “multi-level” ABMs. This multiplicity of offers is the reason why the vocabulary is not completely fixed, making it sometimes difficult to find direct correspondences between the concepts used in complex systems science and the ones proposed in the different ML-ABM approaches. For instance, instead of using concepts of time and space scales, ABM designers have to deal with scheduling algorithms and encapsulation to translate temporal and spatial constraints and transfers between agents that represent levels; similarly, “systemic changes” can be represented by different aspects: dynamic instantiation of agents, injection of new code, etc.

Implementing these levels and their interactions to represent scales and their links is then dependent on two aspects: on one hand, the choices of the modelers regarding the resulting complexity of the models, which ultimately depends on their usage: should they be kept simple enough to be understood by anyone? Should they exhibit emerging properties only found in “real” complex systems? Should they allow different levels of exploration and explanation?; on the other hand, of course, the technical and computational limitations of the existing languages and computer systems available for implementing the ABM architectures used. Both aspects have evolved over time, and although we can witness a complexification of the questions asked to models in parallel with a considerable increase in computational capabilities devoted to running simulations, it does not mean that “simple” approaches are not relevant in most cases, which explains the diversity of the proposals listed in this review. Morvan and Jolly [21] reviewed those points and demonstrate that it has been applied in numerous different research fields as biomedical research, human flow, social science, ecology, etc.

Some previous study did draw some categories of ML-ABM trying to give a reading grid over previous modeling questions. Notably, Mathieu et al. [22] extracted four categories based on modelers' coupling choices (following a decision tree) between two levels. This approach represents a valuable tool for modelers building ML-ABM but does not give more detail on the overall internal model mechanism. This review provides a complementary analysis and aims to see existing ML-ABM architectures, frameworks, and approaches from the point of view of level control in the global model.

As displayed in Figure 1, we propose to present the evolution of level control along a continuum. This continuum goes from completely rigid (Zoom) without any autonomy of the levels to levels not completely autonomous but running under rigid control (Russian Dolls) or autonomous levels with relatively flexible control (Collaboration). Those three milestones are not impermeable, and there are some models and work in between them.

These levels' control differences can come from the conceptualization, creation, and organization of levels in a multi-level architecture, which is handled differently between the different approaches presented here. The Section 2.2 design pattern which coordinates levels as different models, each built with a single temporal and spatial scale, uses functions to allow switching from one level to another. Some other approaches aim to have a more discrete composition between levels and include each level into another one in a spatio-temporal hierarchical order. This extension can keep ABM's scheduling concept, like with the Section 2.3 pattern which allows building an integrated model at the cost of losing the level's inner control, or, without this control lost, as in the Section 2.4 pattern weak coupling model's level.

All these level control difference creates some extra complexity in the exploitation of those models (point 5 from Elsawah et al. [8]). When a model starts to have different levels (made or not of different independent models), it becomes more complicated to capture and analyze emerging patterns from the global system or in-between described levels. This analytic point can be needed by modelers which require to keep detailed information from each level and ensure interaction between them. Russian Dolls and Collaboration patterns provide different approaches to do so. The first one uses similar analysis tools as classical ABM as this multi-level architecture remains very close to it. The second one uses some tools inspired by software engineering.

But, some multi-level models do not need this to analyze and capture those changes, as in the first multi-level model presented by Gil-Quijan et al. [5] about the growth of a cancerous tumor. At the beginning of the simulation, the micro-level (at the cell scale) is important as it lets emerging a cluster of cells that can be observed and identified as the tumor. However, those cells can be aggregated into a bigger agent at a higher scale and lose the detailed level to see this tumor evolution over organs. The micro-level was useful to see and be precise on the creation of the tumor, once it has been created, this level is not useful in the model anymore and can, therefore, be removed.

Therefore, the choice of a pattern in the continuum to build an ML-ABM will have to be chosen by modelers following some modeling constraints or some more general software development limitations. In the first case, as in the previous example of the tumor model, modelers can choose a simpler construction of levels as in-between level emergence is not the main interest. Concerning software limitations, these can, for example, come from the fact that a model is composed of sub-models of different natures (ABM, EBM, etc.), which can be complicated to integrate into a Russian doll type architecture; in this case, it is simpler to use a Zoom or Collaboration type architecture. Also, the choice of an ML-ABM pattern can be a combination of both modeling and software choices.

Finally, integrating the human dimension as a dedicated level in an ML-ABM, as pointed out in Section 3, is the next big challenge of SES using agent-based models. Current solutions make it mostly impossible to escape the “unique hierarchy of viewpoints” to describe and simulate models. This prevents the implementation and coordination of multiple simultaneous viewpoints, which usually require multiple hierarchies (e.g., authors in the reviewing process) to define the functions and structures of a complex socio-environmental system.

In order to highlight the benefice of each approach as well as illustrate which questions can be answered by which multi-level coupling solutions, we propose to use a simple agent-based model of pedestrian evacuation (composed of People agents walking from left to right) along this review over every pattern presented.

2.2. Zoom

2.2.1. Problematic faced in SES

On the first end of the continuum, the most rigid control is the one represented by the so-called “Zoom” pattern. Each level is an independent model using a level of abstraction on the system using different kinds of models (agent-based, equation-based, else) with any scale. In this pattern, only a single level (i.e., model) is processed at a time, and modelers describe how the model will switch from one level to another using some transition function allowing to aggregate or disaggregate elements from one level to another.

In other words, while zooming in, the model will (1) call a transition function (defined by modelers) over the current level, (2) give this result to initialize the new level, then (3) destroy the previous level now unused. The system coherence over levels is insured by modelers with the transition function chosen.

However, as levels are not persistent in the execution of the model, the second SEs' problem of capturing systemic changes is very limited (or impossible), and the Zoom pattern is not used to answer this need.

2.2.2. Definition

To describe it in a more formal way: the Zoom pattern corresponds to a representation where levels are explicitly described (usually as agents) in the model with destructive behavior. Level behaviors, scales, transfer functions, etc. are also described explicitly. However, only one level is active at a time in the simulations. This ensures that there is no competition or conflict between the levels of representation during the execution of the model. Each level has an explicit aggregate and disaggregates transfer function that explains how to go from one level to the next one. Therefore, the use of this function will destroy the source level currently in use, to create the next one.

Figure 2 illustrates the global architecture of this pattern. In this illustration, the model has three levels: one equation-based model and two agent-based models. As shown, every level represents a different scale and is spatially (and temporally) limited by the level above. Finally, it is possible to move from one level to another, following some transfer functions (aggregation, disaggregation, etc.). This construction leaves each level independent, but only one level can be executed at a time.

If this Zoom pattern is used on the evacuation example model, it allows us to model and simulate the crowd either as an agent-based model (i.e., the initial model) or as a fluid mechanics equation (as in Henderson [23]). Zoom does not allow processing them at the same time; therefore modelers will have to define transition functions to move from one level to another. In this example, the mean-field approximation of agents [24] can be used to move from the ABM level to the EBM one as displayed in Figure 3.

2.2.3. Literature review

Differently from other patterns presented later, the Zoom implementation is simple and does not require a real framework to create this kind of ML-ABM. Most of the implementations are ad hoc (i.e., dedicated to the model it has been used for, and not generic to any model), then instead of listing them, this review will list how and which mathematical functions are used to allow switching from one level to another.

A note can nevertheless be made to highlight that this pattern is preferred in the modeling field of road traffic [25], as well as in crowd simulation [26, 27].

Some of the most commonly used transfer functions are aggregation approaches including mean-field theory [24, 28] and aggregation of variable methods [29]. In those approaches, the studies of the behavior of complex dynamics at the micro level with many agent components interacting with each other and with the environment is approximated by a single average dynamics of the system at the macro level. Those approaches have been applied to a wide range of fields such as physics, artificial intelligence, epidemics, ecology, biology, and game theory. However, they have the drawback of not keeping a detailed representation of the system, and, in particular, do not offer any flexibility in information transported between levels of the system due to the principle of mathematical functions.

One way to overcome the limitation is to use a graph-representation as an intermediate level which allows more flexible information transfer from different levels of a complex system either as bottom-up or top-down information flow through levels. This idea was introduced in Nguyen [30] and in some related and extended work [31–33]. This methodology was first tested with the theoretical case study in population systems/ecology; and later rapidly applied to fishery systems, epidemiology systems [34], soil systems [35], and waste management systems [36].

2.2.4. Strengths and limitations

It is possible to extract some common strengths and limitations from the use of the Zoom pattern by modelers.

The greatest strength of this pattern lies in its ease of use and implementation by modelers. This simplicity is found in the independence of the levels between them. Actually, they are only linked by transition functions to switch from one level to another. In fact, this independence leaves, among other things, the modelers free in the nature of the models that can be used as levels (agent-based, equation-based, etc.).

Moreover, due to this autonomy of the levels, the development of the layers of a multi-level model can be done in parallel by letting a multi-disciplinary team work on different models (in their nature, temporal and spatial scales, etc.) which can be easily joined in a Zoom-type ML-ABM. Also, the development and extension of a single level can be done independently of the global nesting of this level in the rest of the model.

Two more strengths result from the independence of the levels, namely that they are, by nature, reusable in other models and can be arranged at the discretion of the modelers. The reusability allows retrieving highly specialized models (ABM, EBM, etc.) from another team of modelers to improve and enrich the model currently developed. These models can be independent or come from another ML-ABM of the Zoom type. The choice in the arrangement of levels between them allows one to follow the vision and the architecture that the modelers wish to apply. It also reinforces the independence of the levels because they do not necessarily have to fit together according to scales or any other constraint (except for the feasibility of the transition function, which is virtually always feasible).

A final strength of this pattern lies in the resource saving it allows. Indeed, by only computing one level at a time, the addition of new levels will not impact the resources consumed by the execution of this multi-level model.

These forces create, nonetheless, a strong limitation in the use of this pattern. The destruction of unused levels (positive for the model resources' consumption) inevitably creates a loss of information from these levels and prevents the appearance and exploration of emergent phenomena at multiple scales and between these levels.

2.3. Russian Dolls

2.3.1. Problematic faced in SES

As a reminder, SES is using ML-ABM to try to deal with abstract representation at different scales (spatial and temporal) and to capture systemic changes in the whole model and in-between levels.

The Russian Dolls pattern tackles both problems by offering a less rigid control over the levels : they are provided with the possibility, when needed, to act on their own, but this follows a very hierarchical coordination. Each level is developed in a given scale and space, then they are all coordinated (i.e., scheduled as defined in ABM) together at the model point of view in regard to each scale. In other words, the temporal scale means that a level executed at a minute scale will be executed 60 times between steps of a level working at an hour scale; as for the spatial scale, each level is limited and bounded in a sub-space of the above level, like a house (sub-level) in a city (above-level).

As a result, this pattern forces levels to be used in a strict hierarchical order with each level smaller, or equal, in its temporal and spatial scale to the level above. Also, it is important to highlight that levels are no longer independent, making this ML-ABM very comparable to integrated models. Thus, compared to the previous Zoom pattern, no level (then nor information) is lost or destroyed which allows capturing the emergence and systemic changes of each level of the model and in-between executed levels.

This architecture is based on a software engineering concept called “tight coupling” (or “strong coupling”) [37]. A component (in ML-ABM, it is level) in this concept is usually highly dependent on each other and needs to know a lot about other components. Changing the internal logic of one component in a tightly coupled application often requires changes to several other components. But it offers the advantage of a very stable whole system and a strong integration from one component with the others.

2.3.2. Definition

This pattern, as well as its corresponding implementations in different platforms, offers support for representing multiple levels simultaneously. However, it must respect the constraint of strict control over how the levels behave and exchange information. This can be implemented using scheduling schemes similar to those in the holonic formalism [38].

In this holonic-like approach, levels are globally rescheduled to execute the full model by level (i.e., one level is executed at a time and the whole model is considered as a simple single-level ABM). It requires mixing processes and structures (which are central to systems thinking). The organization on multiple levels impacts both the structural and functional aspects of the model. First, in the structural aspects, elements (i.e., agent, level, etc.) belong to others, or are contained in others; second, in the functional aspects, elements depend on others' dynamics. These generally correspond to an implementation of concepts of spatial and temporal scales (playing a central role in multi-level mechanisms), and the extension or modification of some properties from ABM: agents now can “belong” to others or be contained in others, which makes those agents being “executed” by others and lost their own control.

In the example illustrated in Figure 4, the ML-ABM rescheduled following the Russian Doll pattern will be executed as follows. The macro-level (i.e., the city scale level) is executed where cars are moving according to a traffic model. Then, each building will be executed at the meso-level, calculating its energy consumption. Finally, for each room in the building, the micro-level is scheduled and runs each agent following another model. Once every level has been executed, the model starts a new cycle, following this same process.

Furthermore, each level can have a reciprocal effect within and with other levels, leading to some emerging patterns in the multi-level model. For example, the meso-level could be influenced by actions done at the micro-level: if a room is cooking, doing a videoconference, or reading a book, this will influence differently the energy consumed at the meso-level. Moreover, if we were to add a pollution model at a higher level than the current macro-level, it could be fed by our macro-level (with car's pollution) and meso-level (with electricity's generation pollution).

The global scheduling explained in the example can be re-adapted by modelers depending on their needs. It can start from the smallest level to the biggest (some Bottom-Up scheduling), or the opposite (Top-Down scheduling). Thus, all levels are synchronized and computed together, keep their agent's details, and can dynamically draw emerging properties in between levels.

This holonic implementation is very similar and comparable to what is called “tight coupling” (or strong coupling) in software engineering. It is a type of coupling which describes a system in which software parts are not only linked together but are also highly dependent on each other.

In this comparison, Russian Dolls' parts are its levels and share many developmental characteristics with parts of tightly coupled systems, which are often seen as disadvantages:

• A change in one module (level) usually forces a cascade of changes in the other.

• The software architecture composition of the modules (levels) might require more effort and time due to the increased inter-module dependency.

• A particular module (level) might be more difficult to reuse and test because dependent parts must be included.

Coming back to the case of the pedestrian example model, the use of the Russian doll architecture allows for example to integrate of this model, which allows simulating pedestrian movements on a single road, in a larger scale model simulating for example the traffic in the whole city. This coupling would benefit the bigger level to have a more precise simulation of critical points with a finer simulation of people evacuating an area. Those two levels would, however, lose their self-execution scheduler and be considered as a whole by the multi-level model. This way, considering that the city (i.e., big-scale level) has a step of 1 h, and the evacuating crowd (i.e., small-scale level) of 30 min, the model would execute the crowd one extra time between each city's execution as displayed in Figure 5.

2.3.3. Literature review

Different tools and frameworks have been developed over years to ease this kind of ML-ABM by modelers. Some of them have been made and used in the field of urban traffic as Multi-Level Mesa [39] provides methods to help manage the complex interactions of agents and modules of agents (e.g., groups) across multiple hierarchies (i.e., levels).

But, some more generic platforms have also been created for modelers either to apply some formalism or to extend already existing ABM platforms like:

• GEAMAS (GEneric Architecture for MultiAgent Simulation) [40] is a pioneering ML-ABM framework integrating three levels of description (micro, meso, macro). The micro and macro levels represent, respectively, agent and system points of view, while the meso level represents an aggregation of agents in a specific context. Levels communicate with each other asynchronously. It has been applied to a variety of fields as thermodynamic models [41].

• ML-DEVS (Multi Level DEVS) [42] is an extension of DEVS formalism [43] which allows the simulation of multi-scale models (and not only coupled models in which the behavior of a model is determined by the behaviors of its sub-models). There are two types of relation between levels: information propagation and event activation. However, ML-DEVS focuses on multi-scale modeling and therefore, only supports pure hierarchies of models. This extended formalism has been implemented in the framework of James II [44] to develop multi-level models.

• CRIO (Capacity Role Interaction Organization) [45] is an organizational meta-model dedicated to ML-ABM based on the concept of holon. It has been specially developed for multi-scale simulations of pedestrian flow. It had been used to model human activities [46] as well as a base to build normative holonic metamodel for multi-agent systems [47].

• LevelSpace Hjorth et al. [48] is an extension for NetLogo [49] which allows modelers to dynamically create models inside other models (in the same way as agents are created inside models), which implies a strong hierarchical structure.

• Capture Release Vo [18] is a GAMA Platform extension [50] that allows aggregate agents in the above level's agent. This process changes the behavior of captured agents which will be controlled by the bigger agent. It has been used, for instance, on an evacuation model [20] and several toy models displaying the clusterization of boids [51] as flocks [52].

Finally, some ad hoc implementations have also been used in different projects and research fields with similar approaches as in medicine with some tumor modeling [53], in urban traffic [54, 55] or even in hydrological modeling project [56].

2.3.4. Strengths and limitations

It is possible to extract some common strengths and limitations from the use of the Russian Dolls pattern by modelers.

Level's persistence is one of the strengths of this pattern in regard to the fifth SES challenge from Elsawah et al. [8]. It allows modelers to analyze and capture emergence between levels and explore modeling problems over multi-level representations of the system.

Another strength of this architecture is its natural hierarchical architecture. The spatio-temporal imbrication of levels is what is perceived from our reality and makes the conceptualization of those multi-level models simple.

However, this strength also creates one of the major drawbacks of this pattern as levels are not independent, which creates several problems shared with tight coupling in software engineering. As levels are integrated into the full model, extending or debugging them becomes more complex.

This design pattern is pretty close to classical ABM as it does not bring any new concept or method to build or use ML-ABM but extends some (like ABM's scheduling) for a multi-level explicit purpose. Therefore, it has been very straightforward for the modeling community to develop this pattern and create a wide variety of simple and powerful tools to build Russian Dolls.

But, this strength also creates a limitation in the nature of usable levels. As the full model will be close to a simple ABM, it is difficult to add levels that are not agent-based (but not impossible as in Chapuis et al. [57]). Furthermore, it will be impossible to use anything else than an ABM for an intermediate level (like in a multi-level model architecture with some ABM at the macro level, equation-based at the meso, and ABM at the micro). This kind of structure is possible with the last pattern (Section 2.4).

2.4. Collaboration

2.4.1. Problematic faced in SES

As a reminder, SES is using ML-ABM to try to deal with abstract representation at different scales (spatial and temporal) and to capture systemic changes in the whole model and in-between levels.

At the other end of the continuum, we find a pattern of Collaboration that represents a loose control between autonomous levels. Each level is, therefore, considered as an independent model using any level of abstraction on the system using different kinds of the model (agent-based, equation-based, else) with any scale.

However, to ensure catching emerging patterns in and in-between levels, those are not destroyed and created during the model's simulation run. Similar to the Russian Dolls, they all are collaborating and processed at the same time.

This architecture derives from software engineering concepts called “loose coupling” or “weak coupling” (and is opposed to “tight coupling”) [58]. The idea behind this concept lies in two main aspects: 1) components (for ML-ABM it is level) are weakly associated (i.e., have a breakable relationship) with each other, making changes in one component least affect the existence or performance of another component; and 2) each component has little (or no) knowledge of the definitions of other separate components. Those points protect level autonomy and independence, especially the second one preventing levels to lose their own scheduling/execution control.

The drawback of this flexibility is the complexity of the use of such a pattern. Even though this kind of coupling has been experienced for years and is widely adopted by the software development field, it requires some technical software work to connect each level together with some unusual concepts for modelers.

2.4.2. Definition

Similar to the Russian Dolls pattern, the last pattern, Collaboration, also addresses some modeling needs which are not addressed by Zoom. It keeps the level's information from being lost. However, to ensure the independence and self-control of each coupled level, this pattern applies a loose coupling (or weak coupling). Therefore, the main difference between the second pattern and Collaboration is how they both solve the coupling problem between levels.

Collaboration allows us to get rid of the global rescheduling constraint from the previous pattern. Here, the architecture is coupling independent models as levels and allows reciprocal information exchange between them. As this interaction is bidirectional, levels have a direct influence on each other.

In computational and systems design, a system is said to be loose when one component has a breakable relationship with other components, and components have no knowledge (or little) of the definitions of other separate components in the system. In the Collaboration pattern, components are model's level which are all communicating by software messages and considering each other level as a black box. Moreover, since the levels are loosely coupled and considered a black box, each level can be seamlessly replaced by another model. It is also possible to extend a given Collaboration model with another pre-existing model as in Chapuis et al. [59].

In Figure 6, each level corresponds to an independent model that sends information (the output result of each processed cycle) to other levels (as the input value for the processed cycle) in directions chosen by modelers. Those messages are not limited by the hierarchical order of levels in the model, e.g., the micro-level can directly interact with the macro-level, and reverse.

Therefore, by changing the way levels are coupled, this pattern no longer requires modelers to focus on or work on level scheduling. Because each level is considered a black box, they are all scheduled by themselves, each keeping control of the evolution of the model's level. This control distribution among the levels is in particular interesting in terms of the representation of systems where functions are, therefore, also distributed.

However, this distribution comes at the cost of a greater complexity in the models, as the interactions between levels are less structured, and all the related problems of coherence between levels can raise.

In the example evacuation model (Figure 7), it is possible to apply this pattern in order to evacuate the crowd on two exit points respecting a fluid physics model [60]. Therefore, the modeler will create an interface allowing to send output data (e.g., Person agent entering this area) which will be taken as input by the second model. That second model will process its fluid simulation then send back information on which path agents are taking (Figure 7). This way, instead of representing the crowd as agents making a choice, the modeler can consider it as a fluid that will force agents to one exit following a more macro-scale crowd dynamic.

2.4.3. Literature review

Different tools and frameworks have been developed over years to ease this kind of ML-ABM by modelers. Some of them have been specifically made and used in the field of urban traffic as

• SimMobility Lu et al. [61] is structured in three components (long-, mid-, and short-term) and follows a multi-level approach based on the time aspect. Modeling aspects are distributed across the three components and brought together into a single database.

• TraSMAPI (Traffic Simulation Manager Application Programming Interface) [62] is a system architecture integrating (with weak coupling) SUMO [63] and JADE [64] frameworks to build ML-ABM. JADE agents represent drivers (managing micro-behavior such as lane changing) that are linked to vehicles in SUMO (managing meso travel modes).

Some generic tools have also been created for modelers either to apply some formalism or to extend already existing ABM platforms like:

• GEAMAS-NG David et al. [65] is an updated version of Russian Dolls' GEAMAS, where the framework provides tools to detect and reify emergent phenomena between different kinds of models (like MAS and CBR [66]). It has been used for modeling urban area evolution.

• Co-modeling Huynh [67] is a GAMA Platform extension changing the base concepts of ABM formalism by allowing agents to be models themselves. With it, the ML-ABM consists of a collection of models that are only loosely coupled. It has been used to couple an ABM with a hydrological model in an evacuation model [59].

Finally, as for other patterns, some ML-ABM has been developed using this design pattern but without any framework in an ad hoc implementation. It is the case in fields such as biological systems, e.g., in many works on the multi-level modeling of tumor development [68, 69], urban traffic [70] or resource management models [71]. An interesting usage is made by Yang et al. [72] who use the flexibility of this pattern to easily swap the micro level and explore its impact without having to modify the rest of the model.

2.4.4. Strengths and limitations

These pattern offer qualities from both previous patterns. Similarly to the Zoom pattern, it allows to use and reuse of any kind of independent model as a level in the built multi-level model, and, as with the Russian Dolls, it preserves levels during the whole model's execution allowing the exploration of emergence in a multi-level system. Additionally, due to its loose coupling base, it is possible for modelers to conceptualize and coordinate levels in a hierarchical order of their wish.

However, this pattern is also the most complicated for non-developer scientists and modelers to use. For example, the comodeling framework requires three extra technical files to ensure communication and coherence between two levels (making a total of five files). This creates a great deal of complexity when trying to develop models.

Also, its overall complexity lets modelers precisely explicit the coherence of levels' execution, leading to greater complexity in the development and can make multi-level wrong because of it.

Because of this overwhelming cumulative complexity, few tools currently exist that are capable of hiding this complexity and allowing a wide range of modelers to use and explore this pattern.

3. Discussion

3.1. Limitations in the reviewed continuum

Each of the approaches and ML-ABM architecture along the continuum presented in this review have demonstrated its utility, usability, and interest. They all have been used by modelers in different fields to answer different scientific questions and allow, different, to model complex systems in the ABM model over several levels of abstraction. Therefore, they all achieve the same goal to allow modelers to create, experiment on, and work with multi-level models.

However, they also share a strong limitation in regard to SES requirements: None of them propose ways to explicitly represent spatial or temporal scale within or between levels. This limitation is induced by the use of ABM in which those concepts are missing.

Thus, the Zoom pattern simply forgets this issue between levels by considering only one structural and functional level at a time. The spatial scale can, in a way, be expressed in the transfer function between levels, but the temporal scale is always induced in the model and modelers are responsible for respecting this coherence between levels of the model they are building. The two other patterns, Russian Dolls and Collaboration, do have some workarounds to address those needs, though they are limited. The temporal constraint and coherence can be translated with scheduling and the spatial constraint is translated by the encapsulation of levels in each other.

Those solutions allow us to address and translate these spatial and temporal constraints between levels in a somewhat limited way, although they are sufficient in most cases. For example, complex system models following the natural single hierarchical imbrication, like in urban traffic models [54] or biological models [53], totally fits with implicit this spatio-temporal scales in ML-ABM; but some SES models do not (e.g., the scientific reviewing process).

Since SES are interested in the agent's behavior, which can be used as a level but not indexed in any natural spatio-temporal hierarchy, it requires the possibility to explicitly define those constraints. This point reveals another strong limitation shared by each of those patterns: they are all restricted to represent only one hierarchy at a time. It forces modelers to mix functional and structural properties in describing levels and forces them to express only a single hierarchy, despite the fact that several hierarchies of points of view and abstraction may be necessary to understand the functioning of a complex system and, moreover, socio-environmental ones.

This limitation prevents modelers from working on a wide range of models. We can illustrate this with EasyChair's organization: how is it possible for the same person (i.e., agent) to be an author, reviewer, and member of the program committee, at the same time? Taking back this review's example, evacuating agents can, in the real world, also have several roles as Pedestrians with the only will to go to the exit point, Parents varying this first behavior with a will to keep their children close and safe, Police Officer trying to evacuate everyone before himself, or a combination of each. How can it be simply implemented by modelers over the pre-existing model, and how can modelers choose which role have the greatest influence on the agent? Over which conditions?

Each of those roles brings a single physical agent to have different behavior (when it is an author, it will listen to comments and try to please reviewers, but this comportment will be entirely reversed when the agent is wearing the reviewer role), interaction range (interacting with other agents from the same group with the same role), etc. This pictures the limitation of models allowed by the use of a single hierarchy and, if used, needs to over simplify the model in a wrong abstraction.

3.2. Toward a more faithful representation of multiple-hierarchy in ABM

Although ABM does not really provide a way to represent the entirety of the concepts used in complex system science (notably simultaneous hierarchies of viewpoints), its software basis, multi-agent systems (MAS), has explored some interesting paradigms that could be used to enrich the patterns presented above.

Multi-agent systems alone provides a good basis for building complex agent-based systems, but it is one of its extensions, the organization centered multi-agent system (OCMAS), which can serve as a real base to address current SES ML-ABM drawbacks.

The concept of “organization” is hard to define, but Ferber et al. [73] described its main features as follows:

• An organization is constituted of agents (individuals) that manifest a behavior.

• The overall organization may be partitioned into groups (partition) that may overlap.

• Behaviors of agents are functionally related to the overall organization activity (concept of role).

• Agents are engaged in the dynamic relationship which may be “typed” using a taxonomy of roles, tasks, or protocols, thus describing a kind of supra-individuality.

• Types of behaviors are related through relationships between roles, tasks, and protocols.

An important element of organizations is the concept of role. A role is a description of an abstract behavior of agents. A role describes the constraints (obligations, requirements, skills) that an agent will have to satisfy to obtain a role, the benefits (abilities, authorization, profits) that an agent will receive in playing that role, and the responsibilities associated with that role.

A role is, then, a description of patterns of interactions that an agent will have to perform. Organization can be considered at two different levels: at the organizational (or social) level (i.e., as a composition of roles) and the concrete (or agent) level (e.g., a crowd) [74].

Another important principle described by Ferber et al. [73] is that the organizational level describes the “what” and not the “how.” The organizational level imposes a structure onto the pattern of agents' activities, but it does not describe how agents behave. In other words, the organizational level does not have a “code” that can be executed by agents, but it provides specifications using rules and expectations placed on the agents' behavior.

3.2.1. Agent, group, and roles

One interesting approach to treating this is the Agent/Group/Roles (AGR) paradigm [75]. When used, it is possible to extend the behavior of the agents belonging to roles that are changing or adding behavior to their existing ones. The AGR formalism is, thus, defined as:

• An agent is defined as an active communicating entity that takes on roles in groups. An agent becomes a member of a group by taking on a role defined by this group. It can then simultaneously take charge of other roles of this group, or roles defined in other groups, of which it then also becomes a member. No assumption is made about the architecture of the agents.

• A group consists of a set of agents interacting through their roles. At an abstract level, it defines a set of roles and relations between these roles that realize a global function. A group is then instantiated by the assumption of its roles by an agent, but all the roles in it may not be assumed, and the same role may be instantiated several times on different agents. The group is, therefore, a description structure with a very high level of abstraction and is capable of generating a great diversity of realizations. Finally, groups are opaque to each other: agents can only interact if they belong to the same group.

• A role is defined as the abstract representation of the function of an agent in a group. Roles are defined in groups and encapsulate how an agent should act within the group.

To allow the application of the AGR principle, an in-depth re-work of how Agent-Based Models conceptually (following the AGR paradigm) and technically (in how frameworks and software tools implement models) work is needed. Indeed, the current structure of the ABM is based on the composition of agents that are owned and compose groups. The architecture of ML-ABM is based on this composition of the group of agents.

Following the new AGR structure, it needs to be changed to allow agents to be owned and fed by different roles. Then, those roles could compose groups on which it would be possible to re-implement the current ML-ABM design (Figure 8).

This modification would deeply change the way levels are composed. They will no longer be spatially bounded to the agents composing them, and will instead let modelers create levels that rely on non-hierarchical, non-spatial, and non-temporal interactions between agents and roles. This new architecture allows the creation of complex SES models as the EasyChair's organization previously (Section 3.1) described in Figure 9.

In the AGR paradigm, illustrated in Figure 9, a person (i.e., agent) can have several roles, so it can be an author, a reviewer, and a member of the program committee. Each role would cumulatively enhance the behavior of the agent. Agents can also be grouped by role allowing committee members to communicate together and know other members of the group, which allows removing any spatial need constraint (from the agent and the group).

This new structure allows models to escape the single vertical hierarchy between levels. In Figure 9 1) levels are neither spatially nor temporarily limited by other ones, and 2) several levels can operate at the same hierarchical level as different evaluation groups in the diagram.

3.2.2. AGR review

Several frameworks have been proposed following this new architecture proposition, including some by J. Ferber himself:

• AALAADIN Ferber et al. [76] is the first meta-model of an artificial organization implementing AGR created by J. Ferber. It allows for building multi-agent systems with different forms of organizations such as market-like [77] and hierarchical organizations as well as in general modeling like in complex industrial process modeling. This first implementation has been later extended to a generic Java library called MaDKit [78, 79].

• ORIGAMI (Original Roles Identification, Groups and Agents Modeling Itinerary) [80] is based on the use of UML diagrams to build organizational representations of a system based on the AGR formalism. It allows structuring the analysis of a system through the identification of its structural units, the functions described, and the entities that constitute it with the use of diagrams. It has been demonstrated on a toy model reimplementing a collective management model of water irrigation system [81].

As the AGR formalism deeply change the structure of (ML-)ABM, some frameworks aims to extend it to make it more compliant with classical ABM and avoid having to change it:

• AGRE (AGR + Environment) [82] is an extension of the AGR organizational model, which includes physical (or simply geometrical) environments called “area.” This extension is based on the concept of a space which can be seen either as a physical area or as a social group.

• IRM4S (Influence Reaction Model for Simulation) [83] is an ABM meta-model based on the Influence Reaction model which views action as a two step process: first, agents produce “influences” (i.e., individual decisions) according to their internal state and perceptions, then, the system “reacts” (i.e., computes the consequences of influences) according to the state of the world [84]. The relations of perception and influence between levels are specified with digraphs.

In its presentation paper, this framework have been demonstrated on robot soccer collaboration model and the predator prey model [83] and reused as a base for a smart traffic simulator [85].

• IRM4MLS (Influence Reaction Model for Multi-Level Simulation) [21] is a multi-level extension of IRM4S. It relies on a generic vision of multi-level modeling. Therefore, interactions between levels are not constrained. It has been successfully applied to some real cases transportation systems [86].

Both frameworks aim to address the limitation discussed in Section 3.1 of ML-ABM, which does not fit some new multi-level models. They all extend agents with the new concept roles. Roles can be used in two directions: 1) to describe groups (which can be used as levels) and 2) to describe the aggregation of roles to describe agents. This model's structure enhancement removes the constraint to have agents belonging to only one super-level (as roles describe levels and one agent can belong to several roles) but also creates greater complexity.

Unfortunately, none of them is actually implementing AGR or proposing convincing tools for modelers to use and apply this innovative paradigm. Regardless of their usability, each of them tried to fit the AGR paradigm into either an existing framework or the general ABM architecture. However, as discussed in this review (Section 3.1), using both methods fundamentally cannot work. Then, most of the reviewed frameworks do keep spatial and temporal constraints between levels and failed to tackle the first described ML-ABM limitations.

Therefore, research and experimentation into tools that would allow modelers to truly use the AGR formalism is a topic of interest and could lead to some great new models.

4. Conclusion

This review explored how agent-based multilevel modeling (ML-ABM) has evolved and what solution it has offered to meet the representation needs of modelers. Three high-level architectures (called Design Patterns in this review) were described as landmarks on a continuum that allowed us to classify ML-ABM tools and frameworks for modelers from very simple approaches giving control to only one level at a time to more complex approaches allowing for the representation of relatively sophisticated feedback loops between autonomous levels.

The first landmark, called (Section 2.2), is the easiest to implement and allows modelers to move from one level to another with so-called “destructive” disaggregation/aggregation functions. It has the advantage of being simple to implement for the modelers and can be used with any type of model, but it only allows representation of one active level at a time, which limits the possibilities to represent and focus on the emergence relations between levels. It also presupposes a complete knowledge of how the different levels are implemented, neither allowing reuse of legacy models nor imagining separating them to make autonomous models.

The progression along the continuum is logically done in terms of the autonomy of the individual levels; the second landmark, which we call Section 2.3, marks the possibility of describing levels that can be active simultaneously but organizing them within a rigid holonic structure, where time and space scales are necessarily hierarchically nested. It allows modelers to describe the levels in a relatively natural way at the cost of a certain rigidity of the whole and a lack of autonomy of the individual levels as models.

The last one, which we have called Section 2.4, is the most complicated to implement for the modelers, but it offers the most flexibility in the description of the relations between levels. Based on the principle of “weak coupling” from software engineering, this approach considers each level as a black box that communicates through predefined inputs and outputs, which allows describing their interactions and feedback loops in a very free way. The strength of the approaches that follow this pattern is that they do not lock the modeler into a too rigid approach, and allow much better reuse of existing models. Their weakness, on the other hand, is that the design and validation of the models are made much more complex.

It is important to emphasize that this continuum is only used to classify the studies we have reviewed. It obviously does not represent a qualitative value that would express that some multilevel approaches are “better” than others, as this can only be judged in relation to the objectives of the modelers wishing to answer a specific question in their representation of SES. In some cases, simply capturing the transition from one level to another (appearance of a structure or behavior causing the transition to a higher level model) is sufficient, and this is what the vast majority of published study is limited to. In other cases, cyclic behaviors of aggregation/disaggregation, emergence/constraint, or more complex feedback loops must be implemented between levels to allow modelers to answer the questions posed to them, and this is what approaches ranging from Russian Dolls to Collaboration allow.

In terms of SES representation, the review has nevertheless highlighted (Section 3.1) two important limitations shared by all the approaches presented: 1) the lack of convergence on a common and documented representation of time and space scales, which remains at the sole discretion of the modelers and is often not explicit, making the reuse of models representing these levels quite problematic; 2) the impossibility of representing more than one hierarchy at a time, which prevents the implementation and coordination of several simultaneous points of view, sometimes necessary to define the functions and structures of a complex socio-environmental system, in particular when multiple social groups—and their overlaps and interactions—must be represented.

We hypothesize that as the questions posed to modelers become more complex, these two limitations will become real obstacles to a more professional use of ABM. On the one hand, because, as multilevel modeling becomes more widespread, it will be necessary to be able to easily reuse the models of levels, which will have to make their time and space scales explicit; on the other hand, because the possibility of taking into account and interweaving different points of view is one of the prerequisites of interdisciplinary approaches, which become widespread in the handling of SES problems. In order to make progress on these two points, we believe it is necessary to revise the classical structure of agent-based modeling, which is still too closely linked to the limitations of object-oriented programming, from which it is inherited, in order to offer a more faithful representation of the multiple levels and to link them more explicitly to scales of time and space. The use of the AGR paradigm (Section 3.2) is in this respect potentially interesting. It allows one to define an agent as a dynamic set of roles and to define the groups of agents (the levels) in terms of the grouping of these roles. An agent can thus be described as belonging to several distinct levels simultaneously. The conditions for aggregating, disaggregating, or sharing control are defined by the group itself, which also allows, in theory, to add or subtract them from the model dynamically and also to build, eventually, easily reusable libraries of levels.

This proposal implies modifying the “classical” metamodel of agent-based models while keeping their easily accessible side for non-experts and continuing to ensure that their implementation is not too complex or constrained. This is not an easy challenge to meet, as it is also this simplicity that has made them successful in recent years. But as agent-based models become indispensable tools in decision support processes, moving away from the “toy model” aspect that has often been reproached to them, it seems important to us to make sure that their development becomes more professional. We hope that this review, these proposals, and the study that we are going to do on them will allow us to progress in this direction.

Author contributions

AB was the principal designers of this document and is responsible for much of the writing. AD and DN-N brought some complementary references and helped to structure the whole review. All authors have worked together, each in their own specialty, to design, and describe the research presented in the paper.

Funding

This work was supported by the ANRS (https://anrs.fr) through the funding of a COVID-19 SOUTH project (COV15) and the IRD Vietnam (https://vietnam.ird.fr).

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References