description

We have built a system that allows simple multiagent simulations with detailed network simulation. The contribution of the system is identifying a clean interface and plug in point to the ns2 network simulator for a multiagent system to use. It demonstrates how a system could integrate sending agent messages with sending network messages. It also shows a simple method of synchronizing time slices between the simulators.

 

It was built with a focus on multiagent reinforcement learning at the network routing level, and environment sensing and data aggregation at the agent level. The agents communicate complex messages. They sense information from the environment and know their location. The multiagent system simulates time for the agents and environment, delivers messages.

 

The main system philosophy is that reuse does not yield enough benefit for the large effort that goes into enabling it. Develop specific agents, environments and modify previous simulations to suit your needs. Attempt to make pieces which will be used from simulation to simulation lean. Mostly, make things small and high quality enough that they can easily be understood and altered from the ground up. In practice this results in using the both simulation infrastructures as is; modifying exisiting agents, task structures, and routing protocols; and implementing environments from scratch.

 

So far, we have developed rough simulations for weather sensing for CNAS agents and solar harvesting simulation. These implementations serve as a reference guide for future usage and evolution.

 

The original goal of the project was the identify, customize and use an existing multiagent system with detailed network simulation. We did not find such a system, so we sought to identify an existing multiagent system to integrate with the popular ns2 network simulator.  The goals for the multiagent system were flexibility in modifying agent goals and coordination. Since the experiment domains considered had simple task structures, we built a simple multiagent system and focused on ns2 integration and facilitating implementing custom routing protocols.

 

The system is currently being completed with a component for expressing and simulating richer task structures. It will incorporate ideas from the activity world view in discrete event simulation, and use TAEMS for describing tasks. Running tasks will mostly follow MASS's lead. Each agent has a local view of tasks, simulator has a global view.

 

As the simulation progresses, sensing the environment might trigger the creation of a goal or the simulation will have scheduled task assignments specified when created. An agent will consult its local view and scheduler to devise a what tasks will be performed and by whom. It will register this schedule with the simulator. The simulator will attach intertask relationship information to each task. The simulator will activate each agent when each task begins, providing it with the outcome. The outcome will be calculated by the simulator using the global view, runtime data, and relationship effects. The agent will then proportionally update state and resources according to the outcome as time progresses.

 

brief arch:  

 

 

logical module stack

--------------------------------------------------------

| agent | plan | multiagent-task-sim |

----------------------------------------------------------------

     | task-analysis |            | network-sim |

       --------------------                 --------------------

                                       | route-proto |

                                        --------------------

 

 

install:

USE BASH

tcl8.4.19 - cd internal/tcl8.4.19/unix, ./configure , make ,  [make install]

tk8.4.19 - cd internal/tk8.4.19/unix, ./configure , make , [make install]

otcl-1.13 - cd internal/otcl-1.13, ./configure --with-tcl=../tcl8.4.19 --with-tk=../tk8.4.19 ,  make , [make install]

tclcl-1.19 - cd internal/tclcl-1.19, ./configure --with-tcl=../tcl8.4.19 --with-tk=../tk8.4.19, make , [make install]

ns-2.33 - cd ns-2.33, ./configure --with-tcl=../internal/tcl8.4.19 --with-tk=../internal/tk8.4.19 --with-otcl=../internal/otcl-1.13 --with-tclcl=../internal/tclcl-1.19 --without-defaultoptions ,

              temporarily,    export  LD_LIBRARY_PATH=../internal/tcl8.4.19/unix:../internal/tk8.4.19/unix

              make , [make install]

*ns-2.33 needs LD_LIBRARY_PATH pointing to tcl libraries, PATH pointing to ns2 root directory. see "ns2-env.sh" for example.

[um-olsr-0.8.8 - place decompressed folder in ns2, apply patch - already done]

[taems - 1.3.1 - build (add correct version of utilities.zip(?))  - already done]

sisc-1.16.6_all - (get - i used deb pckg. for red hat, use tgz or jdk w/ src) - install sisc, or run "sisc" with full path

*sisc needs CLASSPATH to taems and external libraries (javacc.jar distributed with teams). see "sim-env.sh" for example.

 

download: net3.85 (60 MB, june 09)

 

directories layed out as follows.

.../

      *.scm

      *.otcl

      *.sh

      /ns-2.33

            ...

            Makefile.in

            /olsr

      /task-analysis*

             (taems contents: classes/, lib/, ...)

     /internal

            /tcl...

             /tk...

             /otcl-...

             /tclcl-...

             /sisc-...

 

use:

     term 0, synchronization server (run first or second):

          run "./synch-go.sh"

          kill with ctrl-c until shell prompt appears, after each simulation

     term 0.5, qroute server (run first or second):

          run "./qroute-go.sh"

          simulation ends, but scheme objects persist for inspection at scheme prompt

          type " (exit) " to kill

     term 1, ns2 (run third):

          first use of BASH shell, run ". ns2-env.sh" (sets up ns2 path and ld_library_path for tcl, tk) (NOTE: if didnt make install tcl, need to create ln -s to tcl8.4.19/library/init.tcl as well)

          run "ns"

          at tcl prompt, use "source ns2-script.otcl" to start

          simulation ends, but tcl objects persist for inspection at tcl prompt

          type "exit" to kill tcl prompt

          more out.tr and outstat.tr to view traces

          after each simulation, run "ns2", etc

     term 2, simulator (run fourth):

          first use of BASH shell, run ". sim-env.sh" (setup java / sisc classpath)

          run " ./internal/sisc-*/sisc dishwasher-sim-script.scm -e "(run SIM)"  "  (or something close to that)

          simulation ends, but scheme objects persist for inspection at scheme prompt

          type "(exit)" to kill

          after each simulation, run "sisc ...", etc

 

 

 

hack: 

 

...guide for end user scripting of simulation instances

 

 

documentation references: 

 

-sisc manual(recommend: 5. i/o - bulk character i/o, string ports, socket operations, tcp, file manipulation. 7. types/objects - standard types, generic procedures, defining generic procs, defining methods, object system, classes, instantiation  . 9. java + scheme - calling java from scheme/*  10. addt'l - hash tables , records , creating libraries)

-r5rs doc ( 6. standard procedures )

-srfi. see shared-include for srfi's to review

 

-ns2 manual ( 3.otcl. 4. nodes, forward, 10. agents, 12. packet, 16. mobile, 30. unicast routing, 35.tcp, 39. apps + transport api  ) , my ramblings about internals, code - /common, /tcp, /routing, /tcl/lib, /tcl/rtglib, /tcl/ex, /dsdv, /tcl/mobility

 

-tcl reference (file, socket, fconfigure, gets, puts, flush, expr, mathop, format, list, if, foreach, while, after, vwait, proc, incr, array, global, source) , man page

-otcl reference (tutorial. object - destroy, init, instvar, proc, next, set. class - instproc, superclass.)

-tclcl code (no docs. instead see key files tcl-object.tcl, tcl.cc )

-um-olsr (install.)

 

-taems guide ,  taems home , mass home , mass/jaf internals overview

 

 

background/history: 

 

"Focus more on what the system is capable of doing. Also focus more on the agent structure and what kind of agent we are able to develop and how TAEMS comes into the picture."

before start

-glance at a multiagent simulation environment, familiarize with issues in creating one -> robocup rescue internals review, write one from scratch for simple task allocation

-look at some past experiments and needs, review ideas in marl + routing -> huzaifa's routing + organization knowledge, abdallah's Multiagent Reinforcement Learning and Self-Organization in a Network of Agents 2007, kumar's thesis cqrouting, boyan + littman's routing w/ marl nips 94 paper, bellman distance vector routing paper 58

-find mas development or simulation library that integrates detailed network simulation -> none found, some have stats on total messages

-find mas development or simulation library that is flexible in modifying agent behavior; check promas, web surveys, paper surveys of systems -> compare 2apl, ooa, zues, cougaar, madkit - features, approach, internals. implement prototype with 3apl. select ooa because of prolog and java interoperability, lean but useful api for declaring goals and services. small, modifiable implementation

start

-brief look at network simulator features -> ns2/c++, swans+jist/java

-review concepts in discrete event simulation, network routing concepts -> languages, systems, libraries, world views - activity, event, process; routing - centralized link state and distributed distance vector

-select development or simulation library -> base on taems/mass because of past precedent of usage in lab, probabilistic outcomes, expressiveness of task relationships

-quick review of usage of robocup rescue -> see email description

development...

-glue a network simulator together with multiagent simulation ->

   -sketch out components, points of integration, what portions of mass, jaf, taems, and ns2 to use

> The simulator is so abstract, and focused on purely

> executing events, that it can be ignored for our purposes. Taems will be

> used as the basis for each agents knowledge of the world as well as an

> agent's scheduler input. Jaf can be ignored - many irrelevant features,

> focused on building agents with plug-ins, providing a gui.

   -use sisc/jvm scheme to wrap taems, agent implementation + multiagent system  + talk over sockets with ns2 about messages sent. implementing routing protocols and talk with ns2.

-make ns2 my bitch, unhealthy obsession ->

   -ns2 overall workings, sending packets, events, agents,

   -routing protocols - understand manet tutorial

   -implement wired routing outside of ns2, integration of ns2 and external routing library

   -understand dsdv, olsr

   -review implementation of ns2 wireless routing

   -scripting ns2 simulation sessions

   -implement qrouting

-implement integration of ns2 msg server and sending messages in multiagent system

-implement multiagent system with mailbox, time slices, running agent

some usage...

-implement agents for weather sensor simulation w/ roles, teams, aggregation of data

-implement agents for solar simulation w/ sensing, environment, sharing readings, learning w/constraints

development...

-implement synchronization server to speed up , correct coordination between ns2 and multiagent

-test unit level and record scaling performance

-create taems view wrapping

development...

-simulator incorporate task knowledge, handle interrelationships, delivering outcomes

-agent has planner to reason with taems view

 

 

 

 

 

projects:

 

test larger task structures (effort: medium, skill: intermediate)

-define some larger task structures for agents, writing method implementations

-create simulation scripts

-run and note behavior, any errors

-fix or modify behavior of system to liking, if needed

 

test contrasting local task structures (effort: medium, skill: intermediate)

-same as "test larger task structures" except create some local and non-local methods with varying accuracy in outcomes.

 

throw major exceptions (effort: small, skill: beginner)

-browse system, identifying important portions of logic

-declare and throw exceptions around these important portions for typical runtime exceptions expected

 

unique handling of assignments (effort: small, skill: beginner)

-decide on how agents or system should select which agent for a given role/type should handle a scheduled task assignment or environment change

-currently all agents are notified of an environment change and all agents can act as they wish

-currently all agents of a given role/type are notified of a task assignment and each of them can handle assignment as they wish

 

implement remaining interrelationships (effort: medium, skill: advanced)

-produces, consumes, uses, facilitates and hinders

-the planner needs to recognize and plan with such interrelationships in task structures

-the runtime task simulator needs to enforce

-the implementations of agent's methods need to allocate execution outcome values

 

implement complete planner (effort: medium, skill: advanced)

-with specific tradeoff strategies

 

flesh out handling of differences between plans and actual execution (effort: medium, skill: intermediate)

-when more accurate runtime task execution causes a plan to fail, decide on how system should react

 

complete qroute implementation (effort: small, skill: beginner)

-qrouting basics are implemented, but some specific parameters and behavior need to be completed

 

ns2 synchronization with custom scheduler (effort: medium, skill: intermediate)

-rewrite ns2 synchronization to use custom scheduler with built in synchronization points instead of default scheduler with synchronization events

 

cleanup distributed synchronization (effort: small, skill: beginner)

-implement distributed synchronization to more cleanly perform "round-robin synchronization"

-currently ns2 and task simulator synchronize by acquiring and releasing a mutex

-the server controlling the mutex ensures round robin access between ns2 and task simulator

 

finalize logging (effort: small, skill: beginner)

-use defined tracing functions to finalize overall logging and provide logging levels

-change some of existing tracing functions to suit

 

cleanup taems logging (effort: small, skill: beginner)

-silence or minimize logging from taems