API-User's Guide

Information relevant to API-Users.  Relevant for people who want to program against MATSim without being part of the regular refactorings.

MATSim API Status

There is currently an API unter org.matsim.api.core.v01 . 

This is what should be used, although it may not yet be complete.

The API is relatively stable, but not fully stable.  Use releases if you need stability.

MATSim API Use Cases

The MATSim API was/is designed with the following use cases in mind:

  1. Initial demand generation
  2. Behavioral modules (such as TimeAllocationMutator, Planomat)
  3. the "mobsim" (network loading)
  4. analysis (postprocessing)

A 5th case, starting the controler out of a java pgm (including capability to set controler listeners), may have to be added ... it is possible to do this, but not api-based.

In apr'11, api-based network generation was added as use case 0.

Case 0, i.e. "network generation", is roughly there, but needs a number of "utilities" that do not have stable paths.

Case 1, i.e. "initial demand generation", in our view works quite well.  Please talk to us if things do not work or if you desire additional functionality.

Case 2, i.e. "behavioral modules", is not tested, and needs an entry in the core.  Testers are welcome; please talk to us.

Case 3, i.e. "alternative mobsims", is not tested.  Testers are welcome; please talk to us.

Case 5, i.e. "analysis/postprocessing", is not tested.  Testers are welcome; please talk to us.

Case 0: Network generation

In general we recommend to start with data from openStreetMap (see here).  There are, however, not so few cases where data comes from other sources (e.g. some emme or visum model, or some navteq or teleatlas data).  In some cases, there may be existing matsim converters for this (check here).  This section will give pointers especially for situations where some programming is necessary (since these should, in theory, be api use cases).

emme/2

emme/2 network files can be exported in some ascii format (probably tab separated).  These can be read in again by normal parsers.

Unfortunately, the emme/2 network information is not typed strongly enough that it is possible to write automatic parsers. 

Some initial material can be found under tutorial.unsupported (javadoc, code).

More material can be found by searching for "emme/2" in the playground (after revision 14899).

Case 1: Initial demand generation

The short story

Start with something like

Config config = ConfigUtils.createConfig();
Scenario sc = ScenarioUtils.createScenario(config);

From there on, you can use command completion on sc and anything else that you have found in order to find out about additional commands.

At the end, you write to file by

MatsimWriter popWriter = new org.matsim.api.core.v01.population.PopulationWriter(population) ;
popWriter.write( filename ) 

The slightly longer story

Config config = ConfigUtils.createConfig();
Scenario sc = ScenarioUtils.createScenario(config);

Network network = sc.getNetwork();
Population population = sc.getPopulation();

PopulationFactory populationFactory = population.getFactory(); // (*)

Person person = populationFactory.createPerson(sc.createId("someId"));
population.addPerson(person) ;

Plan plan = populationFactory.createPlan();
// do something with the plan, fill it with Activities and Legs.
person.addPlan(plan);


MatsimWriter popWriter = new org.matsim.api.core.v01.population.PopulationWriter(population, network);
popWriter.write(filename);

It is important to note the (*) syntax, i.e. the creational methods are in the factory.

 

The long story

An example is under src/tutorial/programming/example8DemandGeneration.  This should also be used to obtain hints with respect to the correct syntax ... since it is automatically part of all refactorings.

Case 2: Your own mental module

The short story

A mental module

  • extracts information from the events stream
  • modifies plans when they are given to it

You can write your own mental module and plug it into MATSim.

The slightly longer story

Essentially:

public class MyModule implements PlanStrategyModule {
        MyModule() {
        }

	public void prepareReplanning() {
	}

	public void handlePlan(Plan plan) {
            // when given a plan, decide here what to do with it
	}

	public void finishReplanning() {
	}
}

public class MyStrategy extends PlanStrategy {
    public MyStrategy(Scenario s) {	
        this.addStrategyModule(new MyModule());
    }
}

Additional options can be found from the examples (see below).

You can then specify the class name of your module in the config-file as a replanning strategy, e.g.

<module name="strategy">
  <param name="ModuleProbability_1" value="0.1" />
  <param name="Module_1" value="your.package.MyStrategy" />
  ...add other modules here...
</module>

If you run a simulation with the default MATSim-Controler, it will then load your mental module and use it to modify the plans. This API is not yet complete, in the sense that it only offers limited functionality. We are working on a more powerful way to add your mental modules, but this may take quite some time.

Notes:

  •  Please forgive the confusion between "modules" and "strategy".  In the code, a "strategy" is composed of a plan selector, and several (strategy) modules.  In the config file, what is called a "strategy" in the code is called a "module".

Examples

One example is under tutorial.programming.example10PluggablePlanStrategy in the source repository (under "matsim").

Case 3: Your own mobsim

The short story

The "mobility simulation", or short "mobsim", is what executes all selected plans, and writes events.

The slightly longer story

You can do

Controler controler = new Controler() ;
controler.setMobsimFactory( new MyMobsimFactory() ) ;

plus

public class MyMobsimFactory implements MobsimFactory {
    public Simulation createMobsim( Scenario sc, EventsManager ev ) {
        return new MyMobsim( sc, ev ) ;
    }
}

plus

class MyMobsim extends Simulation {
    Scenario sc ; EventsManager ev ;
    MyMobsim( Scenario sc , EventsManger ev ) {
        this.sc = sc ; this.ev = ev ;
    }
    public void run() {
        Network net = sc.getNetwork() ;
        Population pop = sc.getPopulation() ;
        ...
        EventsFactory ef = ev.getFactory() ;
        ev.processEvent( ef.createXXXEvent( ... ) ) ;
        ...
    }
}

This is fully api-based (except that some material is still under core.api.experimental) .

You then need to make sure that

  • you execute all selected plans
  • you generate agent activity end events
  • you generate agent/vehicle departure events
  • you generate link leave events
  • you generate link enter events
  • you generate agent/vehicle arrival events
  • you generate agent activity start events

If you do this, your mobsim should work inside the MATSim iterative framework.

Case 4: Events

Events are a central concept in MATSim, as they are the only way for feedback from the mobility simulation to the behavioral modules.

Event Types

Event Overview and Order

Overview graphic not available

Event types and order in which they are thrown.

Event Types

The following types of events are currently supported and should be generated by a mobility simulation:

id name data description
0 arrival time, agent-id, link-id An agent arrives at a link at the specified time
2 left link time, agent-id, link-id An agent leaves a link at the specified time. This builds a group together with 5-entered link. An agent moves through the network by entering and leaving link until it reaches its destination link.
3 stuckAndAbort time, agent-id Used for deadlock-prevention. If a deadlock occurs at some place, we solve it be removing some "stuck" agents, aborting them, such that other agents can continue.
4 wait2link time, agent-id After an agent departs, it must wait until it finds a space to enter the road. This is the moment the agent really enters the road. Often at the same time as 6-departure, but not always.
5 entered link time, agent-id, link-id Issued if an agent crosses a node (see 2-left link)
6 departure time, agent-id, link-id when an agent departs at a location
7 actstart time, agent-id, link-id, act-type when an agent starts performing an activity
8 actend time, agent-id, link-id, act-type when an agent ends performing an activity
9 agentMoney time, agent-id, amount an agent earns some amount of money (e.g. -5 if it has to pay a toll)

The following example shows the events of one agent, starting at home and driving to work:

T_GBL   VEH_ID  LEG_NR  LINK_ID FROM_NODE_ID    EVENT_FLAG      DESCRIPTION
21610   1       0       1       0               8               actend home
21610   1       0       1       0               6               departure
21612   1       0       1       0               4               wait2link
21640   1       0       1       0               2               left link
21640   1       0       2       0               5               entered link
21740   1       0       2       0               2               left link
21740   1       0       3       0               5               entered link
21820   1       0       3       0               2               left link
21820   1       0       4       0               5               entered link
21890   1       0       4       0               0               arrival
21890   1       0       4       0               7               actstart work

 

Generating Events

Some remarks about generating events, e.g. in a custom traffic simulation:

  • Events must be in the correct temporal order per agent, but not globally.
    An event e1 at time t1 of agent a1 must not be created after an event e2 at time t2 of the same agent a1 if t1 < t2. However, an event e3 at time t3 of agent a2 can be created after e1, even if t3 < t1. It is still good practice to ensure that all events are chronologically ordered.

Handling Events

All events, that were issued in a simulation-run, should be passed along to an instance of the class org.matsim.evens.Events. This is done by creating a new event and then calling computeEvent() method of your instance of Events.

Example:

Events myEventManager = new Events();
EventLinkEnter test = new EventLinkEnter(time,  agentId,  linkId);
Events.computeEvent(test);

The Events instance will then take the event and forward it to every EventHandler, that wants to receive events of this kind.

To receive events out of the simulation, you have to write your own event-handler as described below.

Building an event handler

Building a new eventhandler that uses already existing events

If you just want to receive any of the already defined events, it is an easy task:

  1. Simlpy write your own EventHandler class, that implements every EventHandler...I interface that you want to get to know about.
  2. Then you have to make a new instance of your handler-class and add it to your Events instance by using Events.addHandler(EventHandlerI handler).
  3. We are ready now, Events will call your EventHandler for every event that he listenes to.

Hazard: Be aware, that if you overload some special EventHandler ...I interfaces and the BasicEventHandlerI than the special interfaces will receive the appropriate events and the handleEvent(BasicEvent event) receives the SAME event a second time. This will get optionally in a later verion.

Remark: Instead of overloading several EventHandler...I interfaces it sometimes might be easier just to overload BasicEventHandlerI and to check the events in an if clause like this:

if(event.getClass() == EventAgentNoRoute.class)  {
  ...
} else if(event.getClass() == EventAgentArrival.class)  {
  ...
} else if(event.getClass() == EventLinkLeft.class)  {
  ...
}

Example:

 

class CountLinkTravelTimes implements
        EventLinkEnterHandlerI,
        EventHandlerLinkLeft {
  public void handleEvent (EventLinkEnter event) {
    ...
    id = event.linkId; // do something with it
    ...
  }
  public void handleEvent (EventLinkLeft event) {
    ...
    id = event.linkId; // do something with it
    ...
  }
}
...
Events myevent = new Events();
CountLinkTravelTimes myhandler = new CountLinkTravelTimes();
myevents.addHandler(myhandler);
EventLinkEnter test = new EventLinkEnter(time,  agentId,  linkId);
Events.computeEvent(test);

Building your own specialized event

Building your own specialized event based on an existing event:

Now we have to additionally define a specialized event class, let's say

class EventAgentStuckAndReplanning extends EventAgentStuck {
  private double replanning_probability;
  // some other special things...
}

Now we are ready to receive this Event from all EventHandlerAgentStuckI! We can check for this special event by the means of

if(event.getClass() == EventAgentStuckAndReplanning.class) {...}

If for some reasons we want to write a handler that only receives this special Event StuckAndReplanning, we would have to add a new interface that extends EventHandlerI

e.g.

public interface  EventHandlerAgentSuckAndReplanningI extends EventHandlerI {
  handleEvent(EventAgentSutckAndReplanning event);
}

and ready you are... just implement that interface in your Handler and the handler will start receiveing only the appropriate events!

Building your own specialized event based on nothing at all:

It is simply as above, only that you inherit from BasicEvent. All Events should inherit from BasicEvent, which only defines, that there has to be a time and an agentId. One might think of events that don't even have an agentId, maybe you would leave this field blank (e.g. Integer.MIN_VALUE) or whatever. We included agentId into BasicEvent as the wast majority of events will need an agentID.

Event Types - Future

This article is work-in-progress!

The current event types are less and less suited for describing the simulation output. Especially when activities take place in facilities (instead of links as now) and when public transport is included in the simulation, new event types have to be introduced. This page proposes a new set of events that should be suitable for the future development of MATSim.

Key Concepts of New Events

One of the main concepts are the differentiation between Persons and Vehicles. Persons are part of the population/scenario, specified in the plans/population-file. Vehicles are created (at the moment) within the simulation. The simulation only moves vehicles, never persons. Pedestrians should thus to be enclosed into a "Pedestrian-Vehicle".

Proposed Events

Event Name Attributes Notes
EndActivity person, facility, actType  
EnterVehicle person, vehicle, facility  
DepartureAtFacility vehicle, facitlity  
FacilityToLink vehicle, link  
LinkToIntersection vehicle, link  
IntersectionToLink vehicle, link  
LinkToFacility vehicle, link  
ArrivalAtFacility vehicle, facility  
LeaveVehicle person, vehicle, facility  
StartActivity person, facility, actType  
VehicleStuck vehicle, link  

 

File Format: TXT

The TXT format is the currently most used file format for storing events. It consists of a plain text file with the following header:

T_GBL   VEH_ID  LEG_NR  LINK_ID   FROM_NODE_ID    EVENT_FLAG      DESCRIPTION
  • T_GBL is the time of the event, in seconds after midnight
  • VEH_ID corresponds to the agent-id (there's no such thing like a vehicle-id yet!)
  • LEG_NR unused, set to 0
  • LINK_ID the link of the id the event took place. If not used, set to 0
  • FROM_NODE_ID unused, to to 0
  • EVENT_FLAG the id of the event-type (see Event Types)
  • DESCRIPTION unused, usually set to the name of the event-type (see Event Types)

The fields are separated with tabs. Make sure to also include a tab between EVENT_FLAG and DESCRIPTION, even if the DESCRIPTION is empty!

The following is an example, how an event-file could look like:

T_GBL   VEH_ID  LEG_NR  LINK_ID  FROM_NODE_ID    EVENT_FLAG      DESCRIPTION
21610   1       0       2        0               8               actend h
21620   2       0       3        0               6               departure
21630   3       0       4        0               4               wait2link
21640   4       0       5        0               2               left link
21650   5       0       6        0               5               entered link
21660   6       0       7        0               0               arrival
21670   7       0       8        0               7               actstart w
21680   8       0       9        0               3               stuckAndAbort

 

Parallel Event Handling

When running a MATSim simulation, unused cores can be utilized to make the simulation faster. The event handler which is used by default, is running in the same thread as the simulation. For this reason, the presented 'parallelEventHandling' module can make your simulation much faster.

How to use the 'parallelEventHandling' Module

Parallel event handling can be turned on by inserting the follwing module into the config XML file of MATSim:

<module name="parallelEventHandling">
    <param name="numberOfThreads" value="1" />
</module>

The only required parameter in this module is 'numberOfThreads'. The value of this parameter specifies, how many threads (cores) should be assigned to handling events.

There is an additional optional parameter 'estimatedNumberOfEvents', which can be optimized to your simulation resulting in slightly faster runs. But its usage requires an estimate of the number of events which will occur in one iteration.

<param name="estimatedNumberOfEvents" value="5000000" />

Hints and Pitfalls

  • Don't make any  assumptions on the order, in which the handlers are executed (as they are executed in parallel).
  • Don't write the data in one handler and read that data in a different handler. Although this is possible, special care is required as synchronization between threads is needed for this. Furthermore this could lead to a degradation of the speed up.
  • There is no point in using parallel event handling, if you have just one core available on your machine.
  • Always make sure, that one thread is needed for the single cpu simulation. This means, if you have 4 cores, you can set the parameter 'numberOfThreads' to maximum 3.
  • If 2 handlers have been added to the simulation, then there is no point in assigning 3 threads to parameter ''numberOfThreads', because it won't make the simulation faster (but rather could slow it down in some cases).
  • The number of handlers can be bigger than 'numberOfThreads'. In this case automatically if possible, each thread is assigned the same number of handlers. 
  • If the simulation is quite slow (compared to event handling), you won't be able to make full use of parallel event handling.

The actual speed up you get depends on many factors, especially on those mentioned in 'hints and pitfalls'. Experiments on a 16 core machine with different numbers of handlers have shown that the parallel event handler can reduce the simulation time with a very low overhead.

Further Reading

Some Java specific implementation aspects of JDEQSim and parallelEventHandling are described in the following paper:

Waraich, R.,  D. Charypar, M. Balmer and K.W. Axhausen (2009) Performance improvements for large scale traffic simulation in MATSim, paper presented at the 9th Swiss Transport Research Conference, Ascona, September 2009. Download from here.

 

 

 

 

File Format: XML

Description of the data-format to write (and read) Matsim-Events

Events should be stored in XML-Files.

There is no DTD for Events because each event could use other attributes as needed. As long as it is not possible to defined something like "free additional attributes" for tags in DTD, there will be no DTD for events.xml. Therefore, the events-fileformat ist very simple to reduce the possibility of errors.

The file contains:

  • xml-Header
  • root-tag <events>
  • <event>-tags with only attributes, otherwise empty (<event> is a leaf in the xml/dom-tree)

event-attributes:

The following attributes are required:

  • time, in seconds past midnight
  • type, an identifying string, see also Events

Most events will also have the following attributes:

  • agent, the ID of an agent
  • link, the ID of a link in the network

Additional event-types may define additional attributes.

Example of an events.xml-file:

<?xml version="1.0" encoding="utf-8"?>
<events xml:lang="de-CH">
<event time="21600" type="departure" agent="759062" link="47" /> <!-- time: 06:00, in seconds -->
<event time="22980" type="arrival" agent="759062" link="122" /> <!-- time: 06:23 -->
<event time="21720" type="departure" agent="966916" link="25" />
<event time="22865" type="arrival" agent="966916" link="312" />
</events>

Additional event-attributes:

  • "activity", the number of the activity starting with0
  • "act_type", the type of the activity, e.g. "home", "work"
  • "leg", the leg number starting with 0

Case 5: On-the-fly analysis

The short story

Idea: Attach an events listener to the running iterations, and use the events data flow for any kind of analysis.

The slightly longer story

public final class MyControler {
	public static void main(String[] args) {
		Controler controler = new Controler( args ) ;
		final Scenario sc = controler.getScenario() ;
		ControlerListener startupListener = new StartupListener() {
			@Override
			public void notifyStartup(StartupEvent event) {
				EventHandler myEventHandler = new MyEventHandler() ;
				event.getControler().getEvents().addHandler(myEventHandler);
			}
		};
		controler.addControlerListener(startupListener);
		controler.run();
	}
}

and then

class MyEventHandler implements ActivityEndEventHandler {
    public void handleEvent( ActivityEndEvent ev ) {
        // do any kind of api-based analysis with the event
    }
}    

This second part is fully api-based (except that some material is still under core.api.experimental) .

The long story

Some material (not fully consistent with the above) is under src/tutorial/programming/example6EventsHandling.  This should also be used to obtain hints with respect to the correct syntax ... since it is automatically part of all refactorings.

ObjectAttributes

This is not truly "public api".  But it is meant as code to be used if one needs additional data attributes within matsim.

See here for code.