Framework workshop PAK346 – 26 nd November 2008, Hohenheim – Huigen PAK346 Framework workshop Marco Huigen (P8)

Framework workshop PAK346 – 26nd November 2008, Hohenheim – Huigen

PAK346 Framework workshop

Marco Huigen (P8)


Content

Introduction (20 min.)Expert N: needs and requirements

Putting this workshop in PAK-perspective (30 min.)P8: OverviewThe world according to me

Framework thinking: Living between worlds (30 min. - 1 hr)

DANUBIA theoretical (1 hr) DANUBIA practical (45 min.)


The PAK346 P8 project


Objectives and responsibilities of P8

1. Facilitate project communication on integrationIntegration-on-Demand approach

2. Facilitate technical implementation of the ‘overall land system model’

Danubia framework

3. Facilitate ‘Integrative simulation experiments’Experimental frame

4. Data Management Simulation input and output

Model coupling and data management - Project P8


2 Facilitate technical implementation P8 will support the model integration by using

the Danubia framework*„Framelets“ (knowledge manager, model manager,

experiment manager)

Every project group is responsible for programming its model, its model wrapper and its model interfacesP8 assists in finding and designing the required

components (in the IoD approach)


* Open source, clearance will be requested from developer group


3 Facilitate ‘Integrative simulation experiments’ P8 will set up and maintain the integration

environment for running simulations (without dynamic P1).Framework control (e.g. model-sequencer, time-

controller)Maintain calculation nodes hardwareData storage capacity



4 Data Management of simulation input and output P8 will set up and maintain a Data Management

System (DMS) for simulation input and outputAs much as possible and useful simulation data will be

stored in universal database format (approachable with SQL)

Allows for analyzing scenariosMaintain DMS software (MySQL) and hardware



Framework Thinking


DANUBIA framework

If your model-coupling project:Requires multiple model-coupling configurationsHas long time horizonRequires re-usabilityRequires (scientific and technical) adaptability

• Scientific functionality model could (easily) change• Technical functionality might change

…

Then: Use a computational architecture


DANUBIA Theoretical


DANUBIA: System Structure and Framework Technology

Research group “Computer Science” Ludwig-Maximilians-Universität München

Rolf Hennicker, Stephan JanischAndreas Kraus, Matthias Ludwig


The DANUBIA system is designed as an integrativeplatform for

• coupled simulations of various models of natural-science and socio-economic disciplines

• analysis of transdisciplinary effects of mutually dependent processes (including "acting" entities like households, farmers, ...)

• integrates 17 simulation models developed by the Danube groups

• runs on a computer cluster with more than 32 processors

• the simulation models run in parallel and exchange data at run-time


Common graphical modeling language (UML) used by all project groups for the documentation of interfaces, concepts and designs

Framework technology to facilitate the integration of simulation models

Object-oriented approach in all development phases (system analysis, design and programming)

Formal software engineering methods to verify critical parts of the integrated DANUBIA system

System Development Principles


DANUBIA – Architecture


The Framework Idea

Extract common properties and rules which hold for all DANUBIA models and implement them in a general abstract pattern (template).

The model developer must only implement the open pieces of the template by model specific realizations.


The DANUBIA Developer FrameworkCommon properties of all DANUBIA models

<<extends>> <<extends>> <<extends>> <<extends>>


Common Behaviour of all DANUBIA Models

run()getData()compute()commit()

getData

computecommit


DeepActor Models

Deep actor models integrate into their simulations decision-making entities, called actors (e.g. households, water suppliers, farmers).

Any actor has a repository of potential plans that the actor can execute (initial plans) .

In each computation step an actor decides which of the initial plans should actually be executed (active plans).

To support decisions each actor has sensors through which he can observe the "environment" and a history to remember previous decisions



options

rating

execute filter

querymodelGetData

modelPreCompute

modelPostCompute

run()getData()compute()commit()

getData()compute()commit()modelGetData()…

modelGetData()…


• A model is blocked (waitForGetData) if the coordination condition for getData is not satisfied (similarly, waitForCommit).

• The correctness of the coordination can be formally verified with model checking techniques.


Conclusion: DANUBIA framework

Common modelling language (UML) has been proved to provide a valuable tool for integrative system modelling.

Framework technology

supports model developers to integrate (conceptually and technically) their simulation models into the overall system structure of DANUBIA

provides general rules (templates) which support the trustability and quality of the system

Additional provisions for software quality:Formal verification of the correct coordination of concurrently running simulation models according to their local time scales.


Conclusion: DANUBIA framework Uses Object Oriented technology

Gives and keeps a very clear, transparent overview of complex models/components

Completely written in JAVAEach model/component is packed into a compiled jar-file

Component based Each component can be plugged on and off in a simulation

Uses JAVA R(emote)M(ethod)I(invocation) technology Individual model components may run on different computers

and platforms Models and Components (=client) log on to the framework server.

The server controls the data exchanges and does the time-management


Conclusion: DANUBIA framework Badly chosen name

Framework is generically applicable, not only for Danube area Framework is spatially explicit

By using configuration files users may set up any ‘simulation environment’ based upon ASCII grid-files.

The extent and resolution (= proxel size) are user-defined in the configuration. Framework allows for partial area testing.

Framework is temporally explicit In each model configuration file users indicate the model time step. The time scale is flexible (can be expressed in seconds, min, hours, days, months,

years). Framework does not support itself dynamic coupling within a time step, but due to

time scale flexibility this can be overcome. Framework allows for distributed and parallel processing (task based)

Uses JAVA RMI (OS independent)


Conclusion: DANUBIA framework Framework has a high learning curve

Due to its many functionalities and advanced level of programming, users will find it difficult to understand the ‘inner-working’.

Framework is transparent The architecture, although evolved drastically over the years, has a very logical

built-up Framework requires models to have ‘plug-points’

In a simulation run the framework controls all flows and needs therefore control over the components.

Framework supports model development and ‘testing’ very well. Has extensive logging capacities and j-unit support

Framework is open source Although open source, it is unlikely that our project will be able to structurally

adapt the code due to lack of required skills


DANUBIA Practical


Our DeepFarming model consists of the following components: 1. ACRE: An agricultural optimization model

Programmed in GAMS, Calculates on district level, Calculation annual interval, Calculation provides input for agents to create a new land use plan

2. ACRE wrapper: a java program controlling the GAMS model Responsible for file maintenance; zips, deletes and moves files, Responsible for data maintenance; reads and writes GAMS from/to MySQL.

3. DeepFarming Core model: a deepactors model implementation Responsible for data maintenance; reads and writes our input and output

from and to a MySQL database. Responsible for all data interactions with other models in the simulation

environment. 4. DeepFarming District models: a deepactors model implementation

Responsible for actual calculations of our agents. Every model represents a set of districts and has around 6000 farm actors. Because of this sub-model architecture, a higher level of distributed

calculation is achieved. 5. MySQL database: a Relational Data Management System 6. mydfm connector: a controller for unified access to the mysql RDMS 7. Several tools for data modification and analysis


Deep Farming

In the Danubia framework our DeepFarming model directly interacts with the models

- Biological (crop growth)- Soil (nutrient flows)- WaterSupply (animal drinking water, irrigation)- Meteorology (weather)

DeepFarming actors:- Total of more than 50.000 actors ‘in sim’- Each actor ‘lives’ on a proxel- 28 farm system types (e.g. CashSugMaize, MeatBreedSum)- Each actor has statistically based size of land (consisting of arable

and/or grassland)- Each farm system type has unique crop rotation system (17 land use

types are represented)- Each farm system type has unique combination of husbandry (9

animal types are represented)


Deep Farming

Per simulated day our actors may perform various actions. Whether these actions take place depend on:

1. Actors’ plan2. Environmental sensors (weather

conditions)3. Crop conditions

Yr2Yr1

Daily decisions:Daily decisions:• Planting (weather forecast, vegetation time demands)Planting (weather forecast, vegetation time demands)• Fertilizing (Crop stage, weather conditions)Fertilizing (Crop stage, weather conditions)• Harvesting (Crop stage, weather conditions)Harvesting (Crop stage, weather conditions)• Cutting (biomass increase, weather conditions)Cutting (biomass increase, weather conditions)• IrrigateIrrigate

•Run ACRERun ACRE•Make district plan per year (after crop silage is planted)Make district plan per year (after crop silage is planted)•Disaggregate plan of ACRE to AgentsDisaggregate plan of ACRE to Agents


Deep Farming

The farm actors derive their crop rotation plan based on their interpretation of the ACRE output


Deep Farming

Per crop advanced farm management decision rules have been programmed, taking into account weather conditions, environmental conditions, crop requirements and conditions.

e.g.

Planting depends on:• vegetation period in days, • vegetative rest, • temperature sum, • minimum germination temperature,• „Temperaturgrenzwert“ for germination

and• “Befahrbarkeit” of land

Do not plant today

Requirements:· the crop is in the crop rotation of the actor· the crop was not planted yet

Does the actor want to plant winter wheat?

No

Yes

Requirements:· no heavy rain predicted· land is accessible by machinery

Crop requirements:In the development stage BBCH 10-30 (Leaf development to stem elongation) the temperature should be below 5°C

Crop requirements:During the BBCH stage 39-49 (end of stem elongation until the beginning of heading) enough rainfall is required

The actual weather situation is acceptable?

Vernalization

Winter Wheat has to get into the stage of tillering before winter

dormant.

In the period of heading rainfall is enough?

Plant today

No

No

No

No

Yes

Yes

Yes

District: “climate history“

DANUBIA: Current state


Crop requirements:Planting date should be 0-30 days before the appearance of the first frost


Decision tree: Planting of Winter Wheat

Yes


-150

-100

-50

0

50

100

150

200

250

3001 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

1 2 3 4 5 6 7 8 9 10 11 12

-200

-100

0

100

200

300

4001 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

1 2 3 4 5 6 7 8 9 10 11 12

Weather forecast

Is it time to drill winter wheat?

Requirements of winter wheat:- Germination temperature- Vernalisation- Enough rainfall during heading

heading germination temperature

Vernalisation

Deep Farming

An actor dynamically adjusts its plan based on experiences regarding weather in combination with crop requirements.

Documents

Framework workshop PAK346 – 26 nd November 2008, Hohenheim – Huigen PAK346 Framework workshop Marco Huigen (P8)