MODPI: A parallel MOdel Data Passing Interface for integrating legacy environmental system modelsA. Dozier, O. David, Y. Zhang, and M. Arabi

Motivations

Different languages, frameworks, operating systems

Two-way feedbacks between legacy models

Refactoring legacy code is difficult

To program

Many lines of code

Reproducing model outputs correctly

To maintain

Updating model versions

To adopt

Little to no community adoption of code

When refactored by third party

When significantly changed

Goals & Objectives

Goal

“to facilitate and abstract the legacy model integration process to include complex, multi-directional interactions between models… of different architectures and maintain model individuality”

Objectives

Design abstract interface for simplicity

Apply the interface

Comparison between two methods of integration

Calling one as subroutine Using MODPI

Model 2

Read inputs

Execute

Write outputs

Model 1

Read inputs

Execute

Write outputs

Execute

Write outputs

Model 1

Read inputs

Execute

Write outputs

Model 2

Read inputs

Execute

Write outputs

MPI networkcommunication

Model 2 Wrapper

MODPI Implementation

Model 1 Wrapper (User-specified)

MODPI Implementation

modpi_init()- Subscribes to

events by name- Initializes MPI

modpi_finalize()

MOdel Data Passing Interface (MODPI)

Model 1

Start

Daily Loop

…Perform work…

End Daily Loop

End

On Start

On Top Loop

On Bottom Loop

On End

Model 2

Start

Daily Loop

…Perform work…

End Daily Loop

End

On Top Loop

Senders

Rece

ivers

Even

ts

mpirun or mpiexec

Sample implementation

DayCent-HYDRUS

DayCent estimates biogeochemical fluxes

HYDRUS provides physically-based representation of soil water content

Two different implementations

“SUB”: call HYDRUS as a subroutine within DayCent

“MODPI”: DayCent-HYDRUS linked using MODPI

Sample implementation

Code changes

Sample implementation

Runtime and overhead

Discussion

Advantages

Framework and language independent

Multi-lingual and (hopefully) multi-platform

Non-intrusive event-based system

Built-in parallelization and communication via MPI

Limitations

May produce overhead in virtualized environments

Requires MPI

Framework must be compatible with MPI

May require some knowledge of MPI programming

Future Work

Use “reflection” to lookup variable by string

Use a range of processes to broadcast values

Automate data transformations

Run tests across Windows and Linux

Generate Fortan or C code for initial model integration wrapper

Questions?

References

Images

Language

Stop and listen

Advantages

Simple concept

Simple programming

Disadvantages

Computationally inefficient

Convergence of iterations?

Option 1 – Iterative

Model 1

Model 2

Write new inputs

Converged?

Read outputs

Exit

No?

Yes!

Model 2

Read inputs

Execute

Write outputs

Advantages

Computationally efficient

Disadvantages

Medium to hard concept

Medium to hard programming

Maintain separate code base

Option 2 – Call model as sub-model

Model 1

Read inputs

Execute

Write outputs

Execute

Write outputs

Implementing MODPI

Add events

Minimal model refactoring

Build sender and receiver subroutines

Build subroutine to point to events within the model

Build a wrapper program

Call modpi_init()

Run model

Call modpi_finalize()

Define input text files

Run mpirun or mpiexec