Asynchronous Functions In C++

ACCU 2005© Schalk W. Cronjé

Asynchronous Functions in C++

The Generic Approach

Schalk W. Cronjé

20 April 2005

ACCU 2005© Schalk W. Cronjé

Why asynchronous processing?

● All “real-time” systems require some form of asynchronous architecture.

● Anything that needs to do background processing whilst attending to other tasks or user input require an asynchronous architecture of some sort

ACCU 2005© Schalk W. Cronjé

Cost of Switching Architecture

In any standard implementation there is usually a significant cost of effort and time in switching between different asynchronous mediums.

ACCU 2005© Schalk W. Cronjé

McCall's Quality Factors

● Correctness● Reliability● Usability● Maintainability● Portability● Efficiency

● Testability● Flexibility● Integrity● Reusability● Interoperability

ACCU 2005© Schalk W. Cronjé

Not just Threads!

Asynchronous execution extends far beyond just threads.

Threads are but a part of a much bigger picture.

Since threads are a known quantity they remain a good metaphor to explain the concepts surrounding asynchronous C++ functions

ACCU 2005© Schalk W. Cronjé

Prior Art

● Kevlin HenneyACCU 2003 / 2004http://www.two-sdg.demon.co.uk/curbralan/papers/accu/MoreC++Threading.pdf

● Drazen DotlicC++ Users Journal, Nov 2004

● Boost Threads

● Boost Signals & Slots

ACCU 2005© Schalk W. Cronjé

Design Goals

● The ability to execute a free function, member function or functor asynchronously without regard to the asynchronous medium

● The ability to change asynchronous mediums with as little code change as possible

● Keep it portable

ACCU 2005© Schalk W. Cronjé

The 5 Essentials

● Launching a function asynchronously

● Knowing whether the function has completed

● Waiting for a function to complete

● Collecting the result

● Asynchronous notification

ACCU 2005© Schalk W. Cronjé

Asynchronous Primitives

● Threads (+mutexes etc.)● Async IO● C Signals

Of these three only threads can be ported easily and can play nicely with C++. Therefore threads can be used to build asynchronous mediums without being used as one itself.

ACCU 2005© Schalk W. Cronjé

Asynchronous Mediums

● Threads ● Thread Pools● Task Queues● Spawned Processes● XML-RPC & SOAP● Interprocess Message Queues

ACCU 2005© Schalk W. Cronjé

To detach or not

● A detached task is one on which synchronous waits cannot be performed

● Win32 & Pthreads distinguish between detached and non-detached threads.

● Non-detached threads require a cleanup to be performed after thread has terminated

● boost::threads uses d-tor to detach a thread● It is debatable whether all tasks should

automatically be created in a detached state

ACCU 2005© Schalk W. Cronjé

Building a Generic Interface

ACCU 2005© Schalk W. Cronjé

Creating a Task

async_id create_task( Medium, Functor );async_id create_task( Medium, Functor );

template <typename Medium>create_task(

Medium const& async_medium_,

);

typename async_traits<Medium>::id_type

typename async_traits<Medium>::functor_type f_

ACCU 2005© Schalk W. Cronjé

Asynchronous Traits #1

template <typename Medium>struct async_traits {

};

template <typename Medium>struct async_traits {

};

typedef implementation-defined id_type;typedef implementation-defined functor_type;static id_type create(Medium,functor_type);

id_type must be non-native, but lightweight copyable

ACCU 2005© Schalk W. Cronjé

// Simple Medium exampleclass thread_id;class SimpleThreader{

public:

// Creates a thread, runs the functionthread_id create( boost::function< int() > ) const;

};

ACCU 2005© Schalk W. Cronjé

Asynchronous Traits #1

template <>struct async_traits<SimpleThreader> {

};

template <>struct async_traits<SimpleThreader> {

};

typedef SimpleThreader::thread_id id_type;typedef boost::function< int()> functor_type;

ACCU 2005© Schalk W. Cronjé

boost::function

The Boost.Function library contains a family of class templates that are function object wrappers. The notion is similar to a generalized callback.

http://www.boost.org/doc/html/function.html

ACCU 2005© Schalk W. Cronjé

// boost::function makes easy work of wrapping// pointers to functionsint my_func( int,int );boost::function< int(int,int) > f1 = &my_func;std::cout << f1( 3, 4 );// and even member functionsusing std::string;boost::function< string::size_type(string const*) > f2= &string::size;string s2(“Hello, World”);std::cout << f2(&s2);

ACCU 2005© Schalk W. Cronjé

Asynchronous Traits #1

template <>struct async_traits<SimpleThreader> {

};

template <>struct async_traits<SimpleThreader> {

};

typedef SimpleThreader::thread_id id_type;typedef boost::function< int()> functor_type;

static id_type create( SimpleThreader const& medium_, functor_type f_ ){return medium_.create(f_);}

constness is not a requirement

ACCU 2005© Schalk W. Cronjé

// Calculate information flow of all source files// in a directoryint calc_IF4( const char* directory_ );

SimpleThreader threader;

thread_id id= create_task( threader, boost::bind( &calc_IF4,”/myproj” )

);

ACCU 2005© Schalk W. Cronjé

Completing create_tasktemplate <typename Medium>typename async_traits<Medium>::id_typecreate_task(

Medium const& async_medium_,typename async_traits<Medium>::functor_type f_

){

}

return async_traits<Medium>::create(async_medium_,f_

);create_task can becomplemented by a version taking a mutable reference to the asynchronous medium

ACCU 2005© Schalk W. Cronjé

Restricting the return typetemplate <typename Medium>typename async_traits<Medium>::id_typecreate_task(

Medium const& async_medium_,typename async_traits<Medium>::functor_type f_

){

}

return async_traits<Medium>::create(async_medium_,f_

);

BOOST_STATIC_ASSERT(( boost::is_object<typenameasync_traits<Medium>::id_type>::value ));

ACCU 2005© Schalk W. Cronjé

// Example thread_idclass thread_id{

friend class SimpleThreader;public:

typedef SimpleThreader medium_type;thread_id();thread_id(thread_id const&);bool done() const;void wait() const;int const* data() const;

private:class low_level_impl;boost:shared_ptr<low_level_impl> m_pImpl;thread_id(low_level_impl*);

};

ACCU 2005© Schalk W. Cronjé

Waiting for a Task

void wait_task( task_id );bool task_completed( task_id );

template <typename TaskID>voidwait_task( TaskID const& );template <typename TaskID>booltask_completed( TaskID const& );

ACCU 2005© Schalk W. Cronjé

Asynchronous Traits #2

template <typename Medium>struct async_traits {

};

template <typename Medium>struct async_traits {

};

static void wait(id_type);static bool completed(id_type);static bool detached(id_type);

typedef implementation-defined id_type;typedef implementation-defined functor_type;static id_type create(Medium,functor_type);

ACCU 2005© Schalk W. Cronjé

Asynchronous Traits #2

template <>struct async_traits<SimpleThreader> { typedef SimpleThreader::thread_id id_type;

};

template <>struct async_traits<SimpleThreader> { typedef SimpleThreader::thread_id id_type;

};

static bool completed( id_type const& id_ ) { return id_.done();} static void wait( id_type const& id_ ) { id_.wait(); } static bool detached( id_type const& id_ );

ACCU 2005© Schalk W. Cronjé

// Having started a task to calculate IF4// we can check whether a task has completed

if( !task_completed( id ) ) {

// do something else first}// or just simply wait for task to completewait_task( id );

ACCU 2005© Schalk W. Cronjé

Completing wait_tasktemplate <typename TaskID>voidwait_task( TaskID const& task_id_ ){

}

typedef typename async_traits_of<TaskID>::type traits;

if( !task_completed(task_id_) ){

if( traits::detached(task_id_) )throw invalid_operation;

elsetraits::wait(task_id_);

}

ACCU 2005© Schalk W. Cronjé

async_traits_oftemplate <typename TaskID>struct async_traits_of{

typedef typename async_traits<typename TaskID::medium_type

> type;};

Easy to specialise if necessary

ACCU 2005© Schalk W. Cronjé

Collecting the Result

result_pointer task_result( task_id );

template <typename TaskID>task_result( TaskID const& task_id_ );typename async_pointer_of<TaskID>::type

If task has not completedresult is null

ACCU 2005© Schalk W. Cronjé

Asynchronous Traits #3

template <typename Medium>struct async_traits {

};

template <typename Medium>struct async_traits {

};

typedef implementation-defined result_type;typedef implementation-defined result_pointer;static result_pointer get_result(id_type);

typedef implementation-defined id_type;typedef implementation-defined functor_type;static id_type create(Medium,functor_type);static void wait(id_type);static bool completed(id_type);static bool detached(id_type);

ACCU 2005© Schalk W. Cronjé

Asynchronous Traits #3

template <>struct async_traits<SimpleThreader> { typedef SimpleThreader::thread_id id_type;

};

template <>struct async_traits<SimpleThreader> { typedef SimpleThreader::thread_id id_type;

};

typedef int result_type; typedef int const* result_ptr; static result_ptr get_result( id_type const& id_ ) {return id_.data();}

ACCU 2005© Schalk W. Cronjé

Completing task_resulttemplate <typename TaskID>typename async_pointer_of<TaskID>::typetask_result( TaskID const& task_id_ ){

}

typedef typename async_traits_of<TaskID>::type traits;

if( !task_completed(task_id_) )return async_pointer_of<TaskID>::type();

elsereturn traits::get_result(task_id_);

BOOST_STATIC_ASSERT(( !boost::is_void<typename traits::result_type>::value ));

ACCU 2005© Schalk W. Cronjé

Detaching & Notification

● In order to achieve true asynchronous functions notifications must be implemented

● If a function is launched in a detached mode the only way to know when it has finished is via a callback or alternative notification

● It is important that notifications are asynchronous safe – at least for the medium on which they are applied

ACCU 2005© Schalk W. Cronjé

Creating a Detached Task

async_id create_task( Medium, Functor, Notifier );async_id create_task( Medium, Functor, Notifier );

template <typename Medium>create_task(

Medium const& async_medium_,

);

typename async_traits<Medium>::id_type

typename async_traits<Medium>::functor_type f_, boost::function<void( typename async_traits<Medium>::id_type )> notify_

ACCU 2005© Schalk W. Cronjé

Asynchronous Traits #4template <typename Medium>struct async_traits {

};

template <typename Medium>struct async_traits {

};

typedef boost::function<void(id_type)> notification_type;static id_type create_detached (Medium,functor_type,notification_type);

typedef implementation-defined id_type;typedef implementation-defined functor_type;typedef implementation-defined result_type;typedef implementation-defined result_pointer;static id_type create(Medium,functor_type);static void wait(id_type);static bool completed(id_type);static bool detached(id_type);static result_pointer get_result(id_type);

ACCU 2005© Schalk W. Cronjé

Asynchronous Traits #3

template <>struct async_traits<SimpleThreader> { typedef SimpleThreader::thread_id id_type; typedef boost::function< int()> functor_type;

};

template <>struct async_traits<SimpleThreader> { typedef SimpleThreader::thread_id id_type; typedef boost::function< int()> functor_type;

};

typedef boost::function< void(id_type) > notification_type; static id_type create_detached( SimpleThreader const&, function_type, notification_type );

ACCU 2005© Schalk W. Cronjé

Asynchronous Traits Summarytemplate <typename Medium>struct async_traits {

};

template <typename Medium>struct async_traits {

};

typedef implementation-defined id_type;typedef implementation-defined functor_type;typedef implementation-defined result_type;typedef implementation-defined result_pointer;typedef boost::function<void(id_type) notification_type;static id_type create(Medium,functor_type);static id_type create_detached (Medium,functor_type,notification_type);static void wait(id_type);static bool completed(id_type);static bool detached(id_type);static result_pointer get_result(id_type);

ACCU 2005© Schalk W. Cronjé

Asynchronous Function Summary

id_type create_task( Medium, Functor );id_type create_task( Medium, Functor, Notifier );void wait_task( id_type );bool task_completed( id_type );result_pointer task_result( id_type );

ACCU 2005© Schalk W. Cronjé

Additional Considerations

● Task identifiers must be lightweight copyable

● A completed task's result must be available until the last task identifier for that task has been removed.

● boost::shared_ptr generally the easiest way to accomplish both the above

ACCU 2005© Schalk W. Cronjé

Extending the Traits

● Some mediums might not be detachable. ● This can be handled by adding an additional

is_detachable constant.● create_task(m,f,n) can assert on this during

compile time.

ACCU 2005© Schalk W. Cronjé

Extending Traitstemplate <typename Medium>struct async_traits {

BOOST_STATIC_CONSTANT(bool,is_detachable=true);// ...

};template <typename Medium>typename async_traits<Medium>::task_idcreate_task( /* parms omitted for brevity */ ){

BOOST_STATIC_ASSERT(async_traits<Medium>::is_detachable);}

template <typename Medium>struct async_traits {

BOOST_STATIC_CONSTANT(bool,is_detachable=true);// ...

};template <typename Medium>typename async_traits<Medium>::task_idcreate_task( /* parms omitted for brevity */ ){

BOOST_STATIC_ASSERT(async_traits<Medium>::is_detachable);}

ACCU 2005© Schalk W. Cronjé

Start with simplethreaded app

Improve performanceby using thread poolsor task queues

Use distributed computing by goingout of process orout of host

ACCU 2005© Schalk W. Cronjé

Concluding

ACCU 2005© Schalk W. Cronjé

A generic approach to asynchronous execution is not a golden solution, but it goes a long way to decoupling the asynchronous architecture from

the business logic.

It allows for selection of an architecture based upon underlying platform without having to modify the overlaying business application