Embed Size (px)
Citation preview
CAPTURING CHANGES IN
ORGANIZATIONAL SOCIAL CAPITAL
AN IMPACT EVALUATION OF A SMALL-GRANT INNOVATION FUND
IDS EVENT “Impact, Learning and Innovation: Towards a Research and
Practice Agenda for the Future” – 26-27 March 2013 – IDS Sussex
Giel Ton
WHAT DO WE WANT TO KNOW
Intervention logic of innovation grant
APPROVED BUSINESS PLANS
(PROCESSING ACTITIVIES)
INVESTMENTS IN BUSINESS
OPPORTUNITY
ORGANISATIONAL MANAGEMENT OF
BUSINESS OPPORTUNITY
BUSINESS PLAN IN OPERATION
FINANCIAL SUSTAINABILITY OF THE ORGANISATION
IMPROVEMENT IN ORGANISATIONAL
CAPACITIES
IMPROVED ACCESS TO FINANCIAL
SERVICES
SELFMANAGEMENT OF THE
ORGANISATION
MOREINCOME TO MEMBERS
BETTER NUTRITIONAL
STATUS
BETTER SERVICES TO MEMBERS
IMPROVED ACCESS TO NATIONAL AND
REGIONAL MARKETS
TRAINING ON BUSINESS
OPPORTUNITY
MORE SENSE OF BELONGINGNESS
WITH THE ORGANISATION
INITIAL ORGANIZATIONAL
CAPACITY
GOOD QUALITY BUSINESS PLANS
EFFICIENT QUALITY
CONTROL AND GRANT
ALLOCATION SYSTEM
APPROVED BUSINESS PLANS
(PROCESSING ACTITIVIES)
INVESTMENTS IN BUSINESS
OPPORTUNITY
ORGANISATIONAL MANAGEMENT OF
BUSINESS OPPORTUNITY
BUSINESS PLAN IN OPERATION
FINANCIAL SUSTAINABILITY OF THE ORGANISATION
IMPROVEMENT IN ORGANISATIONAL
CAPACITIES
IMPROVED ACCESS TO FINANCIAL
SERVICES
SELFMANAGEMENT OF THE
ORGANISATION
MOREINCOME TO MEMBERS
BETTER NUTRITIONAL
STATUS
BETTER SERVICES TO MEMBERS
IMPROVED ACCESS TO NATIONAL AND
REGIONAL MARKETS
TRAINING ON BUSINESS
OPPORTUNITY
MORE SENSE OF BELONGINGNESS
WITH THE ORGANISATION
INITIAL ORGANIZATIONAL
CAPACITY
GOOD QUALITY BUSINESS PLANS
EFFICIENT QUALITY
CONTROL AND GRANT
ALLOCATION SYSTEM
GENERAL EVALUATION QUESTIONS
EXTERNAL EVALUATION OF BUSINESS PLAN IMPACTS
CHECK KEY ASSUMPTION
CHECK IMPACT
CHECK IMPACT?
TYPOLOGIES
INTERNAL MONITORING OF GRANT IMPLEMENTATION
MIXED-METHODS RESEARCH DESIGN
Key impact pathway B: Business plans improve organisational resilience (org. social capital)
Key impact pathway D: Grant investments release constraints in access to financial services
Key impact pathway C: Business plans lead to increased economic performance IN
TER
VEN
TIO
N
Key impact pathway A: Business plans improve capacity to pay organisational costs
INTE
RV
ENTI
ON
Key impact pathway B: Business plans improve organisational resilience (org. social capital)
Key impact pathway D: Grant investments release constraints in access to financial services
Key impact pathway C: Business plans lead to increased economic performance IN
TER
VEN
TIO
N
Key impact pathway A: Business plans improve capacity to pay for collective action
ATTRIBUTION PLAUSIBLE = NET-EFFECT MEASUREMENT FEASIBLE =
QUASI-EXPERIMENTAL DESIGN APPROPIATE
CONTRIBUTION STORY = NEED FOR MONITOIRNG INFORMATION
Data collection for building the impact
contribution story
Intervention
Use operational data for
immediate outcomes
Ultimate outcomes
Reflect on (existing)
secondary data on poverty
Intermediate outcomes
Appropriate proxies for
key outcome areas
Configurations
Process tracing
and monitoring of
interdependencies
Impacts of innovation grants on collective marketing groups
1. Core design elements
Time-series on key indicators:
● Sales: total sales & sales of innovative product
● Patrimony
● Self-funding of recurrent cost
● Access to funds/credit
Treatment and comparison group
● 30 farmer organizations that are funded
● 20 that are not-funded (though eligible)
● Stratified random selection
Statistical tests and econometrics outcome variables:
● Growth rates
● Regression with year after implementation as explanatory variable
2. Contribution story and threats to validity
The grant is assumed to be a vital parts of a ‘package’ of causal factors that together is sufficient to produce the ultimate outcomes.
• E.g. banks show increasing willingness to negotiate with producer organisations that have some patrimony
• E.g. through the investment the access to government procurement (nutritional programmes) is improved but still depend on parti-political dynamics
However, it is possible that the grants are not necessary: do you still see traces of processes (partly) triggered by the intervention
• Most likely, the grant is necessary in some configurations with some type of organizations/markets, but not in all (‘what works for whom under what conditions?’)
• Tackling the threats to validity to the claim of contribution: counterfactual thinking
• Can we explain the realization of the outcomes without making reference to the grant (‘alternative explanations’)?
TOOL FOR MEASURING SOCIAL CAPTAL
Qualitative tool: 38 reports with qualitative
descriptions (approx 15 pp/each)
Feeding learning processes
Escuela de líderes – Leadership course
Question 1 Question 2
INHERENT TENSIONS IN COLLECTIVE MARKETING THAT NEED ORGANISATIONAL MECHANISMS TO RESOLVE THEM
This tension is present in the activities we realize
Hardly present
Never present
We have managed to resolve it with organisational agreements / internal regulations
We are looking for ways to resolve it
We don’t need to resolve it
1- “Regulating member supply” Members sometimes protest that the organisation does not buy all their produce?
2- “Quality assurance systems” Are there members that try to deliver lower quality products than is required?
3- “Reduce the need for working capital”
Members demand cash payment instead of waiting until the organisations has sold the product?
4- “Prevention of disloyal behaviour” Are there members that sell part of their produce to other buyers though they promised to sell to the organisation?
5- “Ways to distribute profits” Do members accept that the organisations does not distribute all its profits?
6- “Differ benefits and services to members and non-members”
Is there preferential treatment (e.g. price) when buying from members compared with non-members?
7- “Decide on investments and activities that do not benefit all”
Did the organisation projects or investments that are only to the benefit of a sub-group of members of the group?
8- “Delegating and supervising marketing tasks”
Do members accept that others in the organisation take decisions on prices of products sold without prior consult to the assembly?
9- “Legal responsibility in contracts and loans”
Do members take responsibility for eventual fines and sanctions related with sale contracts or loans that the board negotiates?
10- “Manage political aspirations of board and staff”
Do members accept that board members or team staff take party political responsibilities?
Quantitative comparative tool
Pregunta 1 Pregunta 2
TENSIONES INHERENTES A LA COMERCIALIZACIÓN COLECTIVA
Esta tensión se presenta en las actividades que realizamos como organización
Se presenta muy poco
no se presenta nunca
hemos logrado resolverlo con acuerdos/arreglos organizativos
estamos buscando la forma de resolverlo
no necesitamos resolverlo
“Regular la cantidad a acopiar de los miembros”
“Sistemas de garantía de calidad”
“Reducir la necesidad de capital de trabajo”
“Prevenir deslealtad en las ventas”
“Maneras de distribuir los excedentes”
“Diferenciar los beneficios y servicios a miembros y no miembros”
“Decidir sobre inversiones y actividades que no beneficien a todos”
“Delegación y supervisión de tareas de comercialización “
“Responsabilidad jurídica en contratos y préstamos”
“Manejo de aspiraciones políticas”
Table 3. Weighing factors used for calculating each tension-containment variable (Tx)
ScoreQ1x
This tension is present
in their activities
Hardly present Never
present
They have managed to resolve 9 6 3
ScoreQ2x
They are looking for ways to
resolve it
6 4 2
They don’t need to resolve it 3 2 0
Tx=ScoreQ1 x *ScoreQ2x
CASE-BASED COMPARATIVE ANALYSIS
Conventional statistical tricks
Table.. Rotated component matrix on organizational performance indicators
Factor 1:
Economic performance
Factor 2:
Organizational size
Factor 3:
Capital intensity
Patrimony per member (Ln) -.025 -.006 .979
Active members (Ln) .189 .874 -.294
Organizational Age (Ln) .059 .897 .224
Sales (Ln) .886 .438 -.126
Sales per member (Ln) .991 -.043 .041
Extraction Method: Principal Component Analysis.
Rotation Method: Varimax with Kaiser Normalization.
Set-theoretic tricks: fQCA
The visual inspection of scatterplots helped to detect and
describe the diversity in the sector of economic farmer
organizations.
To further deepen the analysis, other case-based comparative
techniques are needed to find groupings of organizations with
distinct scores on each of the tensions that compose our
construct, as certain organizations may perform functions
where not all tensions are relevant and for which the absence
is not indicative of weaker capabilities.
We expect to explain success with a configurational analysis:
like Qualitative Comparative Analysis (Rihoux and Ragin,
2008; Schneider and Wagemann, 2012), when we have the
‘success-variable’ late 2013.
CHALLENGES AHEAD
Practical issues:
● Time-series: confidentiality and ‘contentious property’
● Quasi-experimental design: attrition of the comparison group
Fundamental problems
● ‘Fishing’ versus ‘Iterative process to have a closer look at cases’
● ‘Success’ is multifacetal: what is the dependant variable in fQCA?
Thanks!
Sietze Vellema
Giel Ton
