EVALUATION AS A MEANS OF ENHANCING ACCOUNTABILITY … · evaluation as a means of enhancing accountability and beneficiary participation within nonprofit organisations iii The different

EVALUATION AS A MEANS OF ENHANCING ACCOUNTABILITY AND BENEFICIARY PARTICIPATION WITHIN NONPROFIT

ORGANISATIONS

Kylie Louise Kingston

B.Teach., B.Ed.St., Grad.Dip.Ed (ece), M.Bus. (Philanthropy and Nonprofit Studies)

Dr Craig Furneaux (Principal) Dr Laura de Zwaan (Associate)

Dr Lyn Alderman (External)

Submitted in fulfilment of the requirements for the degree of

Master of Philosophy (Accountancy)

School of Accountancy

QUT Business School

Queensland University of Technology

2018

evaluation as a means of enhancing accountability and beneficiary participation within nonprofit organisations i

Keywords

accountability, agonistic democracy, beneficiary, critical accounting, dialogic

accounting, evaluation, not-for-profit organisations, participation, stakeholders,

transformative participatory evaluation

ii evaluation as a means of enhancing accountability and beneficiary participation within nonprofit organisations

Abstract

Traditional constructions of accounting and accountability are typically

embedded in neoliberal logics, which privilege the needs of certain stakeholder groups

ahead of others and therefore are contributory factors for inequality. From within this

critical epistemological position, this research critiques the current privileging

hegemony of accounting primarily to the powerful, with an aim of creating a more

equitable accountability framework. Specifically, this thesis seeks to examine for what

are not-for-profit organisations accountable to their beneficiaries, how, and on whose

terms?

This thesis is informed by theories of agonistic democracy, dialogic accounting,

and transformative participatory evaluation. The motivation of this research is to

support the empowerment of marginalised groups by developing participatory

evaluation frameworks for use within not-for-profit organisations.

Qualitative case-studies of two not-for-profit organisations provide the context

to explore current practices of beneficiary involvement within the accountability and

evaluation processes of these organisations. Data sources include semi-structured

interviews with three stakeholder groups (board, staff, beneficiaries), and secondary

data from publicly available organisational documents, and publically available data

concerning the legislative environment of each case.

The findings of the research show that accountability to beneficiaries within not-

for-profit organisations is complex. The accountability needs of beneficiaries within

this research differ across their organisations, depending upon their timeframe of

engagement (whether ongoing or episodic). These differences call for a consideration

of when accountability should be discharged and when evaluation should occur.

evaluation as a means of enhancing accountability and beneficiary participation within nonprofit organisations iii

The different accountability needs of the beneficiaries within the two cases have

enabled the identification of two modes of downwards accountability, and can be

considered multi-modal. The two accountability modes revealed through the findings

are: 1) an ongoing, relational mode of accountability to beneficiaries, and 2) an

episodic, transactional mode of accountability to beneficiaries.

Based upon these two modes of accountability, two evaluation instruments have

been drafted: an evaluation instrument for a relational mode of accountability, and an

evaluation instrument for a transactional mode of accountability. The evaluation

instruments seek to give voice to beneficiaries and enhance the organisations’

accountability process.

This research has social and practical impacts through the formation of the

instrument, the potential improvement of organisational performance, and the

empowerment of beneficiaries through participation within decision-making

processes. The research contributes to theory development through extending literature

on dialogic accounting theory, not-for-profit accountability, and participatory

evaluation, by introducing a temporal dimension to the discussion. Finally, the

research has potential implications for policy. Instead of a monologic, principal/agent

approach to accountability to powerful stakeholders, a symbiotic, mutually beneficial

relationship between the organisation and the giving of voice by the organisation to

the beneficiaries through accountability structures, is articulated.

iv evaluation as a means of enhancing accountability and beneficiary participation within nonprofit organisations

Table of Contents

Keywords ................................................................................................................................... i

Abstract ..................................................................................................................................... ii

Table of Contents ..................................................................................................................... iv

List of Figures ......................................................................................................................... vii

List of Tables ......................................................................................................................... viii

List of Abbreviations ................................................................................................................ x

Statement of Original Authorship ............................................................................................ xi

Acknowledgements ................................................................................................................. xii

Chapter 1: Introduction ........................................................................................... 13 1.1 Background ....................................................................................................................... 13

1.2 Context .............................................................................................................................. 15

1.3 Purpose .............................................................................................................................. 15

1.4 Significance ....................................................................................................................... 17

1.5 The Research Question ..................................................................................................... 18

1.6 Gaps .................................................................................................................................. 20

1.7 Definitions ......................................................................................................................... 22

Chapter 2: Philosophical Underpinning of the Research ..................................... 25 2.1 Dialogic Accounting Theory ............................................................................................. 26

2.2 Agonistic Political Theory and Agonistic Pluralism ......................................................... 30

2.3 Summary and Implications ............................................................................................... 33

Chapter 3: Literature Review ................................................................................. 35 3.1 Establishing the context .................................................................................................... 36

3.2 An Accountability Framework .......................................................................................... 39

3.3 Accountability to Whom? ................................................................................................. 40

3.4 Accountability for What .................................................................................................... 45

3.5 Accountability on Whose Terms? ..................................................................................... 46

3.6 Accountability How? ........................................................................................................ 47

3.7 Accountability How: Through Evaluation ........................................................................ 53

3.8 Summary and Implications ............................................................................................... 63

Chapter 4: Research Design .................................................................................... 65 4.1 Ontology ........................................................................................................................... 66

4.2 Epistemology .................................................................................................................... 67

4.3 Methodology and Research Design .................................................................................. 68

4.4 Analysis ............................................................................................................................. 79

evaluation as a means of enhancing accountability and beneficiary participation within nonprofit organisations v

4.5 Establishing Trustworthiness and Reliability ................................................................... 84

4.6 Ethical Considerations ...................................................................................................... 87

4.7 Summary ........................................................................................................................... 87

Chapter 5: Case Narratives - Case A ..................................................................... 88 5.1 Findings - Organisational Structure .................................................................................. 88

5.2 Findings - RQ 1.1: For What is a NPO Accountable to its Beneficiaries? ....................... 92

5.3 Findings - RQ 1.2: How is Accountability Currently Discharged to Beneficiaries? ........ 98

5.4 Findings - RQ 1.3: How is Accountability on the Beneficiaries’ Terms Practised? ...... 107

5.5 Summary of Findings Across the Primary and Secondary Data .................................... 114

5.6 Evaluation Summary ...................................................................................................... 116

5.7 Summary – Case A Findings .......................................................................................... 116

Chapter 6: Case Narratives - Case B .................................................................... 118 6.1 Findings - Organisational Structure ................................................................................ 118

6.2 Findings - RQ 1.1: For What is a NPO Accountable to its Beneficiaries? ..................... 120

6.3 Findings - RQ 1.2: How is Accountability Currently Discharged to Beneficiaries? ...... 125

6.4 Findings - RQ 1.3: How is Accountability on the Beneficiaries’ Terms Practised? ...... 134

6.5 Summary of Findings Across the Primary and Secondary Data .................................... 141

6.6 Evaluation Summary ...................................................................................................... 142

6.7 Summary – Case B Findings .......................................................................................... 143

Chapter 7: Cross Case Comparison and Discussion ........................................... 145 7.1 Cross Case Comparison .................................................................................................. 146

7.2 Discussion - RQ 1.1: For What is a NPO Accountable to its Beneficiaries? ................. 155

7.3 Discussion - RQ 1.2: How is Accountability Currently Discharged to Beneficiaries?...161

7.4 Discussion - RQ 1.3: How is Accountability on the Beneficiaries’ Terms Practised? ...167

7.5 Summary of a Relational or Transactional Approach to Accountability ....................... 180

7.6 Evaluation Framework ................................................................................................... 182

7.7 Summary ......................................................................................................................... 187

Chapter 8: Conclusions ......................................................................................... 189 8.1 Overview of the Research ............................................................................................... 189

8.2 Key Findings .................................................................................................................. 192

8.3 Limitations of the Research ............................................................................................ 198

8.4 Future Research .............................................................................................................. 200

8.5 Summary ......................................................................................................................... 201

References ............................................................................................................... 204

Appendices .............................................................................................................. 216 Appendix A – Approved Ethics Research Documents ......................................................... 216

vi evaluation as a means of enhancing accountability and beneficiary participation within nonprofit organisations

Appendix B – Interview Protocol ......................................................................................... 224

Appendix C - Coding Definitions ......................................................................................... 230

Appendix D - Evaluation Instruments .................................................................................. 231

evaluation as a means of enhancing accountability and beneficiary participation within nonprofit organisations vii

List of Figures

Figure 1. The directions of accountability (figure developed by the researcher) ................. 42

Figure 2. Conceptual model of the research ......................................................................... 64

Figure 3. The ontological, epistemological and methodological assumptions of the research .......................................................................................................... 84

Figure 4. The relationship between beneficiary voice and empowerment within Case A. ......................................................................................................... 108

Figure 5. The potential impact of accountability when practised on the terms of the beneficiaries ................................................................................................. 110

Figure 6. Multidirectional impact within Case A ............................................................... 115

Figure 7. Transforming downwards accountability into relational accountability ............. 115

Figure 8. The potential impact of beneficiary evaluation upon social justice .................... 135

Figure 9. The association between beneficiary disempowerment and the need for a relational form of accountability .................................................................. 136

Figure 10. The transaction of accountability ...................................................................... 142

Figure 11. Income source comparison Case A and Case B (Case A, 2017, ACNC, AIS; Case B, 2017, ACNC, AIS) ................................................................. 149

Figure 12. The accountability needs of the beneficiaries interviewed across Relational and Transactional Accountability ............................................... 154

Figure 13. Impact within the Relational Accountability model of Case A ........................ 157

Figure 14. Impact within the Transactional Accountability model of Case B ................... 158

Figure 15. A Relational Accountability approach to participative evaluation ................... 164

Figure 16. A Transactional Accountability approach to participative evaluation .............. 165

Figure 17. The relationship between dialogic accounting, accountability when, and relational and transactional accountability ................................................... 172

Figure 18. Upwards and downwards accountability processes can be mutually supportive ..................................................................................................... 175

Figure 19. The accountability directions with the CEO as central ..................................... 176

Figure 20. The symbiotic relationship between upwards and downwards accountability when viewed on the beneficiaries' terms. ............................. 177

Figure 21. Typology of beneficiary evaluation processes within an NPO’s evaluative environment ................................................................................ 184

Figure 22. An Evaluation Framework within Relational Accountability ........................... 185

Figure 23. An Evaluation Framework within Transactional Accountability ..................... 186

Figure 24. An accountability framework on the beneficiaries' terms ................................. 195

viii evaluation as a means of enhancing accountability and beneficiary participation within nonprofit organisations

List of Tables

Table 1. Variance between principles of a monologic approach to accounting and a dialogic approach to accounting (table adapted from Blackburn et al., 2014, p. 91-92) .............................................................................................. 30

Table 2. Summary of key accountability descriptions and considerations from the scholarly literature ......................................................................................... 38

Table 3. Benefits and challenges of beneficiary participation within the literature (adapted from Mercelis, Wellens and Jegers, 2016) ..................................... 52

Table 4. The compatibility of the critical and transformative paradigms ............................ 62

Table 5. Details of each interview conducted across the two cases ..................................... 76

Table 6. Timeframe of case participation and role of interview participants ....................... 76

Table 7. Secondary data used in each case .......................................................................... 78

Table 8. The matching of data sources to the research questions and objectives ................ 79

Table 9. Coding Hierarchy – Codes and Themes of the Analysis ....................................... 82

Table 10. Beneficiary Quotes signifying the importance of Case A in their lives ............... 92

Table 11. Case A coded references across all stakeholder groups to RQ 1.1: For what is a NPO accountable to its beneficiaries? ............................................ 94

Table 12. Case A document analysis - RQ 1.1: For what is a NPO accountable to its beneficiaries? ............................................................................................ 96

Table 13. Avenues of beneficiary participation within Case A ......................................... 100

Table 14. Case A staff member’s questions for an evaluation instrument ......................... 104

Table 15. Case A cross stakeholder group comparison within RQ 1.2: How is accountability currently discharged to beneficiaries? ................................. 105

Table 16. Case A document analysis coding summary from RQ 1.2: How is accountability currently discharged to beneficiaries? ................................. 106

Table 17. Case A cross stakeholder group analysis of coded references within RQ 1.3: How is accountability on the beneficiaries’ terms practised? .............. 111

Table 18. Case A document analysis of coded references within RQ 1.3: How is accountability on the beneficiaries’ terms practised? .................................. 112

Table 19. Case A cross stakeholder group analysis of coded references within all RQs .............................................................................................................. 114

Table 20. Case B coded referencing to RQ 1.1: For what is a NPO accountable to its beneficiaries? .......................................................................................... 123

Table 21. Case B document analysis summary RQ 1.1: For what is a NPO accountable to its beneficiaries? .................................................................. 124

Table 22. Case B possible barriers to beneficiary participation in evaluation presented by the staff group ........................................................................ 128

Table 23. Case B cross stakeholder group comparison within RQ 1.2: How is accountability currently discharged to beneficiaries? ................................. 132

Table 24. Case B document analysis within RQ 1.2: How is accountability currently discharged to beneficiaries? ......................................................... 133

evaluation as a means of enhancing accountability and beneficiary participation within nonprofit organisations ix

Table 25. Case B cross stakeholder group analysis of coded references within RQ 1.3: How is accountability on the beneficiaries’ terms practised? ............... 139

Table 26. Case B document analysis of coded references within RQ 1.3: How is accountability on the beneficiaries’ terms practised? .................................. 139

Table 27. Case B cross stakeholder group analysis of coded references within all RQs .............................................................................................................. 141

Table 28. Organisational characteristics of Case A and Case B ......................................... 147

Table 29. The beneficiary evaluation points within Case A and Case B. ........................... 152

Table 30. Timeframe of beneficiary participation within each Case ................................. 152

Table 31. Cross Case Analysis coded references to RQ 1.1: For what is a NPO accountable to its beneficiaries? .................................................................. 156

Table 32. Relational and Transactional Accountability comparison - RQ 1.1: For what is a NPO accountable to its beneficiaries? .......................................... 160

Table 33. Cross Case Analysis coded references to RQ 1.2: How is accountability currently discharged to beneficiaries? .......................................................... 161

Table 34. Relational and Transactional Accountability comparison- RQ 1.2: How is accountability currently discharged to beneficiaries? .................................. 167

Table 35. Cross Case Analysis coded references to RQ 1.3: How is accountability on the beneficiaries’ terms practised? .......................................................... 168

Table 36. The enactment of dialogic accounting principles through this research (table adapted from Blackburn et al., 2014) ................................................. 174

Table 37. Relational and Transactional Accountability comparison - accountability on whose terms? ........................................................................................... 180

Table 38. Similarities and Differences between Relational Accountability and Transactional Accountability (as demonstrated within the research findings) ....................................................................................................... 181

Table 39. Theoretical underpinnings of the evaluation frameworks. ................................. 183

x evaluation as a means of enhancing accountability and beneficiary participation within nonprofit organisations

List of Abbreviations

• ACNC Australian Charities and Not-for-profit Commission

• IRP Industry research partner

• NPO Non-profit organisation

• SDGs Sustainable Development Goals

evaluation as a means of enhancing accountability and beneficiary participation within nonprofit organisations xi

Statement of Original Authorship

The work contained in this thesis has not been previously submitted to meet

requirements for an award at this or any other higher education institution. To the best

of my knowledge and belief, the thesis contains no material previously published or

written by another person except where due reference is made.

Signature:

Date: November 2018

QUT Verified Signature

xii evaluation as a means of enhancing accountability and beneficiary participation within nonprofit organisations

Acknowledgements

I am indebted to those who helped me during the process of conducting this

research and writing this thesis. I owe a huge thanks to my supervisory team of Craig,

Laura, and Lyn, who guided me throughout, with knowledge and patience, and were

still there on the other side.

I would also like to thank everyone at the QUT School of Accountancy and the

ACPNS for their advice, encouragement, friendships, cups of much needed coffee, and

financial support through the accelerate scholarship, without which I would not have

been able to undertake this research. This research would also not have been possible

without the research participants, and to them I am truly grateful, for their time,

generosity, and knowledge.

I would like to dedicate this thesis to Angus and Alice, for repeatedly restoring

my self-confidence, offering shrewd insights, and taking me for daily walks throughout

the process of this research.

Chapter 1. Introduction 13

Chapter 1: Introduction

The purpose of this chapter is to provide an overview of the thesis. The chapter

will begin in Section 1.1 by presenting the background of the research, which includes

establishing its need. This will be followed by Section 1.2 with the presentation of the

research context, the research purpose (in Section 1.3) and the research significance in

Section 1.4. Section 1.5 details the central research question, the secondary research

questions, and the objectives of the study. Section 1.6 highlights the gaps the research

seeks to fill. This is followed by the definitions of the key terms used in Section 1.7.

Finally, Section 1.8 gives an outline of all the chapters within the thesis.

1.1 Background

1.1.1 Sustainable development goals.

In 2015, the heads of state, and government representatives met under the

umbrella of the United Nations. Together they established and adopted a series of goals

aimed at ending poverty, protecting the planet and ensuring prosperity for all, by the

year 2030 (United Nations, 2015). These goals have been termed the Sustainable

Development Goals (hereafter SDGs). The SDGs are comprised of 17 goals, each with

a series of targets for achievement. The SGDs operate at a global, regional, national

and subnational level (United Nations, 2015, p. 33). It is within the subnational level

that this research is positioned, when it takes on the call to action espoused by the UN

themselves in saying, “For the goals to be reached, everyone needs to do their part:

governments, the private sector, civil society and people like you.” (United Nations,

n.d.).

In taking up this challenge, this study seeks to undertake research which can

inform two of the 17 goals, goals 10 and 16, at a subnational level. Goal 10 being, “to

reduce inequality within and among countries”; and goal 16 being, to “promote

14 Chapter 1. Introduction

peaceful and inclusive societies for sustainable development, provide access to justice

for all and build effective, accountable and inclusive institutions at all levels” (United

Nations, 2015).

In order to not be overwhelmed by this ambition, this research embraces

Deleuze and Guattari’s metaphor of the rhizome. A rhizome being a subterranean stem,

that takes multiple forms, from underground tubers to “ramified surface extension”

(2013, p. 5). One is never sure where a rhizome starts and ends, will surface, nor the

impact it has, and is having. Metaphorically, a rhizome “ceaselessly establishes

connections between semiotic chains, organizations of power, and circumstances

relative to the arts, sciences, and social struggles” (Deleuze & Guattari, 2013, p. 6).

Although the pathway of the rhizome can never be truly known, the rhizomatic impact

of small changes at the individual or organisational level, can be large changes at the

collective or global level. That being said, small change at the individual or local level,

is a worthy achievement in and of itself. A key opportunity may be that the evidence

base developed by this research, can be used to inform practice specifically.

In its attempt to play a part in addressing the SDGs, this research seeks to action:

1. SDG 10.2 - “Empower and promote social, economic and political inclusion of all, irrespective of age, sex, disability, race, ethnicity, origin, religion or economic or other status.” (United Nations, 2015, p. 21)

2. SDG 16.6 – “Develop effective, accountable and transparent institutions at all levels.” (United Nations, 2015, p. 25)

3. SDG 16.7 – “Ensure responsive, inclusive, participatory and representative decision-making at all levels.” (United Nations, 2015, p. 25)

According to Bebbington and Unerman (2018), the SDGs have only recently

begun receiving attention within academic accounting literature. They argue that

accounting academics have a role to play in embedding policy and action at an

organisational level, in order to contribute to the meeting of the SDGs (Bebbington &


Unerman, 2018, p. 3). This research is keen to heed this call, and contribute to the

development of an evidence base which might inform organisational and

Governmental policies and practices, that can lead to a reduction in social inequality,

and the promotion of more inclusive societies.

1.2 Context

As this research has a core motivation to address social injustices by empowering

marginalised groups, it is positioned within a critical accounting contextual realm. It

attempts to takes up the task of critical accounting’s central tenant, “to challenge and

ultimately change existing social relations; moving towards a more emancipated and

equitable social order” (Catchpowle & Smyth, 2016, p. 220).

Contextually, the central focus of the research is upon beneficiary groupings

within not-for-profit organisations (hereafter NPO). The research assumes the position

that beneficiary groupings are frequently underrepresented within an NPO’s

accountability structures, and that this is to the detriment of both the beneficiary and

the organisation itself. The validity of this assumed position will be examined within

the extant literature and presented within this research. This research posits beneficiary

participation in non-profit organisational evaluation processes, as a means of

increasing beneficiary representation, and ultimately the empowerment and equality

of the beneficiary group. The validity of this position will also be examined through

the extant literature, and researched empirically within this study.

1.3 Purpose

From a critical accounting perspective, this research proposes that traditional

ways of conceptualising accounting and accountability, that focus on the perspectives

of the capital market and shareholders, and thereby promote neo-liberal logics and

policies (Brown & Dillard, 2015a), are at odds with the democratic sentiments of


participation. Neo-liberal policies are those underpinned by an intent to draw on

market mechanisms and private actors to regulate the economy and distribute goods

and services (Chiapello, 2017, p. 50). These are the policies upon which contemporary

accounting focuses. Gray, Brennan and Malpas (2014) contend that ironically,

“contemporary accounting has uncritically and unreflectively developed into a

monolithic practice that in effect prevents any genuine accounting, especially of its

own practice” (p. 271).

In order to challenge this, and promote increased democratic participation, a

dialogic theory of accounting has been drawn upon (Bebbington, Brown, Frame &

Thomson, 2007). Within this theory, referred to as dialogic accounting theory, there is

a call to move away from the traditional monologic approach to accounting, towards a

more dialogic approach, which embraces heterogeneity, and rejects the notion of a

universal narrative (Brown, 2009).

In responding to the call for research that critically examines traditional

approaches to accounting and accountability (Brown, 2009), and to embrace a more

dialogic approach, this research critically explores stakeholder involvement in

accounting and accountability processes and systems, situated within NPOs. Dillard

and Ruchala (2005, p. 611) contend that although not operating for the benefits of

shareholders, “instrumentally rational hierarchical accounting and control systems are

no less prevalent and controlling in the public and the not-for-profit sectors”, than in

the private sector. Of particular focus within this research, is the stakeholder group that

receives a service from the NPO, referred to hereafter as the beneficiary (Wellens &

Jegers, 2017, p. 196).

This research posits the need to develop accounting and accountability systems

that enable accountability to flow downwards, to the beneficiaries of NPOs’ services.


It is theorised that doing so will:

1. “Facilitate informed citizen participation in the decision-making process.” (Brown & Dillard, 2015a, p. 252)

2. Make power relations more transparent and embrace critical pluralism (Brown, 2009, p. 319).

3. Create new visibilities, through enhancing beneficiary impact and voice (Brown & Dillard, 2015a, pp. 251-252).

The involvement of beneficiaries within evaluation processes and systems is

proposed within this research, as a method of increasing accountability to

beneficiaries. Of emphasis here, is the use of participatory evaluation (Greene, 1997)

and transformative participatory evaluation theory and processes (Mertens, 2009).

These theories and processes are used to assist in the development of accounting and

accountability systems, that enable beneficiary participation in NPO evaluation.

The purpose of this research is to enhance accountability practices to

beneficiaries within NPOs. This will be done by exploring current practices of

beneficiary participation within NPOs and, based upon that exploration, proposing

evaluation frameworks, aimed at enhancing beneficiary impact and voice; thereby

creating new visibilities, thus seeking to practically and directly create more dialogic

accountability practices within the two cases in this study. As noted above, more

dialogic accounting facilitates participation, shared power and enhanced voice for

beneficiaries.

1.4 Significance

This research has at its core a motivation to address social injustices by

empowering marginalised groups through their participation in evaluation and

accountability processes. Often overlooked within accounting theory and

accountability systems, beneficiaries of NPO’s services are frequently marginalised

and/or vulnerable groupings of people, with little avenue to have their voice heard or


be listened to within their organisation. It is anticipated that through listening to the

voices of the beneficiaries (and other stakeholders) of NPOs, and developing systems

or mechanisms to address accountability to beneficiaries, this marginalised group will

have increased avenues for organisational impact, and increased empowerment within

their lives.

Research into accountability within NPOs is vast, and frequently is framed

through the use of Ebrahim’s (2016) conceptual framework for considering

accountability. This accountability framework considers accountability to whom, for

what and how? When considering to whom a NPO is accountable, beneficiaries are

considered as a stakeholder within the accountability framework. The question missing

from the discussion, is on whose terms (Brown & Dillard, 2015a, p. 250) is

accountability to beneficiaries practised? In asking on whose terms are beneficiaries

accounted for, a dialogic approach to accountability is framed.

Framing Ebrahim’s (2016) accountability framework, with dialogic accounting

theory, enables the problematisation of its accountability elements (Bebbington et al.,

2007). In doing so, to whom an NPO is accountable, for what, and how is challenged

by questioning, on whose terms are we accounting and accountable (Brown & Dillard,

2015a, p. 250). In this way, a dialogically informed framework for NPO accountability

is presented. These concepts will be further explored within Chapters Two and Three.

1.5 The Research Question

In consideration of the research background and purpose, the central research

question this study seeks to answer is:

RQ 1. How can evaluation enhance accountability to beneficiaries within

NPOs?


This research question articulates the research motivation to enhance

accountability structures directed towards the beneficiary stakeholder group, within

NPOs. The research question articulates the desire to explore the use of evaluation as

a possible means of achieving this.

1.5.1 The Research Objectives.

The central ambition of this research is in keeping with the very aim of dialogic

accounting itself, “to enable a diverse range of societal actors to account for things that

traditional accounting ignores” (Brown & Dillard, 2015a, p. 250).

In this light, the objectives of the research are:

1. To explore and document current practices of beneficiary involvement within accountability and evaluation processes within NPOs.

2. To propose systems that enhance accountability to beneficiaries within NPOs, which create new visibilities, and acknowledge beneficiaries’ rights to have contested narratives.

1.5.2 Secondary Research Questions.

In order to answer the research question and meet the research objectives,

secondary questions have been developed from the extant literature, based on the

dialogically informed framework for accountability (for what, to whom, how and on

whose terms). These secondary questions will guide the research analysis and

presentation. The secondary research questions are:

• RQ 1.1: For what is a NPO accountable to its beneficiaries?

• RQ 1.2: How is accountability currently discharged to beneficiaries?

• RQ 1.3: How is accountability on the beneficiaries’ terms practised?

The purpose of RQ 1.1 is to generate an understanding of what the research

participants believe an NPO is accountable to its beneficiaries for. In asking the

beneficiary (and other) stakeholder groups this question, it is hoped an understanding

of what the beneficiaries are seeking the NPO to be accountable for, can be gained.


The purpose of RQ 1.2 is to understand the current ways NPOs discharge

accountability to their beneficiary stakeholder groups. In doing so, an understanding

can also be formed of the importance and value the organisation places upon the

discharging of accountability to its beneficiaries.

The purpose of RQ 1.3 is to apply a critical lens to the means of discharging

accountability downwards, within NPOs. In doing so an understanding can be formed

of how accountability can be practised on the terms of the beneficiaries, rather than on

terms directed by other, more powerful stakeholder groups. Further justification for

each research question is presented within the Literature Review in Chapter 3.

Both RQ 1.1 and RQ 1.2 assist in meeting Research Objective One through

exploring and documenting current practices of beneficiary involvement. RQ 1.3 seeks

to assist in meeting Research Objective Two through enabling an understanding of

how accountability systems can be developed on the beneficiaries’ terms, in order to

acknowledge their right to present contested narratives.

1.6 Gaps

In conducting research with beneficiaries and NPOs into current beneficiary

participation, and how this may be increased, this research will contribute to filling the

following gaps identified within the scholarly literature:

1. NPO accountability literature; hearing beneficiary’s voice: Mercelis,

Wellens and Jegers (2016, p. 1447) describe there being a “dearth of research

on accountability towards beneficiaries in general, let alone on beneficiary

participation”. They highlight that the perceptions of beneficiaries on

beneficiary participation are rarely studied (Mercelis, Wellens and Jegers,

2016, p. 1447). This research seeks to fill this gap through directly studying

the perspective of beneficiaries, upon their own participation and proposing

ways of enhancing this within the NPOs studied.


2. NPO accountability literature; beneficiary participation methods: This

research will also contribute to filling a gap identified by Wellens and Jegers

(2016, p. 309) when they call for research into ways to strengthen beneficiary

involvement in NPOs. Similarly, O’Dwyer and Unerman (2010, p. 469) call

for research into accountability in NPO contexts, in order to more fully

comprehend the complexities involved when trying to give voice to

stakeholders.

3. Critical accounting literature: In proposing a critical dialogical framework

for accounting information systems, Dillard and Yuthas (2013) call for field

research to identify interested stakeholder groups, their needs and values, and

ways of communicating with them (p. 118). In researching with beneficiaries,

as an interested stakeholder group, in order to better understand their needs

and values within accounting and accountability, this research presents an

opportunity to engage with this call. It does this by proposing beneficiary

participation in evaluation to be a means of communicating with this

stakeholder group.

4. Critical accounting literature; enactment of dialogic accounting theory:

Much of the extant literature on dialogic accounting theory is theoretical in

its content. In developing evaluation frameworks that ensure “ongoing

engagement, evaluation and accountability” (Blackburn, Brown, Dillard, &

Hooper, 2014, p. 94), this research attempts to enact dialogical accounting

theory, empirically and practically.

5. Critical accounting literature; engagement with SDGs: As discussed,

Bebbington and Unerman (2018) have identified a gap within accounting

literature in relation to how organisational policies and actions can contribute

to the achievement of the SDGs. This research attempts to begin filling this


large gap by: proposing specific ways in which two NPOs can enhance their

practices in order to promote social and political inclusion; developing

effective accountability structures; and attempting to ensure inclusive and

participatory decision-making (United Nations, 2015).

6. Evaluation literature: As a consequence of the neo-liberalisation of

evaluation, Mathison (2018) calls for a reconceptualisation of evaluations to

“speak truth to the powerless, not the powerful” and to “empower the

powerless to speak for themselves” (p. 118). This research will attempt to fill

this gap, by offering a means through which evaluators may be able to enable

beneficiaries’ voices to be heard.

1.7 Definitions

Listed below are the key terms used within the research, and their definitions.

This is to aid in communicating a shared understanding of terminology within the

thesis. These definitions are presented in alphabetic order. Due to the construction of

accountability being complex, a definition for it is not provided within this list. Instead,

an in-depth discussion of what accountability is can be found in Section 3.1.2.

• Adversary: The opponent within democratic politics “with whom one shares

a common allegiance to the democratic principles of ‘liberty and equality for

all’, while disagreeing about their interpretation.” (Mouffe, 2013, p. 7)

• Beneficiary: The stakeholder group that receives a service from the NPO

(Wellens & Jegers, 2017, p. 196).

• Dialogic Accounting Theory:

The involvement of multiple constituencies in developing accounting tools and techniques that create and communicate relevant and timely information, enabling all to participate in decision making and to engage in back and forth dialogue to establish common ground for addressing perceived problems in organisational conduct. (Vinnari & Dillard, 2016, p. 26)


• Evaluation: “The systematic process of gathering and interpreting empirical

information in order to make a judgement of quality and worth about the

program, policy, or practice being evaluated.” (Greene, 2013)

• Hegemony: How society comes to be structured around particular power

formations (Spence, 2009, p. 209).

• Neo-liberal policies: Policies “underpinned by an intent to draw on market

mechanisms and private actors to regulate the economy and distribute goods

and services.” (Chiapello, 2017)

• Participatory evaluation: Evaluation process which “involves program staff

and participants in all aspects of the evaluation process to increase evaluation

understanding and use while also building capacity for future inquiries.”

(Patton, 2015, p. 220)

• Political: The way in which society is symbolically structured (Mouffe,

2012).

• Political pluralism: The importance of recognising marginalised viewpoints

when considering the injustices that arise from dominant hegemonies

(Brown, 2017, p. 37).

• Praxis: Repeated action informed by reflection (Scotland, 2012).

• Transformative participatory evaluation: Participatory evaluation

approaches that see the evaluation as an intervention that “itself seeks to

change a given social order based on particular notions of social justice or

democracy.” (Dahler-Larsen, 2018, p. 877)

1.8 Thesis Outline

Chapter One has presented the research purpose, background and the

significance of the research question. Chapter Two establishes the theoretical

underpinnings of the research, exploring dialogic accounting theory, agonistic political

theory and agonistic pluralism, and their relationship to each other. In doing so the

research is situated from a philosophical position. Chapter Three situates the research

within the extant scholarly literature through providing the literature review. The

literature review includes an examination of literature within the disciplines of NPO


accountability, and evaluation. Chapter Four outlines the means within which the

research will be conducted. This includes establishing the ontological, epistemological

and methodological positioning of the research, and a justification for the chosen

research method of case study. Chapter Four also explains the data collection and

analysis methods.

Chapters Five and Six contain the Case Narratives for each case within the case

study. In Chapter Seven, the two cases of the research are cross analysed, which

includes a discussion of the findings from the Case Narratives and analysis, in light of

the extant scholarly literature. The thesis concludes in Chapter Eight with the summary

of the research conducted, and the presentation of the key findings. Chapter Eight also

contains a discussion of the limitations of the research and recommendations for future

research.

Chapter 2. Philosophical Underpinning of the Research 25

Chapter 2: Philosophical Underpinning of the Research

This chapter presents the theoretical assumptions and underpinnings of the

research. In justifying the research from a theoretical position, it is important to recall

the motivation to address social injustices through the empowerment of marginalised

groups. It is thought that this can be achieved through their participation in evaluation

and accountability processes, enabled through the development of participatory

evaluation frameworks. However, providing new participatory frameworks without

first examining the causes of social inequalities will do little to bring about authentic

change.

In this light, the research positions the traditional monologic conceptualisation

of accounting and accountability, embedded in neo-liberal policies, as being

hegemonic, and a cause of the existing social inequalities. Here, hegemony is

referenced to Gramsci’s (1971) theory of hegemony, and refers to “how society comes

to be structured around particular power formations” (Spence, 2009, p. 209). This

research purports a need to develop frameworks for participation that are motivated by

counter-perspectives, or counter-hegemonies (Tambakaki, 2014, p. 8). A counter-

hegemonic intervention aims not to unveil a ‘true reality’, but to re-articulate a given

situation in a new configuration (Mouffe, 2013, p. 79).

To assist in doing this, the research will be informed by theories of agonistic

democracy and pluralism, and dialogic accounting. These theoretical positions will be

elaborated and discussed throughout this chapter. The chapter begins with a discussion

of dialogic accounting theory in Section 2.1. Section 2.2 introduces agonistic political

theory and agonistic pluralism. The chapter concludes in Section 2.3 with a summary

of the key philosophical underpinnings of the research.

26 Chapter 2. Philosophical Underpinning of the Research

2.1 Dialogic Accounting Theory

From a critical perspective, traditional approaches to accounting and

accountability are grounded within a neoclassical, economic (Dillard, Yuthas &

Baudot, 2016, p. 15) framework, and are relationally symbiotic with capitalism,

governance and governmentality (Gray, Brennan & Malpas, 2014, p. 261). These

approaches presume objectivity, rationality, neutrality and utility maximisation

(Vinnari & Dillard, 2016, p. 25). However, as Lehman, (2013, p. 137) contends,

accounting is always subjective, socially constructed and mutable. These concepts will

be explored further within this section.

As discussed previously (Section 1.3), neo-liberal policies are those underpinned

by an intent to draw on market mechanisms and private actors to regulate the economy

and distribute goods and services (Chiapello, 2017, p. 50). The current accounting

regime, embedded in neo-liberal logics (Brown & Dillard, 2015a; Chiapello, 2017),

frames accounting and accountability from within a monologic perspective. The

monologic perspective being the single (mono) narrative that is dominated by capitalist

perspectives, positivism, and neoclassic economies (Brown & Dillard, 2015a).

Graham (2013, p. 125) describes the finance and accounting industries as a

hegemonic system for the production of truth. It is this production of apparent truth

that allows accounting and accountability to then appear apolitical, ahistorical and

acontextual. Once a truth is established and concretised, it is like it was always there.

It then becomes difficult to see, and appears as objective, as a reality. Vinnari and

Dillard (2016) concur that “accounting does not only reflect a reality, it participates in

the construction of that reality” (p. 25).

Lehman (2013, p. 137) suggests there are “vast implications” of accounting’s

actual subjectivity, when it is presented as objective. He highlights the importance of


questioning what is chosen to be represented and what is chosen to be silenced?

(Lehman, 2013, p. 137). The very act of being able to ask these questions demonstrates

the subjectivity of the phenomena. However, within this now produced truth, a pattern

of power relations is established, and the meaning of social relations is set (Vinnari &

Dillard, 2016, p. 28). This process creates more and less powerful constituents and

stakeholders. Who we are accountable to, for what, and how, is positioned and defined.

Evidence of this can be observed within the Australian Accounting Standards

Board’s document, “The Framework for the Preparation and Presentation of Financial

Statements” (the Framework), which position information (financial reporting) as

being useful to “existing and potential investors, lenders and other creditors”

(Australian Accounting Standards Board [AASB], 2016, p. 29). In this manner, users

are positioned and defined by neoclassical economics (Dillard, Yuthas & Baudot,

2016, p. 16). This is done by a presumption that the decision usefulness of accounting

information is for meeting the needs of those privileged by the neoliberal hegemony,

those being business, investors and capital markets, for profit maximisation and

economic growth (Brown & Dillard, 2015a). Although in recent times the Framework

has sought to include other user stakeholders within NPOs (including donors,

taxpayers, beneficiaries, advisers, and members of parliament) in its discussion

(AASB, 2016), the powerful stakeholders are still clearly positioned and the current

dominant hegemony of economic interests still dictates accountability structures.

It makes sense that when users are constructed from within accounting’s

narrowed neo-liberal framework, stakeholders outside of this construction will be

unaccounted for. A process which gives voice to users defined as requiring relevant,

timely and understandable information, simultaneously takes voice away from those

outside of the definition. Brown and Dillard (2015b) are clear:


If accounting is to address the decision-making and accountability needs of a range of stakeholders, the process and concepts for developing, assessing, committing to and applying accounting technologies require reconsideration. We seek to open up discussion on these issues so as to foster accounts for stakeholders that promote democratic participatory governance. (p. 962)

In response to the call for a reframing of accounting and accountability away

from that which is instituted in neoclassical economics and corporate law (Brown &

Dillard, 2015b, p. 962), a more dialogic approach to accounting has been put forward.

Proposed by Bebbington et al. (2007), dialogic accounting denotes:

The involvement of multiple constituencies in developing accounting tools and techniques that create and communicate relevant and timely information, enabling all to participate in decision making and to engage in back and forth dialogue to establish common ground for addressing perceived problems in organisational conduct. (Vinnari & Dillard, 2016, p. 26)

Burgeoning as a theory, a dialogic approach to accounting and accountability has

been presented and critiqued within extant scholarly literature over the past decade. It

has been advanced predominantly within critical and social accounting circles (for

example Blackburn et al., 2014; Brown 2009; Deegan, 2017). Brown and Dillard

(2015a) state the aim of dialogic accounting to be “to enable a diverse range of societal

actors to account for things that traditional accounting ignores and to develop

accounting that acknowledges divergent ideological positions” (p. 250). As this

research has a core motivation to address social injustices by empowering marginalised

groups through participation in accountability processes, this is a fitting theoretical

base for this research to adopt.

In keeping with a growing body of literature within the critical accounting

paradigm, dialogic accounting theory draws upon Mouffe’s (1999) conceptualisation

of an agonistic approach to democracy (agonistic political theory, agonistic pluralism)

(Bebbington, et al., 2007; Bond, 2011; Brown, 2009; Vinnari & Dillard, 2016). In her

seminal work, Brown (2009) has drawn on agonistic political theory to develop a


conceptual framework for critical dialogic accounting. This framework is an attempt

to democratise accounting, by moving from monologic accounting principles to

dialogic accounting principles (Brown, 2009). This approach interrogates “what is

accounted for, how it is accounted and on whose terms it is accounted” (Brown &

Dillard, 2015a, p. 250). Within this:

Brown seeks to develop an approach to accounting that respects the differing interests and needs of stakeholders, addresses the role of power in privileging some interests over others, and acknowledges the impossibility of consensus around monologic accounting systems. (Dillard, Yuthas & Baudot, 2016, p. 15)

According to Brown (2009), the eight principles with the dialogic framework are:

1. Recognise multiple ideological orientations. 2. Avoid monetary reductionism. 3. Be open about the subjective and contestable nature of calculations. 4. Enable accessibility for non-experts. 5. Ensure effective participatory processes. 6. Be attentive to power relations. 7. Recognise the transformative potential of dialogic accounting. 8. Resist new forms of

monologism. (p. 324-327)

Table 1 (below), adapted from Blackburn et al. (2014) illustrates the main

variance between the principles of a monologic approach to accounting and

accountability and a dialogic approach.


Table 1. Variance between principles of a monologic approach to accounting and a dialogic approach to accounting (table adapted from Blackburn et al., 2014, p. 91-92)

As an underlining philosophical approach, this research is informed by these

principles of dialogic accounting. The principles will be used within the discussion of

the findings (Chapter Seven) as a means of identifying a dialogically orientated NPO,

and to inform the development of dialogically motivated, beneficiary participative

practices. In order to extend the understanding of dialogic accounting theory, and the

theory’s suitability to this research, its underlying assumptions will be explored within

the following section.

2.2 Agonistic Political Theory and Agonistic Pluralism

Dialogic accounting theory draws heavily upon agonistic political theory and

agonistic pluralism. Understanding the political situated-ness of accounting within this

research is important. Without an understanding of the contributing causes of the social

injustices the research seeks to address, it is likely that solutions will be sought that are

embedded within the very political causes of the problem.

Burlaud and Colasse (2011) state that “anyone who knows anything about it at

all knows that accounting is political through and through. It is not a pure technique,

far from it” (p. 44). However, traditional accounting depoliticises accounting through


denying the political (Brown, 2009). Within dialogic accounting theory, accounting

and accountability are firmly positioned as political acts, and this position is strongly

espoused within this research.

Agonistic political theory is an approach to participative democracy which

contrasts a deliberative approach by placing a central role upon difference and conflict

(Brown, 2009). An agonistic approach to democracy “recognizes the complexity of

prevailing power dynamics and that competing perspectives and interests cannot be

resolved through logic or reason” (Dillard & Yuthas, 2013, p. 114). Of importance

within agonistic political theory and the theoretical positioning of this research, is

pluralism. Within a critical ontology and a socially constructed epistemological

perspective (see Chapter Four), this research questions and problematises the existence

of a single, objective reality waiting to be discovered and doing so, assumes a more

pluralistic vision of truth (Dillard & Yuthas, 2013).

However, in embracing pluralism and observing Brown’s (2017) detailed

analysis of the different generations of pluralism, care is taken to situate this research

away from the second generation of neo-liberal pluralism, where “post-war pluralists

saw themselves as apolitical scientific observers” (p. 26). Instead, this research is

positioned within the third generation of political pluralism, where pluralists focus on

“the injustices that arise from dominant hegemonies and the importance of recognising

marginalised viewpoints” (Brown, 2017, p. 37). Pluralist politics recognises the

existence of conflict but enfolds it within democratic activity (Vinnari & Dillard, 2016,

p. 28).

Understanding, or considering, how to deal with conflict or differing values and

perspectives is important within this research. This is especially so because this

research is attempting to address the disempowerment of marginalised groupings of

people. As established, a possible cause of this disempowerment is the very accounting


and accountability hegemony within which the marginalised groupings are positioned.

Therefore, to attempt to engage with pre-existing constructs and ways of being which

have perhaps caused and perpetuate the power dynamics within societal groups in a

deliberative manner (with an aim of consensus), would be engaging with the same

constructs under a new guise. It could also quite possibly do more harm than good to

bring to the table stakeholders with such power asymmetries (for example beneficiaries

and staff), and expect an equality of position capable of leading to rational or

empowered consensus.

Therefore, essential to the critical pluralist positioning of this research is an

understanding of the rejection of notions of deliberative democracy and

communicative rationality (Brown, 2017, p. 30). These assume an ability to reach

consensus and do not give credit to the distribution of power, or the hegemony under

which they are operating.

Brown (2017, p. 31) warns of the Habermasian style of “reasonable pluralism”

as being ineffective in combatting injustices and dominant hegemonies in a

contemporary neo-liberal climate. For this reason, it is important, in embracing a

constructionist ontology, to attempt to establish a counter-reality based upon a

different way of being. In doing so, this research will espouse an agonistic approach to

pluralism and democracy, and be informed by agonistic political theory (Mouffe,

2013).

Agonistic political theory is proposed as an alternative to deliberative forms of

democracy such as that developed by Habermas, where the belief is that consensus is

possible via rational argumentation (Mouffe, 1999). Mouffe’s (1999, 2012; 2013)

agonistic pluralism challenges the existence of a public space in which power and

antagonism have been removed and consensus realised. She argues that a society’s


democratic character:

Can only be given by the fact that no limited social actor can attribute to herself the representation of the totality and claim in that way to have the mastery of the foundation … social objectivity is constituted through acts of power. (Mouffe, 1999, p. 752)

Here, democratic political theory views deliberative consensus as an

impossibility due to “the eradicable antagonisms arising from the incompatibility of

plural values and uneven power distribution” (Vinnari & Dillard, 2016, p. 27). Rather

than attempting to ignore or level out power asymmetries, instead an agonistic

approach facilitates conceptualisations of power that are supportive of democratic

processes (Dillard & Vinnari, 2017, p. 100).

Within the context of this research, which aims to empower marginalised groups,

this theoretical base is particularly relevant. It acknowledges that regardless of the

research outcomes, power asymmetries will still exist amongst stakeholders, but that

does not eliminate the right to participation, nor the worth of the research. Although

beneficiaries have a less powerful position within the NPO in terms of their ability to

hold the NPO to account, they still have a right (under democratic values of social

justice and equality) to voice an opinion that is incompatible with more powerful

stakeholder’s, and that opinion, although contestable within a democratic struggle, is

worthy and valid. Consensus is not necessitated. After all, as Dillard and Vinnari

(2017) aptly state, “one of the privileges of democracy is to disagree and to participate

in the related power struggles and conflict” (p. 100). It is how that can be done in a

manner that respects the rights of all, that is of concern for this research at hand.

2.3 Summary and Implications

In embracing a dialogic approach to accounting and accountability, informed

through principles of agonistic pluralism, this research will redefine users away from

the narrow boundaries set by traditional accounting theory. In acknowledging the


power asymmetries that have allowed certain stakeholder groups to be worthy of

relevant, timely and understandable information at the exclusion of others, this research

will attempt to work outside of these dynamics, to address social injustices.

By focusing on beneficiaries of NPOs, and positioning them as one range of

societal actor that traditional accounting tends to ignore, this research will embrace

difference, open up dialogue and attempt to create a new system for participation. This

system of participation will reposition and redefine accounting and accountability

systems away from a traditional neo-liberal conception, and towards a system that

empowers and gives voice to marginalised groups of people.

Chapter 3. Literature Review 35

Chapter 3: Literature Review

The purpose of this chapter is to explore current academic literature on

accounting, accountability, management, and policy, in order to investigate how

evaluation contributes to this discourse, in light of the participation of beneficiary

stakeholders. The aim of the literature review is to uncover gaps in the current thinking

on accountability to beneficiaries and highlight how evaluation processes may assist

in filling these gaps.

As discussed within this chapter, extant scholarly literature suggests that by

increasing beneficiary participation within NPOs (Murtaza, 2012, p. 112),

beneficiaries’ rights, under international conventions (O’Dwyer & Unerman, 2010, p.

452) will be activated. Through beneficiary participation the NPO will experience

higher levels of accountability by empowering its beneficiaries (Jacobs & Wilford,

2010, p. 799; Kilby, 2006, p. 953; Portela, 2012, p. 78) to be able to hold the

organisation to account (Najam, 1996, p. 345; Kilby, 2006, p. 952; Jacobs & Wilford,

2010, p. 799). This in turn improves organisational learning (Ebrahim, 2009, p. 899)

and allows beneficiaries voices to be heard (McNamara, 2013, p. 56; Schmitz &

Mitchell, 2016, p. 257). Evaluation is posited as a means of increasing beneficiary

participation, and of increasing accountability downwards, to beneficiaries.

In order to progress, after establishing a definition of NPOs and accountability

in Section 3.1, the chapter will be structured using Ebrahim’s (2016) conceptual

framework for deeper analysis of accountability. This framework is presented within

Section 3.2. As previously detailed, this framework, which is also discussed by Jordan

(2007), Murtaza (2012), O’Dwyer and Boomsma (2015), and Schmitz, Raggo, and

Vijfeijken (2012), questions: accountability to whom (Section 3.3), for what (Section

36 Chapter 3. Literature Review

3.4), and how (Section 3.6). In casting the critical lens of this research across

Ebrahim’s (2016) conceptual framework, the addition of accountability on whose

terms (Brown & Dillard, 2015a) is proposed (Section 3.5). In doing this, a conceptual

framework for dialogic accountability (Brown, 2009) is presented.

Through this examination, the role evaluation may play in increasing

accountability to beneficiaries will be investigated (Section 3.7). As such, a review of

current scholarly literature on the topics of rights and participation of beneficiary

groupings, participative evaluation and transformative participatory evaluation is

necessary. The chapter concludes with a summary in Section 3.8.

3.1 Establishing the context

The aim of this section is to provide a clear context for the positioning of the

research. In order to do this, the section will begin by defining what a non-profit

organisation is. It will then proceed to establish a working definition of accountability

drawn from the extant literature.

3.1.1 What is a non-profit organisation?

As detailed by Anheier (2005), the not-for-profit sector can be difficult to

definitively define. The sector and its organisations (NPOs) can be described through

different approaches, including definitions from a legal perspective, a functional

perspective, an economic perspective and a structural-operational perspective

(Anheier, 2005). For the purpose of this research, the definition of an NPO adopted by

the United Nations will be used. This definition includes the following criterion to

define NPOs (United Nations, 2003, p. 18). They are

• organisations that are; • not-for-profit and non-profit distributing; • institutionally separate from government; • self-governing;


• non-compulsory.

To add to this definition, in their research, Cordery and Sim (2017) have

classified Civil Society Organisations. As a part of civil society organisations, NPOs

are included in this classification. The classifications developed by Cordery and Sim

(2017) are: Advocacy, Classic Charity, Infrastructure, Membership, Philanthropist,

and Service Providers. In their quantitative, survey-based research, Cordery and Sim

(2017, p. 16) find that accountability profiles vary across these different organisational

types in relation to who they are accountable, how accountability should be discharged,

and the mechanisms in which to do this.

3.1.2 What is accountability?

Within academic writing, there exist multiple and often competing perspectives

of accountability. Accountability has been referred to as having a ‘chameleon-like’

nature, especially in consideration of its competing constituencies (Sinclair, 1995, p.

231). Najam (1996, p. 350) claims that accountability is a complex and abstract

concept. As such, accountability is identified as having no one true meaning within its

socially constructed nature (Ebrahim, 2003a, p. 194).

For the purpose of this research, the understanding of accountability is

constructed through the intertwining of understandings drawn from the academic

literature. This includes Ebrahim’s (2003a) statement that accountability is “… the

means through which individuals and organizations are held externally to account for

their actions, and as the means by which they take internal responsibility for

continuously shaping and scrutinising organizational mission, goals, and performance”

(p. 194). Murtaza (2012, p. 112) enriches this definition by adding that accountability

also incorporates the rights of stakeholders to participate in all phases and levels of the

performance management cycle of an entity, thereby improving


justice. In contributing to the construction of accountability, Roberts, (1991)

incorporates a social acknowledgement, through recognising “that one’s actions make

a difference to both self and others” (p. 365). Gray, Brennan and Malpas (2014, p. 266)

describe accountability as the responsibility to provide an account. As such,

accountability within this research context considers relationships, stakeholder

participation, rights, power, social justice and responsibility. It is both a process and

an outcome.

Table 2 (below) provides a summary of some key accountability descriptions

and considerations from within the extant scholarly literature. As shown in the table,

there is an awareness within the literature, that accountability considers empowerment.

Accountability is said to be empowering when it opens a NPO up to a degree of control

by its beneficiary stakeholders (Kilby, 2006, p. 953).

Table 2. Summary of key accountability descriptions and considerations from the scholarly literature


This concludes the establishment of the research context. The literature review

will now progress to discuss the positioning of accountability within the extant

literature in relation to an accountability framework.

3.2 An Accountability Framework

As previously detailed, within not-for-profit accountability academic literature

(Ebrahim, 2016; Jordan, 2007; Murtaza, 2012; O’Dwyer & Boomsma, 2015; Schmitz

et al., 2012), questions are posed to enable NPOs to broaden their understanding of

accountability and to facilitate more meaningful discussions of the topic (Najam, 1996,

p. 341). These questions form a conceptual Accountability Framework for considering

accountability within NPOs (Ebrahim, 2016). The questions are:

• To whom are NPOs accountable?

• For what are NPOs accountable?

• How are NPOs accountable?

Another accountability framework is presented within the extant literature. This

framework is referred to as an “accountability regime”, and positioned within a public

accountability discourse (Mashaw, 2005). This framework questions: “who is

liable…to whom; what are they liable to be called to account for; through what

processes accountability is to be assured; by what standards the…accountable

behaviour will be judged; and, what are the potential effects of finding those standards

have been breached” (Mashaw, 2005, p. 118). Whilst this accountability regime

facilitates a similar yet extended questioning of accountability, for the purpose of this

study, the accountability framework presented by Ebrahim (2016) will be of focus due

to its grounding within, and development from the not-for-profit sector. However, in

order to position Ebrahim’s accountability framework more critically (which is


appropriate to the critical accounting position this thesis assumes), the lens of dialogic

accounting theory is applied (Brown & Dillard, 2015a).

Embedding Ebrahim’s (2016) framework within a dialogic perspective, offers a

system which allows for contested narratives to coexist, where multiple viewpoints are

engaged, power dynamics addressed and actor engagement taken seriously (Brown,

2009, p. 317). As such, a dialogically informed conceptual framework for

accountability can extend Ebrahim’s framework by interrogating what is accounted

for, to whom are we accounting, how it is accounted, whilst adding - on whose terms

are we accounting and accountable? (Brown & Dillard, 2015a, p. 250).

Applying the critical perspective of Brown and Dillard (2015a), which

interrogates power, to the accountability framework of Ebrahim (2016), the following

accountability questions for NPOs to consider, are established. These questions will

be discussed throughout the remainder of this chapter. The questions are:

• To whom are NPOs accountable?

• For what are NPOs accountable?

• How are NPOs accountable?

• On whose terms are NPOs accountable?

3.3 Accountability to Whom?

Multiple theories exist from within which accountability to whom can be

positioned. From within the perspective of agency theory organisations are typically

concerned with the funder/provider (or principal/agent) relationship to explain the

performance of an organisation (Ebrahim, 2009; Jensen & Meckling, 1976). This is

done by evaluating the extent to which providers have expectations set by funders

(Campbell & Lambright, 2016).


This relationship is also prevalent within resource dependency theory where the

organisation’s resource dependence upon the funder (or donor) creates an

asymmetrical power relationship (Campbell & Lambright, 2016). This asymmetrical

power relationship is also considered within instrumental stakeholder theory, in which

an organisation “…concentrates on how to organize a credible decision making

process” (Scherhaufer, 2014, p. 452). The outcome of these perspectives is to focus

attention on powerful stakeholders.

It is notable within agency theory, resource dependency theory and instrumental

stakeholder theory, that accountability systems favour the most powerful stakeholders

within NPOs. These are typically the funders, donors or government bodies. Ebrahim

(2005) refers to accountability systems that are heavily weighted towards these

powerful stakeholders as “narrow manifestations of accountability” (p. 57).

In contrast, normative stakeholder theory, reframes accountability systems.

Within normative stakeholder theory, Dawkins (2014) highlights that all stakeholders

(regardless of how much power they hold), are entitled to have input into matters that

affect them. Wellens and Jegers (2014, p. 940) suggest that normative stakeholder

theory can serve as a useful framework for NPO accountability. Here, stakeholders are

defined, and include those who are affected, or can affect an organisation (Ebrahim,

2003b, p. 814; 2005).

Considering the importance of stakeholders within the accountability processes

of NPOs features strongly within academic literature (Cordery & Sim, 2017; O’Dwyer

& Unerman 2007, 2010; O’Dwyer & Boomsma, 2015). Ebrahim (2007), Najam (1996)

and Kilby (2006), stress an NPO’s multiple accountabilities: to themselves, to clients

and to donors. Najam (1996) claims that these multiple accountabilities make

accountability within the not-for-profit sector more complex than for the private or


public sectors. To complicate the dynamic further, these multiple accountabilities are

frequently also competing accountabilities (Ebrahim, 2003b, 2005). Ebrahim (2005)

confirms the asymmetries in the relationships between these constituents are a result

of power difference.

Murtaza (2012) has directionally positioned the multiple stakeholder

accountabilities within an NPO. The accountability directions are illustrated within

Figure 1 (below). Here, accountability can be:

1. Upward = to donors, funders, governments. 2. Inward = to boards, mission. 3. Downward = to communities, beneficiaries, clients. 4. Sideways = to other NPOs.

Figure 1. The directions of accountability (figure developed by the researcher)

Within extant scholarly literature, there is a clear emphasis on the priority NPOs

place towards upwards accountability, over that of other, less powerful directions.

(Research that evidences this claim includes, Campbell and Lambright, 2016;

Ebrahim, 2003b, 2005, 2007, 2009; Jacobs and Wilford, 2010; Jordan, 2007; Murtaza,

2012; Najam, 1996; O’Dwyer and Unerman, 2007; O’Dwyer and Unerman, 2010;

Schmitz et al., 2012).

Jordan (2007) warns that organisations focus on upwards accountability at their

own risk as “accountability mechanisms that emphasize the needs of donors, donor

agencies, and/or governments could jeopardize other important relationships with

other stakeholders” (p. 153). Similarly, Wellens and Jegers (2014, p. 938) draw a


relationship between a focus on upwards accountability and weaker organisational

performance and program sustainability. Jacobs and Wilford (2010) also stress that

mechanisms of upwards accountability tend to weaken downwards accountability.

Najam (1996) observes that NPOs’ most obvious line of responsibility, is for

them to be accountable to the community and clients they are working with. However,

Najam (1996) and Kilby (2006) highlight that because these communities are

frequently impoverished or marginal, they lack the mechanisms for holding the NPO

to account. Jacobs and Wilford (2010) contend, “whereas powerful actors can require

accountability from less powerful actors, less powerful actors cannot so easily require

it of the powerful” (p. 799). Essentially, they lack the power to enable downwards

accountability (to themselves). This is a quality that donors and governments do not

lack, which is evidenced through their ability to hold NPOs to account.

O’Dwyer and Unerman (2007; 2010) call for a move to what they label as social

accountability. Social accountability allows NPOs the “scope to embrace broader

accountability for their wider social impacts” (O’Dwyer & Unerman 2007, p. 450),

with an explicit focus on their key beneficiary constituencies. Similarly, Gray, Brennan

and Malpas (2014, p. 262) view conventional accounting, defined through neo-liberal

logics, as just one way in which accounts can be envisaged. They posit social

accounting as another way. Here, all of social life “might be viewed as the giving and

receiving of accounts” (Gray, Brennan & Malpas, 2014, p. 262).

Downwards accountability is a key aspect of this thesis. This necessitates an

exploration of downwards accountability, particularly accountability to beneficiaries

within the extant scholarly literature. This exploration is presented within the

following section.


3.3.1 Accountability to beneficiaries.

The importance of downwards accountability to beneficiaries is strongly

acknowledged within academic literature. Amongst other literature, downwards

accountability to beneficiaries features within Campbell and Lambright (2016),

Ebrahim (2005), Jordan (2007), Najam (1996), Murtaza (2012), Rey-Garcia, Liket,

Alvarez-Gonzalez, and Maas, (2017), Schmitz et al., (2012), Schmitz and Mitchell,

(2016) and Wellens and Jegers (2017). These papers will be referred to throughout the

literature review.

Wellens and Jegers (2017, p. 197) claim NPOs to be ultimately accountable to

their beneficiaries because they are the stakeholder that receives the service of the

NPO. As such, beneficiaries should have the right to participate in decisions that affect

service delivery, and affect their daily life. However, Schmitz, Raggo and Vijfeijken’s

(2012) research suggests that NPOs’ aspirations to engage in downwards

accountability, are not as yet matched by their practices.

In his seminal work, Najam (1996) cautions of NPOs’ downwards accountability

practices where beneficiary participation is merely a feel good exercise. Najam (1996)

refers to these practices as the “sham-ritual” (p. 346). Ebrahim, (2007) emphasises the

importance of NPOs having processes for addressing unequal power relations, in order

for beneficiary participation to lead to genuine downwards accountability. Before

considering the process of how to be accountable, in order to address unequal power

relations, an NPO must first know what it is accountable for.


3.4 Accountability for What

When considering what an NPO is accountable for, the extant scholarly literature

details a selection of obligations or objects. Murtaza (2012) details standards setting,

performance appraisal and sanctioning. Ebrahim (2003b, p. 815), Murtaza (2012) and

Najam, (1996) all detail two levels of accountability: functional accountability (for

resources, resource use and immediate impacts) and strategic accountability (for

impacts upon other organisations and the wider environment). Murtaza (2012) also

draws attention to “impact oriented criteria” (accountability for the involvement of

communities, effectiveness, equity and sustainability), and “process-orientated

criteria” (accountability for technical standards, financial integrity/legality, efficiency

and coordination with others) (p. 115).

Other scholarly literature details NPOs are accountable for

• financial reporting and efficiency (Ebrahim, 2009; O’Dwyer & Unerman, 2007; Schmitz et al., 2012);

• transparency and disclosure (Schmitz et al., 2012);

• fulfilment of legal obligations (Schmitz et al., 2012).

Ebrahim, (2009) labelled three normative logics, within which what an NPO is

accountable for can be framed. They are:

1. Technocratic or professional – to enhance accountability by improving results, through measuring performance.

2. Coercive or punitive – emphasis on disclosure, and reliance on legislative or regulatory oversight.

3. Adaptive or strategic –focusing on the mission, vision, key activities and how it frames social problems. (Ebrahim, 2009, p. 888-889)

Deliberating on these three logics enables an organisation to gain a broader

consideration of what they are accountable for, rather than focusing on coercive or

technocratic strategies. It also broadens the catchment of the constituencies that are


affected by, or involved in, accountability processes. Within the more complex logic

of the adaptive frame, sits a greater emphasis on organisational learning and

development, strategic choice, critical reflection on mission and possible involvement

of all stakeholders, at all levels (Ebrahim, 2009).

This raises the importance of questioning if the accountability obligations of

NPOs meet the needs of all stakeholders. Are the accountability needs of all

stakeholders met if, as the literature suggests, NPOs are predominantly accountable

for financial reporting, efficiency, transparency, disclosure and the fulfilment of legal

obligations? As discussed, the academic literature suggests NPOs prioritise

accountability upwards. It would be fitting then, that what NPOs are accountable for,

also has its priorities in this direction. Depending upon which stakeholder group is

being considered, what an NPO is accountable for, could be different. What

beneficiaries see as accountable, may be different to what the funders or government

see as accountable. In applying a critical lens which is foundational to this study, and

considering the power asymmetries between stakeholder groups, it becomes important

to question, on whose terms is accountability practised? In doing so, the research

question: (RQ 1.1) For what is a NPO accountable to its beneficiaries, arises.

3.5 Accountability on Whose Terms?

As noted above, when positioned within a critical lens, it becomes apparent that

what an organisation is accountable for will change depending upon who is seen to be

the primary accountable agent. This raises the question of on whose terms are we

accounting? When considering accountability to beneficiaries, NPOs might consider

their reach, that is, the amount of beneficiaries they service, to be a valid measure.

However, for the beneficiary themselves, accountability for reach may not match their

desire for accountability to their individual needs. Indeed, as recently as 2017, Rey-


Garcia et al.’s research involving 2229 Spanish NPOs, confirmed previous research

(Wellens & Jegers, 2016), that a considerable number of NPOs “do not in any way

measure beneficiaries’ needs” (Rey-Garcia et al., 2017, p. 507). This research indicates

that the majority of NPOs surveyed say they implement beneficiary-related

consultation and participation mechanisms, yet many only use it symbolically (Rey-

Garcia et al., 2017). As such, beneficiaries are considered to have a restricted impact

upon decision-making and output (Rey-Garcia et al., 2017). Thus the process of

rethinking who NPOs are accountable to, reframes what NPOs are accountable for,

and necessitates the questioning of the dominant hegemony in which accountability is

defined.

As discussed within Chapter Two, when accountability is unquestionably

defined and framed within neo-liberal policies, positioning the who and what of

accountability is clear. For example, accountability to shareholders for financial

performance is well legitimised and legislated. However, when dominant hegemonies

or ideologies are confronted and power relations challenged, the question: on whose

terms are we defining the who and what of accountability, is raised. In doing so, the

research questions: (RQ 1.2) how is accountability currently discharged to

beneficiaries, and (RQ 1.3) how is accountability on the beneficiaries’ term practised,

arise. The answer to these questions frames how accountability is actioned or

discharged.

3.6 Accountability How?

Within traditional accounting which is embedded within neo-liberal logics (as

discussed in Chapter Two), how accountability is discharged is typically and

powerfully framed. Through the accounting standards and standard setters, definitions


are established which construct ways of looking, and define and build who is visible

and who is not. These constructions determine on whose terms we are accounting.

Accountability typically appears in the form of financial statements and

reporting. These provide financial information to assist users in making decisions

about resource allocation or provision (AASB, 2016). This framing of accounting and

accountability establishes who accountability is for, what accountability is for, and

how accountability is to be discharged. However, in considering a more dialogic

perspective, on whose terms this version of accountability is positioned can be

questioned. When thinking dialogically, how accountability is constructed and

discharged will be different from the monologic approach. From the perspective of the

beneficiary, how accountability appears, on their terms, may be different from

traditional, neo-liberal constructions. How accountability to beneficiaries can be

actioned, on their terms, necessitates asking them.

Mashaw (2014) introduces the concept of time into the scholarly accountability

discussion, from a public accountability and public policy lens. He suggests

considering “how can or should contemporary accountability regimes deal with time

lags, that is, long separations between decisions and effect” (p. 575) and also time “lags

between actions and accountability demands” (p. 586). It is through this perspective,

that Mashaw adds the question of accountability when to the “accountability regime”.

See Section 3.2 for Mashaw’s previous articulation of an accountability regime

(Mashaw, 2005).

To Mashaw, time seems to appear as a resource within the consideration of

accountability. A similar presentation of the need to consider time, and time lags

between accountability demands and actions through evaluations is made by Taut

(2008, p. 229). Postulating on a possible time lag arising within accountability

processes, is valid in planning how accountability to beneficiaries is to be practised.


This is especially important in ensuring the process is empowering to beneficiaries,

and not disempowering due to a time lag.

3.6.1 Accountability how: through a rights-based approach.

As a pathway to the empowerment of beneficiaries, Kilby’s (2006) research

uncovers the need for more structural links in order to deliver stronger empowerment

outcomes, and that these formal structures establish a right for beneficiaries that is

itself empowering. It is perhaps from a rights-based perspective (Jordan, 2007;

McNamara, 2013; O’Dwyer & Unerman, 2010; Schmitz et al., 2012; Schmitz &

Mitchell, 2016) that beneficiaries will be able to hold NPOs to account.

O’Dwyer and Unerman (2010) describe the rights-based approach to

accountability as:

The rights-based approach focuses explicitly on defining people’s rights in relation to various state and non-state actors (such as non-government organisations) as laid down in international conventions, and then empowering them to claim these rights. (p. 452)

Central to this understanding is the accountability relationship between rights’ holders

and the government as the main duty bearer bound by legal obligations under

international conventions (Schmitz & Mitchell, 2016, p. 255). Schmitz and Mitchell

(2016, p. 255) question why those with rights are not claiming their rights, and why

those with duties not contributing to their realisation? If, as previously discussed,

beneficiaries (as rights’ holders) lack the power to hold NPOs to account, it seems

logical that they also lack the power to hold the government to account. Perhaps the

NPO can be the agent of empowerment for beneficiaries at a service level, but also to

the government on a wider level through embracing a rights-based approach. The focus

could be on beneficiary participation within evaluation processes of a NPO. Doing so


would give beneficiaries a pathway for their voice to be heard, referred to hereafter as

‘giving voice’.

3.6.2 Accountability how: through giving voice.

Scholarly literature on giving voice is frequently situated alongside discussions

of rights’ actualisation (see McNamara, 2013; Schmitz & Mitchell, 2016). Schmitz,

Raggo, and Vijfeijken (2012) extend the parameters of the giving voice discourse to

highlight the importance of beneficiaries’ voices reaching those parts of the

organisation that are usually oriented to upwards accountability stakeholders,

including the board, financial management, human resources and procurement units.

This allows the possible result of downwards accountability processes; that is, to give

greater voice to beneficiaries, to impact upon the rights of the beneficiary on a personal

and collective level, and to impact upon organisational learning (acknowledged as a

significant component of the accountability process by Ebrahim, 2009).

Cornwall (2008) stresses that simply being involved in a process of voice-giving

is not equivalent to having a voice. She highlights that a voice “needs to be nurtured”

in order to have influence, and involves effort from within the organisation (Cornwall,

2008, p. 278). Of importance also, is being prepared for the impact giving-voice may

have. In her seminal paper, White (1996) warns of inevitable conflict which should

arise when the voiceless are given a voice, due to the challenging of power relations.

In fact, it is the absence of conflict within participatory processes which should “raise

our suspicions” (White, 1996, p. 15).

White’s (1996) observations here are compatible with Mouffe’s (1999)

reasoning away from a democratic process based upon notions of deliberation, and

towards an agonistic approach that questions the dominant hegemony and seeks to


transform power relations (Mouffe, 2013, p. 8-9). Mouffe’s (2013) positioning of

conflict highlights:

Conflict in liberal democratic societies cannot and should not be eradicated, since the specificity of pluralist democracy is precisely the recognition and the legitimation of conflict. What liberal democratic politics requires is that the others are not seen as enemies to be destroyed, but as adversaries whose ideas might be fought, even fiercely, but whose right to defend those ideas is not to be questioned. (p. 7)

Within this context, adversary refers to the opponent within democratic politics

“with whom one shares a common allegiance to the democratic principles of ‘liberty

and equality for all’, while disagreeing about their interpretation” (Mouffe, 2013, p.

7). Although adversaries might fight against each other within the democratic process,

as each seeks to have their principles become hegemonic, their right to have differing

views, and to contest and defend them is never questioned (Mouffe, 2013). This

“confrontation between adversaries” is what Mouffe (2013) labels the “agonistic

struggle” (p. 7).

White’s (1996) caution of conflict regarding the giving of voice, suggests a

moment of hegemonic acknowledgement, and highlights a need for commitment from

the NPO to genuine participation and organisational learning. Engaging in tokenistic

downwards accountability processes could end up doing more harm than good to both

the NPO, and the beneficiary themselves, through the shutting down of voice, due to

viewing conflict as negative. Embracing an agonistic approach would see the NPO not

be threatened by conflict, but see it as a sign that participation is genuine, dialogic, and

capable of transformation.

A NPO can dialogically reframe how they are accountable, through considering

accountability on the terms of the beneficiaries, as a political player, whose voice, as

a valid adversary, is commonly unheard. Through utilising evaluation processes and


opportunities, as a method of participation of beneficiaries, rights can be actualised

and voice can not only be given to beneficiaries, but it can also be taken by them, as

power is restructured.

3.6.3 Accountability how: through participation.

Mercelis et al. (2016) describe participation as the giving of voice to stakeholders

“through forms of deliberation, consultation and/or mobilisation, designed to influence

planning and decision making” (p. 1448). Participation of beneficiaries is one

component of downwards accountability (Wellens & Jegers, 2016). In their analysis

of the literature, Mercelis et al. (2016), identify various benefits and challenges of

beneficiary participation. This analysis is detailed within Table 3 (below).

Table 3. Benefits and challenges of beneficiary participation within the literature (adapted from Mercelis, Wellens and Jegers, 2016)


In concurring with some of the challenges of beneficiary participation listed in

Table 3, O’Leary (2016) cautions that participative accountability mechanisms that aid

in meeting downwards accountability objectives have been found to be problematic in

terms of their failure to grant beneficiaries the ability to articulate their true and

authentic interests in a meaningful and coherent manner. This might be due to a lack

of examination of the causes of societal inequalities, and an attempt to embed new

participatory mechanisms within pre-existing structures. It is perhaps through a

consideration of the eight principles of dialogic accounting (as detailed within Chapter

Two), within participatory mechanisms aimed at enabling downwards accountability,

that authentic change might occur. Here, beneficiary participative evaluation is a

possible means of enhancing downwards accountability (Ebrahim, 2003b).

3.7 Accountability How: Through Evaluation

Evaluation, which Scriven (1991) defines as a process of understanding the

merit, value, or worth of things, is presented as a tool for enhancing accountability

downwards to beneficiaries (Ebrahim, 2003b). (Also see pages 22 and 58 for more

detailed definitions of evaluation). Yet the relationship between accountability and

evaluation is ambiguous within academic literature.

Portela (2012, p. 74) asserts that accountability and evaluation interweave

dynamically and overlap. Owen (1993, p. 14) lists accountability as a reason for

carrying out an evaluation, and Ebrahim (2005, p. 62) and Portela (2012, p. 74)

describe evaluations as ‘accountability mechanisms’. Similarly, Hoefer (2000, p. 168)

claims that evaluation is an important method to ensure and accomplish accountability.

Hall (2014, p. 309) suggests evaluation can assist in demonstrating accountability to

beneficiaries and donors. Taken further, Benjamin (2008) states that “accountability is

a critical public concern and evaluation plays an important role in addressing this


concern” (p. 338).

Illustrated here is a dynamic relationship between accountability and evaluation,

where evaluation is presented as a method of accountability, and accountability is

presented as a product of evaluation. Claims are made that evaluations provide a

critical link between accountability and organisational learning (Ebrahim, 2005) and

that the degree of organisational learning occurring as a result of an evaluation,

depends upon an organisation’s accountability focus (Ebrahim, 2005, p. 62). In

considering the accountability focus, Benjamin (2008), adds that “evaluators need to

consider the accountability relationship in which their work is taken up and given

meaning” (p. 338).

Evaluators also need to be aware of the hegemony within which their work is

situated. In her powerful address, Mathison (2018, p. 115) details how evaluation has

laboured under neo-liberalism for decades. She warns of evaluations reflecting neo-

liberal values of commodification, competition and privatisation, at the expense of the

“public good” (Mathison, 2018, p. 115). Evaluations are used as tools to rationalise

and normalise state actions within neo-liberal strategies (including, new public

management, performance management and impact measurement) (Mathison, 2018).

Mathison (2018) believes evaluations adopt market speak and measure program

success in relation to profit, and that outcomes are conceptualised in economic terms.

In order to move beyond accountability as a tool to meet the neo-liberal agenda,

which supports the principal/agent upwards reporting requirements, an evaluation

orientation that focuses on democratic participation is suggested (Hanberger, 2004, p.

13). This sentiment is also shared by Benjamin (2008) when describing how evaluation

can elevate the developmental and dialogical function of accountability over more

instrumental functions. An evaluation that focuses on democratic participation is also

in keeping with the principles of dialogic accounting theory.


The value of multiple stakeholder inclusion through democratic evaluation

processes is well documented in academic literature (Bryson, Patton & Bowman, 2011;

Mertens, 1999). Taut’s (2008) research provides a review of the literature on

stakeholder participation in program evaluation. However, at the conclusion of the

literature review, Taut (2008) calls for more “rich, detailed descriptions of successful

and unsuccessful evaluations involving stakeholders” (p. 229). Similarly, Mathison

(2018) also calls for evaluations that involve “the poor, the homeless, the diseased” (p.

118) as the clients, and hopes that this might enable evaluations to contribute to public

good.

Hanberger (2004) theorises of the different types of democratic evaluations that

attempt to activate citizenry participation. Here, caution is raised that democratic

evaluations can simultaneously legitimise democracy and reinforce powerful actors in

the policy field (Hanberger, 2004). Similarly, Fetterman, Rodriguez-Campos,

Wandersmand and O’Sullivan (2014) discuss the importance of understanding

different evaluation approaches to stakeholder inclusion in evaluation.

Examples of research involving marginalised stakeholders includes Baur, Abma

and Widdershoven’s (2010) use of responsive evaluation to study the challenges of

involving stakeholders with unequal relationships in evaluation. They find that

organisations must find ways to place clients’ (or beneficiaries’) stories about their

“daily experiences on the policy agenda” (p. 242). Davidsdottir and Lisi (2007) use

case study methodology to research four Icelandic schools. They found that evaluation

is optimal when schools take a democratic stance. Ryan and Destefano (2001) used

case study methodology to research dialogue as a democratising practice within

evaluation. They found that evaluators must privilege the dialogic process in order to

keep it going, and for dialogue in evaluations to be “rich and perplexing” (p. 199).


Hreinsdottir and Davidsdottir (2012) use the deliberative democratic evaluation

approach to involve preschool aged children in evaluation research, in order to listen

to their views and values. Here, the process of involving marginal stakeholders enabled

the revelation that the children had other priorities than the parents and staff. House

(2006) argues that evaluation is political. House (2006) presents research that

approaches evaluation from a democratic stance, that involves face-to-face meetings

with key stakeholders. House (2006) argues that democratic evaluation can increase

transparency and participation, and reduce the reinforcement of powerful actors in the

policy field.

In learning from the evaluation literature, if NPOs are to embrace the importance

of downwards accountability to their beneficiaries as a significant stakeholder group,

the inclusion of beneficiaries in the evaluation process could be a place to start. After

all, framing an evaluation from the perspective of the beneficiary, would allow

“different conclusions about an organization’s effectiveness” (Ebrahim, 2005, p. 63)

to come about.

Evaluations are essentially about making a judgement (Greene, 2013). Unlike

traditional accounting, which exists in a positivist paradigm, seemingly capable of

being objective, rational and neutral (Vinnari & Dillard, 2016, p. 25), or complete,

neutral and free from error (AASB, 2016), evaluations invite the questioning of on

whose terms are we accounting. Evaluations invite subjectivity and bias, and summon

a repositioning of qualitative characteristics like relevance, faithful representation,

usefulness, comparability, understandability, timeliness and verifiability (AASB,

2016) towards for what, to whom, how, and on whose terms. For the purpose of this

research, the following quote by Greene (2013) is used to position the understanding

of evaluation:


Evaluation is the systematic process of gathering and interpreting empirical information in order to make a judgement of quality and worth about the program, policy, or practice being evaluated. As a judgmental practice, evaluation is always saturated with values. Evaluation always serves someone’s interests but not the interests of others. (p. 750)

In summary, through beneficiary participation in evaluation, it is theorised that

the voice of the beneficiary can be instrumental in shaping new directions of

accountability, away from linear models (upwards, downwards, sideways, inwards),

to something more circular and three-dimensional (even spherical). This gives rise to

accountability that is dialogical, reciprocal, and open, and that makes visible the

dominant hegemony, whilst at the same time presenting counter hegemonies as

possible.

3.7.1 Participatory evaluation.

Participatory evaluation, has been defined as an evaluation process which

“involves program staff and participants in all aspects of the evaluation process to

increase evaluation understanding and use while also building capacity for future

inquiries” (Patton, 2015, p. 220). Conroy (2005) acknowledges the importance placed

on stakeholder involvement in evaluations of NPOs by both evaluators and the NPO

constituency, highlighting that “there has been a growing acceptance of participatory

evaluation where … beneficiaries are the key actors of the evaluation process, not the

mere objects of the evaluation” (p. 110-111). Chouinard (2013) confirms that

participatory evaluation is about the engagement of participants in the evaluation

process, but adds that it is not defined by any specific set of techniques or methods. It

considers whose voice to include, how, and who will speak for them (Chouinard,

2013). From this perspective, participatory evaluation is indeed a political process

(Chouinard, 2013).


Like Wellens and Jegers (2016), Ebrahim (2003b) also claims that downwards

accountability can be enhanced through participatory evaluation, which combines the

tools of evaluation with the processes of participation. He suggests that the

accountability mechanism of participation can be used to create systematic

involvement of beneficiaries in the evaluation of NPOs (Ebrahim, 2003b, p. 819).

While Ebrahim (2003b) argues that evaluation is an important mechanism of

accountability, he also raises a series of barriers to the implementation of evaluation

within NPOs including

• conflicts amongst NPOs and funders about whether the evaluation should be measuring processes or products;

• the relevance of evaluation for NPOs including the time and cost to conduct the evaluation;

• the tendency of donors to focus on discrete projects rather than long term processes;

• the limited staff available; • the rewarding of successful projects by funders, and punishment of riskier,

innovative approaches; • the tendency to equate evaluation with performance measurement or

assessment.

Thus, while acknowledged as a mechanism of accountability, evaluation is seen

within the literature to be difficult to implement. However, as Ebrahim (2003b, p. 819)

suggests, through participatory evaluation, complex downwards accountability can

develop. This approach supports democracy, and arises from questioning the power

that “derives from access to evaluation and the implications for society if only the

powerful have such access” (Patton, 2015, p. 223). After all, as Ebrahim (2003b, p.

819) emphasises “without some mechanism for addressing unequal power relations,

participation appears unlikely to lead to downwards accountability”.

Therefore, the evaluation method of participatory evaluation (Greene, 1997) can

assist NPOs to engage in downwards accountability practices by giving voice to


beneficiaries through involvement in the evaluation processes of their service.

Specifically, participatory evaluation can provide a process to meet the accountability

needs of beneficiaries, and help to overcome the barriers raised by Ebrahim (2003b).

Considered of importance within a NPO’s beneficiary participative evaluation

framework, is the inclusion of processes that enable the closing of the feedback loop

(Jacobs, 2010; World Vision UK, INTRAC, Social Impact Lab, CDA, 2016). Here,

importance is given to a 3-step process of:

1. Gathering beneficiary evaluative data via an evaluation instrument.

2. Using this data to enable organisational learning.

3. Communicating a response back to the beneficiary.

(World Vision et al., 2016, p. 2).

If any of the steps in the process are missing, the feedback loop is said to be

broken. The importance of closing the feedback loop within beneficiary participation

in evaluation, is that doing so ensures beneficiaries’ voices, heard through the

evaluation instrument, are used to enable learning and development by the NPO.

Importantly, this learning and development is then reported back to the beneficiary.

The risk without such a loop, is that involving beneficiaries in evaluation could become

the end goal (Gigler, Bailur, Naylor, Pradhan, & Rajani, 2014), resembling a feel good

exercise. Instead, the aim is for the NPO to use the beneficiaries’ evaluation to

contribute to change and organisational learning.

3.7.2 Deliberative democratic evaluation.

As mentioned earlier (in Section 3.7), within evaluation scholarly literature,

there exists a type of evaluation which takes a deliberative democratic approach

(Davidsdottir & Lisi, 2007; Ryan & Destefano, 2001; Taut, 2008). Developed by

House and Howe (2000) the deliberative, democratic approach to evaluation centres


on the principles of inclusion of all stakeholders, dialogue amongst evaluators and

stakeholders, and deliberation amongst all parties to reach conclusions (House, 2006,

p. 124).

As an approach to evaluation which attempts to include stakeholders,

deliberative democratic evaluation has many merits. However, this research takes the

position that an approach which seeks an outcome of consensus through deliberation,

has not considered the inability of less powerful stakeholders (beneficiaries) to express

a voice, or have it heard, in a context where consensus with more powerful

stakeholders (staff) is an aim. Therefore, and in keeping with Mouffe’s rejection of

notions of deliberation within democratic processes, this research advocates

theoretically informing its evaluation framework with the principles of agonistic

pluralism, and agonistic democracy. In doing so an agonistic approach to democratic

evaluation is proposed, or what could be termed - Agonistic Democratic Evaluation.

However, in order to move beyond the sham-ritual (previously discussed within

Section 3.3.1) identified by Najam (1996, p. 346), and towards an acknowledgement

of the power imbalances that drive and support the dominance of upwards

accountability, the need is to move past mere participation, and towards transformative

participation (O’Leary, 2016).

3.7.3 Transformative Participatory Evaluation.

Transformative participatory evaluation is defined within the scholarly literature,

as participatory evaluation approaches that see the evaluation as an intervention that

“itself seeks to change a given social order based on particular notions of social justice

or democracy” (Dahler-Larsen, 2018, p. 877). This consciously brings in the voices of

those most oppressed in order to bring about social change (Mertens & Wilson, 2012,

p. 181). These evaluations seek to include people who are most


marginalised (Mertens, 1999; Mertens & Wilson, 2012, p. 211) and in doing so give

more voice (Guba & Lincoln, 1989) to beneficiaries, thereby embracing principles of

social justice. As such, transformative participatory evaluation has many parallels with

the principles of dialogic accounting.

Standard evaluation practice does not normally consider lived experience and

the promotion of social justice (Whitmore, Guijt, Mertens, Imm, Chinman, &

Wandersman, 2006, p. 340). However, transformative evaluation is presented within

the extant literature as an evaluation process motivated toward another social order

(Dahler-Larsen, 2018, p. 877; Whitmore et al., 2006, p. 340). Within this perspective

“transformative thinking, fairness, justice and democracy cannot wait until the

evaluation is complete but must be embodied in the very evaluation process” (Dahler-

Larsen, 2018, p. 877). Transformative participatory evaluation tries to “create

conditions where participants can empower themselves” (Cousins & Whitmore, 2007,

p. 91). The following key concepts underpin transformative participatory evaluation:

1. Consideration of who creates and who controls the production of knowledge. 2. In terms of process, the distance between researcher and researched is broken

down. All participants are contributors collectively. 3. Importance of critical reflection. Requiring participants to question, to doubt

and to consider a range of social factors. (Cousins & Whitmore, 2007, p. 91).

Evaluations conducted in this way are said to have the potential to be

transformative, because they place “central importance on the lives and experience of

the least powerful and illuminates and questions asymmetrical power relations”

(Cooper, 2014, p. 148). This research aims to give voice to, and empower beneficiaries

within the accountability structures of their NPO, through involvement in evaluation

processes. As such, transformative participatory evaluation is an apt theoretical base.


Transformative participatory evaluation is positioned within the transformative

paradigm of evaluation research (Mertens, 2009). From an ontological,

epistemological and methodological perspective, transformative participatory

evaluation poses a strong alignment to the critical nature of the research. Research

methodologies within both the critical and transformative paradigms seek to critique

hegemony and reveal power imbalances (Campbell & Bunting, 1991, p. 9). Therefore,

not only is there no conflict presented through the juxtaposition of these paradigms,

but there is also great justification for employing transformative participatory

evaluation as a process within this research from a methodological perspective. A

comparison of the critical and transformative paradigms is presented within Table 4

(below).

Table 4. The compatibility of the critical and transformative paradigms


3.8 Summary and Implications

This chapter presents a literature review of academic research within the

discourses of NPO accountability and evaluation. Key concepts which have emerged

from the review include accountability, beneficiaries, stakeholder engagement,

empowerment, downwards accountability, giving voice, participation and

transformative participatory evaluation.

The literature review has revealed that NPO organisations have historically

prioritised upwards accountability to donors and/or government, while espoused

downwards accountability to beneficiaries as stakeholders within NPO organisations

is not widely practised. The importance of downwards accountability to the

empowerment of beneficiaries and organisational outcomes has been highlighted.

Through applying a critical perspective: to the traditional ways of framing

accountability in NPOs (that is, to whom, for what, and how); to the traditional ways

of conceptualising to whom NPOs are accountable (that is, the accountability

directions of upwards, downwards, inwards, and sideways); and to democratic

evaluation theory, the question arises of on whose terms are we accounting and

evaluating? It is from this questioning, that this research is posited, and the central

research question: How can evaluation enhance accountability to beneficiaries, arises.

This conceptual modelling is represented in Figure 2 (below).


Figure 2. Conceptual model of the research

It is posited that by engaging in downwards accountability processes through the

involvement of beneficiaries within the evaluation of their organisations, that

beneficiaries will be empowered and have their voices more strongly heard. At an

organisational level, this will impact upon organisational learning and development,

and service delivery.

Chapter 4. Research Design 65

Chapter 4: Research Design

The purpose of this chapter is to explain the ontological, epistemological, and

methodological, premises of the research. The chapter will also describe the methods

used to gather and analyse the data, in order to answer the research questions and meet

the research objectives.

As explained and justified throughout this chapter, this research is situated within

the paradigm of critical theory. This is an appropriate paradigm because it seeks to

address issues of social justice and marginalisation (Scotland, 2012, p. 13), which is in

keeping with the research motivation. Scotland (2012) describes the critical paradigm

as:

Culturally derived, historically situated and influenced by political ideology, knowledge is not value free. The critical paradigm asks the axiological question: what is intrinsically worthwhile? Thus, the critical paradigm is normative; it considers how things ought to be; it judges reality. The utopian aspirations of the critical paradigm may never be realized but a more democratic society may materialize. (p. 13)

This chapter will progress as follows, Section 4.1 outlines the ontology of the

research, and Section 4.2 explains the epistemological perspective of the research. The

following Section 4.3, discusses the methodology and research design. Section 4.4

presents the data, its means of collection and analysis. Section 4.5 details an overview

of the research paradigm. The trustworthiness and reliability of the research are

established within Section 4.6. Section 4.7 discusses ethical issues and limitations of

the research. Section 4.8 concludes with an overview of the chapter.

66 Chapter 4. Research Design

4.1 Ontology

Ontology is the “study of being” (Crotty, 1998, p. 10) or the study of what is -

what constitutes reality? (Scotland, 2012, p. 9). Whilst being situated within a socially-

constructed ontology, this research takes a critical positioning (Guba & Lincoln, 1998,

p. 210), by acknowledging that whilst reality is constructed, it “has been shaped by

social, political, cultural, economic, ethnic and gender values” (Scotland, 2012, p. 13).

Here, human beings are capable of rational self-critique (Campbell & Bunting, 1991,

p. 9), and reality is historically constituted, produced, and reproduced by people

(Myers, 2009). There is an acknowledgement that “language does not passively label

objects”, but language “actively shapes and moulds reality”, and contains power

relations used to empower or weaken (Scotland, 2012, p. 13).

Rexhepi and Torres (2011) believe critical theory accepts that it is an

intellectual’s role to create a social imaginary:

The creation of social imaginary implies (necessitates) a moral responsibility and a political commitment. A moral responsibility exists to imagine social scenarios through which people can deliberate and construct mechanisms of participation that may expand the workings of democracy and a political commitment to create an autonomous sphere of public debate… which is neither controlled by the market nor controlled by the State. (p. 690)

Rexhepi and Torres, (2011) suggest that critical theory can “herald a liberatory

education that empowers stakeholders” and provides a “means for successful bottom-

up, top-down engagement” (p. 684). When considering the topic of this research as the

need to create opportunities for the participation of beneficiaries, in order to create new

visibilities and new constructions of knowledge through engagement, this is a fitting

theoretical paradigm.


4.2 Epistemology

Epistemology concerns what it means to know, or how knowledge can be created

and communicated (Scotland, 2012, p. 9). This research acknowledges a socially

constructed epistemological view of subjectivity, and the understanding that “social

reality is not separate from us” (Cunliffe, 2008, p. 124). Instead, “knowledge itself is

socially constructed and facts are social products” (Cunliffe, 2008, p. 125).

However, this research frames this understanding within a critical paradigm. It

does this by understanding that knowledge is shaped by societal structures (Campbell

& Bunting, 1991, p. 5) and the dominant hegemony (Patton, 2015, p. 692), thereby

acknowledging the importance of considering power structures. As such, the research

understands that “knowledge is both socially constructed and influenced by power

relations from within society” (Scotland, 2012, p. 13). Knowledge is historically

positioned, entwined in power relations (Wedeen, 2010, p. 260), and marked with

inequality (Scotland, 2012, p. 13).

This philosophical position, which seeks to understand social life and lived

experience (Schwandt & Gates, 2018, p. 344), and address issues of social justice and

marginalisation, is fitting to the purpose research. From within this epistemological

perspective, this research is informed by dialogic accounting theory - drawn from

agonistic political theory (Brown, 2009, p. 315), contemporary democratic theory

(Brown & Dillard, 2015b, p. 965), and transformative participatory evaluation theory

- to situate the research motivation, purpose, and methodology. These theories

emphasise the viewpoints of marginal groups and the interrogation of systemic power

structures, in order to further social justice (Mertens & Wilson, 2012, p. 160).


4.3 Methodology and Research Design

4.3.1 Methodology.

The methodology of critical theory calls the current ideology (or hegemony) into

question, and tries to instigate action in the name of social justice (Crotty, 1998, p.

157). In this process, values and assumptions are interrogated, conventional social

structures challenged, and social action engaged (Crotty, 1998, p. 157). Therefore, the

inquiry is inseparable from the political, and the aim of the methodology is to

emancipate the disempowered (Scotland, 2012, p. 13; Campbell & Bunting, 1991, p.

4). This is in keeping with the previously stated research aim of empowerment and

social justice of marginalised groupings (in this context beneficiaries of NPOs). Within

the critical paradigm, the starting point is often preconceived, “finding out is the

means, change is the underlying aim” (Scotland, 2012, p. 13).

The realisation of this aim of change, within the methodology of the critical

paradigm, frequently involves a level of praxis, enabling an iterative relationship

between the theory, the data, the research questions and the analysis (Scotland, 2012,

p. 13). The recursive relationship within this critical methodology positions this

research as abductive in relation to theory development, as the approach moves back

and forth between the theory and the data (Saunders, Lewis & Thornhill, 2016, p. 148).

Abduction is appropriate for research projects where data is used to explore a

phenomenon, in an attempt to identify themes, and explain patterns that can lead to the

generation of new theory, or the building upon existing theory (Saunders et al., 2016).

As an exploratory research project, seeking to locate themes and patterns within the

data into a conceptual framework (Saunders et al., 2016), and build upon or modify

existing theory (Eisenhardt, 1989), this is a fitting approach. As such, the use of theory

is important in the opening stages of the research to create an initial theoretical

framework which informs the topic and approach (Walsham, 1995, p. 76).


Furthering the discussion on praxis, due to the limited time frame of this research

project, reaching a level of praxis with the beneficiaries is difficult. However, the

second objective of the research is to develop an evaluation instrument which the

research participants can choose to use to evaluate their NPOs. This process creates a

tool for praxis, developed out of the research data and findings, to enable repeated

action informed by reflection, by the NPO and its beneficiaries.

4.3.2 Research design.

For the purpose of this research, an understanding of the research design will

be informed through the following quote by Ponelis (2015). “A research design is the

logic that links the research purpose and questions to the processes for empirical data

collection, data analysis, in order to make conclusions drawn from the data” (p. 539).

4.3.2.1 Method: Qualitative case study.

This research used interpretive qualitative case studies (Keutel & Werner, 2011;

Ponelis, 2015; Stake, 1995; Walsham, 1995) as the research methodology. Case study

is defined as a “research strategy that involves the empirical investigation of a

particular contemporary phenomena within its real-life context, using multiple sources

of evidence” (Saunders et al., 2016, p. 711). Case studies focus on exploring dynamics

present within settings (Eisenhardt, 1989), and are a suitable research methodology for

answering how questions (Walsham, 1995, p. 74; Yin, 2014, p. 9). Recalling the

research question of this study, how can evaluation enhance accountability to

beneficiaries within NPOs, this is a suitable methodology, and it is fitting to the critical

positioning of the research (Ridder, Hoon & Mccandless Baluch, 2014, p. 373).


4.3.2.2 Multiple case studies.

In order to contribute to theory development, the research used a multiple case

study approach. Here, two cases, as empirical units (Patton, 2015, p. 259) of analysis

(Yin, 2014, p. 32), were studied. The use of two cases allows a richer representation

of the uniqueness of each case to be formed (Stake, 2006, p. 121), through creating the

ability to understand each case in juxtaposition to the other. This offers the potential

for a deeper understanding of the processes and outcomes of the cases (Miles,

Huberman & Saldana, 2014, p. 30). Multiple case studies are said to provide for

stronger analytical conclusions than that of a single case study design (Yin, 2014, p.

64).

Extant literature, particularly that from within a positivist standpoint, frequently

advocates for a higher amount of cases to increase the robustness of the research (Yin,

2014, p. 57). However, for the purpose of this research, the focus was on understanding

the experience of actual cases operating in authentic situations (Stake, 2006, p. 3), in

order to develop a rich understanding of the phenomena being examined. As such, it

was decided that whilst two cases would allow for cross case comparison, it would be

restrictive enough to enable an in-depth study of the phenomena within each setting,

in the given limited timeframe. A description of the sampling process used to select

the cases is detailed in the following section.

4.3.2.3 Participants.

This research involved the participation of an industry research partner (hereafter

IRP) to assist in the selection of the two cases. The IRP, is a certified ‘B Corp’

company that provides online information services to NPOs (the IRP clients). ‘B Corp’

companies are “businesses that meet the highest standards of verified social and

environmental performance, public transparency, and legal accountability to balance


profit and purpose” (B Lab, 2018). The IRP was introduced to the researcher through

the research project’s principal supervisor. The principal supervisor had previously

engaged in research with the IRP. The IRP was believed to be a suitable addition to

the research project due to the nature of their service and mission to assist human

services, which are in keeping with the research motivations.

The involvement of the IRP was crucial in gaining access to NPOs that were

independent of the research team. The IRP acted in a facilitative/introductory capacity

by passing on details of the research proposal and ethically approved documents (see

Appendix A) to potential case participants who were their clients. Of the three clients

approached by the IRP, two volunteered to participate in the research. At the direction

of the client, the names and contact details of the chief executive officer of each NPO

were passed on via email to the researcher.

The researcher then contacted each chief executive officer via email and follow-

up telephone call, to arrange a meeting to further assess their willingness to participate,

and their understanding of the research project. The two contacted NPOs became the

research cases. They will hereafter be referred to as Case A and Case B. Neither case

was known to the researcher or the supervisors prior to the commencement of the

research. From this point the IRP was no longer involved in the research project.

At the initial meetings with each case representative, the research aims and

process were discussed. The meetings were used as a means of ascertaining the

willingness of the NPO to voluntarily participate in the research. Once this willingness

was ascertained, practical elements of the interview participants and processes were

discussed. The meeting with Case A included the executive director, another staff

member and the researcher. The meeting with Case B included the chief executive

officer, three other staff members, and the researcher. After each meeting, a secondary


contact person to liaise with the researcher in respect to the research project was

nominated by the executive director and chief executive officer respectively.

4.3.4 Instruments.

In order to understand the dynamics within each case (Eisenhardt, 1989) the

research used two sources of data; primary data in the form of interview transcriptions,

and secondary data in the form of publically available organisational and legislative

documents. Using multiple sources of evidence within each case enables triangulation

of the data, which Yin (2014, p. 119) states is needed to create converging lines of

inquiry, when using case study as a method.

Triangulation of the data in this way, strengthens the confirmability and

credibility of the research (Shenton, 2004, p. 65-72). Shenton (2004, p. 66) highlights

that using supporting data obtained from documents can help explain attitudes or

behaviours of participants, and also assists in verifying details provided by

participants.

4.3.4.1 Interviews.

Interviews are considered a primary source of data in qualitative research and

interpretive case studies (Ponelis, 2015, p. 541; Walsham, 1995, p. 78; Walsham, 2006,

p. 323). Within each case, the research used semi-structured interviews as a data

collection method from participant stakeholders. Saunders et al. (2016, p. 727) define

semi-structured interviews as interviews in which the interviewer begins with a set of

interview themes or questions, but can vary the order or ask new questions depending

upon the context of the interview situation.

Semi-structured interviews are appropriate to the research paradigm and research

questions, due to their ability to make better use of the potential of dialogue as a

knowledge-producing agent (Brinkmann, 2018, p. 578). Brinkmann and Kvale


(2015, p. 6) intensify the definition of semi-structured interviews through adding that

they are “an interview with the purpose of obtaining descriptions of the life world of

the interviewee in order to interpret the meaning of the described phenomena” (p. 6).

Again, this is a good fit to the desire to understand the lived the experience of the

participants espoused within the epistemology of the research.

Denzin (2001, p. 24) discusses interviews as dialogic conversations that connect

us to a larger moral community. Denzin (2001, p. 24) also stresses that “words matter”,

and imagines a world where language empowers. In keeping with these sentiments,

interviews are an appropriate research instrument. They support the transformative

motivation of the research, and the desire to empower beneficiaries through evaluation

frameworks and instruments aimed at giving-voice (or through language). Here,

interview transcriptions will be viewed as “critically empowering texts”, which

“criticize the world the way it is, and offer suggestions about how it could be different”

(Denzin, 2001, p. 24).

4.3.4.2. Interview participants and sampling process.

Within each case, participants from three stakeholder groups were identified as

the key informants of the research. Babbie (2015, p. 189) defines key informants as

people who are well versed in the social phenomena under study, and are willing to

talk about it. The use of three different stakeholder groups as informants strengthens

triangulation via data sources (Shenton, 2004, 66). In this way, “viewpoints and

experiences can be verified against others”, generating a rich picture of attitudes, needs

and behaviours (Shenton, 2004, p. 66). The three stakeholder groups identified are:

1. Beneficiary. 2. Staff. 3. Board.


The research has a focus on hearing the voices of the beneficiary stakeholder

group, and as such, this stakeholder group takes preference in number within the

interview sample sizes. As an intended result, the number of staff and board members

interviewed is less than the number of beneficiaries interviewed. However, across all

stakeholder groups the sample size was dictated by the willingness of voluntary

participants.

Snowball sampling was used to identify participants through initial engagement

with each case’s research liaison person. Babbie (2015, p. 188) highlights that

snowball sampling is an appropriate nonprobability sampling technique when

members of a population are hard to locate, and for exploratory research, such as this.

In this research, each case’s liaison person was asked to pass on the ethically approved

research participation information sheets, interview participation sheets, and research

information flyer (located in Appendix A) to members of each stakeholder group.

Due to the nature of the service delivery within Case A, the liaison person

suggested the most optimal way of gaining interview participants was to attend the

NPO on a specified date when many beneficiaries are present, in order to be able to

conduct interviews using the snowballing process. This proved to be a successful

strategy as five beneficiary interviews were conducted at the NPO site, within that day.

In keeping with a snowballing process, the liaison person then passed on contact details

of staff and board representatives also willing to participate. These additional three

interviews (one board representative, and two staff representatives) were held at the

NPO site, on subsequent dates. Consequently, Case A’s interview participants

consisted of five beneficiaries, one board member and two staff members, totalling

eight interviews. All were conducted face-to-face, audio recorded and within the NPO

site. All possible key informants who indicated a willingness to be interviewed were

interviewed.


In keeping with the nature of the service delivery of Case B, a different approach

to sourcing interview participants was needed. Here, after distributing the research

details to potential participants, the case liaison person emailed the contact details of

all respondents who indicated a willingness to participate in an interview to the

researcher. The researcher then individually contacted each key informant and

arranged an interview time and location that suited them.

In similarity to Case A, all respondents who indicated a willingness to participate

were contacted by the researcher and subsequently interviewed. In Case B, three

representatives of the beneficiary group, two staff members and one board member

were interviewed, totalling six interviews. All interviews were audio recorded. Within

the beneficiary group and at the request of each beneficiary, one interview was

conducted at the interviewee’s place of work, one was conducted in a private

boardroom at the researcher’s university faculty, and one was conducted at a branch

of the NPO. All interviews were conducted face-to-face. Within the staff group, at their

request, both interviews were conducted at the NPO branch site where each staff

member worked. Both were face-to-face. The board interview within Case B was

conducted over the telephone at the request of the interviewee.

The amount of participants interviewed totalled 14 across the two cases. Whilst

a higher number of interviewees was sought by the researcher, the amount of

interviewees was limited by those willing to participate. However, Miles et al. (2014,

p. 31) highlight that within qualitative research, researchers typically work with small

samples of people located within their case. This is in contrast to typical quantitative

researchers who seek statistical significance through having larger samples (Miles et

al., 2014, p. 31). Table 5 (below) lists the interview details for each participant.


Table 5. Details of each interview conducted across the two cases

Table 6 (below) lists each interview participants’ timeframe and role within their

case. The details within this table are referred to in the discussion of the findings in

Chapter Seven.

Table 6. Timeframe of case participation and role of interview participants


At the commencement of each interview, the interviewer (the researcher) began

by reiterating the purpose and objectives of the research, and explaining the details on

the interview consent form. This included discussing the voluntary nature of the

interviewee’s participation, their ability to leave the interview at any time, the process

of audio-recording the interview, the process of transcribing the recording, and their

ability to withdraw from any involvement within two weeks of the completion of the

interview. Once the interviewee indicated a verbal understanding and willingness to

progress, and signed the consent form, the interview was started.

The interviews began by following an interview protocol. Each stakeholder

group had a different interview protocol (located in Appendix B). The interview

protocols outlined a set of interview questions which the interviewer used as a guide

to structure the interview process. In keeping with the flexibility of semi-structured

interviews, the interviewer was able to use these questions to ensure the key themes

were covered whilst also being able to probe more deeply into responses made by each

interviewee. In doing so, each interview was able to become a dialogue between the

interviewer and interviewee, and tangents relevant to the research were able to be

explored as they arose.

Immediately after each interview was conducted, the audio recording was sent

to an independent professional transcription service to be transcribed. The transcriber

signed an ethically approved confidentiality agreement (see Appendix A), thereby

safeguarding and protecting the privacy of the interviewees. Using this service ensured

consistency within the transcribing process across all interviews and assisted in

enhancing the reliability of the research due to the professional nature of the service

and experience of the transcriber. Using the service also assisted in reducing the time

needed to transcribe the audio-recordings which is of consideration within a restricted


research time period (Saunders et al., 2016). Accuracy of the transcriptions was

confirmed by the researcher checking each for errors by matching them to the

recording. All transcriptions were anonymised prior to analysis.

4.3.4.3 Secondary data.

Secondary data, in the form of publically available annual reports,

organisational documents, financial statements, and legislative documentation allowed

for a more in-depth understanding of each case (Keutel & Werner, 2011) and more

significant insights to emerge (Ponelis, 2015, p. 541). The use of secondary data also

enables triangulation of the data to occur (Ponelis, 2015, p. 541). Triangulation helps

to identify the multiple realities present within each case (Stake, 2006, p. 38). In

triangulating the data, the findings will have been verified by more than one source of

evidence (Yin, 2014, p. 121). The secondary data used in each case analysis is

summarised in Table 7 (below). Of note within Table 7 is the differing publically

available documents within each case. All available documents were accessed for the

research. Where the documents differ across the two cases signals a lack of availability

of comparable documents, or a difference in production of internal organisational

documentation.

Table 7. Secondary data used in each case


Data sources were chosen that enabled the answering of the research questions.

Table 8 presents a matching of the data sources to the research questions and the

research objectives.

Table 8. The matching of data sources to the research questions and objectives

4.4 Analysis

The interpretation and analysis involved the formation of case records which

pulled together and organised the data (Patton, 2015, p. 537). The case records

included the raw data and its analyses through thematic and document analysis across

the data set, with the aim of generating a picture of each case in relation to the research

topic. These were then used to form the case study narratives (see Chapters Five and

Six) which tell a story about each organisation (Patton, 2015, p. 538) in relation to the

research objectives. According to Patton (2015, p. 538) the case study narrative is a

descriptive story about the organisation, which allows the reader to understand its

uniqueness.

The analysis concludes with a cross case comparison (see Chapter Seven).

According to Stake (2006), whilst comparison is a “search for the similarities and

differences between the cases” (p. 82), it cannot be limited to a simplistic comparative

description. Instead the across case comparison must attempt to generate a thick

description (Geertz, 1973), which enhances the “situationally and complex interaction


of case knowledge”, and in doing so deepens our understanding of the individual cases

(Stake, 2006, p. 83).

In order to ensure the confidentiality of each case within the narratives, when

referring to information obtained from publically available (online via websites such

as the ACNC website, and the organisation’s website) legislative and organisational

documents will not be referenced by their full title but by substituting ‘Case A’ or

‘Case B’ in place of the organisation’s name. Similarly, to ensure confidentiality of the

interview participants, any reference to direct quotes will be abbreviated to refer to the

case letter (A or B), the stakeholder group (B – beneficiary, S – staff or Bd – board) and

a random number to represent the person (see Table 5 for a list of interview

abbreviations used). Individuals will also be referred to as s/he rather than he or she

because in some stakeholder groups only one male beneficiary was interviewed.

Referring to that person as him would enable identification of that individual to persons

within the case.

4.4.1 Thematic analysis and coding.

Both the primary and secondary data sets were analysed using thematic

analysis. To distinguish between the primary and secondary data within the findings

and discussion of the research, the thematic analysis of the secondary data is referred

to as the document analysis. Thematic analysis is a method used to analyse qualitative

data. It involves the search for themes and patterns across the data set (Saunders et al,

2016, p. 729). The analysis was assisted by the use of the QSR Nvivo software

program. Qualitative data analysis software tools such as QSR Nvivo, are said to be

helpful in supporting data analysis, arrangement and management (Ponelis, 2015, p.

545), and in justifying findings by providing evidence to link findings back to the

original data source (Keutel & Werner, 2011, p. 11).


To help strengthen the confirmability of the research, the analysis followed the

phases of thematic analysis suggested by Braun and Clarke (2006). These phases are:

1. Familiarise yourself with the data.

2. Generate initial codes.

3. Search for themes.

4. Review themes.

5. Define and name themes.

6. Produce the report.

After listening to recordings and reading the transcripts and documents

repeatedly, to become familiar with the data, the interview transcripts and secondary

documents were uploaded to QSR Nvivo and analysed through coding to a

predeveloped list of theory-generated codes. During the coding process, new inductive

codes were developed as they arose from the data. Three initial rounds of thematic

analysis through coding took place in order to develop a deep and rich understanding

of the data. Through these rounds, patterns and commonalities within the initial codes

emerged. These were identified as leading themes within the analysis. Codes were

grouped to form the themes of the research. As coding rounds progressed, patterns

within these themes emerged, and they were subsequently grouped according to their

relationship to the three secondary research questions. An additional theme was

identified and labelled organisational information. Coded references within this theme

were used to deepen the researcher’s understanding of each case, and to develop the

information contained within the organisational structure (Sections 5.1 and 6.1) of the

Case Narratives. Table 9 below presents the coding hierarchy of the research.

Definitions of each code are provided within Appendix C.


Table 9. Coding Hierarchy – Codes and Themes of the Analysis

After the initial reduction coding, based upon the coded results across all three

stakeholder groups and the secondary data, a further round of reduction coding was

initiated. This round was to engage in reduction coding and analysis within each

separate stakeholder group, within each case, separately. The purpose of this round of

reduction and analysis was to pick up upon nuances particular to each stakeholder

group within their unique case setting. These results were then analysed across the

cases but within each stakeholder group, to discover similarities and differences

amongst each stakeholder group.


In order to see the data from multiple viewpoints and assist in minimising

researcher bias in the analysis process (Leech & Onwuegbuzie, 2008), tables

displaying the frequency of coding references to particular codes were generated using

QSR Nvivo software. These are used throughout the Case Narratives and the Cross

Case Comparison Chapters. Although not typically a step used in thematic analysis,

using multiple types of analysis in qualitative research is suggested as a means of

further triangulating the results (Leech & Onwuegbuzie, 2008). Numerically

representing qualitative themes in scores such as this, is said to help interpret and more

fully describe a phenomenon (Sandelowski, 2001, p. 231). It is hoped that through

juxtaposing the qualitative abductive analysis with references to the coded frequencies

of each code across stakeholder groups and across cases, that a richer analysis can be

formed, that lessens bias and increases triangulation.

4.4.2 Language structure and triangulation within the Case Narratives.

In order to present the findings of the research with clarity of expression, a

language structure has been employed. The language structure assists in the

identification of data triangulation. The language structure identifies levels of

agreement or similarity across the stakeholder groups and data set, and uses the

following words to represent conditions:

• Confirmation= first level of agreement across two stakeholder groups.

• Alignment= second level of agreement across three stakeholder groups.

• Significance= third level of agreement across the entire data set (interviews and documents).


Figure 3 (below) summarises the ontological, epistemological and

methodological assumptions of the research. In doing so the foundations of the

research paradigm are presented.

Figure 3. The ontological, epistemological and methodological assumptions of the research

4.5 Establishing Trustworthiness and Reliability

4.5.1 Credibility.

Mertens (2009, p. 236) positions validity within research which purports to be

transformational as a controversial concept. She prefers to follow Guba and Lincoln’s

(1989) lead, and consider the credibility of research over its validity. In this regards,

credibility is considered when questioning the quality of measurement and data

collection (Mertens, 2009). Similarly, Shenton (2004), also places a focus on the

credibility of qualitative research, rather than its internal validity (a construct more

closely aligned to positivist investigations).


Shenton (2004) presents a list of criteria that a qualitative researcher may use to

promote confidence in their research’s ability to present findings that are congruent

with reality. In order to promote confidence within the credibility of this research,

Shenton’s list will be discussed in the remainder of this section.

First, the adoption of the research methods used within this study are well

established (Shenton, 2004, p. 64). Case studies are a well-known form of social

science research (Yin, 2014). Second, an early familiarity with the culture of the

participating cases was established through conducting preliminary visits to the case

sites and consulting appropriate documents (Shenton, 2004). Third, the researcher was

removed from the case sampling process through the use of the IRP. Although not

purely random, the use of the IRP helps to negate charges of researcher bias in the

selection of case organisations (Shenton, 2004). Fourth, triangulation of methods was

achieved through the use of multiple sources of data, that is, primary data from

interviews, and secondary data from organisational documents and legislative

documents. Triangulation within the data set was also achieved through having a wide

range of informants across stakeholder groups. The use of three stakeholder groups

within the interview participants enables triangulation via data sources (Shenton,

2004). Fifth, tactics were used to help ensure honesty of the informants including

ensuring interview participants knew their participation was voluntary, which ensured

they were willing to take part (Shenton, 2004). Sixth, the researcher used iterative

questioning to enable clarification of information presented (Shenton, 2004). Seventh,

the researcher had frequent debriefing sessions with the supervisory team, where

alternative approaches and perceptions were discussed (Shenton, 2004). In this light,

the researcher also had frequent peer scrutiny of the research through discussions with

other research candidates and academics outside of the research team (Shenton, 2004).

The researcher also attempted to create a thick description of the phenomena under


study (Shenton, 2004). This helps to build credibility through conveying the actual

situation and context more fully. Finally, in an abductive manner, previous research

findings were examined in light of the current study, which helped to evaluate the

research at hand (Shenton, 2004). Engaging in these activities, helped to build the

credibility of the research.

4.5.2 Dependability.

The dependability of the methodology was increased by using the software QSR

Nvivo. This qualitative research tool assisted in treating the research data in an

unbiased manner (Wellens & Jegers, 2017, p. 201). Recording the interviews increases

reliability through its ability to provide a full description of what was said, which

reduces the partiality presented by note-taking alone (Walsham, 1995, p. 78). Having

the recording professionally transcribed by an external party also strengthened the

reliability of the data, as it prevents researcher bias.

4.5.3 Confirmability.

An awareness is evident within qualitative research of the difficulty in obtaining

objectivity due to the dependence upon human perception. Whilst this research does

not attempt to obtain an objective position through its critical positioning, it does seek

to obtain confirmable findings. In order to promote more confirmable findings, steps

were taken to ensure the findings are the result of experiences of the participants, rather

than researcher bias (Shenton, 2004). An example of these steps, is the use of pre-

tested and researched data analysis techniques. This includes using Braun and Clarke’s

(2006) phases of thematic analysis. The use of empirically researched, thematic

analysis processes such as this, helps to build the confirmability of the research.


4.6 Ethical Considerations

This research has been approved by the Office of Research Ethics and Integrity

at Queensland University of Technology (QUT), Approval Number 1700000820. It

was deemed low risk and met the requirements of the National Statement on Ethical

Conduct in Human Research. All processes for seeking research participants followed

the ethically approved agreements, and informants were sought through the use of the

ethically approved forms, found in Appendix A.

During the process of analysing the organisational documents of one case

organisation, it was found that the treasurer of the case organisation was also a research

candidate known to two members of the research team. Upon this knowledge, the

primary researcher reported this to the principal supervisor, and to the QUT Office of

Research Ethics and Integrity, and completed a disclosure of conflict of interest form.

The potential conflict of interest was managed through not involving the treasurer in

the research, including not being an interview participant. In this way this conflict was

addressed. No further ethical issues arose during the research process.

4.7 Summary

This chapter has presented the ontological and epistemological perspectives of

the research, and the research design. The chapter has positioned the research within

the critical theory paradigm, and justified the use of qualitative interpretative case

study as the methodology used to address the research questions. The chapter

progressed to outline the data collection processes undertaken, and the methods used

to analyse the data. The chapter presented the means within which the research sought

to build credibility, dependability and confirmability, and concluded by detailing

ethical considerations.

88 Chapter 5. Case Narratives - Case A

Chapter 5: Case Narratives - Case A

The purpose of this chapter is to present the Case Narrative for Case A. The

chapter begins in Section 5.1 with the presentation of the findings in relation to the

organisational structure of the case. Sections 5.2 to 5.4 present the findings within the

three secondary research questions - RQ 1.1: For what is a NPO accountable to its

beneficiaries, RQ 1.2: How is accountability currently discharged to beneficiaries, and

RQ 1.3: How is accountability on the beneficiaries’ terms practised? Each section

includes a discussion of the thematic analysis findings from the beneficiary group, the

staff and board groups, and the document analysis. This enables triangulation across

the data set, to report for similarities and differences. Section 5.5 presents a summary

of the analysis of the findings across the primary and secondary data and Section 5.6

reports how evaluation can be used to further enhance Case A’s accountability to

beneficiaries. The chapter concludes with a summary of the findings from the Case

Narrative in Section 5.7.

5.1 Findings - Organisational Structure

Established in 2000, Case A is a not-for-profit, public benevolent institution,

incorporated as an association, with charity status. Classified as a financially medium

sized NPO by the Australian Charities and Not-for-profits Commission (Australian

Charities and Not-for-profits Commission, 2018), it has an annual income between

$250,000-$999,999 (Case A, 2017, ACNC AIS).

5.1.1 Mission of the organisation.

The mission of Case A is to provide psycho-social rehabilitation for adults

experiencing a mental illness. For the purpose of this research, these adults are the

beneficiary group. Case A operates five days per week, from one location within an

Chapter 5. Case Narratives - Case A 89

Australian capital city. The NPO rents premises from which it operates, and owns

property as part of its housing program, which provides short and long-term housing

to beneficiaries. Case A has a beneficiary group consisting of approximately 2,100

beneficiaries.

5.1.2 International accreditation.

Case A is part of an international body of NPOs, who all operate under the same

evidence-based rehabilitation model. Membership of this international body is granted

through an accreditation process. All member NPOs agree to operate under the

guidelines and standards of the international body (Case A, International Standards).

The standards operate as a ‘bill of rights’ for beneficiaries of the service, and a code

of ethical conduct for staff, board and administrators. Beneficiaries of the service are

known as members. All beneficiaries are associate-members of the organisation, and

as such are entitled to participate in the governance of the NPO. The International

Standards are reviewed biannually by a committee comprised of beneficiaries and staff

from the NPOs internationally.

The standards detail the structure of the rehabilitation program. The focus of the

day-to-day operation is the way beneficiaries and staff work together to run all aspects

of the operation of the NPO. This is done through organising the operations into ‘work

units’. Each unit has one staff member and many beneficiaries who work side-by-side

to manage the everyday tasks associated with the running of the NPO. All beneficiaries

elect to join a unit where they work during the day, but can change units at any time.

The participation of beneficiaries within a work-ordered day is the central part of the

rehabilitation program.


5.1.3 Staff.

In keeping with the international standards, Case A has enough staff to engage

with beneficiaries within the various work units, but few enough to ensure that

beneficiary participation in the operation of the units is essential. In this way, without

the participation of the beneficiaries, the organisation would not be operational. The

staff members interviewed have a mixture of experience and qualifications within the

social work, health and the corporate business sectors.

5.1.4 Board.

The governing board of Case A is referred to as the Management Committee.

There are nine members on the board, two of which are beneficiary members. A

beneficiary interviewed, who had previously been on the board, stated “our president

actually has been the same since 2000, he’s been the same president…oh he’s a lovely

man” (AB1, 2017). This quote suggests the beneficiary has a positive perception of the

president.

5.1.5 Funding and income.

Case A receives funding from the health department within their State

government, and federal funding through the Australian Government Department of

Health’s ‘Day to Day Living in the Community’ program. This federally funded

program is transitioning to the National Disability Insurance Scheme under which Case

A now falls. As such, the funding streams for Case A will be altered once the National

Disability Insurance Scheme is fully operational (Australian Government Department

of Health, 2018).

The implementation of the National Disability Insurance Scheme is identified as

a major external event in the life of Case A. However, it is internally viewed as a

positive change. The senior staff member interviewed described the National

Disability Insurance Scheme as a “fantastic thing”, that they are “real happy for” (AS2,


2017). This staff member believes Case A is well positioned for the roll out of the

National Disability Insurance Scheme because “It's not our only income, it's going to

[only] be part of our income.” (AS2, 2017). (See figure 10 for a comparison of income

sources.)

Other sources of income for Case A include funds received through donations,

grants and fundraising. In the interviews, beneficiaries detailed being heavily involved

in fundraising events for example sausage sizzles and raffles, as well as the writing of

grants. Case A’s social enterprises also create an income stream for the NPO, as does

rent from its housing program.

5.1.6 Participation of beneficiaries.

In keeping with the mission of the organisation, beneficiaries participate in Case

A through regular, weekly and day-to-day engagement with the service, typically over

many years. In this way, the participation of beneficiaries within Case A is usually

ongoing and over a long-term time period.

Beneficiaries participate in management, are members of the board, and are

involved in budgeting, financial decision-making and fundraising. Other beneficiary

participation includes involvement in recreational events, a transport program, an

employment program, and a housing program that provides housing for homeless

beneficiaries. Under the guidance of the NPO, beneficiaries also operate a social

enterprise that provides a wage to the participating beneficiary. As such, beneficiaries

are involved in all aspects of the governance and operations of the NPO.


5.2 Findings - RQ 1.1: For What is a NPO Accountable to its Beneficiaries?

The findings from the thematic and document analysis in consideration of what

the NPO is accountable for to its beneficiaries are presented within this section.

Initially, the findings from the beneficiary interviews will be discussed followed by

the findings from the staff and board interviews. The section will conclude with a

presentation of the findings from the secondary data. Triangulation of the findings is

presented throughout the discussion using the language structure presented in Section

4.4.2.

5.2.1 Beneficiary stakeholder group.

In establishing the beneficiaries’ perspectives of what a NPO is accountable for

to them, it is important to gain an understanding of the role the organisation plays in

their lives. It is important to understand this from the beneficiary perspective, through

their language and their voice. Frequently the perspective of the role an NPO plays in

beneficiaries’ lives comes from the organisation, rather than from the beneficiary.

When an understanding of how beneficiary views the service is formed, a better

analysis of their accountability needs can be made.

Throughout the thematic analysis of the interview transcripts, it became clear

that Case A is viewed as an essential part of the lives of the beneficiaries.

Overwhelmingly, the beneficiaries of Case A portray a sense of the organisation being

responsible for saving them during difficult, (even suicidal) periods in their lives. This

is illustrated through the quotes in Table 10 (below).

Table 10. Beneficiary Quotes signifying the importance of Case A in their lives


The accountability needs or demands of the beneficiaries of Case A, must

consider the importance of Case A within their lives. An understanding of their

accountability needs can be formed through the beneficiary quote: “members

[beneficiaries] really need to know what’s going on” (AB3, 2017). This quote reflects

the beneficiaries’ need to know what is going on to enable their participation and assist

their rehabilitation. More importantly, understanding what is going on provides

reassurance that the NPO is viable and able to continue providing support to the

beneficiary group. Framed in this way, the NPO is accountable to beneficiaries for

demonstrating their ongoing operational sustainability, in a manner that is meaningful

and understandable to the beneficiary.

Financial accountability is suggested by the beneficiaries to be less essential

because, “there’s some stuff that’s just not relevant to us” (AB4, 2017). However,

accountability to beneficiaries for non-financial matters presents strongly. These

matters include accountability for the overall happenings of the NPO, its structure,

accessibility of information, decision-making, the achievements of the NPO, and the

impact it is having on beneficiaries’ lives.

5.2.2 Staff and board stakeholder groups.

Two of the eight interviews conducted within Case A were with staff members.

The shared perspective of the staff members confirms the beneficiary view that

beneficiaries “really need to know what’s going on” (AB3, 2017). This is evidenced

through a staff member saying that, “we have their [beneficiary] input in everything,

in any sorts of decisions” (AS1, 2017). In confirmation with the beneficiary group, the

one board member interviewed spoke of a focus of beneficiaries needing

accountability for non-financial information.


During the interviews, when upwards accountability responsibilities were

discussed by the staff (for example adherence to the International Standards), they are

framed as being important for the benefit of the beneficiaries. In doing so, an upwards

accountability responsibility, is transformed into a tool for downwards accountability.

This is evidenced through the following quote:

We're ensuring that people are adhering to the standards at an international level and in turn what happens there is that they're actually legally and properly giving those opportunities to our membership. It's kind of like a quality assurance of the program which is very important. (AS2, 2017)

5.2.3 Cross stakeholder group comparison - RQ 1.1: What is a NPO accountable for to beneficiaries?

Table 11 (below) depicts the total coded references to RQ 1.1: For what is a NPO

accountable to its beneficiaries, across the three stakeholder groups. This table, and

others throughout the chapter, display a measure of the frequency of coded references

recorded at each code. The data shows alignment that the downwards accountability

needs felt by the beneficiaries are supported by the actions of the staff and board. The

corresponding qualitative data reveals a theme of financial reporting being not as

relevant as non-financial reporting. However, the beneficiaries’ desire for a higher

level of non-financial accountability is matched by the organisation’s structure, and the

staff and board’s willingness to provide this.

Table 11. Case A coded references across all stakeholder groups to RQ 1.1: For what is a NPO accountable to its beneficiaries?

The beneficiaries of Case A want the NPO to be accountable for information that

enables a dialogue, and helps to build a relationship between the beneficiary and the


NPO. When interrogating the data across all stakeholder groups a two-way

interchange, of beneficiaries needing to receive an account, and staff and board eager

to enable the giving of that account is presented. In this scenario, the staff and board

acknowledge that Case A is accountable to beneficiaries for governance, management,

and the organisation’s longevity. In doing so, accounting and accountability processes

within Case A display a multi-directional, reciprocal dialogue.

In placing an accountability need upon the organisation for information that

enables a dialogue and builds a relationship between the NPO and the beneficiary, the

beneficiaries appear to be calling for a more relational approach to accountability.

Within this scenario, the beneficiaries want to be involved in accountability processes

that build and strengthen organisational relationships. Rather than being discharged as

finite, accountability is ongoing as it cycles from the beneficiary to the organisation,

and back again, consistently strengthening relationships.

This relational approach to accountability that the beneficiaries of Case A are

seeking enables dialogic accountability. It does this by challenging the traditional

norms of accountability, and by requiring an accountability structure that involves

genuine stakeholder participation. In this way, a relational approach to accountability

embraces dialogic accounting theory’s aims of accounting “for things that traditional

accounting ignores” and developing “accounting that acknowledges divergent

ideological positions” (Brown and Dillard, 2015a, p. 250).

5.2.4 Document analysis - RQ 1.1: For what is a NPO accountable to its beneficiaries?

Table 12 (below) presents the coded references from the document analysis

within RQ 1.1: For what is a NPO accountable to its beneficiaries? The analysis reveals

the beneficiary stakeholder group to be represented strongly within the documents,


particularly within the 2015 Annual Report (the most recent non-financial annual

reports available publically). This document consistently references beneficiaries, and

in doing so establishes an environment where beneficiaries are presented as the

significant stakeholder group deserving of an accountability structure. The document

acts as a downwards accountability enabler, where the NPO provides an account of the

value it places upon the beneficiaries. Little space is allocated to upwards or sideways

accountability. Included are accounts given by beneficiaries. Here, the beneficiaries’

accounts are presented alongside the president’s and CEO’s.

Table 12. Case A document analysis - RQ 1.1: For what is a NPO accountable to its beneficiaries?

The constitution for the NPO lists the objectives of the association. These

objectives detail the beneficiaries as the key stakeholder group to whom an account is

owed. Half of the 14 objectives of the organisation are dedicated explicitly to the

organisation’s responsibility to beneficiaries, including beneficiaries’ rights, inclusion,

access and equality, and how the organisation will advocate and act on their behalf

within the wider community. For example, Case A will “provide equal access for all

[beneficiaries] to every opportunity, and to guarantee the rights of [beneficiaries] to a


place to come, to meaningful work, to meaningful relationships, and to a place to

return” (Case A, 2008, Constitution document).

The Member Handbook confirms beneficiaries are to be called members, not

patients or clients. It also sets the directive that beneficiaries and staff work side by

side, and share responsibility for all aspects of the organisation (Case A, Member

Handbook). These ways of interacting with the beneficiary group institute a unique

downwards accountability need within Case A. Here, account giving and receiving

(between beneficiaries, and the staff and board), again supports a dialogic approach to

accountability.

A focus on accountability for financial information is evident within upwards

accountability reports (ACNC Annual Information Statements, and the audit reports).

These documents have an emphasis on providing financial data to government and

other reporting bodies. Conversely, the organisational documents focus on providing

non-financial accountability. This includes a presentation of accountability for the

impact the NPO is having on the beneficiary. It is evident within the 2015 Annual

Report, that the NPO demonstrates responsibility for accountability to the outcomes of

its program to beneficiaries.

The document analysis of the secondary data substantiates the thematic analysis

of the primary interview data. Strong alignment is demonstrated within the

organisational documents to confirm the words of the beneficiaries, staff and the board.

5.2.5 Summary - RQ 1.1: For what is a NPO accountable to its beneficiaries?

There were many key findings found within RQ 1.1: For what is a NPO

accountable. Firstly, the beneficiaries seek Case A to be accountable for information

that enables a dialogue and builds a relationship between the NPO and its beneficiaries.


This presents as a relational approach to accountability. In addition, the beneficiaries

seek Case A to be accountable for the impact the NPO has on their lives. Also, it was

found that non-financial accountability is more relevant than financial accountability.

Additionally, the findings revealed Case A to be accountable for demonstrating

ongoing operational sustainability to beneficiaries. Finally, upwards accountability

requirements were found to be able to transform upwards accountability reporting into

a tool for downwards accountability.

5.3 Findings - RQ 1.2: How is Accountability Currently Discharged to

Beneficiaries?

The findings from the thematic and document analysis considering how

accountability is currently discharged to the beneficiaries is discussed within this

section. This is important to understanding the current downwards accountability

practices within the NPO, in order to learn how evaluation might be able to enhance

accountability to beneficiaries.


The thematic analysis findings from the beneficiary interviews in relation to

beneficiary participation, mechanisms of accountability, and evaluation, are presented

within this section.

5.3.1.1 Participation.

The discharging of accountability through the participation of beneficiaries

within all aspects of the management and governance of Case A, was detailed within

the interviews. The beneficiaries emphasised the discharging of accountability by the

NPO through verbal communication. All of the five beneficiaries interviewed spoke

of a multi-level system of verbal communication amongst the staff and beneficiaries.

The system enables beneficiaries to be involved in decision-making, and the giving of


ongoing feedback on the day-to-day management of the NPO. There are avenues for

formal or informal, and group or one-to-one meetings.

This communication structure includes

• a morning meeting of all staff and beneficiaries to discuss happenings and raise issues;

• a weekly meeting for all staff and beneficiaries. Here reports are presented by each work unit;

• informal discussions with staff and beneficiaries throughout the day.

Together the beneficiaries interviewed referred to the meetings within Case A

on 33 occasions. The beneficiaries highlighted the significance they place on these

meetings through language such as: “All the information that we need to access, is at

the morning meeting” (AB2, 2017), and “What happens at the meetings definitely does

impact what happens in [Case A]” (AB4, 2017).

Informal, verbal communication was emphasised as an important means of

accountability. All beneficiaries interviewed discussed the value of being able to talk

to any of the staff members and each other throughout the day, stating, “it’s a big thing,

the [Case A] communication” (AB1, 2017). When asked if there was a way to provide

feedback to the NPO, one beneficiary responded “I’d just go up and talk to [the CEO],

and tell him how I feel!” (AB2, 2017). These avenues of one-to-one communication

are important to one beneficiary interviewed because s/he feels uncomfortable talking

in a large meeting setting, but comfortable talking one-to-one with a staff member.

The beneficiaries discussed other avenues for participation within Case A. These

are listed within Table 13 (below). The beneficiaries detailed non-verbal methods the

NPO uses to give information, including a quarterly newsletter, email, social media

pages (e.g. Facebook), and the organisation’s website.


Table 13. Avenues of beneficiary participation within Case A

Two barriers to participation were identified by beneficiaries. The first was

writing. The second was talking in a large group setting, to quote, “but if I have to talk

in front of that table full of people looking at me, I sort of clam up a bit” (AB5, 2017).

These barriers suggest that informal participation in evaluation processes via group

meetings is not suitable for all beneficiaries. Similarly, formal participation in

evaluation processes that require written feedback is also not suitable for all

beneficiaries. The importance is to have a mixture of options available to beneficiaries,

dependent upon their skills and needs.

5.3.1.2 Mechanisms of accountability.

Beneficiaries spoke often of ways that they give and receive information within

Case A, confirming a strong giving and receiving of accounts. As described within the

organisational structure (Section 5.1), accountability to beneficiaries is discharged

through beneficiary representation on the board of the NPO. Two of the beneficiaries

interviewed had previously been board members, one of whom was the secretary.

5.3.1.3 Evaluation.

When asked of the current evaluative means within Case A, the beneficiaries

emphasised the verbal giving of feedback via the structured meeting times and through

informal dialogue with staff. In addition to the verbal giving of feedback, one

interviewee referred to an evaluation of staff by the beneficiaries, where a “member

[beneficiary] is chosen to evaluate a certain staff to see how they think they’re meeting


everyone’s needs” (AB1, 2017). Although one beneficiary referred to a written

feedback form which she had completed once in the year she has been involved with

the NPO, the four other beneficiaries did not speak of this.

When questioned on whether a form of written feedback or evaluation would be

of value to the beneficiaries, two of the five beneficiaries agreed. Of the three that did

not agree, one beneficiary stated that not all beneficiaries would be comfortable with

writing. Another thought having a written feedback form was not needed unless it was

for a complaint, and the other felt that written feedback was not necessary, due to all

the other verbal feedback opportunities currently in place. To quote, “because we kind

of give feedback every week, but we give feedback after every activity, after every

event. So, it’s not in the form of an evaluation, but it is feedback in that it gets minuted,

it gets documented” (AB4, 2017).

The beneficiaries’ report that feedback and evaluation occurs on a day-to-day

level through verbal communication. However, written evaluation of the NPO by the

beneficiaries occurs infrequently. An opportunity to provide written evaluation, as an

additional evaluative element for those that prefer writing and/or are not comfortable

in expressing themselves publicly, has been identified. This could further strengthen

the existing portfolio of means in which beneficiaries are able to provide their

account/s.


The thematic analysis findings from the staff and board interviews in relation to





The communication and participation practices detailed by the staff and board

members aligns with the accounts given by the beneficiaries. Both staff and board

members made reference to participative activity or methods frequently. In

confirmation with the beneficiary group, staff and board members highlighted the

extensive meeting processes. The board member confirmed that “everything is

discussed” (ABd1, 2017), and stated that beneficiary “input is more important than

anyone else’s, we encourage that” (ABd1, 2017). At meetings, if a decision needs to

be made, it is made by a consensus process, “if the majority can live with it…then

that’s the decision and we will go with that” (AS2, 2017), and it “always comes down

to consensus” (ABd1, 2017).

Both staff members discussed the importance of one-to-one conversations with

beneficiaries. This is important in building relationships, and also for those

beneficiaries not comfortable to talk in a group setting. Staff highlighted the

importance of beneficiaries forming relationships with each other in order to build self-

esteem and confidence, which in turn allows them to feel “comfortable to speak out”

(AS1, 2017).

When asked if the staff members could identify any barriers to participation by

the beneficiaries, one staff member thought that travel and a lack of transportation

could impact on beneficiaries. S/he also thought, that attendance at board meetings,

could be impeded due to their night time scheduling. Overwhelmingly both staff

members felt that participation by beneficiaries was a positive thing.


Both staff and board members discussed the multiple mechanisms the NPO has

in place which support the discharging of accountability to beneficiaries. For example,


involvement in board meetings and the AGM, budget development, accessing annual

reports, having a voice in the activities of the organisation, chairing meetings, minute

taking, and contributing to a regular newsletter. “Every piece of the [Case A] day-to-

day running has member [beneficiary] involvement. There isn’t one part of the [Case

A] that’s not accessible to a member [beneficiary]” (AS2, 2017).

The staff and board interviews provided alignment with reports given by the

beneficiaries in relation to beneficiaries’ participation within the governance structures

of Case A. One staff member reported that beneficiaries participate at all levels of the

NPO including sitting on the board, making executive decisions and policy

amendments. The board member spoke of the beneficiaries on the board being able to

pass information back to other members, “a member can go up to the board member

and say - Did you ask about – whatever? What happened?” (ABd1, 2017). The board

member suggested that in knowing there are beneficiaries on the board, other

beneficiaries feel more secure. In confirmation with the staff reports, the board

member stated that currently a policy review process is in place. The policy review

committee includes five to six beneficiaries. This provides further evidence of

accountability being discharged downwards.

The board member discussed processes that enable the giving and receiving of

accounts, in a reciprocal relationship between the NPO and the beneficiaries. This

illustrates the opening up of accountability processes within a dialogic structure. A

dialogic structure embraces the principles of a dialogic approach to accountability

including recognising multiple ideological orientations, including stakeholder

participation, embracing subjectivity, being attentive of power relations and being

accessible by non-experts (Brown, 2009). Rather than being duty based, downwards


accountability mechanisms in Case A are processes that encourage and support an

exchange of accounts through dialogue and participation.

5.3.2.3 Evaluation.

In confirmation with the beneficiary statements, the staff and board of Case A

detailed a consistent informal, evaluative environment within the NPO. When asked

of beneficiaries’ involvement in formal evaluation processes, one of two staff members

and the board member referred to the use of an annual evaluation survey. This confirms

the beneficiary who reported completing an evaluation survey. When asked what

information the staff think is important to gain from a beneficiary, the questions in

Table 14 were put forward. Here, a theme of beneficiary empowerment is presented.

Table 14. Case A staff member’s questions for an evaluation instrument

The dialogic structure of accountability within Case A is reflected in the

organisation’s approach to evaluation. The staff and board members presented barriers

to participation in evaluation processes, including poor literacy skills, “IT know-how”

(ABd1, 2017), and the medications that affect cognitive processing.

5.3.3 Cross stakeholder group comparison RQ 1.2: How is accountability currently discharged to beneficiaries?

Table 15 (below) displays coded references across all stakeholder groups within

RQ 1.2: How is accountability currently discharged to beneficiaries? Here a pattern

can be observed where all groups have coded strongly to the participation theme, when

compared to the evaluation theme. Proportionally, and with relevance to the number

of participants in each group, the coding at each theme is consistent across the groups.


Table 15. Case A cross stakeholder group comparison within RQ 1.2: How is accountability currently discharged to beneficiaries?

Within Case A, participation is an important means of discharging accountability

to beneficiaries. As observed, the interviewees across all stakeholder groups detail the

importance of beneficiary participation. It is the foundation of the mission of the NPO.

The beneficiaries’ involvement in the management and governance of the NPO enables

their participation within the organisations’ accountability structure.

5.3.4 Document analysis - RQ 1.2: How is accountability currently discharged to beneficiaries?

The document analysis of the secondary data within RQ 1.2: How is

accountability currently discharged to beneficiaries is displayed Table 16 (below).

Strong coded references to the theme participation again reflects the participatory

nature of the organisation, and aligns with the interview data. The analysis of the

secondary data reveals a strong presentation of avenues for beneficiary participation

within the organisational documents, however little reference is made to this within

the upwards reporting – legislative documents.


Table 16. Case A document analysis coding summary from RQ 1.2: How is accountability currently discharged to beneficiaries?

5.3.5 Summary - RQ 1.2: How is accountability currently discharged to beneficiaries?

There were many key findings found within RQ 1.2: How is accountability

currently discharged to beneficiaries. Firstly, the beneficiaries of Case A are highly

participative and have extensive involvement in the governance and management

practices of the NPO. In addition, the beneficiaries seek accountability to be discharged

through verbal communication structures. Also, in order to provide for equity across

the beneficiary group, evaluation instruments should take various formats including

written and verbal. The findings reveal the staff of Case A want to know how the NPO

can better meet the needs of beneficiaries. Finally, a dialogic approach to downwards

accountability necessitates instruments that encourage and support an exchange of

accounts through participation.


5.4 Findings - RQ 1.3: How is Accountability on the Beneficiaries’ Terms

Practised?

This section presents the findings from the thematic and document analysis in

reference to understanding how accountability on the beneficiaries’ terms is practised.

Understanding how accountability is practised on the beneficiaries’ terms will provide

insights into how evaluation can enhance accountability to beneficiaries.

5.4.1 Beneficiary Stakeholder Group.

The discussion within this section begins with a consideration of social justice,

power, empowerment, and impact, from the perspective of the beneficiaries. The

discussion then proceeds to consider these elements from the staff and board’s

perspectives. The section concludes with a discussion of these elements within the

secondary data.

5.4.1.1 Social Justice, Power, Empowerment and Impact.

The opportunity to have a voice is significant when considering how

accountability on the beneficiaries’ terms is practised. All of the five beneficiaries

interviewed discuss the importance of having a voice within the organisation, to quote,

“If members don’t have a say it will fall apart” (AB3, 2017). This statement indicates

the beneficiaries’ voice is heard and listened to, and that this is integral to the NPO’s

success. The value of being heard and listened to is further expressed by a different

beneficiary, to quote, “you can suggest stuff, and you know that you’re actually heard

and that it’s implemented and that you’re listened to” (AB4, 2017). This indicates a

degree of control felt by the beneficiaries within the NPO, through giving voice.

The beneficiaries spoke of the transformative impact the NPO has had upon

themselves personally. Of significance to two of the five beneficiaries is the impact of

feeling needed by the NPO, to quote, “we’re needed here” (AB1, 2017), “the fact that


you need to feel wanted and needed, it makes you want to come back and it makes you

want to participate” (AB4, 2017).

The beneficiaries consistently use language that demonstrates their

empowerment within Case A, for example “I can have an idea and formulate it myself

here” (AB1, 2017). Four of five beneficiaries interviewed exhibit a high degree of

empowerment within Case A which could be attributed to their level of participation,

which leads to increased control and ownership. The only beneficiary interviewed that

did not speak of feelings of empowerment has been part of the service for a short period

of time (one week). This infers a relationship between empowerment and the length of

beneficiary participation.

Accountability is said to be empowering when it opens a NPO up to a degree of

control by its beneficiary stakeholders (Kilby, 2006, p. 953). Feelings of beneficiary

empowerment appear to arise within Case A through the increased control the

beneficiaries feel. This has a transformative impact on the beneficiaries’ lives through

increased beneficiary self-confidence. This relationship between beneficiary voice and

empowerment can be illustrated within Figure 4 (below).

Figure 4. The relationship between beneficiary voice and empowerment within Case A.

Case A’s rehabilitation model is structured to transform beneficiaries’ lives. The

inclusion of the beneficiaries within the accountability structures of the organisation is

part of the rehabilitation model that enables downwards accountability to be enacted

on the beneficiaries’ terms. In this process, Case A’s downwards accountability is


transformed. The empowered beneficiaries are now able to seek accountability to

themselves from the staff and board.

Within the thematic analysis, no coded references have been made to the codes

control, disempowerment or neo-liberalism within the beneficiary interviews. This

absence of coding is compatible with the high level of empowerment through

participation the beneficiaries feel.


The discussion within this section presents a consideration of social justice,

power, empowerment, and impact, from the perspective of the staff and board.

5.4.2.1 Social justice, power, empowerment and impact.

Within the interviews, the two staff of Case A advocated for beneficiaries to have

a say in the operation of the organisation. In acting upon this desire, and in

confirmation with the beneficiaries, the NPO is structured to enable the giving of voice

to the beneficiaries. The giving of voice in this context enacts accountability on the

beneficiaries’ terms. This claim is evidenced by the following staff quote, “They [the

beneficiaries] have buy in to it [the NPO], so it’s once again, it’s their [organisation],

it’s their space, it’s their opportunity to have their say” (AS2, 2017).

The board member spoke of the importance of beneficiaries being heard and

listened to. This is in confirmation with the beneficiary group. “You know, when

members know that they've been listened to, when they know that their opinion is

listened to and taken on board, I think that gives them a bit of self-confidence, too, and

builds their self-esteem.” (ABd1, 2017). In confirmation, one of two staff members

also emphasised the building of relationships, as important for beneficiaries’

“confidence” and “self-esteem” (AS1, 2017).


The words of the board member align with the staff and beneficiaries, in relation

to the impact the highly participative organisational structure has in generating

accountability on the terms of the beneficiaries. The board member believes

beneficiaries feel increased ownership as a result of their high level of participation in

the giving and receiving of accounts. Both staff reported their equality with

beneficiaries, to quote, “we walk the partnership with them” (AS2, 2017). This

equality appears to contribute to the practice of accountability on the beneficiaries’

terms. Sentiments of equality, empowerment through confidence, and transformation

were expressed by both the staff and board.

Within Case A, beneficiary equality with staff, enables the giving of voice to

beneficiaries. This leads to increased self-confidence and self-esteem within the

beneficiary, and subsequent empowerment. This empowerment is transformative in its

impact on the beneficiaries’ lives. This relationship is depicted in Figure 5 (below).

Figure 5. The potential impact of accountability when practised on the terms of the beneficiaries

5.4.3 Cross stakeholder group comparison - RQ 1.3: How is accountability on the beneficiaries’ terms practised?

Table 17 (below) presents a summary of the coded references within RQ 1.3:

How is accountability on the beneficiaries’ terms practised, across the three

stakeholder groups. As revealed by the data, empowerment is a significantly coded

theme, within the beneficiary group. This indicates empowerment through

participation is of high importance to the beneficiaries. The frequency of coded

references by the beneficiary group to the code empowerment appears to have an


association to the frequency of coded references to the code social justice, by this

stakeholder group. This could suggest a sense of increased social justice is felt by the

beneficiary group through increased empowerment.

Table 17. Case A cross stakeholder group analysis of coded references within RQ 1.3: How is accountability on the beneficiaries’ terms practised?

It is notable within this table that staff have coded references within the power

theme, and the other stakeholder groups do not. The four coded references to power

by the staff all refer to quotes describing beneficiaries’ lack of control or power within

society in general, and the way Case A attempts to give control and power back to the

beneficiaries. They do this through their rehabilitation program.

The staff group code highly within the social justice theme, indicating an

awareness of social justice issues of inequality, and the importance of giving voice to

the beneficiaries. Although the board group comparatively code at a lower level, an

awareness of the importance of empowering beneficiaries is evident. However, the

impact of empowerment and social justice issues are not as strongly evident within this

group.

5.4.4 Document analysis - RQ 1.3: How is accountability on the beneficiaries’ terms practised?

The document analysis of the secondary data within the main theme: How is

accountability on the beneficiaries’ terms practised is displayed within Table 18

(below). The data shows an absence of coding within the themes empowerment,

impact, power and social justice within the legislative documents. These legislative


documents (ACNC reporting, audit reporting) are largely upwards accountability

mechanisms.

Table 18. Case A document analysis of coded references within RQ 1.3: How is accountability on the beneficiaries’ terms practised?

An absence of coding to this theme within the legislative documents indicates a

lower presence of accountability on the terms of the beneficiaries within upwards

accountability mechanisms. Within the organisational documents there is a high

proportion of coding within the themes empowerment and social justice. No coding

references are made to the theme power within any secondary documents. This reflects

the focus on empowerment within these documents. It is understandable that

accountability on the beneficiaries’ terms is more widely evident within the

organisational documents that seek to engage with beneficiaries, in a downwards

approach. However, perhaps it is within upwards accountability mechanisms that

beneficiaries’ voices could be more strongly heard.

Within the secondary data there is evidence of Case A giving voice to

beneficiaries (Case A, 2015 Annual report). Here, three beneficiaries present reports

on their engagement within Case A. Similarly, within the 2017 Newsletter, three

beneficiaries present their experiences of their involvement within the housing


program of Case A. In addition, within the Brochure, one beneficiary is quoted, saying,

“I wanted life back. I needed good people in it. [Case A] has given me all this and

more” (Case A, Brochure).

The voices of the beneficiaries presented within these organisational documents,

confirms the participative structure of the NPO. This is significant across all

stakeholder groups. It aligns with the words of the staff and board members that voices

of the beneficiaries are indeed valued within the organisation. The high number of

coded references to the themes of empowerment and social justice aligns with the

thematic analysis of the interviews.

5.4.5 Summary - RQ 1.3: How is accountability on the beneficiaries’ terms practised?

There were many key findings found within RQ 1.3: How is accountability on

the beneficiaries’ terms practised. Firstly, the analysis revealed the participative

structure of Case A to enable the practice of accountability on the beneficiaries’ terms.

Additionally, the organisational equality within Case A enables the giving of

beneficiary voice, which impacts on beneficiary confidence, empowerment and social

justice. This appears to have a transformative impact on beneficiaries’ lives. Also, non-

financial organisational documents were found to provide strong avenues for

contributing to accountability on the beneficiaries’ terms. Currently, upwards

accountability reporting documents contain little avenue for contributing to

accountability on the beneficiaries’ terms. However, the findings revealed that they

could contribute to providing accountability on the beneficiaries’ terms, if they gave

voice to beneficiaries.


5.5 Summary of Findings Across the Primary and Secondary Data

Table 19 (below) presents the total number of coded references across the data

set, and the secondary research questions. The data suggests that within the beneficiary

group accountability of the NPO to beneficiaries is important, there are many avenues

of beneficiary participation within accountability structures, and accountability on

beneficiaries’ terms is critical.

Table 19. Case A cross stakeholder group analysis of coded references within all RQs

Through presenting the findings of the research across the data, it appears that

Case A engages in a dialogic approach to accountability. Here beneficiaries are

involved at all levels of the NPO. This is enabled by a highly participative structure,

occurring at three levels: governance, managerial and service recipients.

Presented is an accountability structure that goes beyond a downwards giving of

accounts to beneficiaries. In this case, beneficiaries are in receipt of accounts and also

givers of accounts. Beneficiaries are listened to, and their voices impact upon the

service. This has resulted in beneficiaries having a great sense of ownership within the

NPO.

The beneficiaries seek to have an impact upon the organisation, and they seek

the organisation to be accountable for the impact it is having upon their lives. In this

regard, impact is multi-direction. The impact the NPO has on the beneficiary’s life is

reflected in the impact the beneficiary is having on the NPO. In this way,


accountability is multi-directional. This multi-directional impact is illustrated within

Figure 6 (below).

Figure 6. Multidirectional impact within Case A

The dialogic structure of Case A has given rise to beneficiaries feeling needed,

having higher self-esteem and more self-confidence. In Case A, beneficiary

empowerment has increased participation and transformed lives. Here, downwards

accountability is transformed from being one-directional (NPO to beneficiary), to

being a reciprocal giving and receiving of accounts, in doing so, accountability is

enacted relationally. This is depicted in Figure 7 (below).

Figure 7. Transforming downwards accountability into relational accountability


5.6 Evaluation Summary

As discussed, all stakeholder groups believe beneficiaries are able to provide

verbal evaluative feedback through the current participative structure. The

beneficiaries report being involved in the provision of informal verbal feedback to the

staff.

Within formal evaluation processes, Case A has a written evaluation survey for

beneficiaries, which they endeavour to use annually. This 3.5-page survey asks for

both qualitative and quantitative feedback from the beneficiaries on their attitudes

towards, and involvement within the NPO. Examples of questions on this survey

include:

• Why do you attend Case A?

• What makes it hard for you to attend Case A?

• What do you enjoy/not enjoy doing at Case A?

• What would you change at Case A?

As discussed in Section 5.3.1.3, an evaluation instrument could offer an

additional avenue of evaluation, for those who are not comfortable to talk in groups or

publically. It is for this reason that an evaluation instrument is suggested as a means of

extending the already rich participative and evaluative environment of Case A.

5.7 Summary – Case A Findings

The main findings reported throughout the Case A narrative centre on the

beneficiaries long-term and ongoing timeframe of participation with the NPO. This

particular timeframe has resulted in the beneficiaries seeking accountability to be

discharged to them throughout their engagement with Case B. In doing so enabling the

strengthening of the relationship between the beneficiary and the NPO. In this way

accountability is primarily relational.


An evaluation instrument can contribute to the building of the relationship and

the discharging of accountability to beneficiaries if it is structured to be used

periodically throughout the beneficiaries’ engagement with the NPO. Here, the

evaluation instrument is an additional tool within the evaluative environment of Case

A. In seeking accountability throughout their engagement the beneficiaries are

detailing when accountability needs to be discharged to them. When the accountability

needs of the beneficiaries are met, the relationship is strengthened.

Chapter 6. Case Narratives - Case B 118

Chapter 6: Case Narratives - Case B

The purpose of this chapter is to present the Case Narrative for Case B. The

chapter begins in Section 6.1 with the presentation of the findings in relation to the

organisational structure of the Case. Sections 6.2 to 6.4 present the findings within the

three secondary research questions - RQ 1.1: For what is a NPO accountable to its

beneficiaries, RQ 1.2: How is accountability currently discharged to beneficiaries, and

RQ 1.3: How is accountability on the beneficiaries’ terms practised? Each section

includes a discussion of the thematic analysis findings from the beneficiary group, the

staff and board groups, and the document analysis. This enables triangulation across

the data set, to report for similarities and differences. Section 6.5 presents a summary

of the analysis of the findings across the primary and secondary data and Section 6.6

reports how evaluation can be used to further enhance Case A’s accountability to

beneficiaries. The chapter concludes with a summary of the findings from the Case

Narrative in Section 6.7.

6.1 Findings - Organisational Structure

Established in 1986, Case B is a not-for-profit, public benevolent institution,

incorporated as an association, with charity status. Classified as a financially large

sized NPO by the Australian Charities and Not-for-profits Commission (Australian

Charities and Not-for-profits Commission, 2018), it has an annual revenue greater than

$1,000,000 (Case B, 2017, ACNC AIS).

6.1.1 Mission of the organisation.

Case B is a not-for-profit organisation (NPO), with a mission to represent the

concerns of adult recipients of a housing service (hereafter referred to as the

beneficiary of the service) through the provision of specialist advice services, and


advocacy leading to potential law reformation. Case B seeks to improve the housing

related conditions of beneficiaries and to help beneficiaries understand and exercise

their legal rights (Case B, website).

Operating five days per week, Case B is State-based with a central CBD location

within an Australian capital city and 14 regional offices, seven of which are run by sub-

contracted partner NPOs (Case B, Annual Report 2016-2017). Notably Case B has a

documented client base of 15,112 beneficiaries that used the service within the 2016-

2017 financial year (Case B, 2017 Annual Report).

6.1.2 Staff.

Case B employs 23 full-time, 37 part-time and 3 casual staff across its head office

and regional locations (Case B, 2017, ACNC AIS). Both of the two staff members

interviewed for this research have social work qualifications and experience.

6.1.3 Board.

The governing board of Case B is referred to as the steering committee. There

are eight members on the board. Although beneficiaries are able to become members

of the association, this does not occur regularly in practice. There are currently no

beneficiaries holding positions on the board.

6.1.4 Funding and income.

Case B receives government funding from the State Department of

Communities, the Commonwealth Department of Housing and Public works, and the

State Department of Justice and Attorney General. Income is also received through

donations, memberships and grants.


6.1.5 Participation of beneficiaries.

In keeping with the mission of the organisation, the beneficiaries of Case B

participate within the NPO through accessing advice and advocacy services. Typically,

this begins when beneficiaries access the telephone advice service, to help them

respond to a housing related issue. Participation typically ends when the issue reaches

a conclusion, and advice is no longer needed. Beneficiary involvement may reoccur in

the future, if another housing related issuing arises and advice is again sought. In this

way, beneficiary participation is over a short-term time period and is typically

episodic.

Beneficiary participation avenues also include the receipt of community

education resources and activities (Case B Annual Report 2016-2017). The

beneficiaries of Case B are internally referred to as clients.

6.2 Findings - RQ 1.1: For What is a NPO Accountable to its Beneficiaries?

The findings from the thematic analysis in consideration of what the NPO is

accountable for to its beneficiaries, are presented within this section. Initially the

findings from the beneficiary interviews will be discussed, followed by the findings

from the staff and board interviews. Triangulation of the interview findings will be

presented throughout the discussion, using the language structure presented in Section

4.4.2.

6.2.1 Beneficiary Stakeholder Group.

What beneficiaries believe Case B is accountable for to them became apparent

within the thematic analysis of the interview transcripts. Throughout the analysis it

became clear that the beneficiaries did not position accountability for financial

performance highly. This is evidenced within the quote, “I think getting into knowing


what funds and all that kind of stuff might be a little bit over people's heads” (BB2,

2018). One beneficiary stated that s/he did not feel the need to question how Case B

used their funds. Another thought that if beneficiaries had knowledge of the financial

situation it could be discouraging if they did not get a good service, and yet the NPO

was generously funded. In this example, it is suggested that financial knowledge could

in fact disempower the beneficiary.

Rather than seeking financial accountability, the beneficiaries seek

accountability for the quality of the content of the service provided to them, and staff

training. Good staff training was seen as necessary to the provision of a quality service

that would optimise successful beneficiary outcomes. Hence, accountability to the

beneficiaries for quality service provision is taken to be significant. To quote:

Given that they’re legal matters that a lot of the patrons of the service will be unlikely to have a lot of experience with, the legal expertise in the [staff] team would be important, and [staff] presenting the legal obligations and implications of various actions is paramount. (BB3, 2018)

A distinction was made by the beneficiaries between the quality of the service

provision or advice given, and how the service was provided. How the service was

provided was of less concern. This is evidenced through the quotes, “a lot of people

who are dealing with [housing] issues will probably be less concerned with how the

service is provided. It’s having a home and having [housing] issues sorted is a big issue

for a lot of people” (BB1, 2017), and “well they [the service providers] don’t have to

be friendly, but as long as they’re relatable and can provide all the information…to

maximise the successful outcome of the case” (BB3, 2018).

These quotes suggest these beneficiaries are seeking the NPO to be accountable

for the outcome to the beneficiaries, of the service provided. This is of greater

importance than the relationship between the NPO and the beneficiary.

122 Chapter 6. Case Narratives - Case B


In confirmation with the beneficiary group, staff and board consider

accountability for the finances to not be of great concern to the beneficiaries. This is

in alignment across the three stakeholder groups.

The board member discussed the importance of Case B having an operational

structure that enables the ability to have “constant contact” (BBd1, 2018) with

beneficiaries. This structure allows Case B to best represent beneficiaries’ interests.

This presents a direct line of accountability downwards, enabled through dialogue.

It is apparent from the board interview transcript that Case B is accountable for

acting in the beneficiaries’ best interests. The organisational structure that supports

Case B to have “very good exposure to the breadth of [beneficiaries’] issues and

concerns” (BBd1, 2018) through constant, direct contact is an enabler of accountability

downwards.

It is evident through the interviews that staff members’ thoughts on what the

NPO is accountable for to beneficiaries confirms those of the beneficiaries. This is

particularly relevant to the impact the organisation has had upon the beneficiary. Here,

a dialogue between the NPO and the beneficiary emerges, where the beneficiaries seek

accountability for outcomes, and the staff seek accountability for the impact the service

has had upon the beneficiary.

This dialogue resembles an exchange, where the meeting of the accountability

needs of both stakeholders enables a transaction of accountability to occur. One way

of viewing this is that if staff are able to provide the beneficiary with an account for

their outcomes, and the beneficiary is able to provide an account to staff for the impact

the NPO has had upon them, an accountability transaction will have occurred that

meets the needs of both stakeholder groups.


6.2.3 Cross stakeholder group comparison RQ 1.1: What is a NPO accountable for to beneficiaries?

The coding references within RQ 1.1: For what is a NPO accountable to its

beneficiaries, are presented within Table 20 (below). This table, and others throughout

the chapter, display a measure of the frequency of coded references recorded at each

code. As evidenced through the data, there is an alignment of the importance of

downwards accountability across all stakeholder groups. Non-financial accountability

to beneficiaries dominated within the interview transcripts. However, the beneficiaries

suggest that financial accountability can be improved by allowing the beneficiaries to

provide feedback upon the non- financial aspects of the organisation. This is evidenced

through the quote, “I think feedback is always good, and it is public funding, so it’s

always good to know how they’re using public funding” (BB1, 2017). In this quote,

the beneficiary is presenting the importance of beneficiary feedback in validating the

use of public funding. In this sense, s/he is suggesting that downwards accountability

(through beneficiary voice) can enhance upwards accountability reporting (through

financial acquittals) to funders.

Table 20. Case B coded referencing to RQ 1.1: For what is a NPO accountable to its beneficiaries?

6.2.4 Document Analysis - RQ 1.1: For what is a NPO accountable to its beneficiaries?

Coded references to RQ 1.1: For what is a NPO accountable to its beneficiaries,

from the document analysis are depicted within Table 21 (below). Whilst the table

shows a preference towards upwards accountability within the secondary data,


downwards accountability is present within these documents, particularly within the

two annual reports. These largely qualitative reports present more avenues for

downwards accountability than the upwardly focused financial reports aimed at

meeting legislative requirements. This substantiates the thematic analysis finding that

non-financial reporting is more relevant to the beneficiaries than financial reporting.

Non-financial accountability is more strongly coded than financial

accountability within the secondary data. The prioritising of non-financially related

accountability within the 2017-2022 Strategic Plan Summary is significant.

Table 21. Case B document analysis summary RQ 1.1: For what is a NPO accountable to its beneficiaries?

6.2.5 Summary - RQ 1.1: For what is a NPO accountable to its beneficiaries?

There were many key findings found within RQ 1.1: For what is a NPO

accountable to its beneficiaries. Firstly, the beneficiaries of Case B seek accountability

for the quality of the service provided. In addition, the staff of Case B seek

accountability for the impact the service has had upon the beneficiary, and the board

of Case B seeks accountability for acting in the beneficiaries’ best interests. Also, non-

financial accountability is more relevant than financial accountability to beneficiaries.

Finally, transactional accountability was found to occur when the accountability needs

of multiple stakeholders are met during the exchange of accounts.


6.3 Findings - RQ 1.2: How is Accountability Currently Discharged to

Beneficiaries?

The findings from the thematic and document analysis considering how

accountability is currently discharged to the beneficiaries of Case B is discussed within

this section. It is important to understand the current downwards accountability

practices within the NPO in order to learn how evaluation might be able to enhance



The thematic analysis findings from the beneficiary interviews in relation to




The main methods of beneficiary participation within Case B are

• telephone support; • email communication; • face-to-face meetings with Case B staff.

Participation is centred on the beneficiary’s housing related issue, and typically

ends upon its resolution. In this way, participation of beneficiaries is short-term and

episodic. Participation resembles an exchange or transaction of information, rather

than the forming of a relationship. Within the interviews, the beneficiary group did not

speak of negative aspects or barriers to increased beneficiary participation within the

evaluation and accountability structures of the NPO. Similarly, the beneficiaries of

Case B did not speak of any involvement within the board or governance structures of

the NPO. However, the beneficiaries indicated that increased participation in an

evaluative way is important.



The beneficiaries of Case B spoke of methods available to give and receive

information. The information given was in direct relation to the beneficiary’s issue,

and not about the NPO itself. An exception to this was expressed by one beneficiary

when s/he was given a bit of “inside information” (BB2, 2018) in relation to a funding

change. This beneficiary believes that the giving of this organisational information

resulted from the close relationship s/he has built with the staff member. The

beneficiary highlighted that the giving of an account in this way, by the NPO, has been

valuable.

6.3.1.3 Evaluation.

The beneficiaries spoke often of evaluation and evaluative instruments. When

asked about the current means of giving information, feedback or evaluation to the

NPO, all three beneficiaries spoke of a lack of awareness of any beneficiary evaluation

or feedback avenues within Case B. However, all three beneficiaries interviewed were

clear that they would value an opportunity to evaluate the NPO and/or to provide

feedback upon the service they received. This is evidenced through the quote, “I think

some people would feel that it is important to be able to pass on that advice or things,

ideas that could better the organisation” (BB2, 2018).

This quote infers that beneficiary involvement in evaluation improves service

delivery. The beneficiaries all believe they could have spoken to a staff member if they

had had any concerns with the service they were receiving, but did not need to do this.

Evaluation methods suggested by the beneficiaries include a survey, a feedback

sheet, or a form. These methods indicate the beneficiaries believe an avenue for

providing written feedback would be most suitable. The three beneficiaries were clear

in what questions or information should be asked of them, including:


• How do you feel about the service provided to you? • Do you feel that staff were well trained in knowing the legislation

necessary to maximise positive outcomes for the beneficiaries? • Do you believe you received all the information necessary to maximise

the changes of a positive outcome? • Were staff relatable? • Was the information you received presented clearly, and to your

knowledge, was it accurate? • Do you feel like you were well looked after and no gaps were present? • Do you have any concerns? • Are there any areas for improvement? • Do you have any general feedback? • Did you have a positive outcome in your case?

Within these questions, there is a focus on service provision, and the quality of

information provided by the staff. The focus of the account the beneficiaries want to

give to the NPO is not in relation to the actual outcome of their housing dispute. The

beneficiaries separate service provision from positive or negative housing dispute

outcomes. The addition of the final question above, presents as enabling a more

rounded understanding of the beneficiary’s experience within the NPO, rather than

being that of a grading instrument.


The thematic analysis findings from the staff and board interviews in relation to




In confirmation with the beneficiary group, the staff of Case B discussed the

beneficiary participation avenues. In addition to information giving and receiving

methods already presented by the beneficiary group, staff and board spoke of

brochures, posters, stalls at festivals, fact sheets and social media as information giving

mechanisms.


One of two staff interviewed spoke of beneficiaries being able to become

members of Case B and participate in governance structures. The staff member noted

that this does not happen often. S/he spoke of the benefit of having beneficiary

members as:

I guess for me having a strong membership [of beneficiaries] would be a nice way to have their involvement and understand in what we do, and showing also that we're acting in the best interest of the [beneficiaries]. (BS2, 2017)

In contrast to the beneficiary group, staff highlighted many possible barriers to

beneficiary participation in evaluation. These barriers are presented within Table 22

(below). Of prominence are issues around timing and beneficiary ability. In

confirmation with the staff group, the board member also spoke of the need to be

cognisant of timing when requesting an evaluation. The board member presented

limited access to the internet as an additional barrier to beneficiary participation in

evaluation.

Table 22. Case B possible barriers to beneficiary participation in evaluation presented by the staff group


In reference to downwards accountability instruments, one staff member

discussed their “semi-regular client satisfaction surveys” (BS1, 2017) and their need

to complete a beneficiary satisfaction survey this year in order to meet upwards

accountability (funding) requirements. This suggests that upwards accountability

requirements are a motivational strategy to increase the involvement of beneficiaries

within accountability structures. Similarly, the board member identified the potential


for downwards accountability mechanisms to improve upwards accountability

reporting. In doing so, downward accountability through beneficiary participation

meets the organisation’s upward accountability requirements. The impact is of dual

benefit.

This dual benefit is reasonable, however it is important to consider the power

differential between stakeholders (beneficiaries and government/funders) positioned

within a neo-liberal hegemony. This has the potential to corrupt the framing or value

of downwards accountability mechanisms, in and of their own right. One beneficiary

alluded to this sentiment when s/he considered what Case B would find useful to know

from the beneficiary group (in an evaluative sense), in saying “there’s going to be my

view of what I think they would think is important, their view of what they think is

important, and the things that they actually get rewarded; so their deliverables” (BB3,

2018). This statement suggests that the beneficiary believes the organisation’s

deliverables (or its upwards accountability requirements) are more valued than the

beneficiaries’ views of what is important, as this drives their income.

6.3.2.3 Evaluation.

In confirmation with the beneficiary group, the staff spoke often of evaluation

and evaluation processes. Although Case B currently does not provide avenues for the

formal participation of beneficiaries in evaluation, the staff and board members spoke

of past instruments, including a telephone survey. Case B also keep a compliments and

complaints register.

Despite an absence of formal downwards evaluation structures, the staff and

boards members spoke positively of developing such structures and of beneficiary

participation in evaluation in general. The staff detailed benefits of beneficiary

participation in evaluation, including:


• new learning; • gaining a broader insight into Case B’s effectiveness; • knowing the individual beneficiary’s situation better; • improved understanding of the beneficiary group as a whole; • development as an organisation in relation to training, skills and service

location; • better service delivery; • learning how to better respond to beneficiary’s needs; • continuous improvement as an organisation.

The importance of knowing what is being evaluated, and what the evaluation

instrument is reporting, were highlighted by one staff member (BS1, 2017). Both staff

members interviewed reflected on the importance of understanding the difference

between the beneficiary’s satisfaction with the service provided by Case B (or the input

the service has provided), and the actual outcome of the beneficiary’s housing issue or

dispute. The staff thought that these two elements (the input and the outcome) can be

hard to separate in the eyes of the beneficiary. As such, a negative outcome of the

beneficiary’s housing issue would negatively affect the beneficiary’s opinion of the

service provided. Conversely, a positive outcome could positively sway a beneficiary’s

opinion of the service provided. Staff also spoke of the frequent time difference

between the beneficiary’s involvement in the NPO and the outcome. For this reason,

the timing of involving beneficiaries in evaluation is necessary to consider.

Staff suggestions for ways to involve beneficiaries in evaluation include a

written satisfaction survey, a verbal survey and a telephone feedback line. One staff

member believes it is essential to have different feedback modes in order to capture

the different capabilities of the beneficiary group. Here the suggestion is to ask the

beneficiary to identify a preferred mode of feedback.


The staff and board members interviewed provided insights into the type of

information they think would be valuable to be able to receive from the beneficiaries,

and possible questions to do this. These questions are:

• Does the beneficiary feel better informed and more knowledgeable due to their involvement with Case B?

• If the same housing related issued reoccurred, would they be able to act independently to resolve it, due to their interactions with Case B and the knowledge they have acquired?

• Is there agreement between what the staff said they were going to do or achieve with the beneficiary, and what actually occurred? If not, was there a reason for this?

• What were the achievements? • Did the beneficiary fully understand the problem? • Were they given a range of strategies to employ and were they satisfied with

the options? • Did they understand and implement the strategies provided? • Were they satisfied with the communication?

A theme of empowerment felt by the beneficiary as a result of their involvement

with Case B, is presented within these questions (see the first three questions). The

majority of the remaining questions relate to the quality of the service provision and

the beneficiary’s evaluation of it.

6.3.3 Cross stakeholder group comparison - RQ 1.2: How is accountability currently discharged to beneficiaries?

Table 23 (below) presents the coded references within the interview data across

all stakeholder groups, within RQ 1.2: How is accountability currently discharged to

beneficiaries. The low number of coded references suggests beneficiaries have less

awareness of avenues for participation, than the other groups. The comparatively high

number of coded references by the staff group to the theme evaluation suggests a strong

desire by this stakeholder group for increased avenues of participation in evaluation

by beneficiaries.


Table 23. Case B cross stakeholder group comparison within RQ 1.2: How is accountability currently discharged to beneficiaries?

There is considerable alignment within the evaluation theme across all

stakeholder groups. This includes alignment of

• a current lack of beneficiary participation in evaluation;

• the importance of beneficiary participation in evaluation;

• the questioning on an evaluation instrument relating to service quality.

As a point of difference, the staff group seek to understand, through an

evaluation instrument, if the beneficiaries have become empowered due to their

involvement with Case B. This information is seen as important to organisational

learning and improvement.

The staff group demonstrated concern in the beneficiaries’ ability to separate the

quality of the service provided to them, from the outcome of the beneficiary’s issue.

However, in consideration of the evaluation instrument questions posed by the

beneficiary group, this concerns appears to be unwarranted. The beneficiaries’

questions illustrate an ability to separate these two areas, with a focus on an evaluation

of the quality of the service provision.

As a point of difference, the beneficiary group focus on an evaluation instrument

that is in written format; however, the staff group highlight the importance of offering

both written and verbal formats. This difference could be attributed to the staff having

a wide engagement with beneficiaries across all socio-economic and demographic


levels, and knowing the different needs and abilities of their beneficiary group as a

whole.

6.3.4 Document analysis - RQ 1.2: How is accountability currently discharged to beneficiaries?

The document analysis of the secondary data within RQ 1.2: How is

accountability currently discharged to beneficiaries (as depicted within Table 24

below), reveals the annual reports and the Strategic Plan Summary to be central

documents. All other documents present an absence of coded references within this

research question. This substantiates the thematic analysis findings of a lack of

beneficiary participation in evaluation and of downwards accountability processes in

general.

Table 24. Case B document analysis within RQ 1.2: How is accountability currently discharged to beneficiaries?

Within the evaluation theme, beneficiary participation within evaluation is

discussed in the 2016 Annual report. Here reference is made to focus groups that

enabled beneficiary participation on a discussion paper. The 2017 Annual Report

contains a quote illustrating positive beneficiary feedback. Whilst not specifically

focussed on beneficiaries, the Strategic Plan Summary contains objectives of

continuously improving the quality and the accessibility of the service. These

objectives help to illustrate an evaluative environment, striving for improvement. The


inclusion of a beneficiary evaluation instrument, into this evaluative environment is

fitting.

6.3.5 Summary RQ 1.2: How is accountability currently discharged to beneficiaries?

There were many key findings found within RQ 1.2: How is accountability

currently discharged to beneficiaries. Firstly, it was found that beneficiary

participation in Case B is typically short-term and episodic. In addition, beneficiary

participation was found to take a transactional form through the giving and receiving

of information. All stakeholder groups in Case B view the development of a

beneficiary evaluation instrument positively, however there are currently limited

avenues for beneficiary participation in evaluation. Within the evaluation instrument

the beneficiaries want to give feedback on the quality of the service received, and are

able to separate this from their own case outcome, while the staff want to know

beneficiaries’ perceptions of the service quality, and if the beneficiary has been

empowered due through their involvement with Case B. In order to provide for equity

across the beneficiary group, the evaluation instrument should take various formats

including written and verbal. Finally, it was found that downwards accountability can

increase financial accountability and enhance upwards accountability, whilst upwards

accountability agendas can increase downwards accountability structures.

6.4 Findings - RQ 1.3: How is Accountability on the Beneficiaries’ Terms

Practised?

This section presents the findings from the thematic and document analysis in

reference to understanding how accountability on the beneficiaries’ terms is practised.

Understanding how accountability is practised on the beneficiaries’ terms will provide

insights into how evaluation can enhance accountability to beneficiaries.



The discussion within this section begins with a consideration of social justice,

power, empowerment, and impact, from the perspective of the beneficiaries. The

discussion then proceeds to consider these elements from the staff, and board’s

perspectives. The section concludes with a discussion of these elements within the

secondary data.


The giving of voice to beneficiaries within Case B is important across the

beneficiary group. One of three beneficiaries interviewed discussed feeling a lack of

equality within society, arising specifically from the housing-related circumstance that

all beneficiaries of Case B share. This housing circumstance leads this beneficiary to

feel disempowered within society. S/he emphatically stated “we want some justice”

(BB2, 2018). Although Case B’s mission is to help beneficiaries gain social justice,

this beneficiary believes that the giving of voice through evaluation will increase the

positive outcomes of the NPO and lead to increased social justice. This cause and effect

relationship is illustrated in Figure 8 (below).

Figure 8. The potential impact of beneficiary evaluation upon social justice

Being empowered through participation is important to this beneficiary. The

beneficiary believes that her/his feelings of being disempowered could be lessened

through participative evaluation, throughout the progression of her/his involvement

with Case B. It is believed by the beneficiary that ongoing feedback would enable an

exchange of accounts that would strengthen the relationship between the beneficiary

and the staff member. In this way, the beneficiary indicates that ongoing involvement


in evaluation could transform a transactional type of accountability into a more

relational type of accountability. In contrast, the other beneficiaries interviewed, who

do not feel disempowered within society, suggest that participation in evaluation at the

conclusion of their involvement with Case B is optimal. In this way, a transactional

form of accountability remains.

An association appears between the beneficiary’s feelings of disempowerment

and the desire for relational accountability from the NPO (rather than transactional

accountability). Here, it is theorised, that greater feelings of disempowerment (in

society at large) lead to an increased desire for empowerment (from the NPO).

Empowerment is then enhanced through beneficiary participation in ongoing

evaluation. This strengthens the relationship between the beneficiary and the NPO. This

interaction is illustrated within Figure 9 (below).

Figure 9. The association between beneficiary disempowerment and the need for a relational form of accountability

6.4.1.2 Neo-liberalism.

As discussed within Chapter Two, neo-liberal policies are those underpinned by

an intent to draw on market mechanisms and private actors to regulate the economy

and distribute goods and services (Chiapello, 2017). These are the policies upon which

contemporary accounting and accountability practices are based. Within the

interviews, one of three beneficiaries made statements that support the theory that neo-

liberal policies are at odds with the democratic sentiments of participation, which leads

to the isolation and disempowerment of individuals who are positioned within that

hegemony. Private actors that draw on market mechanisms are most often the cause of


Case B’s beneficiaries seeking support from them. In this sense, neo-liberal policies

contribute to the creation of Case B’s beneficiary group. The statements made by the

beneficiary exemplify someone feeling disempowered through neo-liberal agendas. To

quote:

It's a tough industry, and people who are paying out other people's mortgages, and then when the interest rates go down, does the renter ever get to - you know? They don't. You ever heard of rent going down? (BB2, 2018)

A different beneficiary illustrates a possible outcome of neo-liberal policies

through the quote:

The service is ultimately based around needs of the beneficiaries, so them having a quality experience and a positive one is important, but again, it’s not like we’re paying out thousands and thousands of dollars each for it; so I think as always I went into this not expecting the world. (BB3, 2018)

Although unintended, this beneficiary suggests that because the service is

publically funded and s/he is not paying for it, it is appropriate for beneficiaries to

receive a lower quality service, that beneficiaries could expect that. From this

perspective, the beneficiaries, some of whom already feel like they are the “plebs of

the Earth” (BB2, 2018), feel that they actually do not deserve any better.


The discussion within this section considers social justice, power,

empowerment, and impact, from the staff, and board’s perspectives. This assists in

developing an understanding of how accountability is and can be practised on the terms

of the beneficiaries.


The board member discussed the importance of giving voice to beneficiaries, in

order to hear the beneficiary’s point of view. To the board member, listening to the

beneficiary’s voice is “potentially really helpful to us. It’ll improve the quality of what


we do, to continuously improve” (BBd1, 2018). The board member stated the

importance of knowing the beneficiaries’ stories in order to advocate for them to

politicians and the public. To know their stories, their voices first need to be heard.

The empowerment of beneficiaries through their engagement with Case B is of

significance to all the staff and board members interviewed. Beneficiary empowerment

to deal with future housing related disputes, is a key objective of the organisation. This

can be individually through education, or through advocacy leading to law reform.

Within the interviews, staff discuss the powerlessness felt by the beneficiaries, and

how giving voice to beneficiaries can help them have more control. The achievement

of empowerment is presented as a key understanding the staff would like to gain from

a beneficiary evaluation instrument. One staff member suggested that through the

giving of voice to beneficiaries within the NPO, they will feel less helpless, and more

empowered.

The staff acknowledge the importance of providing quality advice in order to

empower beneficiaries, and help them transform their lives. In confirmation with the

beneficiary group, both staff members referred to the importance of staff training in

order to provide quality advice, and optimise the chances of a positive outcome for the

beneficiary.

6.4.3 Cross stakeholder group comparison - RQ 1.3: How is accountability on the beneficiaries’ terms practised?

Table 25 (below) presents a summary of the coded references within RQ 1.3:

How is accountability on the beneficiaries’ terms practised, across the three

stakeholder groups. As can be observed within this table, accountability on the

beneficiaries’ terms is important to the beneficiary stakeholder group. The

beneficiaries’ desire for empowerment and social justice is notable. The impact the


organisation has upon the beneficiary group codes highly to the staff group, indicating

that to them, this is important knowledge to gain. Proportionally, the social justice of

the beneficiaries is highly coded to the staff and board. This presents an alignment of

the importance of social justice across all groups.

Table 25. Case B cross stakeholder group analysis of coded references within RQ 1.3: How is accountability on the beneficiaries’ terms practised?

6.4.4 Document analysis - RQ 1.3: How is accountability on the beneficiaries’ terms practised?

Table 26 (below) presents a summary of the coded references within the themes

empowerment, impact, power and social justice, from the document analysis. A pattern

emerges when observing this data of accountability on the terms of the beneficiaries

being represented within the qualitative reporting of Case B (principally the annual

reports and the strategic plan summary). This suggests financial reporting, such as that

found in audit reports, does not support the social justice of beneficiaries through

empowerment. Nor does it allow for the giving of voice to this stakeholder group.

Table 26. Case B document analysis of coded references within RQ 1.3: How is accountability on the beneficiaries’ terms practised?


The high number of coded references to social justice and empowerment within

the largely qualitative organisational reports (the annual reports and the strategic plan

summary) align with the staff and board perceptions of the importance of beneficiary

social justice and empowerment. This is substantiated across the thematic and

document analysis.

6.4.5 Summary - RQ 1.3: How is accountability on the beneficiaries’ terms practised?

There were many key findings found within RQ 1.3: How is accountability on

the beneficiaries’ terms practised. Firstly, the giving of voice to beneficiaries is verified

as important across all stakeholder groups, and can increase accountability on the

beneficiaries’ terms. Giving voice to beneficiaries was found to increase the positive

outcomes of beneficiaries, their sense of social justice, beneficiary empowerment, and

service quality. Additionally, the beneficiaries were found to feel empowered by Case

B. This empowerment increases the impact on beneficiaries’ lives. It was found that

disempowered beneficiaries can feel empowered through ongoing participatory

evaluation, and ongoing participatory evaluation can transform transactional

accountability to relational accountability. However, it was found that not all

beneficiaries seek relational accountability. Instead, the beneficiaries predominately

seek participation in evaluation at the conclusion of their involvement with Case B, in

a transactional manner. Neo-liberal policies were found to contribute to the creation of

this beneficiary group, and increase beneficiaries’ sense of disempowerment. Finally,

non-financial organisational documents enable accountability on the beneficiaries’

terms, more than financial documents.


6.5 Summary of Findings Across the Primary and Secondary Data

Table 27 (below) presents the total number of coded references across the

primary and secondary data, within the secondary research questions, for Case B

Although what an NPO is accountability for to beneficiaries coded highly within the

secondary data, it is important to refer to the previous discussion (see table 20)

detailing the coding to upwards accountability within these documents. The

quantitative data reveals the staff group to have spoken often about how accountability

to beneficiaries is discharged. This suggests their desire for increased downwards

accountability mechanisms.

Table 27. Case B cross stakeholder group analysis of coded references within all RQs

The thematic and document analysis of the data set presents Case B as an

organisation eager to engage in increased downwards accountability. The structure of

the organisation in relation to its mission of providing episodic advice and support to

beneficiaries as needed, presents particular beneficiary accountability needs.

Within Case B beneficiaries predominately seek accountability in order to

complete their engagement with the NPO, as a point of closure. As such, a transaction

of accountability is needed to fulfil the beneficiaries’ accountability needs. This

transaction can be enabled through the use of an evaluation instrument at the

completion of the beneficiaries’ participation. However, a different approach to

accountability may be called for by a beneficiary. This would be in addition to the


transactional approach at the completion of the engagement. In this instance a

relational approach to accountability can supplement a transactional approach.

The findings of the Case B analysis illustrate how a dialogic approach to

accountability can enhance beneficiary participation in accountability structures. As

found, the accountability dialogue between the NPO and the beneficiary within Case

B resembles a transaction. This is represented within Figure 10 (below).

Figure 10. The transaction of accountability

An appropriately developed evaluation instrument can actualise the exchange

of accounts (what the beneficiaries want to say, and what the NPO seeks to hear), in

order to complete the transaction. In this way, a transactional approach to

accountability also embraces dialogic accounting theory’s aims of accounting “for

things that traditional accounting ignores” and developing “accounting that

acknowledges divergent ideological positions” (Brown and Dillard, 2015a, p. 250).

6.6 Evaluation Summary

As discussed within this case narrative, participation of beneficiaries within the

evaluation structure of Case B is important across all stakeholder groups. However,

there are currently no active means to achieve this. Therefore, an opportunity for the

development of an evaluation instrument that enables beneficiaries to participate in the

giving and receiving of accounts is presented.


In analysing the data presented within the case narrative, the following points

are important to consider in the development of the evaluation instrument

• as a method of completing an accountability transaction, it is designed to be used at the completion of the beneficiaries’ engagement with the organisation;

• it seeks to be equitable through being in either written or verbal formats; • the instrument focuses on beneficiary evaluation of the quality of the service

they received; • the instrument allows staff to assess the achievement (or lack) of beneficiary

empowerment resulting from service participation.

The content of the evaluation instrument should enable the beneficiaries to give

an account of their experiences within the NPO, for example, beneficiary perspectives

on

• the quality of the service provided, and their satisfaction with it; • the ability of staff; • the presentation of information; • the relatability of staff; • any concerns; • areas for improvement; • achievements made; • general feedback.

The content of the evaluation instrument should enable staff to gauge beneficiary

empowerment resulting from their involvement with Case B, for example, beneficiary

perspectives on

• the level of knowledge gained throughout their involvement with Case B; • their understanding of the problem; • the actions they would take in the future if similar housing issues arose.

6.7 Summary – Case B Findings

The main findings reported throughout the Case B narrative centre on the

beneficiaries short-term and episodic timeframe of participation with the NPO. This

particular timeframe has resulted in the beneficiaries seeking accountability to be


discharged to them at the conclusion of their engagement with Case B. In doing so

enabling the fulfilling of an accountability transaction.

An evaluation instrument can act as an enabler of the fulfilment of the

accountability transaction between the NPO and the beneficiary. Through seeking

accountability at the conclusion of their engagement the beneficiaries are detailing

when accountability needs to be discharged to them. The fulfilment of the

accountability transaction, on the beneficiaries’ terms, is thought to lead to increased

beneficiary empowerment and social justice.

Chapter 7. Cross Case Comparison and Discussion 145

Chapter 7: Cross Case Comparison and Discussion

The purpose of this chapter is to present the cross case comparison and

discussion of the research findings in relation to the extant scholarly literature. To

provide consistency, the cross case comparison (commencing in Section 7.1) uses the

same structure as the Case Narratives (Chapters Five and Six). In doing so, Section

7.1.1 compares the organisational structure of the two Cases, and Section 7.1.2

compares the organisations’ missions. Through this, the approaches of Relational

Accountability and Transactional Accountability, stemming from the findings, will be

introduced, compared, and discussed.

Sections 7.2 to 7.4 are structured using the three investigative research questions.

Section 7.2 compares the findings from each Case Narrative within RQ 1.1: For what

is a NPO accountable to its beneficiaries? Section 7.3 compares the findings from each

Case Narrative within RQ 1.2: How is accountability currently discharged to

beneficiaries? Section 7.4 compares the findings from each Case Narrative within RQ

1.3: How is accountability on the beneficiaries’ terms practised? Throughout these

sections, a richer understanding of Relational and Transactional accountability will be

developed.

Within Section 7.5, a summary of the similarities and differences between

Relational Accountability and Transactional Accountability, developed throughout the

proceeding discussion, is presented. The remainder of the chapter (Section 7.6)

describes the evaluation frameworks developed for each case, Section 7.7 summarises

the chapter.

146 Chapter 7. Cross Case Comparison and Discussion

7.1 Cross Case Comparison

As discussed within Section 4.3, the purpose of the cross case comparison is to

further understand the uniqueness of each case (Stake, 2006). This is done through the

juxtaposition of the cases, to compare for similarities and differences. The intention of

which, is theory development, through the building of new insights (Eisenhardt, 1989).

Of importance within theory-building from case study research, and abductive research

of this type, is the search for cross-case patterns and the comparison of emergent

theories with extant literature (Eisenhardt, 1989). The comparison of emergent theories

with extant literature will occur throughout the discussion within the cross case

comparison, in order to facilitate theorising.

7.1.1 Organisational structure.

Table 28 (below) provides legislative and organisational details of Case A and

B. Notably both cases are incorporated associations within the same Australian state

jurisdiction, and registered as Public Benevolent Institutions, having purposes

beneficial to the public or advancing social or public welfare. Case A was formed in

2000 and is financially classified as a medium sized entity with annual revenue ranging

from $250,000 to $999,999. Forming in 1986, Case B is financially classified as a large

sized entity with annual revenue in excess of $1,000,000. Both cases are endorsed with

deductible gift recipient’s status, receive tax concessions, have a membership body,

and receive income from like sources. In keeping with the financial size of the NPO,

Case A has a smaller staff body with seven employees, and Case B has 46 employees

in total.


Table 28. Organisational characteristics of Case A and Case B

Using Cordery and Sim’s (2017) civil society organisations categorisation, both

cases present as “Service Providers”. Here, Service Providers mainly gain income from

the delivery of goods and services in health, legal and other sectors, they mainly

receive government funding, they have a non-profit motive, and members only weakly

direct operations (Cordery & Sim, 2017). Although the beneficiary members of Case

A present as being strongly participative in the NPO’s operations and governance, the

beneficiaries’ ability to direct structural change is limited. Case B also presents

characteristics aligned with an “Advocacy” civil society organisation (Cordery & Sim


2017, p. 3). Here organisations advocate for improved corporate and/or government

policies for underprivileged groups, by speaking powerfully with or for the

disadvantaged (Cordery & Sim, 2017, p. 3).

The comparison of the organisational characteristics of each case reveals a

dominance of legislative and operational similarities. Notably the two NPOs differ in

financial and organisational size, with Case B being significantly larger in both

regards. As discussed within Sections 5.1 and 6.1, Case A operates from one CBD

location, whereas Case B operates throughout its state, from multiple locations. This

difference affects each organisation’s staffing level and beneficiary reach. Case B

reported a client base of 15,112 beneficiaries in the 2016-2017 financial year (Case B,

2017 Annual Report). Whereas Case A’s beneficiary group is significantly smaller, at

approximately 2,100. This difference in beneficiary group size, appears as a

contributory factor to the different structures of beneficiary participation evidenced

within the Case Narratives. (See Sections 5.3 and 6.3 for an analysis of the participative

engagement of beneficiaries within each case).

A closer comparison of income sources is detailed within Figure 11 (below). It

can be observed in this figure that although each organisations’ financial size varies

greatly, both organisations are heavily dependent upon government funding. Case A

does have a prominent secondary source of income from charging for some services it

provides.


Figure 11. Income source comparison Case A and Case B (Case A, 2017, ACNC, AIS; Case B, 2017,

ACNC, AIS)

7.1.2 Organisational mission.

As discussed within Section 3.3.1 there is an awareness within the extant

scholarly literature of the importance of discharging accountability downwards to

beneficiaries within NPOs (Campbell & Lambright, 2016; Ebrahim, 2005; Rey-Garcia

et al., 2017). Cordery and Sim (2017) have demonstrated that it is also important to

consider the diversity of the NPO sector in relation to organisational structure, funding

and activities, when considering an NPO’s accountability demands. As an extension,

this research presents that not only is there diversity within the NPO sector’s

accountability needs, but there is also diversity within individual beneficiary group’s

accountability needs.

This research has found that differences in NPO service activity, lead to

differences in impact upon beneficiaries. In isolation the uniqueness of each case is

less evident. It is through the juxtaposition of each case that idiosyncrasies emerge,

which shape the participative needs of beneficiaries and their involvement within


current and future evaluation processes. As Cordery and Sim (2017) have shown, there

is a need to consider the diversity of the NPO sector and the different classifications

of NPOs when questioning the accountability demands upon individual organisations.

However, as shown through the findings of this research, there is also a need to

consider the classification of the NPO when studying the specific downwards

accountability methods most suitable to meet the unique needs of the beneficiary

stakeholder group. When it comes to methods of beneficiary participation in

evaluation, the findings show (see Sections 5.7 and 6.7) that one approach will not suit

all beneficiary groups.

Instrumentally driving the nature of beneficiary participation within each case is

the mission of the organisation. The mission of an NPO is its principal purpose and the

reason for its existence (Anheier, 2005). It establishes frameworks around which,

beneficiary participation is constructed and bounded. Of essence is how the mission is

responsible for the construction of a timeframe, within which beneficiary participation

is moulded. This timeframe is instrumental in shaping what an organisation is

accountable for, how and on whose terms. The mission establishes an NPO’s purpose

and goals, the needs it fills, its values and aspirations (Anheier, 2005). It is these

elements that shape the stakeholder groups (specifically who the beneficiary group

are), and determine the resultant timeframe within which the service to the beneficiary

is provided. For the purpose of this discussion, arising from the findings (see Sections

5.1.6 and 6.1.5), it is the timeframe the mission presents that will be focal.

7.1.2.1 Timeframe of beneficiary participation within the NPO.

As discussed within Section 5.1.6, Case A has a mission of providing a voluntary

rehabilitation service to its beneficiaries. This mission supports a long-term timeframe

of consistent beneficiary participation. Here, beneficiaries are engaged weekly and

participation is ongoing (see Table 6 for the timeframe of ongoing participation of each


beneficiary). As discussed within Section 5.5, this type of engagement appears to

foster a relational form of accountability, as beneficiaries seek accountability in order

to strengthen the relationship between themselves and the NPO. Here, accountability

to beneficiaries is enhanced through the giving and receiving of accounts throughout

the beneficiaries’ participation within the NPO. In this way, accountability is ongoing.

As discussed within Section 6.1.5, Case B has a mission of providing advocacy

and advice to its beneficiaries which supports a short-term timeframe of episodic

beneficiary participation. Here, beneficiaries are typically engaged with the NPO

whilst they are seeking episodes of advice and support to resolve a housing related

issue. When the episode concludes, participation typically ceases, (see Table 6 for the

timeframe of episodic participation of each beneficiary). This type of beneficiary

participation appears to foster a transactional form of accountability, where

beneficiaries are calling for accountability to be fulfilled through a transaction that

signals the completion of their episode of engagement. The transaction appears as the

giving and receiving of accounts (or feedback) at the conclusion of the interaction

between the NPO and the beneficiary (see Section 6.5). This transaction enables the

fulfilment of the accountability needs of the beneficiary within the NPO. In this way,

accountability is episodic.

The ideal ongoing (Case A) or episodic (Case B) evaluation points within each

organisation are represented through Table 29 (below). The engagement of

beneficiaries is represented through the shaded areas. As can be seen within this figure,

evaluation within Case A occurs regularly throughout the beneficiaries’ engagement

with the NPO. In Case B, evaluation occurs at the end of each episode of engagement

with the NPO.


Table 29. The beneficiary evaluation points within Case A and Case B.

The differences presented in the accountability needs of the beneficiaries, result

from the unique missions and service types of each case, which form the timeframe of

beneficiary participation within each NPO. These differences have a pronounced

impact upon when accountability occurs as well as the ability of evaluation to enhance

accountability to beneficiaries. The impact on accountability caused by the different

timeframes of the two cases is depicted in Table 30 (below).

Table 30. Timeframe of beneficiary participation within each Case


Therefore, a difference in the accountability needs of the beneficiaries has been

identified. Through identifying the impact of the timeframe of beneficiary participation

on beneficiary accountability needs, and on how the organisation can meet them, this

research has been able to extend the current understanding of NPO accountability

within the scholarly literature. This research has done this in two ways:

1) Through providing empirical research to substantiate the importance of

accountability when within an accountability framework. Within this research,

accountability when arises in consideration of when accountability to beneficiaries is

to be discharged, in order to meet the needs of the beneficiaries. This presents as a

different application of accountability when, than that proposed by Mashaw (2014)

(Section 3.6). Arising from the findings of this research, the timing of accountability

necessitates a deeper questioning of when accountability occurs, rather than only

considering the time lag between actions and accountability demands, as discussed by

Mashaw (2014, p. 586). Accountability when also involves consideration of the

timeframe, and the duration of accountability.

2) The identification of different types of accountability to beneficiaries (that is,

relational and transactional), means that downwards accountability takes more than

one form. It is multi-modal. This multi-modality, which presents as either Relational

accountability or Transactional accountability for these beneficiary groups, will be

explored in the following section.


7.1.2.2 Relational and transactional accountability across Case A and B.

It is possible for beneficiaries from either case to seek the other form of

accountability towards themselves. This is observed within Case B (see Sections

6.4.1.1 and 6.5), where although the NPO fosters a transactional form of accountability

to beneficiaries, one beneficiary desires accountability to take a more relational form.

This stems from the beneficiary’s desire to build a relationship with the staff of the

organisation. It appears to be connected to the beneficiary’s feeling of

disempowerment within society at large. This however does not detract from these two

cases exhibiting characteristics that support either a Relational or Transactional mode

of accountability. Figure 12 (below) plots the accountability needs of the beneficiaries

interviewed across a transactional/relational divide.

Figure 12. The accountability needs of the beneficiaries interviewed across Relational and Transactional Accountability

The findings of the narratives for Case A and Case B will now be probed for

similarities and differences across the three secondary research questions. In doing so,

an understanding of the differences and similarities between Relational Accountability

(as presented within Case A) and Transactional Accountability (as presented within

Case B) will be strengthen. These will be displayed in table form at the end of each

discussion and summarised in Section 7.5.


7.2 Discussion - RQ 1.1: For What is a NPO Accountable to its Beneficiaries?

The findings presented within the Case Narratives indicate the participants of the

research believe an account is owed to the beneficiaries of each service (see Sections

5.2, 5.4, 5.5, 6.2, 6.4). This is in alignment with current academic literature (Wellen &

Jegers, 2014). The discussion within the literature review revealed that extant scholarly

literature largely views what NPOs are accountable for through a neo-liberal lens

(Brown & Dillard, 2015a; Chiapello, 2017). In framing accountability in this way, the

focus is on processes of standard setting, performance appraisal, sanctioning,

efficiency, financial integrity, disclosure and the fulfilment of legal obligations

(Murtaza, 2012; Schmitz et al., 2012). However, when considering what an NPO is

accountable for to its beneficiaries, the findings have shown that traditional forms of

accountability become questionable (see Sections 5.2 and 6.2). Traditional

technocratic and coercive (Ebrahim, 2009) ways of framing accountability may not be

useful to this stakeholder group. The Case Narrative discussions have revealed that

what beneficiaries want the NPO to be accountable for is nuanced, and dependent upon

the type of involvement they have with the organisation. This will be further explained

throughout this section.

Table 31 (below) displays the total coded references across both cases within RQ

1.1: For what is a NPO accountable to its beneficiaries. It is notable within the tabulated

data that Case B codes higher at all themes than Case A. However, the coding patterns

are proportionally consistent across both cases. This data will be discussed more

closely throughout this section.


Table 31. Cross Case Analysis coded references to RQ 1.1: For what is a NPO accountable to its beneficiaries?

7.2.1 Accountability for financial and non-financial information.

When comparing what an organisation is accountable for to beneficiaries across

the two cases, it is clear that non-financial accountability is cited more often than

financial accountability, to these beneficiary groups (see Sections 5.2.1 and 6.2.1). If

NPOs are seeking to increase their downwards accountability, this is significant to

recognize, especially as extant literature highlights accountability in NPOs is

frequently linked to financial reporting and efficiency (Ebrahim, 2009; O’Dwyer &

Unerman, 2007). Thus, enhanced accountability could consider how to better articulate

non-financial outcomes.

When considering the non-financial accountability needs of these two groups of

beneficiaries, differences have arisen within the narratives. The findings suggest that

this is attributed to whether the beneficiaries are seeking Relational or Transitional

Accountability. This will be discussed in the following sections.

7.2.2 Accountability for impact.

The literature review revealed NPOs are accountable for specific performance

standards, that can be divided into impact orientated criteria and process orientated

criteria (Murtaza, 2012, p. 115). It has become apparent within the Case Narrative

discussions, that within downwards accountability, impact orientated criteria is more

relevant (see Sections 5.2, 5.4, 5.5, 6.2, and 6.4), particularly in relation to the criterion


of effectiveness (Murtaza, 2012), or the impact the NPO had upon the beneficiary.

However, the findings of the research show that what the beneficiaries are seeking in

relation to accountability for impact is different across the two cases (see Sections 5.5

and 6.2.2).

Within Case A, beneficiaries exhibit highly participative involvement, in the

day-to-day management and governance of the NPO, over an ongoing period of time.

Here (as discussed in Section 5.5), the beneficiary group seeks accountability from the

NPO for information that enables a dialogue, and builds a relationship between the

NPO and the beneficiary. In this way, impact within what this research refers to as

Relational Accountability is multi-directional (also see Figure 6). Beneficiaries seek

accountability from the NPO for the impact the service has on their lives, and for the

impact they are having on the NPO. This is illustrated in Figure 13 (below). The staff

and board seek to meet the beneficiaries’ accountability needs, through providing

increased participative means. In Relational Accountability, when the accountability

needs of the beneficiaries are met by the NPO, a relationship occurs.

Figure 13. Impact within the Relational Accountability model of Case A


Within Case B, beneficiaries exhibit little to no involvement in the organisational

structure, management or governance of the NPO, and involvement is typically

episodic and over a short-term period. Here (as discussed within Section 6.2.2), the

beneficiary group seeks accountability from the NPO for the quality of the service

provided to them, and how this impacted upon their outcomes. In what this research

refers to as Transactional Accountability, the beneficiaries are not seeking

accountability for the impact they have had on the NPO. This is illustrated in Figure

14 (below). This is in contrast to Case A. In Case B, staff and board seek to be accountable

for the impact the NPO has upon the beneficiary. In Transactional Accountability,

when the accountability needs of the beneficiaries are met by the NPO, a transaction

occurs.

Figure 14. Impact within the Transactional Accountability model of Case B

These subtle yet distinct differences in the impact sought by each beneficiary

group contributes to an enriched understanding of the impact oriented criteria of

accountability proposed by Murtaza (2012). It is clear that both beneficiary groups

within the research are seeking accountability for the impact the organisation has had

upon themselves. The difference arises in consideration of the impact the beneficiary


has had on the NPO, which corresponds with the timeframe of beneficiary

participation.

7.2.3 Accountability for organisational sustainability and service quality.

The findings show that the beneficiaries of Case A seek accountability for the

ongoing operational sustainability of the organisation (see Section 5.2.1). This is due

to the dependence the beneficiaries have developed upon the organisation.

Accountability for sustainability is a criterion of impact oriented accountability

(Murtaza, 2012). Accountability for organisational sustainability presents as a

characteristic of Relational Accountability. Here, beneficiaries have developed

dependencies based upon relationships with the NPO. Case A’s richly participative

structure, allows beneficiaries to be part of the operational sustainability. In this way

they are meeting this accountability need.

This is in contrast to the findings from the Case B narrative, where beneficiaries

did not talk of the organisational sustainability of the NPO. Within Case B, the

beneficiaries are seeking accountability for the quality of the service provided, rather

than organisational sustainability, which is in keeping with their short-term and

episodic engagement (see Section 6.2.1).

7.2.4 Summary of findings within RQ 1.1: What is a NPO accountable for to beneficiaries?

In summary, a presentation of the findings across the two Cases has shown that

although accountability for non-financial information is more relevant to the

beneficiary groups across both cases, what an organisation is accountable for to

beneficiaries, differs dependent upon the type of participation the beneficiaries have

within the NPO. This difference necessitates the questioning of accountability when?


Within Case A, the accountability needs of the beneficiaries are ongoing (when).

Here, beneficiaries seek accountability for organisational sustainability. Within Case

B, the accountability needs of the beneficiaries are episodic (when). Here, beneficiaries

seek accountability for the quality of the service provision.

The beneficiaries of both cases seek accountability for impact, but what this

means in practice, is different relative to the answering of the accountability when

question. Within Case A, beneficiaries seek accountability for the impact they have on

the NPO and the impact the NPO has upon them. Within Case B, beneficiaries seek

accountability for the impact the NPO has had upon themselves, but are not seeking

accountability for the impact they have upon the NPO.

Table 32 (below) presents a summary of the key similarities and differences

between Relational and Transactional Accountability, within RQ 1.1: For what is a

NPO accountable to its beneficiaries?

Table 32. Relational and Transactional Accountability comparison - RQ 1.1: For what is a NPO accountable to its beneficiaries?

The discussion will now progress to consider the findings in relation to RQ 1.2:

how is accountability currently discharged to beneficiaries. In doing so, the cross case

analysis of the two cases continues.


7.3 Discussion - RQ 1.2: How is Accountability Currently Discharged to

Beneficiaries?


1.2: How is accountability currently discharged to beneficiaries? Of note within this

table is the significant difference between the cases to coded references within the

participation theme. This data supports the highly participate environment found to be

within Case A.

Table 33. Cross Case Analysis coded references to RQ 1.2: How is accountability currently discharged to beneficiaries?

7.3.1 Accountability through beneficiary participation.

As evidenced through the findings of the Case A narrative, downwards

accountability is currently discharged through a highly participative environment (see

Sections 5.1.6 and 5.3). This is in support of the needs of the beneficiaries, the mission

of the organisation and a relational form of accountability, which occurs through the

process of giving and receiving accounts. In contrast, the accountability needs of Case

B’s beneficiary group exhibit a less participatory relationship, and a more transactional

demand (see Sections 6.1.5 and 6.3). These different beneficiary needs call for

different means of discharging accountability by each NPO.

Within Case A, accountability is being discharged to beneficiaries through

beneficiary participation in the governance and the day-to-day management of the

service (see Section 5.3). Participation within this structure is largely collaborative. As

shown, this participative structure enables an evaluative environment where

beneficiaries are able to provide feedback and evaluate the organisation in an ongoing,

process driven manner.


Within Case B, the methods of beneficiary participation are targeted towards

enabling the exchange of information from the NPO to the beneficiary (see Section

6.3). In this structure, participation is directed towards facilitating a positive result for

the beneficiary, and as such, is outcome focussed. Participation within this structure is

largely at an individual level. In contrast to Case A, this participative structure does

not foster the creation of an evaluative environment (see Section 6.3). Instead,

evaluation is seen as being most valuable at the conclusion of their involvement, as a

means of completing the information exchange (or transaction).

7.3.2 Accountability through beneficiary participation in evaluation.

In Section 7.1.2.3, the categorisation of NPOs relative to their timeframe of

beneficiary participation was discussed. This discussion suggested NPOs, such as Case

A, that provide an ongoing service over a long-term timeframe, might embrace a mode

of Relational Accountability, and NPOs, such as Case B, that have missions to provide

a service episodically over a short-term timeframe, might embrace a mode of

Transactional Accountability. The findings show that it is important for NPOs to

consider these different modes of accountability in order for the accountability process

to match the needs of individual beneficiary groups. A lack of attempt to match

downwards accountability practices to beneficiary group needs could suggest an

NPO’s engagement in the downwards accountability sham-ritual, warned of by Najam

(1996), where beneficiary participation is merely a feel good exercise. This research

has been able to importantly demonstrate that beneficiary groups seek different modes

of downwards accountability, and that all approaches to accountability (either

relational or transactional) are valid and vital. If organisations are not directing their

accountability practices towards the needs of the beneficiary groups, accountability

may not be fulfilled.


An important tool within a NPOs downwards accountability processes is a

participative evaluation mechanism (Ebrahim, 2003b). An NPO’s ability to understand

the different accountability needs of their beneficiary group (that is either relational or

transactional) enables the development of evaluation mechanisms or instruments that are

better targeted to meet their beneficiary group’s needs. As two different approaches to

beneficiary engagement have been identified, it makes sense that the evaluation

instruments might also be different.

The discussion within the Case Narratives (see Sections 5.3 and 6.3) indicates

that participants believe beneficiary involvement in evaluating NPOs is valuable and

important, to both the beneficiary and for the organisation. Yet both cases detailed

limited examples of beneficiary participation in formal evaluation processes. This

supports Schmitz, Raggo and Vijfeijken’s (2012) claim that organisational aspirations

to engage in downwards accountability are frequently not matched by their practices.

Whilst both beneficiary groups reported a lack of formal involvement in

evaluation processes, the different structural environments and resultant timeframes of

beneficiary participation within Case A and Case B have contributed to differences

within the evaluation findings. The beneficiaries of Case A reported less need to

develop evaluative systems, due to the existing evaluative environment, than the

beneficiaries of Case B. It is thought that the structure of the NPO and its categorisation

as either a Relational Accountability organisation or a Transactional Accountability

organisation, creates this difference.

To the beneficiaries of Case A, who are already highly participative in a

relational manner, the evaluation instrument is designed to further strengthen the

relationship. This is illustrated in Figure 15 (below). Depicted in this figure, the


evaluative instrument enables a strengthening of the relationship between the

organisation and the beneficiary in an ongoing manner. The evaluation instrument is

worded in a way that allows the beneficiaries to provide information regarding their

participation and development within the NPO, over time. The instrument is part of

the ongoing evaluative environment of Case A, and is designed to be used at regular,

cyclical intervals. Here, accountability is part of a relationship, between the NPO and

the beneficiary. An example of an evaluation instrument designed to be used within

Case A (a Relational Accountability NPO) is located within Appendix D.

Figure 15. A Relational Accountability approach to participative evaluation

To the beneficiaries of Case B, who are less participative and more engaged in a

transactional manner, an evaluation instrument is designed to support the fulfilling of

the accountability transaction between themselves and the NPO. In this way, the

evaluation instrument enables the transaction of accountability to occur, within

Transactional Accountability organisations. This is illustrated in Figure 16 (below). In

this approach, the evaluation instrument allows beneficiaries to give an account of the

service quality they have received, whilst simultaneously allowing the NPO to receive


an account of the level of empowerment generated within the beneficiary through their

engagement. In this way, the instrument is able to provide an overview of the outcome

of the beneficiaries’ involvement with the NPO. Here, accountability is part of the

transaction between the NPO and the beneficiary. An example of an evaluation

instrument for Case B (a Transactional Accountability NPO) is located within

Appendix D.

Figure 16. A Transactional Accountability approach to participative evaluation

7.3.3 Accountability through equity.

Despite a call for different means of discharging accountability to beneficiaries,

dependent upon the nature of the beneficiaries’ involvement with the NPO, there are

commonalities presented across the cases in regards to methods of participative

evaluation (Greene, 1997). Both Case Narrative findings (see Sections 5.3 and 6.3)

illustrate a need for beneficiaries to engage with evaluative processes in a way that

enhances equity across that stakeholder group. Developing instruments that support

equity, through allowing equality in access is imperative. The findings across both

cases suggest that in order to do this, evaluation instruments need to support either a


verbal or a non-verbal (written) platform, be formal or informal, and offer individual

or group participation where applicable.

7.3.4 Barriers to participation in evaluation.

The Case Narratives both detail potential barriers to beneficiary involvement in

evaluation (see Sections 5.3 and 6.3). When comparing these barriers across the two

cases, similarities and differences arise. It is proposed that these are a result of the type

of the beneficiary involvement within each case. Within Case B, many barriers

presented by staff arise from the short-term or episodic timeframe of beneficiary

participation. These include difficulty engaging beneficiaries after their issue is

resolved, losing contact with beneficiaries or not having sufficient contact details, the

timing of seeking feedback, and the beneficiary’s time and willingness. It is speculated

that these were not raised as barriers within Case A because of their long-term and

ongoing engagement with the beneficiaries, and the resultant different relationship the

beneficiaries have with the organisation. Conversely, the barriers presented by Case A

have a common focus on access to the NPO by the beneficiary (for example, travelling

at night, or a lack of transport). In similarity, both cases presented literacy skills, and a

lack of technology access or skills, as barriers to participation in written evaluation.

7.3.5 Summary of findings within RQ 1.2: How is accountability currently discharged to beneficiaries?

In summary, a presentation of the findings across the two cases has shown that

how accountability is discharged to beneficiaries differs dependent upon the type of

participation the beneficiaries have within the NPO. This difference necessitates the

questioning of accountability when? Beneficiaries of Case A seek accountability to be

discharged throughout their participation, whereas beneficiaries of Case B seek

accountability to be discharged at the conclusion of their participation. The use of an


evaluation instrument to assist in discharging accountability has been validated across

the two Cases, yet the timing and purpose of the instrument is dependent upon the

consideration of accountability when (as discussed within Section 7.1.2.1, 7.4.4 and

7.4.7). Within Case A, the instrument is used to strengthen the relationship, throughout

the engagement. Within Case B, the instrument is used to complete the accountability

transaction at the end of the beneficiaries’ engagement. Table 34 (below) displays the

qualities of Relational and Transactional accountability found within RQ 1.2: How is

accountability currently discharged to beneficiaries.

Table 34. Relational and Transactional Accountability comparison- RQ 1.2: How is accountability currently discharged to beneficiaries?

The discussion will now progress to consider the findings in relation to RQ 1.3:

how is accountability on the beneficiaries’ terms practised. In doing so, the cross case

analysis of the two cases continues.

7.4 Discussion - RQ 1.3: How is Accountability on the Beneficiaries’ Terms

Practised?


1.3: How is accountability on the beneficiaries’ terms practised? The quantitative and

associated qualitative data represented within this table is used to develop the

discussion within this theme.


Table 35. Cross Case Analysis coded references to RQ 1.3: How is accountability on the beneficiaries’ terms practised?

7.4.1 Accountability on the beneficiaries’ terms through giving voice.

The findings across both Case Narratives (see Sections 5.4 and 6.4) indicate that

the imperative to accountability being practised on the beneficiaries’ terms, is the

giving of voice to beneficiaries. As such, giving voice to beneficiaries is important

across both Relational Accountability and Transactional Accountability. The findings

suggest that the giving of voice to beneficiaries can increase their empowerment both

within the NPO and within life in general. This leads to increased feelings of social

justice within the beneficiary. The findings also indicate that the giving of voice to

beneficiaries can improve the quality of service provided by the organisation, which is

in confirmation with extant scholarly literature (Ebrahim, 2009). Although these

findings are presented across both Case A and Case B, giving voice to beneficiaries is

more strongly practised within Case A. Within Case B the importance of giving voice

is acknowledged as important, yet it is more aspirational.

The practices of giving voice within Case A support Schmitz, Raggo and

Vijfeijken’s (2012) acknowledgement of the importance of beneficiaries’ voices

reaching those parts of the organisation usually oriented towards upwards

accountability (for example participation on the board). This is considered by Ebrahim

(2009, p. 899) to be an important part of the accountability process. However, in


consideration of the power asymmetry between the beneficiaries, and the staff and

board of any NPO (Ebrahim, 2005), Case A’s focus on reaching consensus within the

process of giving voice to beneficiaries could be reconsidered. This is particularly

relevant in light of the impact that the giving of voice can have, and the inevitable

conflict, which White (1996, p. 15) claims should arise. Perhaps the giving of voice in

Case A could embrace an agonistic approach to democracy (Mouffe, 1999), rather than

a deliberative approach.

In enacting an agonistic approach, an NPO would embrace conflicting

participant voices (White, 1996) (whether that be the voice of the staff, board,

beneficiary or other stakeholder), and acknowledge the right of participants to defend

their ideas in a respectful democratic environment (Mouffe, 1999). The NPO would

also be alert to the power asymmetry present between beneficiaries, and staff or board

members within this environment, and in doing so, seek out processes that can create

a transformation of those power symmetries. The embracing of downwards

accountability structures based on the needs of the beneficiary group, rather than on

the needs of the NPO, is a firm starting point in this process. A formal evaluation

instrument can enable the giving of voice to beneficiaries that is safe in its ability to

be heard, and not overpowered by more powerful actors.

7.4.2 Accountability on beneficiaries’ terms leading to empowerment.

The findings of this research confirm Jacobs and Wilford’s (2010) claim that

empowerment is closely related to downwards accountability (see Sections 5.4.1,

5.4.2, 5.5, 6.4.1 and 6.4.2). In focussing this claim, within the Case A narrative (see

Section 5.4.1.1), a relationship was noted between the length of time the beneficiary

has been involved with the NPO and the level of empowerment they display. All

beneficiaries of Case A who have been involved with the NPO, in a highly participative


way, over a long-term period, exhibit strong levels of empowerment. This appears to

be a reflection upon the increased level of control the beneficiaries feel due to long-

term participation. This confirms Kilby’s (2006) sentiment that accountability is

empowering when beneficiaries have some degree of control. As such, an association

between empowerment and Relational Accountability is presented. It is possible that

the absence of feelings of disempowerment displayed by the beneficiaries in Case A,

is also a result of their relationship with Case A. Beneficiary empowerment in Case A

appears to arise throughout their on-going participation.

In contrast, the beneficiaries of Case B are seeking the support of the NPO to

empower them to have a positive outcome in their housing related issue. Here, the

beneficiaries that are seeking Transactional Accountability do not indicate feeling

disempowered, and are satisfied in their transactional engagement with the NPO.

However, the beneficiary that feels disempowered (see Section 6.4.1.1), is seeking a

relational engagement with the NPO. This again supports the claim that there is an

association between Relational Accountability and empowerment. Theoretically, if

this beneficiary is able to develop a relationship with the NPO, her/his feelings of

disempowerment should decrease as that relationship increases. Within Transactional

Accountability and Case B, beneficiary empowerment appears to arise through

completion of the engagement.

7.4.3 The impact of neo-liberal policies upon accountability on the beneficiaries’ terms.

It is notable that within Case A, there is an absence of references made to the

code neo-liberalism. However, within Case B, there are references made (see Section

6.4.1.2). Neo-liberal policies are those underpinned by an intent to draw on market

mechanisms and private actors to regulate the economy and distribute goods and

services (Chiapello, 2017). It was discussed in the Case B narrative (Section 6.4.1.2),


that neo-liberal policies have contributed to the creation of Case B’s beneficiary group.

This is because the NPO exists to provide advice and support to people affected by

market and private actor driven housing conditions.

When government policy delineates people as ‘needy’, and in need of service

provision, the risk is that they could be disempowered, silenced and rendered marginal.

However, counter-hegemonies are possible (Tambakaki, 2014). The findings of this

research have illustrated that the reframing of accounting and accountability towards

a more dialogical approach (Bebbington et al., 2007) is important in beginning to

account for those normally unaccounted for.

7.4.4 Accountability on the beneficiaries’ terms through a dialogical approach to accountability.

When considering the significance of each of the cases to the lives of their

beneficiaries, accountability assumes an urgency. Accountability within this context

goes beyond technocratic or coercive normative logics (Ebrahim, 2009). The words of

the beneficiaries demonstrate that to them, the service of the NPO is vital. This

reinforces the importance of embracing a dialogic approach to the giving and receiving

of accounts (Gray, Brennan & Malpas, 2014), and shines a new light on Brown’s

(2007) principles of dialogic accounting.

When a service makes the difference between life and death, downwards

accountability must go beyond the sham-ritual (Najam, 1996). In applying Brown’s

principles of dialogic accounting, through being attentive to power relations,

downwards accountability can enable non-expert beneficiaries, from varied

ideological or hegemonic orientations, to participate in the giving and receiving of


accounts, by avoiding monetary reductionism, in order to be transformative (Brown,

2007, p. 324-327). In doing this, downwards accountability is transformed from a one-

directional giving of accounts, to a back and forth dialogue, where accounts are given

and received (Gray, Brennan & Malpas, 2014) by both parties. When downwards

accountability is viewed through the lens of dialogic accounting theory, the question

of accountability when arises. Both Relational Accountability and Transactional

Accountability support a dialogic approach to downwards accountability (see Sections

5.2, 5.3, 5.5, 6.2 and 6.5). Here, participation and dialogue through evaluation

encourages an exchange of accounts between the beneficiary and the NPO. Whether

this is ongoing and relational, or episodic and transactional, a dialogue is still fostered.

This relationship is illustrated within Figure 17 (below).

Figure 17. The relationship between dialogic accounting, accountability when, and relational and

transactional accountability

The accounts given by all three stakeholder groups within Case A, indicate an

approach to accountability that is in line with that advocated for within Brown’s (2007)

principles for dialogic accounting. Case A presents as an organisation that enables “a


diverse range of societal actors to account for things that traditional accounting

ignores” (Brown & Dillard, 2015a, p. 250). The accounts given by all three stakeholder

groups within Case B, indicate an organisation seeking to embrace beneficiary

participation through evaluation in order to activate a dialogic approach to

accountability.

It is through this research, that a method of enacting dialogic accounting theory

has been established. This research has extended an understanding of how dialogic

accounting theory can be applied within downwards accountability. This research has

adopted the principles of dialogic accounting theory developed by Brown (2009),

through seeking the perspective of the beneficiaries, as users of accounting

information. Table 36 (below) presents how this research has enacted the principles of

dialogic accounting and accountability. This table is an extension of Table 1 contained

within Section 2.1. As was the same in Table 1, the first two columns of Table 36 are

adapted from Blackburn et al. (2014).


Table 36. The enactment of dialogic accounting principles through this research (table adapted from Blackburn et al., 2014).

7.4.5 Accountability on the beneficiaries’ terms through upwards accountability reporting.

The findings of both Case A and Case B (see Sections 5.2.2 and 6.3.2.2) reveal

that each NPO has engaged in upwards accountability reporting requirements that have

benefitted accountability downwards to beneficiaries. This suggests that when

upwards accountability requires the reporting of how downwards accountability to

beneficiaries has been met, organisations are more likely to engage in activities that

support and increase accountability to beneficiaries. It may be through mandating the

upwards reporting of the meeting of downwards accountability, that increased

downwards accountability will occur.


It is significant to consider Jacobs and Wilford’s (2010, p. 800) caution that

mechanisms used for upwards accountability can undermine downwards

accountability. It is here that care needs to be taken to ensure upwards accountability

is reporting downwards accountability on the beneficiaries’ terms, and not on the terms

of the funders, donors or government (those with power).

The findings indicate that organisational reporting documents can play an

important role in increasing accountability on the beneficiaries’ terms, through being

platforms within which to give voice to beneficiaries (see 5.4.4 and 6.4.4). Currently,

the largely monetary based upwards accountability reporting documents of both

organisations provide little avenue for contributing to accountability on the

beneficiaries’ terms. This confirms Brown’s (2007) importance of avoiding monetary

reductionism, when moving away from the monologic accounting paradigm.

Within the Case Narratives (see Sections 5.2.2 and 6.3.2.2) examples have been

given which illustrate how upwards and downwards accountability processes can co-

opt each other in order to strengthen one’s own agenda. These frequently competing

accountabilities (Ebrahim, 2005) can be repositioned to be considered on the

beneficiaries’ terms. This is illustrated through the examples in Figure 18 (below). In

this figure the arrows represent either upwards or downwards accountability.

Figure 18. Upwards and downwards accountability processes can be mutually supportive


The accountability directions (that is, upwards, downwards, inwards, sideways)

(Murtaza, 2012, p. 114), and the resultant separation of stakeholder groups, is created

by and sustained within the dominant accounting hegemony. This prevents the opening

up of accounting, that would enable dialogic practices to unfold. When accountability

is viewed on the terms of the beneficiaries, it becomes apparent that the traditional

accountability directions position the Chief Executive Officer (CEO) of the NPO

centrally. In this way, accountability is considered on the terms of the CEO. This is

illustrated in Figure 19 (below).

Figure 19. The accountability directions with the CEO as central

To consider accountability on the terms of the beneficiaries, the traditional

directions of accountability will need to change. The findings suggest (see Sections

5.5 and 6.5) a dialogic perspective of accounting and accountability, on the

beneficiaries’ terms, would reframe the separate accountability directions (on the

CEO’s terms) to a mutually supportive accountability network. Where the meeting of

accountability needs of one stakeholder group, can also meet the accountability needs

of other stakeholder groups.

The very language of the traditional accountability directions (for example,

upwards and downwards) perpetuates the power differentials embedded within the

monologic accounting discourse, as it positions beneficiaries as the lowest (and by

implication, least important) stakeholder group. Murtaza (2012) similarly drew


attention to the implied assumptions embedded within the use of this directional

language, when acknowledging where in the NPO hierarchy the downwards direction

falls. However, as shown through these findings, accountability can have a symbiotic

nature, where what benefits one group can also benefit others, when given the

opportunity. Accountability upwards (or downwards, or sideways, or inwards, or even

diagonally) does not only head in that direction. Instead, it permeates the whole.

Accountability is dialogic. To conform accountability into monologic accounting

principles would mean that accountability is capable of being neutral and value-free,

apolitical, quantifiable, dependent upon experts and objective (Brown, 2007). Yet, it

is clear that accountability is none of these.

In assuming a more dialogic view, the traditional directions (Murtaza, 2012) of

multi-stakeholder accountability can be reframed. If upwards and downwards

accountability can be co-opted and mutually beneficial, as illustrated in the findings, a

symbiotic relationship between the directions of accountability exists. Rather than

existing in mutual exclusivity, their interdependence can be embraced. This

relationship between upwards and downwards accountability when the accountability

directions are considered on the beneficiaries’ terms is illustrated within Figure 20

below.

Figure 20. The symbiotic relationship between upwards and downwards accountability when viewed on the beneficiaries' terms.


Within this reframing, the boundaries of accountability to stakeholder groups

blur. Accountability is not neat and tidy. When viewed on the beneficiaries’ terms,

what is accountable upwards is no longer separable from what is accountable

downwards, nor is where accountability in each direction begins or ends evidently

delineated. From an organisational perspective, the symbiotic relationship between the

traditional directions of accountability can be embraced and used to the organisation’s

advantage. It is important that this is done through embracing the principles of dialogic

accounting theory, through casting a critical perspective upon Ebrahim’s (2016)

accountability framework by considering accountability on the beneficiaries’ terms.

Doing so ensures accountability becomes accessible by non-experts, embraces

subjectivity and is attentive to power relations (Brown, 2007). Without this

underpinning, as research has shown, downwards accountability would again be

undermined by the power differentials in upwards accountability (Jacobs and Wilford,

2010).

7.4.6 Accountability on the beneficiaries’ terms through evaluation.

Within the cross case analysis, evaluation has been presented as a possible

method of organisational participation of beneficiaries, which would allow for the

increase of accountability downwards (Ebrahim, 2003b). As shown in the findings of

the research, the beneficiaries want evaluation and participative instruments that allow

their words to impact upon the organisation and the organisation to positively impact

on them (see Sections 5.2.1, 5.3, 6.3, 6.4.1.1 and 6.6). As such, and in support of a

transformative approach to participatory evaluation (Dahler-Larsen, 2018), the

evaluation instrument itself will be a tool for transformation; transformation of the

beneficiary and transformation of the NPO. In this way, the instrument is a

transformative participatory evaluation instrument. The evaluation instrument is


located within an evaluation framework, which constitutes part of the NPOs evaluative

environment. The evaluation framework and instrument for each case will be detailed

in Section 7.6.

7.4.7 Summary of findings RQ 1.3: How is accountability on the beneficiaries’ terms practised?

In summary, a presentation of the findings across the two cases has shown that

how accountability on the beneficiaries’ terms is practised, differs dependent upon the

type of participation the beneficiaries have within the NPO. This difference

necessitates the questioning of accountability when?

The findings of the cross case comparison have demonstrated that when

accountability on the beneficiaries’ terms is practised, the outcomes to the

beneficiaries are different. The practice of accountability on the terms of the

beneficiaries of Case A necessitates the accountability practise to be ongoing. When

this happens, the empowerment of beneficiaries should increase as the relationship

between the beneficiary and the NPO strengthens. Conversely, the practice of

accountability on the terms of the beneficiaries within Case B necessitates

accountability to be practised at the conclusion of their engagement. When this

happens beneficiary empowerment should increase.

In similarity, the cross case comparison has shown that the practice of

accountability on the beneficiaries’ terms, enacts the principles of dialogic accounting

across both cases. The findings have also shown that an evaluation instrument,

developed with an awareness of when accountability is occurring within the

beneficiary group, can enable the practice of accountability on the beneficiaries’ terms.


Table 37 (below) displays the qualities of Relational and Transactional

accountability, found within RQ 1.3: How is accountability on the beneficiaries’ terms

practised.

Table 37. Relational and Transactional Accountability comparison - accountability on whose terms?

7.5 Summary of a Relational or Transactional Approach to Accountability

Table 38 (below) summaries the similarities and differences between relational

(as presented within Case A) and transactional (as presented within Case B)

accountability as evidenced throughout the findings and discussion of the research.

The similarities are presented first. Whilst it is clear that there are many differences

within these two approaches, it is also clear that they are both equally valid modes of

accountability that arise from the differing needs and demands of the beneficiary

group. In this way, both approaches are dialogic, as they are representative of

beneficiary groups that seek involvement within evaluation processes of their

organisations, in order to have their voices consistently or more strongly heard.


Table 38. Similarities and Differences between Relational Accountability and Transactional Accountability (as demonstrated within the research findings)

The discussion will now progress to the development of the evaluation

frameworks and evaluation instruments. The development of these are based upon the

findings of the Case Narratives and the Cross Case Analysis.


7.6 Evaluation Framework

Throughout the discussion of the findings of the research and the presentation of

the cross case analysis, the ability of evaluation to contribute to accountability on the

beneficiaries’ terms has been demonstrated. This is of significance in consideration of

the central research question: How can evaluation enhance accountability to

beneficiaries? The remainder of the cross case analysis will focus on addressing the

second objective of the research. That is: to develop systems that enhance

accountability to beneficiaries within NPOs, which create new visibilities, and

acknowledge beneficiaries’ rights to have contested narratives. This will be done

through the presentation of an evaluation framework, and an evaluation instrument for

each case.

7.6.1 Theoretical underpinnings of the evaluation frameworks.

The evaluation frameworks have strong theoretical underpinnings. Chapters

Two and Three have theoretically situated the research and its resultant evaluation

frameworks. In doing so, there is an acknowledgement that the evaluation frameworks

embrace principles of agonistic political theory, transformative participatory

evaluation theory, dialogic accounting theory and participatory evaluation theory. The

key underpinnings of these theories are contained within Table 39 (below).


Table 39. Theoretical underpinnings of the evaluation frameworks.

7.6.2 Evaluative environment.

As discussed within the Case Narratives and the cross case comparison (see

Sections 5.3, 5.6, 6.3.5 and 7.3), the pre-existing evaluative environment of the NPO

is an important consideration in the development of an evaluation framework.

Although both cases present as having limited formal beneficiary evaluation processes,

the Case Narrative discussion has shown (see Sections 5.3 and 6.3) that the evaluative

environment of an NPO can include a range of beneficiary participatory processes.

Figure 21 (below) presents a summarised typology of beneficiary evaluation processes

(actual and suggested), that are identified within the data set. The shaded boxes within

the figure illustrate where the evaluation instrument developed within this research is

situated, which will fill the downwards evaluation gap identified within the findings.


Figure 21. Typology of beneficiary evaluation processes within an NPO’s evaluative environment

7.6.3 Evaluation framework within a relational accountability approach (Case A).

A framework for evaluation is presented in Figure 22 (below). This framework

is developed for use within an evaluative environment that supports a relational

approach to accountability (as seen in Case A). Within this framework, the evaluation

instrument is used throughout the beneficiaries’ engagement with the NPO in a cyclical

manner. As such, it is an instrument of transformative participation. Data derived from

the instrument enables the strengthening of the relationship between the NPO and the

beneficiary. The process driven, cyclical nature of the evaluation framework actualises

the closing of the feedback loop, as detailed within the literature review (World Vision

et al., 2016).


Figure 22. An Evaluation Framework within Relational Accountability

7.6.4 Evaluation framework within a transactional accountability approach (Case B).

A framework for evaluation is presented in Figure 23 (below). This framework

is developed for use within an evaluative environment that supports a transactional

approach to accountability (as seen in Case B). Within this framework, the evaluation

instrument enables the completion of the accountability transaction, and as such it is

an instrument actualising transformative participation. The data derived from the

evaluation instrument is then used by the NPO to enable learning. In this way the data

has an impact upon the organisation. This impact, which is viewed as an outcome of

the evaluation process, is then communicated back to the beneficiary, which leads to

beneficiary empowerment. In this process the evaluation feedback loop is closed

(World Vision et al., 2016).


Figure 23. An Evaluation Framework within Transactional Accountability

7.6.5 Evaluation instrument.

Evaluation instruments have been drafted to be used within each case. The

instruments are located within Appendix D. Evaluation instrument A is designed to be

used within Case A. Evaluation instrument B is designed to be used within Case B.

The evaluation instruments are able to be used across NPOs that exhibit either a

transactional or relational approach to accountability, with minor wording adjustments

to suit the particular NPO. It is possible for organisations or individual beneficiaries

that seek accountability across both approaches to use both evaluation instruments.

Instrument A would be used at regular intervals throughout the beneficiaries’

involvement with the organisation. Instrument B would be used at the completion of

the beneficiaries’ involvement. Both are designed to enhance organisational


The questions within Evaluation Instrument A are designed with an aim of

building or strengthening the relationship between the organisation and the

beneficiary. Whilst including a consideration of the questions suggested by a staff

member (see Section 5.3.2.3), the questions aim to enable the giving of an account by


the beneficiary which focus on sharing their likes and dislikes, ideas, achievements

and goals for the future. Some of the questions in this instrument are informed by the

‘Start, Stop, Continue’ feedback model. This model is well established and asks

structured questions relating to what the respondent would like to stop, start, and

continue doing at the organisation (Hoon, Oliver, Szpakowska and Newton, 2015).

The questions within Evaluation Instrument B are derived directly from the

questions suggested during the interviews (see Sections 6.3.1.3 and 6.3.2.3). Whilst

focusing on the questions suggested by the beneficiary group, the instrument also

includes questions recommended by the staff group, in order to enable an exchange of

information, or account giving and receiving.

Within the research process, both evaluation instrument drafts were presented to

the beneficiary participants who wanted to give feedback. All beneficiaries that

voluntarily participated in providing feedback on their organisation’s draft evaluation

instrument, responded positively to the instrument and no suggestions of changes were

made.

7.7 Summary

In summary, this chapter has presented the cross case analysis and discussion of

the findings. Through analysing the findings from the Case Narratives in juxtaposition,

the cross case analysis has revealed the timeframe of beneficiary participation to be a

key difference between Case A and Case B. The implications of this difference in

timeframe are felt throughout the beneficiaries’ participation, and also within the

subsequent accountability demands placed upon each organisation. Central to the

different accountability demands is the timing of accountability, which gives rise to

questioning accountability when?

In applying accountability when in practice, the chapter has concluded with the


development of evaluation frameworks based upon the unique timing needs of the

beneficiary groups from each case. In doing so, an evaluation framework applicable

for NPOs whose beneficiary group exhibit the demand for relational accountability,

and an evaluation framework applicable for NPOs whose beneficiary groups exhibit a

demand for transactional accountability has been drafted. The frameworks include an

embedded evaluation instrument for beneficiaries of each service to use as part of the

NPO’s downwards accountability structure (see Appendix D).

Chapter 8. Conclusions 189

Chapter 8: Conclusions

The purpose of this chapter is to present an overall summary of the research and

its key findings. In order to do this, the chapter will begin in Section 8.1 by providing

an overview of the research undertaken. This includes a summary of the key literature

and theoretical positioning, leading to the research questions, and the methodological

and analytical approach used. Section 8.2 will then present the key findings of the

research, and discuss their implications to both theory, practice, and policy. The

chapter will conclude by discussing the limitations of the research in Section 8.3, and

explore possible avenues for further research in Section 8.4. Section 8.5 closes the

research by presenting a summary of the research achievements in light of the original

motivation, to address social injustices by empowering marginalised groups.

8.1 Overview of the Research

This research began with a core motivation to address social injustices by

empowering marginalised groups. This motivation was positioned as matching

elements of the United Nations Sustainable Development Goals (SDGs), particularly

numbers 10 and 16. From this grounding, the purpose of the research was established.

The scope was to research beneficiary participation within NPOs, with the aim of

increasing NPO accountability to beneficiaries, and beneficiary empowerment within

their organisation and in society at large.

The extant scholarly literature examined the conceptualisation and practice of

accountability within the NPO sector. In doing so, accountability was revealed to be

an ambiguous concept, taking many forms. A commonly used approach to the

examination of accountability within NPOs was found to be through the consideration

of the directions accountability takes (upwards, downwards, inwards and sideways,

190 Chapter 8. Conclusions

Murtaza, 2012). The literature review revealed NPOs tend to focus on accountability

upwards to donors, government and funders over accountability downwards to

beneficiaries. Here, a power dynamic is evident, where those with power over the

NPO, are prioritised within a NPO’s accountability practices. This dynamic was

notable within agency theory, resource dependency theory, and instrumental

stakeholder theory. The consequence being, those with less power to hold the NPO to

account, have less representation within the NPO’s accountability practices. Typically,

these are the beneficiaries of the NPO’s service, and are referenced through the

scholarly literature as being the downwards recipients of an NPO’s accountability.

The literature review revealed a second commonly used approach to the

consideration of accountability within NPOs to be Ebrahim’s (2016) accountability

framework. This framework questions accountability to whom, for what, and how.

Used in combination with the accountability directions, the accountability framework

is presented within the literature as a way of understanding an NPO’s accountability

needs and demands.

When positioned within the critical perspective presented within Chapter Two,

through the application of dialogic accounting theory’s attention to accountability on

whose terms, the traditional conceptualisation of accountability within the NPO

literature is challenged. Through casting dialogic theory’s critical gaze over Ebrahim’s

(2016) accountability framework, the addition of accountability on whose terms, is

made. The literature review revealed that doing this enables a NPO’s accountability

focus to shift towards the beneficiaries (and other stakeholders) of the service. The

process of which raises the question, how is accountability practised on the

beneficiaries’ terms?

The literature review progressed to examine the role evaluation plays in NPO’s

accountability structures to beneficiaries. In doing so, it was revealed that although


evaluation can play a role in increasing accountability to beneficiaries, difficulties of

doing this are present within the literature. However, the evaluation approaches of

participatory evaluation and transformative participatory evaluation were found within

the literature review, to offer theoretical and practical approaches to the participation

of beneficiaries within NPO evaluation structures. The aim of which increases both

beneficiary empowerment through participation, and an NPO’s downwards

accountability practices. The evaluation instrument itself becomes a vehicle for

beneficiary transformation (Dahler-Larsen, 2018).

Cautions are evident within the literature when attempting to give voice to those

less powerful. In particular, the appropriate democratic process (agonistic or

deliberative), and the conflict which should occur when power asymmetries are

challenged through the giving of voice to those who traditionally have not had a voice

before (White, 1996).

In consideration of the extant literature, the literature review gave rise to the

central research question, how can evaluation enhance accountability to beneficiaries

within NPOs? In order to answer this central question, three secondary research

questions were formed:

RQ 1.1: For what is a NPO accountable to its beneficiaries?

RQ 1.2: How is accountability currently discharged to beneficiaries?

RQ 1.3: How is accountability on the beneficiaries’ terms practised?

The methodology chosen to answer the research questions within this study, was

interpretative, qualitative, multiple case study (Stake, 1995; Walsham, 1995). This was

deemed the most appropriate in order to understand the phenomena under question.

The methodology used two NPOs as the two cases of empirical study within the

research. Primary data was collected through semi-structured interviews, across three

stakeholder groups, within each case. Secondary data was collected in the form of


publically available, legislative and organisational documents. The data set used

within the research totalled 14 interviews (six hours and 43 minutes of transcribed

primary data) and 22 documents (169 pages of secondary data). Thematic analysis was

used to analyse the interview transcriptions and documents. Triangulation of the data

set helped to identify the diversity of perception within each Case (Stake, 2006).

Triangulation occurred at multiple levels including:

• Triangulation of interview transcripts within individual stakeholder groupings.

• Triangulation of interview transcripts across stakeholder groupings. • Triangulation of documents across the secondary data set. • Triangulation of documents to interview transcripts.

The presentation of findings of the research included the development of a Case

Narrative for each Case, and a cross case analysis including discussion. The findings

evidenced a common theme of beneficiaries wanting to be participative and involved

in the evaluative processes of their service. However, the findings revealed that the

means of doing this are different across the two Cases.

8.2 Key Findings

In recalling the central research question of the study: How can evaluation

enhance accountability to beneficiaries within NPOs, the key findings of this research

are:

• Evaluation can enhance accountability to beneficiaries if NPOs pay attention to the particular timeframe of beneficiary participation within the organisation.

• The timeframe of beneficiary participation within the NPO gives rise to different beneficiary accountability needs within an organisation’s accountability structures.

The following section presents a summative discussion of these key findings. In

doing so a presentation of the research implications, upon theory, practice, and policy

is also made.


8.2.1 Discussion of key findings.

The inclusion of questioning on whose terms are we accounting (Brown & Dillard,

2015a), has enabled the consideration of accountability on the terms of the

beneficiaries. This is not a perspective actively sought within the current monologic

accounting and accountability hegemony. Through positioning accountability

critically, a more dialogic approach to accounting and accountability has been

presented (Bebbington et al., 2007). Viewed in this way, the traditional accountability

directions (Murtaza, 2012) have become questionable and are challenged.

Through positioning the beneficiary at the heart of the accountability process, this

research found that what beneficiaries need, and are calling for within their NPO’s

accountability structure, is dependent upon the type of engagement they have. It has

been demonstrated within this research, that the different timeframe of participation

impacts upon how accountability can be practised on the beneficiaries’ terms.

On the basis of the different timeframes of beneficiary participation, this research

has given rise to two modes of accountability on the beneficiaries’ terms:

1. Relational Accountability.

2. Transactional Accountability.

A thorough presentation of the differences and similarities between these two

approaches to beneficiary accountability has been provided within Chapter Seven, and

is displayed within Table 38. It is important to note, that both of these approaches arise

within a dialogic approach to accounting and accountability, that questions on whose

terms are we accounting? These approaches are grounded within agonistic and

pluralistic political theory, as they attempt to provide places of democratic contestation

for those frequently left out of the democratic process (Vinnari & Dillard, 2016).


In this research, those left out are typically the beneficiaries of NPOs. However,

this research has found that there is a way to include beneficiaries within the

democratic space, but that importantly, it must be done on their terms, and this

necessitates asking them.

This research has shown that the timeframe of beneficiary participation gives

rise to Relational and Transactional Accountability in these ways:

• Relational Accountability arises when beneficiaries’ engagement with the NPO is long-term and ongoing. Here, the beneficiaries seek accountability that helps to form or strengthen their relationship with the NPO. The timing of accountability is throughout the beneficiaries’ engagement with the NPO.

• Transactional Accountability arises when beneficiaries’ engagement with the NPO is short-term and episodic. Here, the beneficiaries seek accountability that helps to complete their engagement with the NPO, thereby enacting an exchange or transaction between the beneficiary and the NPO. The timing of accountability is at the conclusion of the beneficiaries’ engagement with the NPO.

8.2.2 Implication for theory.

8.2.2.1 NPO accountability theory and critical accounting theory.

As discussed, NPO accountability literature focuses on considering

accountability to whom, for what, and how? (Ebrahim, 2016; Jordan, 2007; Murtaza,

2012; O’Dwyer & Boomsma, 2015). The NPO accountability literature also focuses

on considering the accountability directions (Murtaza, 2012). Both of these

perspectives locate the NPO centrally, and in doing so, question accountability on the

NPO’s terms.

Dialogic accounting theory has offered an additional consideration, which

extends the accountability framework and changes the central focus of attention. This

extension is the questioning of accountability on whose terms. When applied to

beneficiaries, as the central focus of attention, this research has questioned accountably

on the beneficiaries’ terms.


Through questioning accountability on the beneficiaries’ terms, the addition of

accountability when is made to Ebrahim’s (2016) accountability framework. This has

enabled the ability to build upon NPO accountability theory, through an extension of

this accountability framework. This research has found, that viewing the accountability

framework on the beneficiaries’ terms, necessitates questioning the timing of

accountability. An accountability framework on the beneficiaries’ terms is depicted

within Figure 24 (below):

Figure 24. An accountability framework on the beneficiaries' terms

In addition, this research has presented an extension to critical accounting theory. It

has done this through proposing a means of enacting accountings that sustain an

ongoing engagement with agonism and dissention (Brown & Tregidga, 2017, p. 19).

Among the many values of beneficiary engagement through participatory evaluation,

framed by agonistic pluralism, is its ability to present an option for what agonistic

social institutions and accountings might actually look like in practice (Brown &

Tregidga, 2017, p. 19).

8.2.2.2 Evaluation theory.

The findings of this research have implications for evaluation theory and

scholarly literature. The necessity of considering the implications of the timeframe of

beneficiary participation within the NPO, upon their evaluation needs, is significant in


building upon participative evaluation theory (Greene, 1997) and transformative

participatory evaluation theory (Mertens, 1999). This is an important contribution to

the corpus of literature calling for beneficiaries to be key actors in evaluation

processes, and not mere objects of it (Conroy, 2005). Importantly, this research details

how beneficiaries can be actors in their evaluations.

In developing evaluation frameworks and instruments positioned from within an

agonistic approach to democracy (Mouffe, 1999), this research presents a challenge to

the evaluation theory grounded within a deliberative democratic space (House &

Howe, 2000). Whilst acknowledging the merits of deliberative democratic evaluation,

this research questions its ability to affect change in the social order for marginalised

participants, when power asymmetries are present within the deliberative process. In

embracing agonistic pluralism, this research proposes an Agonistic Democratic

Evaluation, which supports Mathison’s (2018) call to move evaluation away from the

neo-liberal agenda. An agonistic approach to democratic evaluation acknowledges the

power asymmetries present within evaluation participants, and embraces the provision

of spaces for contestation, rather than for consensus.

Agonistic pluralism does not try to eliminate differences between adversaries.

Rather, it attempts to mobilise difference towards the promotion of democratic designs

(Mouffe, 1999). In this way, an agonistic approach to democratic evaluation can

provide a platform within which the marginally represented can have a place and a

voice within the democratic arena. Here, the arena takes the form of the NPO. In this

regard, the evaluation instruments developed through this research act as invitations

for beneficiaries to participate within the democratic space of their NPO. This presents

a contribution to the extension of evaluation theory.


8.2.3 Implications for practice.

8.2.3.1 NPO accountability practice. The findings of this research present the ability to impact upon NPOs’

accountability practices. In acknowledging that beneficiaries have accountability

needs dependent upon their timeframe of participation, NPOs can better target their

downwards accountability structures. A NPO with beneficiaries who have a long-term,

ongoing engagement of participation, can use the understandings drawn from this

research of relational accountability, to inform their accountability practices. This

could be generalised to include NPOs such as aged care facilities or preschools, where

the beneficiaries’ engagement is typically long-term and ongoing. Embracing

relational accountability would enable a strengthening of the relationship between the

beneficiary and the NPO, through the use of the relational accountability evaluation

instrument within its downwards accountability structure (see Appendix D). Similarly,

the reverse can be applied to transactional accountability NPOs, for example legal

advice centres. Here, downwards accountability structures can be influenced through

an understanding of the characteristics of transactional accountability, and the use of

the transactional accountability evaluation instrument (see Appendix D).

8.2.3.2 NPO evaluative practice.

The findings of this research have significant potential to impact upon evaluation

practice. The evaluation frameworks and instruments developed from the findings of

the research, that support either a relational or transactional approach to accountability,

can be used, or built upon within the NPO sector. They provide sound starting points

from which NPOs can begin to involve beneficiaries in evaluation. If staff and board

of NPOs are to consider the timeframe of their beneficiaries’ participation, and match

this to either a relational or transactional approach, they can develop evaluation

structures that enable the empowerment of their beneficiary group. This can be done


by involving their beneficiaries within evaluation structures that meet the particular

accountability needs of that beneficiary group.

8.2.4 Implications for policy.

The findings of the research also may have implications for policy. Traditional

service delivery and accountability technologies have typically been framed in terms

of the principal/agent relationship which privileges the powerful stakeholders.

However, as discussed within Section 7.4.5, when the traditional directions of

accountability (Murtaza, 2012) are viewed critically, on the terms of the beneficiaries,

the discrete accountability directions begin to blur, and a symbiotic, mutually

beneficial relationship emerges. When upwards accountability is framed from the

terms of the beneficiary, it can become an instrument of downwards accountability.

Similarly, downwards accountability can also be beneficial to meeting upwards

accountability demands.

Currently accountability is predominately tied to those with the most power over

the NPO (for example government, donors and other funders), and linked to funding

which NPOs clearly need to be sustainable. If, however, those current power holders

were able to shift that power towards downwards accountability, an act of

empowerment of beneficiaries could occur. This could be done by organisational or

government policy acknowledging the importance of hearing beneficiary voice and

explicitly calling for the involvement of beneficiaries within accountability processes,

such as evaluation.

8.3 Limitations of the Research

Some studies indicate that case study research has limitations in regards to a lack

of replicability due to the highly contextual nature of the inquiry, and problems relating

to control mechanisms needed to account for rival explanations (Recker, 2013, p. 95).


The sample size of two NPOs within the case study presents as a limitation to the

generalisability of this research outside of the research arena. However, this qualitative

research is not trying to be statistically generalisable. Instead, this research is aimed at

exploring relationships and providing insights that can be used to build theory

(Saunders et al., 2016). As such, the research is focused more upon achieving analytic

generalisability, where researchers strive to generalise from particulars to broader

constructs or theories (Polit & Beck, 2010, p. 1453). In this regard the small sample

size has been beneficial in allowing for a more in-depth study of the phenomena present

within each case. This is frequently the goal of qualitative research, as researchers

strive to provide a “rich, contextualized understanding of human experience” through

the study (Polit & Beck, 2010, p. 1452). However, the desire to be able to generalise

the approaches of relational and transactional accountability across the wider sector of

NPOs, provides an avenue for future research, and will be discussed within Section 8.4.

The use of the industry research partner (IRP) to provide access to research

participants, was important in reducing researcher bias. It ensured the NPOs were

independent and not known to the research team prior to participation. The use of the

IRP also assisted in providing access to marginal groups through a safe passageway,

due to the pre-existing relationship between the IRP and the NPO. However, a possible

limitation is presented in consideration of the nature of NPOs that access the service

of the IRP. This is because the IRP provides a service to assist NPOs to measure

outcomes for clients. From this perspective, NPOs that engage the services of the IRP

could be considered biased towards the benefits of evaluation at the outset. This raises

concerns of participation bias (Saunders et al., 2016, p. 397). That being said, the

perspectives of the beneficiaries from each case are outside of this bias.


8.4 Future Research

Although presented as a limitation (Section 8.3), the findings of this research in

relation to the generalisability of the relational and transactional approach to

accountability, present a notable opportunity for further research. Further research

could enable the ability to address this limitation, through the use of a wider sample of

cases. Researching the applicability of relational and transactional accountability

across a wider sample of NPOs will enable the testing of this theory. Researching with

a wider sample size of NPOs, will also allow the possibility of other approaches to

accountability with beneficiaries to arise.

Another area for future research is presented through Case A’s and Case B’s

future use of the drafted evaluation instruments with their beneficiary groups.

Researching the implications of the use of the evaluation instrument within each NPO

will help to ascertain its value, and the impact (if any) that it had upon the beneficiary

group and the NPO’s service delivery.

The theoretical positioning of this research also generates avenues for further

research particularly within the critical accounting area. The application of dialogic

accounting theory to participative evaluation theory presents opportunities for a more

thorough examination of an agonistic approach to democratic evaluation. The

intertwining of these two disciplines presents opportunities for theoretical and practical

extensions.

Similarly, the addition of accountability when to Ebrahim’s (2016)

accountability framework, presents an opportunity for further study. Whilst Mashaw

(2014) considers accountability when within his accountability regime, this is in

relation to a time lag, or “long separation between decisions and effect” (p. 575). This

study positioned beneficiaries at the centre, and identified a need to question when


accountability should be discharged to them. Consideration of accountability when

essentially adds a temporal dimension to accountability research. It could well be that

accountability when is important to consider in questioning the needs of all stakeholder

groups.

8.5 Summary

This research began with a motivation to address social injustices by

empowering marginalised groups through their participation in evaluation and

accountability processes. It sought to explore and document current practices of

beneficiary involvement within accountability and evaluation processes within NPOs,

and to develop systems that enhance accountability to beneficiaries. The research was

attempting to create new visibilities, and acknowledge beneficiaries’ rights to have

contested narratives (Brown, 2009).

Although minor in comparison to the duress experienced daily by marginalised

groups of people within the world at large, this research is contributing to social

change. This research has given voice to eight marginal people, in an attempt to

provide benefit for many more, and has designed an instrument in line with the

actioning of the SDGs at a subnational level. In listening to and valuing the words of

the beneficiaries (and other participants) interviewed within Case A and Case B, this

research has formed new understandings of what it means for NPOs to be accountable

downwards. It has identified different modes of downwards accountability, in its

attempt to reconfigure the traditional, power based, accountability directions, by

placing beneficiaries front and centre.

This research has exceeded its objectives by identifying the need to consider

accountability when in relation to beneficiary participation in evaluation. In doing so,

it has extended theory and could have implications for practice and policy


development. In keeping with its critical methodological positioning, this research has

enabled praxis through creating evaluation instruments based upon the words of the

beneficiaries, for the beneficiaries to use within their NPOs. By doing so, this research

has given back to the participants in an attempt to reduce the inequalities present

between the researcher and the researched.

Finally, this research has engaged in the elements of critical change. It has looked

into circumstances of injustice, interpreted the findings as a critique of that

circumstance, and used those findings to attempt to mobilise and inform change

(Patton, 2015, p. 692). In embracing a critical perspective, this research has applied

dialogic accounting theory to more traditional ways of viewing NPO accountability.

In doing so, it has responded to the call raised by Brown (2009) for research that

critically examines traditional approaches to accounting and accountability.

References 203

References

Anheier, H. (2005). Nonprofit Organizations: Theory, Management, Policy. London:

Routledge.

Australian Accounting Standards Board [AASB]. (2016). Framework for the

Preparation and Presentation of Financial Statements. Canberra, viewed 8

August 2018, http://www.aasb.gov.au.

Australian Charities and Not-for-profits Commission. (2018). How does ACNC

define charity size? Canberra, viewed 8 August 2018 http://www.acnc.gov.

au/ ACNC/ Manage/Reporting/ SizeRevenue/ACNC/Report/SizeRevenue.

Aspx

Australian Government Department of Health. (2018). Day to Day Living in the

Community Program. Reviewed June 11 2018 http://www.health.gov.au/

internet/ main/publishing.nsf/content/men tal-d2dl

Babbie, E. (2015). The Practice of Social Research (Fourteenth edition). Belmont,

CA: Cengage Learning.

Baur, V., Abma, T., & Widdershoven, G. (2010). Participation of marginalized

groups in evaluation: Mission impossible? Evaluation and Program

Planning, 33(3), 238–245. doi:10.1016/j.evalprogplan.2009.09.002

Bebbington, J., Brown, J., Frame, B., & Thomson, I. (2007). Theorizing engagement:

the potential of a critical dialogic approach. Accounting, Auditing &

Accountability Journal, 20(3), 356–381. doi:10.1108/09513570710748544

Bebbington, J., & Unerman, J. (2018). Achieving the United Nations Sustainable

Development Goals. Accounting, Auditing & Accountability Journal, 31(1),

2–24. doi:10.1108/AAAJ-05-2017-2929

Bechange, S., & Paton, C. (2010). Retracted: Determinants of project success among

HIV/AIDS nongovernmental organizations (NGOs) in Rakai, Uganda.

International Journal of Health Planning and Management, 25(3), 215–230.

https://doi.org/10.1002/hpm.1025

Benjamin, L. M. (2008). Evaluator's role in accountability relationships:

Measurement technician, capacity builder or risk manager?

Evaluation, 14(3), 323-343.

B Lab. (2018). About B Corps. Retrieved from https://bcorporation.net/about-b-corps

References 204

Blackburn, N., Brown, J., Dillard, J., & Hooper, V. (2014). A dialogical framing of

AIS-SEA design. International Journal of Accounting Information Systems,

15(2), 83–101. doi:10.1016/j.accinf.2013.10.003

Bond, S. (2011). Negotiating a “democratic ethos”: moving beyond the agonistic –

communicative divide. Planning Theory, 10(2), 161–186.

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative

Research in Psychology, 3(2), 77–101. doi:10.1191/1478088706qp063oa

Brett, E. (2003). Participation and accountability in development management.

Journal of Development Studies, 40(2), 1–29.

doi.org/10.1080/00220380412331293747

Brinkmann, S. (2018). The Interview. In Denzin, N., & Lincoln, Y. (Eds.). The Sage

Handbook of Qualitative Research (Fifth edition.). Los Angeles: Sage.

Brinkmann, S., & Kvale, S. (2015). InterViews: Learning the Craft of Qualitative

Research Interviewing (Third edition.). Los Angeles: Sage.

Brown, J. (2009). Democracy, sustainability and dialogic accounting technologies:

Taking pluralism seriously. Critical Perspectives on Accounting, 20(3), 313–

342. doi:10.1016/j.cpa.2008.08.002

Brown, J. (2017). Democratizing accounting: Reflections on the politics of “old” and

“new” pluralisms. Critical Perspectives on Accounting, 43, 20–46. doi:10.10

16/j.cpa.2016.11.001

Brown, J., & Dillard, J. (2015a). Opening Accounting to Critical Scrutiny: Towards

Dialogic Accounting for Policy Analysis and Democracy. Journal of

Comparative Policy Analysis: Research and Practice, 17(3), 247-268.

doi:10.1080/13876988.2014.989684

Brown, J., & Dillard, J. (2015b). Dialogic Accountings for Stakeholders: On

Opening Up and Closing Down Participatory Governance. Journal of

Management Studies, 52(7), 961–985. doi:10.1111/joms.12153

Brown, J., & Tregidga, H. (2017). Re-politicizing social and environmental

accounting through Rancière: On the value of dissensus. Accounting,

Organizations and Society, 61, 1–21. doi:10.1016/j.aos.2017.08.002

Bryson, J., Patton, M., & Bowman, R. (2011). Working with Evaluation

stakeholders: A Rationale, Step-Wise Approach and Toolkit. Evaluation and

Program Planning, 34(1), 1–12. doi:10.1016/j.evalprogplan.2010.07.001

Burger, R. (2012). Reconsidering the Case for Enhancing Accountability Via

Regulation. VOLUNTAS: International Journal of Voluntary and Nonprofit

References 205

Organizations, 23(1), 85–108. doi:10.1007/s11266-011-9238-9

Burlaud, A., & Colasse, B. (2011). International accounting standardisation: Is

politics back? Accounting in Europe, 8(1), 23-47. doi:10.1080/174494

80.2011.574412

Callahan, K. (2007). Citizen Participation: Models and Methods. International

Journal of Public Administration, 30(11), 1179–1196. doi:10.1080/019

00690701225366

Campbell, C., & Bunting, C. (1991). Voices and paradigms: Perspectives on critical

and feminist theory in nursing. Advances in Nursing Science, 13(3), 1–15.

doi:10.1097/00012272-199103000-00004

Campbell, D. A., & Lambright, K. T. (2016). Program performance and multiple

constituency theory. Nonprofit and Voluntary Sector Quarterly, 45(1), 150-

171. doi:10.1177/0899764014564578

Catchpowle, L., & Smyth, S. (2016). Accounting and social movements: An

exploration of critical accounting praxis. Accounting Forum, 40(3), 220–234.

doi:10.1016/j.accfor.2016.05.001

Chanrith, N. (2004). Strengthening NGDO accountability through beneficiary

participation: Lessons learned from two Cambodian NGDOs. Forum of

International Development Studies, 25, 183-213

Chiapello, E. (2017). Critical accounting research and neoliberalism. Critical

Perspectives on Accounting, 43, 47–64. doi:10.1016/j.cpa.2016.09.002

Chouinard, J. (2013). The Case for Participatory Evaluation in an Era of

Accountability. American Journal of Evaluation, 34(2), 237–253.

doi:10.1177/1098214013478142

Conroy, D. K. (2005). Non-profit organisations and accountability - a comment on

the Mulgan and Sinclair frameworks. Third Sector Review, 11(1), 103-116

Cooper, S. (2014). Transformative evaluation: organisational learning through

participative practice. The Learning Organization, 21(2), 146–157.

doi:10.1108/TLO-03-2013-0003

Cordery, C., & Sim, D. (2017). Dominant stakeholders, activity and accountability

discharge in the CSO sector. Financial Accountability & Management, 34(1),

77–96. doi:10.1111/faam.12144

Cornwall, A. (2008). Unpacking “Participation”: models, meanings and practices.

Community Development Journal, 43(3), 269–283. doi:10.1093/cdj

/bsn010

References 206

Cousins, J., Whitmore, E. (2007). Framing Participatory Evaluation. In King, J.,

Cousins, J., Whitmore, E., & Mathison, S. (Eds.). Making sense of

participatory evaluation: Framing participatory evaluation. New Directions

for Evaluation, 2007(114), 83–105. doi:10.1002/ev.226

Crotty, M. (1998). The Foundations of Social Research: Meaning and

Perspective in the Research Process. Thousand Oaks, London: Sage

Publications

Cunliffe, A. (2008). Orientations to Social Constructionism: Relationally

Responsive Social Constructionism and its Implications for Knowledge and

Learning. Management Learning, 39(2), 123–139. doi:10.1177/1350507607

087578

Dahler-Larsen, P. (2018). Qualitative Evaluation Methods, Ethics, and Politics

With Stakeholders. In Denzin, N., & Lincoln, Y. (Eds.). The Sage Handbook

of Qualitative Research (Fifth edition.). Los Angeles: Sage

Davidsdottir, S., & Lisi, P. (2007). Effects of Deliberative Democracy on School

Self-Evaluation. Evaluation, 13(3), 371–386. doi:10.1177/13563890070

78628

Dawkins, C. E. (2014). The principle of good faith: Toward substantive

stakeholder engagement. Journal of Business Ethics, 121(2), 283-295.

doi:10.1007/s105 51-013-1697-z

Deegan, C. (2017). Twenty five years of social and environmental accounting

research within Critical Perspectives of Accounting: Hits, misses and ways

forward. Critical Perspectives on Accounting, 43, 65–87. doi:10.1016/j.cpa.

2016.06.005

Deleuze, G., & Guattari, F. (2013). A Thousand Plateaus: Capitalism and

Schizophrenia. London: Bloomsbury Academic

Denzin, N. K. (2001). The reflexive interview and a performative social science.

Qualitative Research, 1(1), 23-46. 10.1177/146879410100100102

Denzin, N., & Lincoln, Y. (2018). The Discipline and Practice of Qualitative

Research. In Denzin, N., & Lincoln, Y. (Eds.) The Sage handbook of

qualitative research (Fifth edition.). Los Angeles: Sage

Dillard, J., & Ruchala, L. (2005). The rules are no game. Accounting, Auditing &

Accountability Journal, 18(5), 608–630. doi:10.1108/09513570510620475

Dillard, J., & Vinnari, E. (2017). A case study of critique: Critical perspectives on

critical accounting. Critical Perspectives on Accounting, 43, 88–109.

References 207

Dillard, J., & Yuthas, K. (2013). Critical dialogics, agonistic pluralism, and

accounting information systems. International Journal of Accounting

Information Systems, 14, 113-119. doi:10.1016/j.accinf.2011.07.002

Dillard, J., Yuthas, K., & Baudot, L. (2016). Dialogic framing of accounting

information systems in social and environmental accounting domains:

Lessons from, and for, microfinance. International Journal of Accounting

Information Systems, 23, 14–27. doi:10.1016/j.accinf.2016.10.001

Ebrahim, A. (2003a). Making sense of accountability: Conceptual perspectives for

northern and southern nonprofits. Nonprofit Management and Leadership,

14(2), 191–212. doi:10.1002/nml.29

Ebrahim, A. (2003b). Accountability in practice: Mechanisms for NGOs. World

Development, 31(5), 813-829. Retrieved from http://search.proquest.com/

docview/231623135/

Ebrahim, A. (2005). Accountability myopia: Losing sight of organizational learning.

Nonprofit and Voluntary Sector Quarterly, 34(1), 56-87

Ebrahim, A. (2007). Towards a reflective accountability in NGOs. In Weisband &

Ebrahim (Eds.), Global Accountabilities (pp. 193-224). Cambridge:

University Press

Ebrahim, A. (2009). Placing the normative logics of accountability in “Thick”

perspective. American Behavioral Scientist, 52(6), 885-904. doi:10.1177/000

2764208327664

Ebrahim, A. (2016). The Many Faces of Nonprofit Accountability. In Renz, D. &

Herman, R. (Eds.). The Jossey-bass Handbook of Nonprofit Leadership and

Management. New Jersey: Jossey-Bass & Pfeiffer Imprints, Wiley.

Eisenhardt, K. (1989). Building Theories from Case Study Research. The Academy

of Management Review, 14(4), 532–550

Elstad, T., & Eide, A. (2009). User participation in community mental health

services: exploring the experiences of users and professionals. Scandinavian

Journal of Caring Sciences, 23(4), 674–681. https://doi.org/10.1111/j.1471-

6712.2008.00660.x

Estrella, M. & Gaventa, J. (1998). Who counts reality? Participatory monitoring and

evaluation: A literature review (Working paper). London: Institute of

Develop Studies

References 208

Fetterman, D., Rodríguez-Campos, L., Wandersman, A., & O’Sullivan, R. G. (2014).

Collaborative, participatory, and empowerment evaluation: Building a strong

conceptual foundation for stakeholder involvement approaches to evaluation (A

response to Cousins, Whitmore, and Shulha, 2013). American Journal of

Evaluation, 35(1), 144-148. doi:10.1177/1098214013509875

Gaventa, J. (2002). Exploring Citizenship, Participation and Accountability. IDS

Bulletin, 33(2), 1–14. https://doi.org/10.1111/j.1759-5436.2002.tb00020.x

Geertz, C. (1973). The Interpretation of Cultures: Selected Essays. New York: Basic

Books

Gigler, B., Bailur, S., Naylor, D., Pradhan, S., & Rajani, R. (2014). Closing the

Feedback Loop: Can Technology Bridge the Accountability Gap? Washington,

District of Columbia: The World Bank

Graham, C. (2013). Teaching accounting as a language. Critical Perspectives on

Accounting, 24(2), 120–126. doi:10.1016/j.cpa.2012.01.006

Gramsci, A. 1971. Selections From the Prison Notebooks. New York: International

Publishers

Gray, R., Brennan, A., & Malpas, J. (2014). New accounts: Towards a reframing of

social accounting. Accounting Forum, 38(4), 258–273. doi:10.1016/j.accfor.

2013.10.005

Greene, J. (1997). Participatory Evaluation. In Mabry L. (Ed.), Evaluation and the

Postmodern Dilemma. London: JAL.

Greene, J. (2013). On rhizomes, lines of flight, mangles, and other assemblages.

International Journal of Qualitative Studies in Education, 26(6), 749–758

doi:10.1080/09518398.2013.788763

Guba, E. & Lincoln, Y. (1989). Fourth Generation Evaluation. CA: Sage.

Guba, E. & Lincoln, Y. (1998). Competing Paradigms in Qualitative Research. In

Denzin, N., & Lincoln, Y. (Eds.), The Landscape of Qualitative Research:

Theories and Issues (pp 195-220). Calif: Sage

Hall, M. (2014). Evaluation logics in the third sector. VOLUNTAS: International

Journal of Voluntary and Nonprofit Organizations, 25(2), 307-336.

doi:10.1007/s11266-012-9339-0

Halvorsen, K. (2003). Assessing the Effects of Public Participation. Public

Administration Review, 63(5), 535–543

Hanberger, A. (2004). Democratic governance and evaluation. Paper presented at the

Sixth EES (European Evaluation Society) Conference (pp. 1-24). Germany.

References 209

Hoefer, R. (2000). Accountability in action? Program evaluation in nonprofit human

service agencies. Nonprofit Management and Leadership, 11(2), 167-177.

doi:10.1002/nml.11203

Hoon, A., Oliver, E., Szpakowska, K., & Newton, P. (2015). Use of the “Stop, Start,

Continue” method is associated with the production of constructive

qualitative feedback by students in higher education. Assessment &

Evaluation in Higher Education, 40(5), 1–13. doi:10.1080/02602938.2014.

956282

House, E. (2006). Democracy and Evaluation. Evaluation, 12(1), 119–127.

doi:10.1177/1356389006064196

House, E., & Howe, K. (2000). Deliberative Democratic Evaluation. New Directions

for Evaluation, (85), 3–12.

Hreinsdottir, A., & Davidsdottir, S. (2012). Deliberative Democratic Evaluation in

Preschools. Scandinavian Journal of Educational Research, 56(5), 519–537.

doi:10.1080/00313831.2011.599426

Jacobs, A. (2010). Creating the Missing Feedback Loop. IDS Bulletin, 41(6), 56–64.

doi:10.1111/j.1759-5436.2010.00182.x

Jacobs, A., & Wilford, R. (2010). Listen first: A pilot system for managing

downward accountability in NGOs. Development in Practice, 20(7), 797-811

Jensen, M., & Meckling, W. (1976). Theory of the firm: Managerial behavior,

agency costs and ownership structure. Journal of Financial Economics, 3(4),

305–360. doi:10.1016/0304-405X(76)90026-X

Jordan, L. (2007). A rights-based approach to accountability. In Weisband, E. &

Ebrahim, A. (Eds.), Global Accountabilities (pp. 151-167). Cambridge:

University Press

Keutel, M., and Werner, M. (2011). Interpretive case study research: Experiences

and recommendations, MCIS 2011 Proceedings, September 3–5, Limassol,

Cyprus

Kilby, P. (2006). Accountability for Empowerment: Dilemmas Facing Non-

Governmental Organizations. World development (0305-750X), 34 (6), p.

951. doi:10.1016/j.worlddev.2005.11.009

Leeuwis, C. (2000). Reconceptualizing Participation for Sustainable Rural

Development: Towards a Negotiation Approach. Development and Change,

31(5), 931–959. doi:10.1111/1467-7660.00184

References 210

Lehman, C. (2013). Knowing the unknowable and contested terrains in accounting.

Critical Perspectives on Accounting, 24, 136-144. doi:10.1016/j.cpa.2012.03.

005

Leech, N., & Onwuegbuzie, A. (2008). Qualitative Data Analysis: A Compendium

of Techniques and a Framework for Selection for School Psychology Research

and Beyond. School Psychology Quarterly, 23(4), 587–604. doi:10.1037/1045-

3830.23.4.587

Leroux, K., Wright, N., Alexander, J., Brudney, J., & Yang, K. (2010). Does

Performance Measurement Improve Strategic Decision Making? Findings From

a National Survey of Nonprofit Social Service Agencies. Nonprofit and

Voluntary Sector Quarterly, 39(4), 571–587. https://doi.org/10.1177/0899764

009359942

Mashaw, J. L. (2005). Accountability and Institutional Design: Some Thoughts on

the Grammar of Governance. In Dowdle. M. (Ed.) Public Accountability (pp.

115-156). Cambridge: Cambridge University Press

Mashaw, J. L. (2014). Accountability and Time. In Bovens, M., Goodin R. &

Schillemans, T. (Eds.), The Oxford Handbook Public Accountability (pp. 574-

588). Oxford: Oxford University Press

Mathie, A., & Greene, J. (1997). Stakeholder participation in evaluation: How

important is diversity? Evaluation and Program Planning, 20(3), 279–285.

https://doi.org/10.1016/S0149-7189(97)00006-2

Mathison, S. (2018). Does evaluation contribute to the public good? Evaluation,

24(1), 113–119. doi:10.1177/1356389017749278

McNamara, P. (2013). Giving voice to children and young people in research:

Applying rights-based frameworks to meet ethical challenges. Developing

Practice: The Child, Youth and Family Work Journal, (37), 55–66

Mercelis, F., Wellens, L., & Jegers, M. (2016). Beneficiary Participation in Non-

Governmental Development Organisations: A Case Study in Vietnam. The

Journal of Development Studies, 52(10), 1446–1462. doi:10.1080/00220388.

2016.1166209

Mertens, D. (1999). Inclusive Evaluation: Implications of Transformative Theory for

Evaluation. Presidential Address. American Journal of Evaluation, 20(1), 1– 14

Mertens, D. (2009). Transformative Research and Evaluation. New York: Guilford

Press

References 211

Mertens, D. & Wilson A. (2012). Program Evaluation Theory and Practice: A

Comprehensive Guide. New York: Guildford Publications

Miles, M., Huberman, A., & Saldaña, J. (2014). Qualitative Data Analysis: A

Methods Sourcebook (Third edition.). California: Sage

Mouffe, C. (1999). Deliberative democracy or agonistic pluralism? Social Research,

66(3), 745–758. Retrieved from http://search.proquest. com/docview/

209669064/

Mouffe, C. (2012). The Ethics and Politics of Democracy: An Agonistic Approach.

In Österreichische, L., Greif, H., & Weiss, M. (Eds.). Ethics, Society, Politics

Proceedings of the 35th International Wittgenstein Symposium, Austria,

Berlin: De Gruyter

Mouffe, C. (2013). Agonistics Thinking the World Politically. London: Verso

Murtaza, N. (2012). Putting the lasts first: The case for community-focused

and peer-managed NGO accountability mechanisms. Voluntas: International

Journal of Voluntary and Nonprofit Organizations, 23(1), 109-125.

doi:10.1007/s11 266-011-9181-9

Myers, M. (2009). Qualitative Research in Business & Management. Los Angeles:

Sage

Najam, A. (1996). NGO accountability: A conceptual framework. Development

Policy Review, 14(4), 339-353. doi:10.1111/j.1467-7679.1996.tb00112.x

Nikkhah, H. & Redzuan, M. (2009). Participation as a medium of empowerment in

community development. European Journal for Social Sciences, 11, 170-176

O’Dwyer, B. (2005). The construction of a social account: a case study in an

overseas aid agency. Accounting, Organizations and Society, 30(3), 279–296.

https://doi.org/10.1016/j.aos.2004.01.001

O'Dwyer, B., & Boomsma, R. (2015). The co-construction of NGO accountability:

Aligning imposed and felt accountability in NGO-funder accountability

relationships. Accounting, Auditing & Accountability Journal, 28(1), 36.

doi:10.1108/AAAJ-10-2013-1488

O'Dwyer, B., & Unerman, J. (2007). From functional to social accountability:

Transforming the accountability relationship between funders and non-

governmental development organisations. Accounting, Auditing &

Accountability Journal, 20(3), 446-471. doi:10.1108/09513570710748580

References 212

O’Dwyer, B., & Unerman, J. (2010). Enhancing the role of accountability in

promoting the rights of beneficiaries of development NGOs. Accounting and

Business Research, 40(5), 451–471. doi:10.1080/00014788.2010.9995323

O’Leary, S. (2016). Grassroots accountability promises in rights-based approaches to

development: The role of transformative monitoring and evaluation in NGOs.

Accounting, Organizations and Society, 63, 21–41. doi:10.1016/j.aos.2016.

06.002

Owen, J. M. (1993). Program Evaluation: Forms and Approaches (1st edition).

N.S.W: Allen & Unwin

Patton, M. Q. (2015). Qualitative Research & Evaluation Methods: Integrating

Theory and Practice (Fourth edition). California: Sage

Polit, D., & Beck, C. (2010). Generalization in quantitative and qualitative research:

Myths and strategies. International Journal of Nursing Studies, 47(11), 1451–

1458. doi:10.1016/j.ijnurstu.2010.06.004

Ponelis. S. (2015). Using Interpretive Qualitative Case Studies for Exploratory

Research in Doctoral Studies: A Case of Information Systems Research in Small

and Medium Enterprises. International Journal of Doctoral Studies, 10, 535–

550. Retrieved from https://doaj.org/article/9e3cca2064aa4186b6035 d98fb1a

98cb

Portela, A. (2012). Accountability and evaluation: Challenge to democracy.

ZARZADZANIE PUBLICZNE, 4(20), 73-85. Retrieved from

http://search.proquest.com/docview/1382985267/

Recker, J. (2013). Scientific Research in Information Systems: A Beginner’s Guide.

Berlin, Heidelberg: Springer Berlin Heidelberg

Rexhepi, J., & Torres, C. (2011). Reimagining Critical Theory. British Journal of

Sociology of Education, 32(5), 679–698. doi:10.1080/01425692.2011.596363

Rey- Garcia, M., Liket, K., Alvarez- Gonzalez, L., & Maas, K. (2017). Back to

Basics. Nonprofit Management and Leadership, 27(4), 493511. doi:10.1002/

nml.212 59

Ridder, H., Hoon, C., & Mccandless Baluch, A. (2014). Entering a Dialogue:

Positioning Case Study Findings towards Theory. British Journal of

Management, 25(2), 373–387. doi:10.1111/1467-8551.12000

Roberts, J. (1991). The possibilities of accountability. Accounting, Organizations

and Society, 16(4), 355–368. doi:10.1016/0361-3682(91)90027-C

References 213

Rossingh, B. (2012). Collaborative and participative research: Accountability and the

indigenous voice. Social and Environmental Accountability Journal, 32(2),

65

Ryan, K. & Destefano, L. (2001). Dialogue as a Democratizing Evaluation Method.

Evaluation, 7(2), 188-203. doi:10.1177/13563890 122209621

Sandelowski, M. (2001). Real qualitative researchers do not count: the use of

numbers in qualitative research. Research in nursing & health, 24(3), 230-

240

Saunders, M., Lewis, P., & Thornhill, A. (2016). Research Methods for Business

Students (Seventh edition.). Harlow, England: Pearson

Scherhaufer, P. (2014). Bridging the gap between the theory and practices of

stakeholder participation in integrated vulnerability assessments of climate

change. Systemic Practice and Action Research, 27(5), 449-463.

doi:10.1007/s11213-013-9294-8

Schmitz, H., Raggo, P., & Bruno-van Vijfeijken, T. (2012). Accountability of

transnational NGOs: Aspirations vs. practice. Nonprofit and Voluntary Sector

Quarterly, 41(6), 1175-1194. doi:10.1177/0899764011431165

Schmitz, H., & Mitchell, G. (2016). The other side of the coin: NGOs, Rights-

Based approaches, and public administration. Public Administration Review,

76(2), 252-262. doi:10.1111/puar.12479

Schwandt, T. & Gates, E. (2018). Case Study Methodology. In Denzin, N., &

Lincoln, Y. (Eds.). The Sage Handbook of Qualitative Research (Fifth

edition.). Los Angeles: Sage

Scotland, J. (2012). Exploring the Philosophical Underpinnings of Research:

Relating Ontology and Epistemology to the Methodology and Methods of the

Scientific, Interpretive, and Critical Research Paradigms. English Language

Teaching, 5(9), 9–16. doi:10.5539/elt.v5n9p9

Scriven, M. (1991). Evaluation Thesaurus (4th edition). Newbury Park, Calif: Sage

Shenton, A. (2004). Strategies for Ensuring Trustworthiness in Qualitative Research

Projects. Education for Information, 22(2), 63–75. doi:10.3233/EFI-2004-

22201

Sinclair, A. (1995). The chameleon of accountability: Forms and discourses.

Accounting, Organizations and Society, 20(2), 219-237. doi:10.1016

/0361- 3682(93)E0003-Y

References 214

Spence, C. (2009). Social accounting’s emancipatory potential: A Gramscian

critique. Critical Perspectives on Accounting, 20(2), 205–227. doi:10.1016/j.

cpa.2007.06.003

Stake, R. (1995). The Art of Case Study Research. Thousand Oaks: Sage

Stake, R. (2006). Multiple Case Study Analysis. New York: Guilford Press

Tambakaki, P. (2014). The Tasks of Agonism and Agonism to the Task: Introducing

“Chantal Mouffe: Agonism and the Politics of Passion.” Parallax, 20(2), 1–

13. doi:10.1080/13534645.2014.896543

Taut, S. (2008). What have we learned about stakeholder involvement in program

evaluation? Studies in Educational Evaluation, 34, 224-230. doi:10.1016/j.

stueduc.2008.10.007

Unerman, J., & O'Dwyer, B. (2006). Theorising accountability for NGO advocacy.

Accounting, Auditing & Accountability Journal, 19(3), 349-376. doi:10.11

08/09513570610670334

United Nations. (2003). Handbook on Non-Profit Institutions in the System of

National Accounts. New York: United Nations Publication

United Nations. (2015). Transforming our world: the 2030 Agenda for Sustainable

Development. New York: United Nations Publication

United Nations. (n.d.). Sustainable Development Goals. Viewed 8 August 2018,

www.un.org/sustainable development/sustainable development-goals

Vinnari, E., & Dillard, J. (2016). (ANT)agonistics: Pluralistic politicization of, and

by, accounting and its technologies. Critical Perspectives on Accounting, 39,

25–44. doi:10.1016/j.cpa.2016.02.001

Walsham, G. (1995). Interpretive case studies in IS research: nature and method.

European Journal of Information Systems, 4(2), 74. doi:10.1057/ejis.1995.9.

Walsham. G. (2006). Doing interpretive research. European Journal of Information

Systems, 15(3), 320–330.

Wedeen, L. (2010). Reflections on ethnographic work in political science. Annual

Review of Political Science, 13(1), 255-272.

Wellens, L., & Jegers, M. (2014). Beneficiary participation as an instrument of

downward accountability: A multiple case study. European Management

Journal, 32(6), 938-949. doi:10.1016/j.emj. 2014.03.004

Wellens, L., & Jegers, M. (2016). From Consultation to Participation. Nonprofit

Management and Leadership, 26(3), 295–312. doi:10.1002/nml.21191

References 215

Wellens, L., & Jegers, M. (2017). Beneficiaries’ participation in development

organizations through local partners: A case study in Southern Africa.

Development Policy Review, 35(S2), O196–O213. doi:10.1111/dpr.12279

White, S. (1996). Depoliticising development: The uses and abuses of participation.

Development in Practice, 6(1), 6–15. doi:10.1080/0961452961000157564

Whitmore, E., Guijt, I., Mertens, D.M., Imm, P.S., Chinman, M., & Wandersman,

A.(2006). Embedding improvements, lived experience, and social justice in

evaluation practice. In Shaw, I., Greene, J., & Mark, M. (Eds.). The SAGE

handbook of evaluation policies, programs and practices. London: Sage

World Vision UK, INTRAC, Social Impact Lab, CDA. (2016). Using beneficiary

feedback to improve development programmes: findings from a multi-country

pilot. https://www.intrac.org/resources/using-beneficiary-feedback-improve-

development-programmes-findings-multi-country-pilot

Yin, R. (2014). Case Study Research: Design and Methods (Fifth edition). Los

Angeles: Sage.

216 Appendices

Appendices

Appendix A – Approved Ethics Research Documents

PARTICIPANTINFORMATIONFORQUTRESEARCHPROJECT–Interview–

Anexplorationofevaluationasamechanismforenhancingparticipationandaccountabilityinnot-for-profitorganisations.

QUTEthicsApprovalNumber1700000820

RESEARCHTEAM

PrincipalResearcher:

MsKylieKingston MPhilstudent

AssociateResearchers:

DrCraigFurneaux PrincipalSupervisor

DrLauradeZwaan AssociateSupervisor SchoolofAccountancy,QUTBusinessSchool

QueenslandUniversityofTechnology(QUT) DrLynAlderman ExternalSupervisor PolicyandEvaluationBranch

CommonwealthDepartmentofSocialServicesDESCRIPTION

This research project is being undertaken as part of aMaster of Philosophy (accountancy)degreebyKylieKingston.The purpose of this research is to enhance accountability practices within not-for- profitorganisationsthroughexploringcurrentpracticesofbeneficiaryinvolvementintheevaluationprocesses of not-for-profit organisations, and developing evaluation frameworks aimed atenhancingbeneficiaryimpact.Youareinvitedtoparticipateinthisresearchprojectbecauseyouareeitherastaffmember,boardmemberorbeneficiaryofanot-for-profitorganisation.PARTICIPATION

Yourparticipationwillinvolveanaudiorecordedinterviewatyourorganisation’slocationorotheragreedlocationthatwilltakebetween15minutesto1hourofyourtime.Questionswillvaryslightlydependingonyourroleintheorganisation.Examplesofquestionscouldinclude:-Asaboardmemberdoyouhaveanyprocessesinplacecurrentlywithinyourorganisationthatallowbeneficiary’sideastoinfluenceorganisationalpractices/policies?Ifso,canyoutellmewhattheyare?

Appendices 217

- Asastaffmember,whatisimportanttoyoutoknowaboutyourbeneficiaries?- Asabeneficiary,whataresomeofyourneedswithinyourorganisation?- Asabeneficiary,canyoutellmeanywaysthatyouareabletotellthemanagementof

yourorganisationaboutyourneedsorideas?- Asabeneficiary,whatinformationwouldyouliketoreceivefromthemanagementof

yourorganisation?Yourparticipationinthisresearchprojectisentirelyvoluntary.Ifyoudoagreetoparticipateyoucanwithdrawfromtheresearchprojectwithoutcommentorpenalty.Youcanwithdrawanytimeduringtheinterview.Ifyouwithdrawwithintwoweeksafteryourinterview,onrequestanyidentifiableinformationalreadyobtainedfromyouwillbedestroyed.YourdecisiontoparticipateornotparticipatewillinnowayimpactuponyourcurrentorfuturerelationshipwithQUTorwithyournot-for-profitorganisation.

Attheconclusionoftheresearch,youcanalsochoosetoparticipateinadebriefingsessiontodiscussthefindingsoftheresearchandgivefeedbackontheevaluationframeworkdevelopedbytheresearcher.

EXPECTEDBENEFITS

Itisexpectedthatthisresearchprojectwillbenefityoudirectlythrough:• Theempowermentofvulnerablebeneficiarygroupings;• Increasingtheorganisation’ssustainabilitythroughstrengtheningrelationshipswithimportant

stakeholdergroups(beneficiaries);• Thedevelopmentofanevaluationframeworkfortheorganisationtousewithintheiraccountability

processes.

RISKS

Thereareminimalrisksassociatedwithyourparticipationinthisresearchproject.Theresearchteamhasidentifiedthefollowingpossiblerisks:

• Within the interviewprocess there isa riskofdiscomfortand/or inconvenience,however, the

severityofharmislow.• Although there is an unequal relationship between the board, staff and beneficiaries of the

organisation,thenatureofthisresearchistoempowerandsupportbeneficiaries.Assuch,it isbelievedthattheriskofharmduetothisunequalrelationshipislowandisnegatedbythebenefitsoftheresearch.Anyriskrelatingtothiswillbeminimisedthrough the nature of the interviewquestionswhicharenotseeking judgementfromyouupontheservice,voluntaryparticipationandtherighttoleaveatanytime.

PRIVACYANDCONFIDENTIALITY

Allcommentsandresponseswillbetreatedconfidentiallyunlessrequiredbylaw.

Astheresearchprojectinvolvesanaudiorecording:

� Therecordingwillbedestroyedafterithasbeentranscribed. � Therecordingwillnotbeusedforanyotherpurpose. � Onlythenamedresearcherswillhaveaccesstotherecording. � Itispossibletoparticipateintheresearchprojectwithoutbeingrecorded.Inthis

instance,theinterviewerwilltakewrittennotesduringtheinterview.

218 Appendices

Any data collected as part of this research project will be stored securely as per QUT’sManagementof researchdatapolicy.Pleasenotethatnon-identifiabledata fromthisprojectmaybeusedascomparativedatainfutureprojects.CONSENTTOPARTICIPATEWewouldliketoaskyoutosignawrittenconsentform(enclosed)toconfirmyouragreementtoparticipate.QUESTIONS/FURTHERINFORMATIONABOUTTHERESEARCHPROJECT

Ifyouhaveanyquestionsorrequirefurtherinformation,pleasecontactoneofthelistedresearchers:

KylieKingston [email protected] 0731381186

CraigFurneaux [email protected] 0731381186

LauradeZwaan [email protected] 0731382850LynAlderman [email protected] 0261462602

CONCERNS/COMPLAINTSREGARDINGTHECONDUCTOFTHERESEARCHPROJECT

QUTiscommittedtoresearchintegrityandtheethicalconductofresearchprojects.However,ifyoudohaveanyconcernsorcomplaintsabouttheethicalconductoftheresearchprojectyoumaycontacttheQUTResearchEthicsAdvisoryTeamon0731385123oremailhumanethics@qut.edu.au.TheQUTResearchEthicsAdvisoryTeamisnotconnectedwiththeresearchprojectandcanfacilitatearesolutiontoyourconcerninanimpartialmanner.

THANKYOUFORHELPINGWITHTHISRESEARCHPROJECT.PLEASEKEEPTHISSHEETFORYOURINFORMATION.

Appendices 219

CONSENTFORMFORQUTRESEARCHPROJECT–Interview–

Anexplorationofevaluationasamechanismforenhancingparticipationandaccountabilityinnot-for-profitorganisations.

QUTEthicsApprovalNumber1700000820

RESEARCHTEAM




STATEMENTOFCONSENT

Bysigningbelow,youareindicatingthatyou:

� Havereadandunderstoodtheinformationdocumentregardingthisresearchproject. � Havehadanyquestionsansweredtoyoursatisfaction. � Understandthatifyouhaveanyadditionalquestionsyoucancontacttheresearchteam. � Understandthatyourparticipationinthisresearchprojectisentirelyvoluntary.Ifyou

doagreetoparticipateyoucanwithdrawfromtheresearchprojectwithoutcommentorpenalty.Youcanwithdrawanytimeduringtheinterview.Ifyouwithdrawwithintwoweeksafteryourinterview,onrequestanyidentifiableinformationalreadyobtainedfromyouwillbedestroyed.

� Understandthatifyouhaveconcernsabouttheethicalconductoftheresearchprojectyou can contact the Research Ethics Advisory Team on 07 3138 5123 or [email protected].

� Understandthatnon-identifiabledatafromthisprojectmaybeusedascomparativedatainfutureresearchprojects.

� Agreetoparticipateintheresearchproject. Pleaseticktherelevantboxbelow:

Iagreefortheinterviewtobeaudiorecorded.

Idonotagreefortheinterviewtobeaudiorecorded.

Iwouldbeinterestedinparticipatinginadebriefingsessionwithotherbeneficiariestogivemyfeedbackontheevaluationframeworkdevelopedbytheresearcher,andhearthefindingsoftheresearch.Mypreferredemailaddressisbelow.

Name

Signature

Date

220 Appendices

SubjectTitle:Participateinaresearchstudy:Anexplorationofevaluationasamechanismforenhancingparticipationandaccountabilityinnot-for-profitorganisations.

DearMadam/SirMynameisKylieKingstonfromtheSchoolofAccountancy,QueenslandUniversityofTechnology(QUT)andI’mdoingaMastersofPhilosophy.YouarereceivingthisemailasmycolleaguesandIareundertakingaresearchprojectexploringbeneficiaryinvolvementinevaluationpractices,innot-for-profitorganisations.Wearelookingforstaffmembers,boardmembersandbeneficiariestoparticipateinaninterviewbetween15minutestoone-hourinlength.Forstaffandboardinterviews,thefocuswillbeonthecurrentinvolvementofbeneficiarieswithintheevaluationprocessesoftheorganisation,andwhattheexpectationsoftheorganisationareinrelationtothis.Forbeneficiaryinterviewees,thefocuswillbeonwhatinformationbeneficiariesthinkisimportanttoreceivefromtheorganisation,whatinformationbeneficiariesthinkisimportantfortheorganisationtoknow;andwhatinvolvementbeneficiariescurrentlyhavewiththeorganisation,whattheywouldliketohaveandwaysthatmightbeachieved.PleaseviewtheattachedParticipantInformationSheetandConsentFormforfurtherdetailsonthestudy,andcontactdetailsoftheteammembers.Ifyouareinterestedinparticipatinginthisresearchproject,pleasecontactmyselfdirectlyviatelephoneoremail(preferred).Also,ifyouhaveanyquestionsaboutthisprojectyoucancontactmyselforanyoftheresearchteam.Anticipatedbenefitsoftheresearchincludetheempowermentofvulnerablebeneficiarygroupings,astrengtheningofrelationshipswithimportantstakeholdergroups(beneficiaries),andthedevelopmentofanevaluationframeworkfortheorganisationtousewithintheiraccountabilityprocesses.PleasenotethatthisstudyhasbeenapprovedbytheQUTHumanResearchEthicsCommittee(approvalnumber1700000820).Manythanksforyourconsiderationofthisrequest.Yourssincerely

KylieKingstonMastersStudent0731381186

kl.kingston@hdr.qut.edu.auDrCraigFurneauxSupervisor0731381186c.furneaux@qut.edu.auDrLauradeZwaanSupervisor0731382850laura.dezwaan@qut.edu.auSchoolofAccountancy,QUTBusinessSchool,QueenslandUniversityofTechnologyDrLynAldermanExternalSupervisor0261462602lyn.alderman@dss.gov.auCommonwealthDepartmentofSocialSecurities

Appendices 221

PARTICIPATEINRESEARCHInformationforProspectiveParticipants

Anexplorationofevaluationasamechanismforenhancingparticipationandaccountabilityinnot-for-

profitorganisations.Researchteamcontacts

PrincipalResearcher: MsKylieKingston MPhilstudentAssociateResearchers: DrCraigFurneaux PrincipalSupervisor

DrLauradeZwaan AssociateSupervisor SchoolofAccountancy,QUTBusinessSchoolQueensland

UniversityofTechnology(QUT) DrLynAlderman ExternalSupervisor PolicyandEvaluationBranch

CommonwealthDepartmentofSocialServices

Whatisthepurposeoftheresearch?The purpose of this research is to enhance accountability practices within not-for-profit organisations through exploring current practices of beneficiary involvement in the evaluation processes of not-for-profit organisations, and developing evaluation frameworks aimed at enhancing beneficiary impact. Areyoulookingforpeoplelikeme?

Theresearchteamis lookingforparticipants fromthreestakeholdergroupswithinanumberofnot-for-profit organisations who are clients of XXXX. These stakeholders would includerepresentativesfromstaff,boardandbeneficiarygroups.

Whatwillyouaskmetodo?

Yourparticipationwillinvolveattendinganinterviewsessionbetween15minsto1hrinlength,atyour organisation’s location or another suitable location of your choosing. For staff and boardinterviews, the principal researcher, Kylie Kingston,will be asking you questions relating to thecurrentinvolvementofbeneficiarieswithintheevaluationprocessesofyourorganisation,andwhattheexpectationsoftheorganisationareinrelationtothis.Forbeneficiaryinterviewees,Kyliewillbeaskingquestionsrelatingtowhatinformationyouthinkis important toreceive fromtheorganisation’smanagement,andwhat informationyouthink isimportantfortheorganisation’smanagementtoreceivefromyou.Kyliewillbealsointerestedinaskingwhatinvolvementyoucurrentlyhavewiththeorganisation,whatyouwouldliketohaveandwaysthatmightbeachieved.

Arethereanyrisksformeintakingpart?

Theresearchteamhasidentifiedthefollowingpossiblerisksinrelationtoparticipatinginthisstudy:

• Withintheinterviewprocessthereisariskofdiscomfortand/orinconvenience,however,theseverityofharmislow.

• Althoughthereisanunequalrelationshipbetweentheboard,staffandbeneficiaries of theorganisation,thenatureofthisresearchistoempowerandsupportbeneficiaries.Assuch,itisbelievedthattheriskofharmduetothisunequalrelationshipislowandisnegatedbythebenefitsoftheresearch.Anyriskrelatingtothiswillbeminimisedthroughthenatureoftheinterviewquestionswhicharenotseekingjudgementfromyouupontheservice,voluntaryparticipationandtherighttoleaveatanytime.

Yourparticipationinthisresearchprojectisentirelyvoluntary.Ifyoudoagreeto

Appendices 222

participateyoucanwithdrawfromtheresearchprojectwithoutcommentorpenalty.Youcanwithdrawanytimeduringtheinterview.Ifyouwithdrawwithintwoweeksafteryourinterview,onrequestanyidentifiableinformationalreadyobtainedfromyouwillbedestroyed.YourdecisiontoparticipateornotparticipatewillinnowayimpactuponyourcurrentorfuturerelationshipwithQUTorwithyournot-for-profitorganisation.

Arethereanybenefitsformeintakingpart?

Itisexpectedthatthisresearchprojectwillbenefityoudirectlythrough:• Theempowermentofvulnerablebeneficiarygroupings;• Increasingtheorganisation’ssustainabilitythroughstrengtheningrelationshipswith

importantstakeholdergroups(beneficiaries);• Thedevelopmentofanevaluationframeworkfortheorganisationtousewithintheir

accountabilityprocesses.

WillIbecompensatedformytime?

No,butwewouldverymuchappreciateyourparticipationinthisresearch.

Iaminterested–whatshouldIdonext?

Ifyouareinterestedinparticipatinginthisstudy,youcancontacttheresearcherfordetailsofthenextstep:

KylieKingston [email protected] 0731381186Youwillbeprovidedwithfurtherinformationtoensurethatyourdecisionandconsenttoparticipateisfullyinformed.

ThankYou! QUTEthicsApproval

Number:1700000820

Appendices 223

TRANSCRIBERCONFIDENTIALITYAGREEMENTFORQUTRESEARCHPROJECT

Anexplorationofevaluationasamechanismforenhancingparticipationandaccountabilityinnot-for-profit

organisations.QUTEthicsApprovalNumber1700000820

RESEARCHTEAMCONTACTS




SchoolofAccountancy,QUTBusinessSchool,QueenslandUniversityofTechnology(QUT)

THEAGREEMENT

As this research involves questioning individuals about sensitive issues, I the PrincipalResearcher in this research project, require you to sign this transcriber confidentialityagreement.Asthetranscriberforthisresearchprojectyoumust:

• Keepallinformationrelatedtothisprojectsecretandconfidential.• Notdisclosetoanypersonormakeknowninanymanneranypartoftheproject’s

information.• Keeptheproject’sinformationinasecureplacetoensurethatunauthorisedpersonsdo

nothaveaccesstoit.

SIGNATURES

ThisAgreementshallbeeffectivewhensignedanddatedbyallparties.

Transcriber Name

Sign

Date

Witness

Name

Sign

224 Appendices

Appendix B – Interview Protocol

StakeholderGroup:BeneficiaryInterviewProtocolKeyInformant:

SpecificRQ Examples Propertiesofthephenomena

Requiredattributesofakeyinformant

Suitableroles

Howcanevaluationenhanceaccountabilitytobeneficiaries?

Notforprofitorganisation(NPO)

AccountabilityEvaluationformatsServiceprovision

Abletogivewillingandvoluntaryconsent

PersonreceivingaserviceorsupportfromaNPO,classifiedasabeneficiaryoftheservice.

Procedures:- Gatherinformedconsent- InterviewwillbeconductedattheNPOofthekeyinformant,orotheragreedlocation,

atatimedesignatedsuitabletothem.- Iwillrecordtheinterview(ifagreedto)onarecordingdevicewhichwillbeplaced

betweenus.- Iwillalsobenote-takingduringtheinterviewprocess.- Theplannedtimeframefortheinterviewisupto1hour.- Thebelowscriptwillbeusedduringtheinterview.

Interviewquestions/probes/script:Hello MynameisKylieKingston.Thankyousomuchforagreeingtotakepartinmyresearchinvestigationandfor:(either)A. agreeingtobeaudiorecordedforuseinmyresearch.TheaudiorecordingandnotesImake

willonlybeusedbymyselfandmysupervisors.Therecordingwillbetranscribedandthendeleted.(Or)

B. andforallowingmetotakenotesduringtheinterview.Theinterviewwillnotbeusedforanyotherpurpose.

ThekeyquestionthatIaminvestigatingwithinthisinterviewis:howcanevaluationenhance

accountabilitytobeneficiaries?AndI’mhopingtobeabletodevelopanevaluationframeworkbaseduponyourideasthat

beneficiarieslikeyourselfcanusetobeabletohaveanimpactupontheservice.Inordertogainyourthoughtsonthistopic,IhaveformedsomequestionswhichI’mgoingtoask

you.IsitokayifIstartaskingyousomequestionsnow?(ifyes–proceed;ifno–asktoelaborateonreasonsandendinterview).

Questions:(Semi-structuredtoallowformorein-depthdiscussionbaseduponresponses.)

1. I’mwonderingifwecouldstartbyyoutellingmeabitaboutyourself?Likeyourname,yourinterests.

NatureofinvolvementinNPO:2. Canyoupleasedescribetomeyourinvolvementwith (nameof

NPO)?3. I’mwonderinghowlongyouhavebeeninvolvedwith andhowthat

involvementstarted?4. Whattypeofservicedotheyprovidetoyou?5. Istheresomeonewithintheorganisationthatyouinteractwithmorethanothers,like

acaseworkeretc.?

ReceivinginformationfromNPO:

Appendices 225

6. Inwhatwaysdoyoureceiveinformationabouttheorganisation?E.g.whattheyhaveachievedthisyear,howtheyhavehelpedpeople,whattheyhavespenttheirfundingon?

7. I’mwondering,ifyoucouldreceivemoreinformationfromtheorganisation,whatinformationwouldbeuseful,interestingorimportanttoyou?

GivinginformationtoNPO:

8. I’mthinkingaboutyourneedswithintheorganisation,orideasthatyoumighthave,thatyouthinkmighthelptheorganisation.Whataresomeofthewaysthatyoucanthinkof,thatyoucouldlettheorganisationknowaboutthem?

e.g.wouldwritingthemdownwork,istalkingtosomeonebetter,couldyouuseacomputeroratablet,whatelse?

Thankyousomuchforlettingmeinterviewyou.Ihavelearntalotandreallyenjoyedmeetingyou.

I’mwonderingifyouwouldliketobeinvolvedinafollowupmeetingalongwithotherbeneficiariesof ,sothatIcansharewithyousomeofthefindingsofmyresearchandalsogetsomefeedbackfromyouontheevaluationframeworkthatIhavedesignedbaseduponyourideas?

- ifyes,whatwouldbethebestwaytocontactyoutoletyouknowaboutthatmeetingdate/time?Thankyousomuchagainforyourtime!

226 Appendices

StakeholderGroup:StaffInterviewProtocolKeyInformant:



Suitableroles





PersonwhoisastaffmemberofaNPOe.g.CEO,manager,casemanager,executive,areamanager,teammanager,supportworker.

Procedures:- Gatherinformedconsent- InterviewwillbeconductedattheNPOofthekeyinformant,orotheragreedlocation,atatime

designatedsuitabletothem.- Iwillrecordtheinterview(ifagreedto)onarecordingdevicewhichwillbeplacedbetweenus.- Iwillalsobenote-takingduringtheinterviewprocess.- Theplannedtimeframefortheinterviewisupto1hour.- Thebelowscriptwillbeusedduringtheinterview.Interviewquestions/probes/script:Hello MynameisKylieKingston.Thankyousomuchforagreeingtotakepartinmyresearchinvestigationandfor:(either)A. agreeingtobeaudiorecordedforuseinmyresearch.TheaudiorecordingandnotesImakewill

onlybeusedbymyselfandmysupervisors.Therecordingwillbetranscribedandthendeleted.(Or)B. andforallowingmetotakenotesduringtheinterview.Theinterviewwillnotbeusedforany

otherpurpose.ThekeyquestionthatIaminvestigatingwithinthisinterviewis:howcanevaluationenhance

accountabilitytobeneficiaries?AndI’mhopingtobeabletodevelopanevaluationframeworkbaseduponyourideasthatwillbe

usefultoyourorganisationinrelationtoincreasingaccountabilitytobeneficiarieswhichissaidbetobeneficialtoorganisationalperformance.

Inordertogainyourthoughtsonthistopic,IhaveformedsomequestionswhichI’mgoingtoaskyou.Is

itokayifIstartaskingyousomequestionsnow?(ifyes–proceed;ifno–asktoelaborateonreasonsandendinterview).

Questions:(Semi-structuredtoallowformorein-depthdiscussionbaseduponresponses.)1. I’mwonderingifwecouldstartbyyoutellingmeabitaboutyourself?Likeyourname,

qualifications,experience.NatureofinvolvementinNPO:2. Canyoupleasedescribetomeyourpositionwithin (nameofNPO)and

howlongyouhavebeendoingthat?3. Whatgroupingofbeneficiariesdoyouworkmostcloselywith?Beliefs

aroundbeneficiaryinvolvementinevaluationprocesses:MyresearchisaroundbeneficiarieshavingastrongervoicewithinNPOsandifthatwouldbeuseful

and/orevenpossible.4. Arethereanynegativeimplicationsyoucanthinkofinregardtobeneficiaryinvolvementin

evaluationprocesses?5. Whataboutpositiveimplicationsofincreasedbeneficiaryinvolvement?

Appendices 227

6. Assomeonewhoworkscloselywithbeneficiaries,whattypesofthingsareimportanttoyoutoknowaboutthem,thatmightimprovetheservicetheyaregettingfromtheorganisation?

Currentpracticesinvolvingbeneficiarieswithinevaluation/accountability:7. I’mwonderingwhattypesofactivitiesyourorganisationcurrently

doestoreceiveinformationfrombeneficiariesabouttheirneedsorideasontheservice?E.g.feedbackforms,interviews,surveys.

8. Canyouthinkofanyotherwaystoinvolvebeneficiariesthatyourorganisationhasn’ttriedyetbutthatmightbeuseful?

9. I’malsowonderingwhattypesofwaysbeneficiariesarecurrentlyabletoreceiveinformationaboutyourorganisation,liketheorganisation’sservices,performance,activities,howfundingisspentetc.?E.g. newsletters,emails,financialreports,annualreports,informalorinformalmeetings,focusgroupsetc.?

Barrierstoinvolvement:10. Andfinally,withinyourorganisation,doyouthinktherearebarriersto

beneficiariesbeingmoreinvolvedinevaluationprocesses?E.g.time,tools,skills.

Iamplanningtoholdadebriefingsessionwithstaffandexecutivestodiscusstheresearchandreportonthedevelopedevaluationframework.Doesitsuityouformetocontactyouinthefutureregardingtheschedulingandattendingofthis?Thankyousomuchforlettingmeinterviewyou,Ireallyappreciateyourtime.Ihavelearntalotandreallyenjoyedmeetingyou.

228 Appendices

Stakeholder Group: Board InterviewProtocolKeyInformant:



Suitableroles





PersonwhoisaboardmemberofaNPOe.g.chair,president,secretary,treasurer,member,otherrepresentativeoftheboard.

Procedures:

- Gatherinformedconsent- InterviewwillbeconductedattheNPOofthekeyinformant,orotheragreedlocation,

atatimedesignatedsuitabletothem.- Iwillrecordtheinterview(ifagreedto)onarecordingdevicewhichwillbeplaced

betweenus.- Iwillalsobenote-takingduringtheinterviewprocess.- Theplannedtimeframefortheinterviewisupto1hour.- Thebelowscriptwillbeusedduringtheinterview.

Interviewquestions/probes/script:Hello MynameisKylieKingston.Thankyousomuchforagreeingtotakepartinmyresearchinvestigationandfor:(either)A. agreeingtobeaudiorecordedforuseinmyresearch.TheaudiorecordingandnotesImakewillonlybeusedbymyselfandmysupervisors.Therecordingwillbetranscribedandthendeleted.(Or)B. andforallowingmetotakenotesduringtheinterview.Theinterviewwillnotbeusedforanyotherpurpose.

ThekeyquestionthatIaminvestigatingwithinthisinterviewis:howcanevaluationenhanceaccountabilitytobeneficiaries?AndI’mhopingtobeabletodevelopanevaluationframeworkbaseduponyourideasthatwillbeusefultoyourorganisationinrelationtoincreasingaccountabilitytobeneficiarieswhichissaidtobebeneficialtoorganisationalperformance.

Inordertogainyourthoughtsonthistopic,IhaveformedsomequestionswhichI’mgoingtoaskyou.IsitokayifIstartaskingyousomequestionsnow?(ifyes–proceed;ifno–asktoelaborateonreasonsandendinterview).Questions:(Semi-structuredtoallowformorein-depthdiscussionbaseduponresponses.)

• I’mwonderingifwecouldstartbyyoutellingmeabitaboutyourself?Likeyourname,qualifications,experienceintheNFPsector.

NatureofinvolvementinNPO:• Canyoupleasedescribetomeyourpositionwithin (nameof

NPO)andhowlongyouhavebeendoingthat?• Whattypeofservice/sdoesyourorganisationprovide?Beliefsaround

beneficiaryinvolvementinevaluationprocesses:• MyresearchisaroundbeneficiarieshavingastrongervoicewithinNFPOsandif

thatwouldbeusefuland/orevenpossible.DoyouthinkinprinciplethatbeneficiarieshavingastrongervoiceinNPOswouldbeagoodthingforthesector?

IfYES,whattypeofinformationwouldbemostusefultoreceivefrombeneficiaries?IfNO,whynot?

Appendices 229

• Arethereanynegativeimplicationsyoucanthinkofinregardtobeneficiaryinvolvementinevaluationprocesses?

• Whataboutpositiveimplicationsofincreasedbeneficiaryinvolvement?Currentpracticesinvolvingbeneficiarieswithinevaluation/accountability:

• I’mwonderingwhattypesofactivitiesyourorganisationdoescurrentlytoreceiveinformationfrombeneficiariesabouttheirneedsorideasontheservice?E.g.feedbackforms,interviews,surveys.

• Doanyoftheprocessesinplacecurrentlywithinyourorganisationallowbeneficiary’sideastoinfluenceorganisationalproceduresorpolicies?Ifso,canyoudescribehowthathappens?

• Canyouthinkofanyotherwaystoinvolvebeneficiariesthatyourorganisationhasn’ttriedyetbutthatmightbeuseful?

• I’malsowonderingwhattypesofwaysbeneficiariesarecurrentlyabletoreceiveinformationaboutyourorganisation,liketheorganisation’sservices,performance,activities,howfundingisspentetc.?E.g.newsletters,emails,financialreports,annualreports,informalorinformalmeetingsetc.?

Barrierstoinvolvement:• ThelastquestionIwantedtoaskis;generally,withinthesector,doyou

thinktherearebarrierstobeneficiariesbeingmoreinvolvedinevaluationprocesses?E.g.time,tools,skills.

Iamplanningtoholdadebriefingsessionwithstaffandexecutivestodiscusstheresearchandreportonthedevelopedevaluationframework.Doesitsuityouformetocontactyouinthefutureregardingtheschedulingandattendingofthis?Thankyousomuchforlettingmeinterviewyou.Ihavelearntalotandreallyenjoyedmeetingyou.Thankyousomuchforyourtime!

230 Appendices

Appendix C - Coding Definitions

Appendices 231

Appendix D - Evaluation Instruments

Evaluation Instrument A

Ifyouwishtoremainanonymouspleasedonotfilloutyourpersonal

details.Name:

Date:

Igivemyapprovaltobecontactedinrelationtothissurvey:(circle)YES/NOWevalueyourparticipationandinvolvementin(organisation).Whatyouwriteonthisformwillonlybeusedbythestaffof(organisation)asawaytoimprovetheorganisation.Whatyouwriteisconfidentialandwillnotbediscussedwith,

orusedbyanyoneelse.Theseformsaredesignedtobeusedeverysixmonths.

Whenfillingoutthisform,pleasewritespecificallyaboutyourinvolvementat(organisation).

1. Ihavebeencomingto(organisation)for (howlong?)2. Ilastfilledoutoneofthesefeedbackforms monthsago.3. WhenI’mat(theorganisation)Ispendmostofmytime:

4. Lately,Ihavebeenreallyenjoyingis,(andIlikedoingthisbecause):

232 Appendices

5. Lately,Ihavenotbeenenjoying,(andIwouldliketochangehowwedothisbecause):

6. Ithink(theorganisation)wouldbebetterifwestarteddoing:

7. Ithink(theorganisation)wouldbebetterifwestoppeddoing:

8. Inthelast6months,myachievementsat(organisation)havebeen:

Appendices 233

9. ThethingsIwanttoachieveat(organisation)withinthenext6monthsare:

10. IhaveotherthingsthatIwouldliketosay,thatdonotfitintoanyofthesequestions,soIamgoingtowritethemhere:

Thankyouforfillingoutthislearningandinvolvementform.

Appendices 234

Evaluation Instrument B Ifyouwishtoremainanonymouspleasedonotfilloutyourpersonaldetails.

Name:

Date:

Contactphonenumber:

Contactemail:

Igivemyapprovaltobecontactedinrelationtothissurvey:(circle)YES/NOWevalueyourfeedbackontheservicethatyouhavereceivedat(organisation).Yourfeedbackwillonlybeusedbythemanagerialstaffof(organisation)asresearchtohelpus improve the service, ourmanagement and our governance. Your feedback andpersonaldetailsareconfidentialandwillnotbedisclosedtoanyotherparties.Wewillcontactyou(withyourapproval)toletyouknowhowyourresponseshavechangedorimprovedtheway(organisation)operates.

1. Whatwouldyouliketosayaboutyourexperiencewith(organisation)?

Onthefollowingpages,pleasecircletheanswerthatyouthinkbestrespondstothequestionandcommentonthereasonsforyourchoice,ifyouwouldliketo.

Appendices 235

2. Areyousatisfiedwiththequalityoftheservicethathasbeenprovidedtoyou?

a. Yes

b. No

c. Unsure

Pleasecommentonyouranswer

3. Areyousatisfiedwiththeadvicethestaffgavetoyou?

a. Yes

b. No

c. Unsure


236 Appendices

4. Wastheadvicegiventoyouclearandunderstandable?

a. Yes

b. No

c. Unsure


5. Didyoufindthestaffinteractionswithyoutobehelpfulandsatisfactory?

a. Yes

b. No

c. Unsure


Appendices 237

6. Arethereanyareaswhereweneedtoimprove?

a. Yes

b. No

c. Unsure


7. Ifasimilarhousingissueoccurredforyouagain,howwouldyourespond?

Answers:

No, not at all

Maybe Unsure I think so

Yes, for sure

1 2 3 4 5

1 I would be able to deal with the issue myself since I now know the processes.

2 I would contact (the organisation) again to get help.

3 I do not know what I would do if a similar issue arose.


238 Appendices

8. Ifadifferenthousingissueoccurredforyou,howwouldyourespond?

Answers:

No, not at all


Yes, for sure

1 2 3 4 5

1 I would be able to deal with the issue myself since I now know the processes.

2 I would contact (the organisation) again to get help.

3 I do not know what I would do if a similar issue arose.


9. Haveyoulearntnewinformationduetoyourinvolvementwith(theorganisation)?

No, not at all


Yes, for sure

Answer: 1 2 3 4 5

Yes, I have learnt new information.


Appendices 239

10. Didyouhaveapositiveoutcomeinyourhousingissue?

a. Yes

b. No

c. I’mnotsureifmyoutcomewaspositiveornegative


11. Doyouhaveanyothercommentsorsuggestions?

Thankyouforcompletingthisfeedbackform.