Common data in the COBRA-research: An outlinegoals and results (other than purely financial goals) Q4. Who evaluates results Structural control Q5. Sanctions and rewards Q6. Regular

Common data in the COBRA-research:

An outline

Revised and updated version – April 2010

For questions and remarks:

Dr. Koen Verhoest (Associate Professor) Public Management Institute

Katholieke Universiteit Leuven

[email protected]

Cobra – proposal for common survey questions 2

Common data set for the COBRA-research

Part I. Aim of document In this document we develop a common data set which the Cobra-partners aim to develop for their own country or area under research. In this document the data set is formulated in questions, as surveys would be the main method for gathering data that has been used by the early COBRA partners (Belgium-Flanders, Ireland, Norway). However, these data can be gathered also by other means, such as documentation analysis (legalisation, year reports and so on) and structured interviews. The aim is ultimately to develop a cross-national compatible database on public sector organisations. We refer such surveys, interviews or document analysis by the term ‘data gathering‘ in the remainder of the document. We distinguish three levels of commonality in data sets. For each cluster of issues we develop a set of questions on each level, drawing from the original Flemish, Norwegian and Irish surveys. By using this document each new partner should be able to construct its own database, taken into account the obliged and optional questions set and the particularities of its own country.

Different levels of commonality in data sets

Level 1-questions: compulsory core issues and questions

• Issues and corresponding set of questions which have to be part of each partners’ data gathering. Each partner must address these questions using the same form and phrasing by survey, interviews or document analysis.

• Extra answer categories may be added to the question in case this is explicitly stated in this document. When making more answer categories, one must make sure that the new categories may be linked (e.g. by aggregation) to the basic answer categories set out in this document.

• Most of these questions are common in the Flemish, Irish and Norwegian survey.

Level 2 –questions: optional issues and related questions

• Issues which may be part of the data gathering, depending on the partners’ preferences. • If a partner chooses to include that issue, he must include the corresponding questions

(form and phrasing) • Extra answer categories may be added to the question in case this is explicitly stated in

this document


Level 3-questions: optional individual (ad hoc) questions

• Individual questions that may be part of the data gathering. These questions were included in the survey of one of the COBRA teams (the Leuven team, the Bergen team or the Irish team)

• Each partner is free to use these questions in its own data gathering

Level 4-questions: additional country-specific questions

• Each partner is free to add other questions which are of interest for him.

Five clusters of issues/ questions:

A cobra database may encompass five clusters of issues and data (but does not have to, see further). In the remainder of the document we use these clusters to structure the different sets of questions at level 1, level 2 and level 3.

1. Identification and environment 2. Autonomy of the organization 3. Steering and control of the organization by parent ministry/minister and other bodies 4. Culture of the organization 5. Performance of the organization

Rationale of the three levels

The analytic scheme of the broader research model is presented in figure 1. The numbers on the figure refer to the five different clusters of the database. The objectives of the broader research can be seen in different stages:

1. to study patterns of autonomy and control of public sector organizations in different countries;

2. to explain patterns of autonomy and control of public sector organizations by referring to contingent factors as policy fields, tasks, age, size…;

3. to study the effects of patterns of autonomy and control of public sector organizations on performance and organizational results (taking into account interaction with organizational culture)

Level 1 + level 2 questions enable a partner to gather data and to make analysis for these three stages. Stage 2 and 3 may involve multiple case studies to elaborate these links more in depth.


Figure 1: the broader research model

The comprehensiveness of the research can be linked to the different levels of questions: Level 1 questions suffice to • Describe patterns of autonomy and control, though in a rather rough way • To link patterns of autonomy and control with antecedent factors such as policy fields,

tasks, age, size… → level 1 questions are a selection of cluster 1, 2 and 3 (see on figure 1) Level 2 questions enable to • Describe patterns of autonomy and control, though in a more sophisticated way • To link patterns of autonomy and control with more antecedent factors such as policy

fields, tasks, age, size…, and additional factors as target groups, competition, networks • To study the link of patterns of autonomy and control and organizational culture • To study the link of patterns of autonomy and control and organizational performance and

results → Level 1&2 questions cover cluster 1-5 (see on figure 1) Level 3 questions are additional questions which mainly give more detailed descriptive data and some data for explanatory purposes.

Structure of document

For each issue and related questions we give • A short motivation for the inclusion of the issue/question in the involved level • The phrasing of the question and the set of answer categories, • If relevant, special points of attention when dealing with this question in data gathering

and data analysis


• If relevant, the logic of the answer categories, the kind of answering box and the possibility for adding extra answer categories

Level 1 and level 2 questions are included in this text. Level 3 questions, definitions of concepts and methodological argumentation are included in appendices.

Some notes on the scope of organisations to be included in the database

As a result, and as a first step, an agency has been defined, for the purpose of this study, as an organisation that has the following characteristics (based on the definition of the Irish Team): • It is structurally differentiated from other organizations • Some capacity for autonomous decision-making • Some expectation of continuity over time • Performs some public function • Has some personnel and • Some financial resources • It is created by government, funded for a major part by government or under

administrative scrutiny by government. In this stage we do not exclude certain groups of agencies of the analysis, like commercial companies, or private law organizations which match abovementioned criteria. We believe that the information in the database will enable to define ex post similar groups for the purpose of international comparison. Sometimes it is to hard to tell what constitutes ‘one’ organisation (based on work of the Norwegian Rokkan Center team): The organisations in the database belong to three different main categories; Single organisations without subordinated units, that cover the territory under review. The database should cover units in this category. • Service that consists of a central (national) unit and regional/local offices.

All central units in services are mapped but not their regional and local offices. Regional and local offices that are reporting to a central national body are not included in the main database.

• Agencies which are subordinate to other agencies The framework of the COBRA database is developed for the relationship between agencies which report to government/minister/department. The application of the framework to relationships between agencies and subordinate agencies/subsidiaries would call for adaptations. Therefore such agencies are not the focus in the main database, because they are not comparable to the other agencies included.

• Group of similar organisations. Bodies with equal tasks reporting directly to a ministry may be grouped together and analysed through the study of one of them, if this would be necessary to reduce the scale of the study (e.g. regional colleges).


Part II. Overview of issues per cluster on the different levels

Cluster 0: Respondent information

Level 1: core issues and related questions

Level 2: free issues and related sets of questions

Level 3: free individual questions Level 4: country-specific questions

Q1. Organization name Q2. Function of respondent

p.m.

Cluster 1: Identification and environment




Q1. Age of organization Q2. Number of FTE Q3. Size of budget Q4. Sources/types of income – most important one Q5. Legal status and kind of affiliation; legal-structural position Q6. Parent department/ministry (core department) Q7. Policy field Q8. Tasks of the organization

Q9. Way of establishment and link to predecessor organizations Q10. Target groups Q11. Competition Q12. Networks Q13. task of organization with respect to the phases in the policy cycle

Application of freedom of information legislation Influence of EU in the organizations’ set up Responsibility for day to day management – different structures Subsidiaries of the organization

p.m.

Cluster2: Autonomy of the organization




Q1. Strategic and operational personnel management autonomy Q2. Financial management autonomy Q3. Policy autonomy

p.m.


Cluster 3: Steering and control of the organization




Ex post Control Q1. Involvement of organization in the setting of goals (other than purely financial) Q2. Document in which goals of the organization are specified Q 20. Kind of indicators measured Q21. Use of indicators for steering and control Q3. Reporting of attainment of goals and results (other than purely financial goals) Q4. Who evaluates results Structural control Q5. Sanctions and rewards Q6. Regular internal or external audit, actors and focus Structural control Q8. Presence of governing board Q10. Composition of board Q13. Who evaluates the CEO Q14. Basis of accountability for CEO Financial control Control by formal and informal contact Q 15. Frequency of formal contact /number and focus of steering meetings Q16. Frequency of informal contact

General questions on control Q17. Extent of influence of different actors on the direction of the organisation Q18. General criteria as the basis of control by minister or parent ministry Ex ante control Q19. Systems of ex ante control Ex post control Q22. Who measures Q23. Who takes initiative of reporting Q7. Ad hoc audits and specific inquiries Structural control Q9. Who appoints board members Q11. Appointment of CEO Q12. Type of contract of CEO Q24. Accountability of board members Q25. Influence through decision making bodies Financial control Q 26. Type van budget Q 27. Type van accounting system Control by formal and informal contact Q 28. Content and focus of formal contact /focus of steering meetings Q29. Importance of informal contact/steering signals

Questions as to general control Questions as ex post control – goal setting Questions as ex post control – indicators Questions as to ex post control - reporting Questions as ex post control - audit/control/regulation Questions as to boards Questions as to accountability of CEO Questions as to financial control Questions as to level of mutual trust

p.m.


Cluster 4: Culture of the organization




Q1. self-assessment of the organizational culture

p.m.

Cluster 5: Performance of the organization




Q1. Use/development of internal steering techniques as conditions for/indications of performance

Q2. Self-assessment of general performance Q3. Self-assessment of performance with respect to different criteria

Q4. Additional question on performance techniques with a focus on quality

p.m.


Part III. Detailed discussion of questions per level for each cluster

Cluster 0: Respondent information


These questions must be addressed each time the respondent fills in a form. When using web based surveys mostly several forms are used which can be send in separately.

Q1. Organization name

Enables the researcher to identify the responding organization and to link an ID-number to it. This ID-number can then be used in data analysis.

Q2. Function of respondent

• Usually the survey is addressed to the CEO of the organization. But it may be that the CEO does not fill in the survey all by himself, but that other employers of the organization are involved in completing the survey. When interpreting outcomes of specific parts of the survey, it is sometimes important to know on what hierarchical and functional level in the organization the respondent is located.

• One can choose between open answer categories or fixed answer categories. Open answer categories allows the respondent to fill his own function, but is harder to classify afterwards. Using fixed answer categories in this question to make ex post analysis easier. The suggested answer categories below are exemplary. They may be expanded or adjusted to fit country-specific needs.

What is your function in the organisation?


Q2bis. Working experience of respondent in this organization (new question – update 2009)

Enables the researcher to identify how long the respondents works in the organization, and hence his or her knowledge about the organization and the reliability of the answers. How much years do you already work in this organisation? … years




Most of the questions in this cluster can be filled in by the researchers themselves, as long as if they invest the time to find the relevant information and documents (like the laws creating the agency, the budget of the agency or other documents / website of the agency)

Q1. Age of organization

• In this question the respondent is asked to fill in the year of establishment. This enables to calculate age of the organization, which can be relevant in descriptive (variation of age between agencies in a country) and explanatory results (e.g. Link between age and performance; age and level of autonomy…)

In what year was your organisation in her present legal status/kind of affiliation established?

Note: update 2009: the original phrase ‘with her present name and form’ has been changed. The reference to a change in name has been dropped, since this happens quite regularly and does not necessarily imply a fundamental change. Creation in the present ‘form’ has been clarified further in the present ‘legal status’, meaning ‘formal-legal form of affiliation’ (kind of agency, being departmental agency, public law agencies, or another kind of body). There are several ways to ask for the age of the organization:

- Since when current name - Since when current legal form or affiliation - Since when current task portfolio - Since when created as some kind of agency (with more autonomy than a

traditional department) - Since when last change in enacting law

We choose to focus on ‘since when current legal form or affiliation’.

Q2. Number of FTE

• Indication of size of the organization • Plea for the use of a common international comparable measure FTE’s (rather than man-

labour years) How many employees did your organization have on 31 december 2003 (give numbers in Full Time Equivalents)?

Q3. Size of budget

• Indication of size of the organization • Focus is on the expense budget.


What is the total size of the budget your organization works with (in million euro) on 31 december 2003?

Q4. Sources/types of income – most important one

• Indication of financial dependency of the organization on transfers from its parent ministry/government or from other governmental levels. Indication of the extent to which its has self-generated income

• The main distinctions in the answer categories ought to be between o self-generated income1 o transfer or subsidies from its own government o transfer or subsidies of other governmental levels

If one provide more answer categories, one should be able to link his own list of categories in a straightforward way to the abovementioned classification in order to enable international comparison. • There should be an indication of the relative importance of each source of income. One

may ask for o the most important source of income (e.g. primary and secondary source in terms

of amount), or o for the roughly estimated share (in percentages) of each source of income in the

total income of the organization (or if the source of income counts for more than half of the agencies’ budget)

What are the sources of income for your organization (more answers possible)? What is the primary (and, if relevant, secondary source) in terms of amount? or Can you give a roughly estimated share (in percentages) of each source of income in the total income of the organization? (or if the source of income counts for more than half of the agencies’ budget) Depending on the kind of question asked the question will look differently in terms of structure and lay-out. For example: What are the sources of income for your organization (more answers possible)? What is the primary (and, if relevant, secondary source) in terms of share of your total budget? 1 The Norwegian survey distinguishes between different types of self generated income: income from taxes and revenues; income from sales and services; project/commission income/ other kinds of self-generated income. The Flemish survey distinguishes between self-raised taxes, income from the sales of services and products, gifts, sponsoring and membership, others…


Primary source of income (please select one)

secondary source of income (please select one)

Does this source of income provides more than half of the total budget of your organisation?

direct budget allocation by the involved oversight government/authority

Yes/no

Transfers by other governmental levels (including EU or other supranational authorities)

Yes/no

Self-generated income Yes/no Other ( please specify):………………………………

Yes/no

Q5. Legal status and kind of affiliation; legal-structural position

• These set of questions gives us a view on the legal and organizational position of the organization within the broader governmental structure

• The set of questions and answer categories should enable to assess the following issues 1. Whether the organization is set up by an act of parliament, an act of the cabinet

(sometimes called an act of the King), or an act of minister, or legislation from another governmental level (e.g. EU legislation), or other

2. Whether the organization has a own legal personality/identity (or own corporate status)

distinct from that of the state

o No own legal personality/ own corporate status separate from the state o Own legal identity / own corporate status vested in public law o Own legal identity / own corporate status vested in private law2 o Optional: Own legal identity / own corporate status under hybrid law system (both

elements from public law and private law) When having no own legal identity separate from the state, what is its structural position within the ministries

o Component of a vertically integrated ministry/department headed by minister(s) (like ordinary administrations/bureaus) (including deconcentrated services)

o has some kind of independent status within a ministry/department which is headed by a minister(s) (cfr. Next steps agencies in UK – no own legal personality)

o has another kind of affiliation outside the ministries (specify…) o Examples of related questions (to be altered taking into account abovementioned

issues) What is the basis of your organization3? 2 optional: additional subclassification -Being a company registered under the Company law -Being another kind of organisation under private law (association, cooperative, mutual…) 3 The Constitution may be incuded as answer category.


Has your organization a legal personality/identity separate from that of the state/parent ministry?

If your organization has an own legal personality/corporate status, is it vested in?

If your organization does not have an own legal personality separate from that of the state/parent ministry, is your organization ?

Q6. Parent department/ministry (core department)

• identification of the parent department/ministry which functions as a key dialogue partner and which has some role in steering and controlling the organization. Of course, this department/ministry will probably not be the only department the organization has to report to. The parent department/ministry normally resides under the parent minister of the organization.


• Since the specific names of parent departments will be country-specific and can change over time, one may use an open answer mode in this question (or use a scroll box with all the department listed)

• One can ask in subsequent questions to the parent administration (as subpart of the parent department/ministry) if relevant.

Given the administrative organization in country X; What department/ministry is the core department (mother department) of your organization, given the ties your organization has with the department/ministry for the execution of your tasks? List of departments/ministries in country X…

Q7. Policy field

• Identification of the policy field in which the organization operates • Each partner can make its own list of policy fields. However, for sake of international

comparison, one should be able to cluster his list of policy fields to the following broad clusters in such a way that one policy field can be linked exclusively and straightforwardly to one cluster

• Each partner can make its own list of policy fields. However, for sake of international comparison, one should be able to cluster his list of policy fields to the following broad clusters in such a way that one policy field can be linked exclusively and straightforwardly to one cluster:

• We could take the United Nations Classification of Functions of Government (COFOG) as standard as it is widely adopted by practitioners. This categorization has ten main types of activities (each having from five to nine subtypes). The subdivision in subtypes can help the cobra-partner to develop a list of policy fields if more narrow categories are necessary. The list of policy fields is to be found on "http://unstats.un.org/unsd/cr/registry/regcst.asp?Cl=4"

o general public services o defence o public order and safety o economic affairs o environmental protection o housing and community amenities o health o recreation, culture and religion o education o social protection

One could include the possibility for the respondent to select more than one option. This is preferably when one works with very narrowly defined single-issue policy fields. See the example below. Example of question: What is the policy field in which your organization operates? (please select up to two options) List in Flemish survey



Q8. Tasks of the organization

• Identification of what the organization does • Again, for sake of international comparison,

o one should take over the suggested list of tasks or o if one uses an own list of tasks, he should at least be able to link each task in his

own list exclusively and straightforwardly to the suggested list of tasks. • It is highly recommended to ask for a ranking of tasks as to their importance since much

organizations combine different tasks (e.g. primary and secondary task // primary tasks and additional tasks – up to two tasks). Please make sure you have at least the primary task, since this information is included in the joint COBRA database

• For definitions see annex What is the primary and secondary task of your organization?

To what extent do the following statements characterise the relationship of the organisation towards the ministry?” Update 2009: please include also the following question

to a very

great extent

to a great extent partly to a little

extent not at all

The organisation has mainly implementing tasks. Ο Ο Ο Ο Ο The organisations has tasks within the area of policy development.

Ο Ο Ο Ο Ο

The organisation can develop proposals for policy development within ist area of responsibility.

Ο Ο Ο Ο Ο

The organisation can independently negotiate with external actors (e.g. interest associations, EU).

Ο Ο Ο Ο Ο

The expertise of the organisation is used by the ministry.

Ο Ο Ο Ο Ο

The second question that could be asked is: “To what extent has the organisation’s relationship towards the ministry changed during the past five years?” Here, the same answers were used, with the following answer categories: (1) strongly increased (2) increased (3) unchanged (4) decreased (5) strongly decreased.



Q9. Way of establishment and link to predecessor organizations

• Important information for research on institutional persistence and decline. Also important when constructing trajectories over time of organizational forms. For the construction of a real database on historical trajectories one should rely on document analysis (cfr. The Norwegian database from 1945 on)

Going back to the establishment of your organization; how was it established, more specifically was it a:

• Optional: one can ask for name and year of setup of the predecessor organizations4. In case of fusion, secession or succession, can you give the names of the predecessor organizations?

Q10. Target groups

• Focuses on the customer/user/target group of the activities or services of the organization. • The answer categories can be changed. However for sake of comparison, one should be

able to link each of his own categories exclusively and straightforwardly to one of the following clusters:

o Public organization(s) o Private organization(s) o Private individual(s)

• Optional and additional: One could ask for ranking of importance of the different target groups

o primary/secondary o to what extent are the following units relevant target groups (to a large extent- to

some extent-to no extent) 4 In this way it may be possible to reconstruct by which kind of actor the functions of the agency were carried out previously (cfr. the Irish survey): by other government departments on the same administrative level, by public bodies other than departments on the same administrative level; by public bodies on other administrative levels; by the same body but on an ad hoc or non-statutory basis; by private bodies…


Which of the following units are relevant customers/users/target groups of the activities, services and/or products of your organization (several answers possible/select up to three options)?

Q11. Competition

• focuses on the competitive environment of the organization, the degree of competition and the kind of competitors

• as to the optional question about the kind of competitors: The answer categories can be changed. However for sake of comparison, preferably one should be able to link each of his own categories exclusively and straightforwardly to one of the following clusters:


Given the main tasks of your organization; are there still other actors/organizations that deliver similar products and/or services in the same area as the one your organization is active?

In case of 'yes', is your organization in competition with these other organizations?

These two last questions can be integrated in one question, if preferred. Optional: In case of 'yes', are the most relevant competitors (several answers possible):


!note that this classification is more extensive than the basic categories mentioned above!

Q12. Networks

• focuses on the network environment of the organization, the degree/strength of network relations (optional) and the kind of participants in this network (optional)

• as to the optional question about the kind of competitors: The answer categories can be changed. However for sake of comparison, preferably one should be able to link each of his own categories exclusively and straightforwardly to one of the following clusters:


Is your organization involved in cooperation networks with other public or private organizations for the planning of or performance of organizational tasks? Do not include the mother department or the mother administration.

In case of 'yes', who are the most relevant participators in the most important network (several answers possible)? Do not include the mother department or the mother administration. !note that this classification is more extensive than the basic categories mentioned above!


Optional: How would you type the relationship of your organization with the most important partner of the network on following aspects?

Q13. task of organization with respect to the phases in the policy cycle

What is the primary and secundary task of your organization in the policy cycle? Alternative phrasing: how widespread are the following activities in the organization in relation to the parent ministry (the entire organization including subordinate units) (1 to a very large extent –7 not relevant) Primary task in policy cycle (please

select one) Secondary task in policy cycle (please select one)

Policy design/preparation

Policy implementation Policy evaluation


Level 3: free individual questions

See annex


Cluster 2: Autonomy of the organization Basic distinction: We define autonomy as the level of decision making competencies (discretion) of an organization. Control focuses upon the constraints which the ministers/departments can develop to influence the actual use of these decision making competencies in order to influence the decisions made. Making an agency more autonomous involves shifting decision making competencies from external actors to the agency itself by delegation, devolution or decentralisation. We distinguish between • managerial autonomy (the choice and use of financial, human and other resources) and • policy autonomy (objectives, target groups, policy instruments, quality and quantity of

outputs, processes and procedures, issuing of general regulations or decisions in individual cases),

each on strategic or operational level.


Motivation: It is important for the conceptual coherence of the questionnaire that all questions remain in the survey. Managerial autonomy (on finances and on HRM, both strategic and operational) and policy autonomy (towards the political oversight authorities and the administrative oversight authorities) are distinct concepts. As to the Flemish data we found that correlations between the items of the different concepts show high and significant values. Items within a concept show high scale reliability and factor analysis reveals two dimensions of autonomy: managerial autonomy and policy autonomy (see appendix). Update 2009: It should be made clear to the respondents that for the questions concerning autonomy listed below

o we need the assessment of the level of autonomy as the respondent perceives the organization to have in practice (so these questions should not be filled in by the researchers)

o we focus on the level of autonomy at organizational level (not of subdivisions of the organization) – what can the organization (incl. the governing board) decide itself with regards to management and policy?

Q1. Strategic and operational personnel management autonomy

Strategic management autonomy with respect to personnel: Can the organization5 without interference from above / without ministerial or departmental influence/ without prior consent of ministers and departments set general policy for the organization concerning6 most aspects of : 5 The organization includes here the governing board, if the organization has such a board 6 In the Irish survey two other answer category are included in this question: • “Staff tenure: i.e. length of contract, such as temporary or permanent.” (which relates to some extent to ‘way

of appointing personnel’) •


Yes No The level of salaries for groups of staff Conditions for promotions for groups of staff Way of evaluating personnel for groups of staff Way of appointing personnel for groups of staff

General criteria of downsizing in the organization

Set staff number (within budgetary limits)

Update 2009: A more refined versions of the same questions is: Can the organization set general policy for the organization concerning most aspects of :

Yes without minsterial or departmental consent

Only with minsterial or departmental consent

No

The level of salaries for groups of staff Conditions for promotions for groups of staff Way of evaluating personnel for groups of staff Way of appointing personnel for groups of staff



Can the organization set general policy for the organization without prior consent of ministers and departments concerning most aspects of :

Yes for all staff7

Yes for most staff

Yes for some staff

No

The level of salaries for groups of staff Conditions for promotions for groups of staff Way of evaluating personnel for groups of staff Way of appointing personnel for groups of staff



The answer categories should at least be recodable to YES/NO.

7 All staff except CEO and board members


Operational management autonomy with respect to personnel8: Can your organization9 decide without prior consent by ministers or departments on the following for most individual members of staff10

Update 2009: a more refined version of the same questions is: Can your organization decide on the following for most individual members of staff

Yes without minsterial or departmental consent

Only with minsterial or departmental consent

No

Increase wage of a specific employee

Promote a specific employee Evaluate a specific employee Appoint a specific individual Discharge/dismiss a single employee

Or a more refined version of the same questions is: Can your organization decide on the following without prior consent by ministers or departments for individual members of staff

Yes for all staff11

Yes for most staff

Yes for some staff

No

Increase wage of a specific employee

Promote a specific employee Evaluate a specific employee Appoint a specific individual Discharge/dismiss a single employee

8 In the Irish survey an other answer category is included (which relates to some extent to ‘appoint a specific individual’) “Staff tenure: i.e. length of contract, such as temporary or permanent.” 9 The organization includes here the governing board, if the organization has such a board 10 On can extend the answer categories: yes for all staff, yes for most staff, yes for some staff, no. 11 All staff except CEO and board members


The answer categories should at least be recodable to YES/NO.

Q2. Financial management autonomy

Financial management autonomy: can your organization12 itself13:

Update 2009: Alternative, less demanding formulation:

Financial management autonomy: can your organization itself:

Yes without prior

approval from above

Only with prior

approval from above

No

Take loans for investments Set tariffs for services or products Engage in participations in private law legal persons Shift between the budgets for personnel– and running costs

Shift between the budgets for personnel or running costs on the one hand and investments on the other hand

Shift between the budgets of different years

‘From above’ means here: prior consent of the minister, parent department or other minister/department (like Minister/department of Finances)

12 The organization includes here the governing board, if the organization has such a board 13 Rules and conditions as mentioned in in the answer categories are referring to specific rules and conditions re. financial management.


Q3. Policy autonomy

Update 2009: - the following three questions can be merged in one, by treating choice of target

group, choice of policy instruments, and prioritization of activities as variables in three columns.

Policy autonomy: which of following statements is most valid? (1) Within the legal framework that applies, which actor makes the exact delineation/choice of the target group of the policy that your organization14 executes15,:

(2) Within the legal framework that applies, which actor makes the choice/selection of the policy instruments (subsidies, … ) of the policy that your organization executes: :

14 The organization includes here the governing board, if the organization has such a board 15 These answer categories differ from the original surveys in order to optimise the scale. The orginal categories were:

One can merge the first and second answer category for doing analysis, as well as the third and the fourth.


(3) Within the legal framework that applies, which actor makes decisions concerning the fulfillment of the tasks (the way the tasks are implemented, the exact prioritization of activities within the tasks) that your organization executes:

Involvement of the organization in the policy process: Update 2009: it was decided to include this question at level 1

To what extent are the following activities carried out by the organisation (including regional or local units if these exist) or the ministry in charge? 1 = only by the ministry 4 = predominantly by the organisation 2 = predominantly by the ministry 5 = only by the organisation 3 = in equal shares by the ministry and the organisation 6 = does not apply 1 2 3 4 5 6 answering of parliamentary questions Ο Ο Ο Ο Ο Ο preparing the minister/secretary of state for parliamentary meetings (e.g. committees, question hour)

Ο Ο Ο Ο Ο Ο

providing other services to the minister/the secretary of state (e.g. preparation of speeches)

Ο Ο Ο Ο Ο Ο

formulation of draft laws or decrees Ο Ο Ο Ο Ο Ο formulation of regulations, guidelines etc. Ο Ο Ο Ο Ο Ο enactment of regulations, guidelines etc. Ο Ο Ο Ο Ο Ο commenting on draft laws or decrees from other ministries Ο Ο Ο Ο Ο Ο supporting political initiatives (e.g. idea catalogues) Ο Ο Ο Ο Ο Ο


implementation of decisions and policy measures Ο Ο Ο Ο Ο Ο participation in negotiations at the EU/international level Ο Ο Ο Ο Ο Ο drafting of strategies for the whole ministry’s area of responsibility (e.g. mission statement, reform strategy)

Ο Ο Ο Ο Ο Ο

evaluation/assessment/feed-back regarding the effectiveness of policy programmes, laws, measures

Ο Ο Ο Ο Ο Ο


No questions


See annex


Cluster 3: Steering and control of the organization Basic definitions: Control (sensu lato) in this research refers to the cycle of guidance, control (sensu stricto and evaluation (Kaufmann et al. 1986), encompassing the mechanisms and instruments used by the government to intentionally influence the decisions and the behaviour of the agency in order to achieve government objectives. In this study three dimensions of control will be distinguished: timing (ex ante – ex post), focus (inputs – outputs) and control techniques (HMN) (see Kaufmann et al. 1986): • Ex ante control: emphasis is on the ‘before-the-fact’formulation of detailed rules,

regulation, standard operating procedures and approval requirements (or nullification rights) that give direction to the actor so that the desired objective (from the viewpoint of the ministers/departments) will be achieved. The intention is ‘to minimize risks and to increase certainty of performance processes before they begin’ (Wirth 1986).

• Ex post control: emphasis is on checking whether the intended organizational goals have been achieved by the agency and whether there is a need for corrective future actions. Elements of ex post control are objective setting, monitoring, evaluation, audit and sanctions.

• Both ex ante or ex post control may have their principal focus on the choice and use of inputs, or the delivery of the outputs, although in practice they mostly appear in pairs (ex ante on input; ex post on outputs).

• The techniques to control may be hierarchical, market-like and/or network-based: for example

o structural control by influencing the agencies’ decisionsthrough hierarchical and accountability lines towards the agency head or through the supervisory board;

o financial control by influencing the agencies’ decisions by changing the level of budget granted to the agency, the composition of its incomes, and changing the level of risk-turnover;

o control by putting the agency into competition with other organizations; o control by enhancing the creation of cooperation networks of which the agency is

part.


Motivation: We argue that what ex post result steering is concerned all elements of the result steering cycle should be involved for conceptual reasons. Therefore, questions towards goal setting, use of indicators, measurement and evaluation of results, and sanctions and rewards are important indicators for assessing the result steering relationship between organization and oversight authority as a complete result steering cycle. Next to that, alternative ways of steering that are often underestimated in research design such as structural steering and financial steering are important to assess if we want to have a more nuanced picture of the complete steering practice.


Ex post Control

Important note: The goals referred to in this section are the strategic performance objectives at organizational level, that the organization has to achieve. These objectives concerns aspects of performance, quality, efficiency or effectiveness. The goals meant here are the objectives, These objectives are other than the purely financial and budgetary objectives, that the organization has to achieve.

Q1. Involvement of organization in the setting of goals (other than purely financial)

• Measure the dominance of the organization or superior political (e.g. minister) or administrative unit (e.g. parent ministry) in the setting of goals

• We look for the predominant tendency or pattern (to what extent is this mostly/predominantly the case)

• Depending on the country specific situation the notion “a political (e.g. minister) or an administrative (e.g. parent ministry) senior unit” can be restricted to minister or parent ministry

Is your organization involved in the setting of these goals (other than purely financial goals) and to what extent is this mostly/predominantly the case (one answer)?

Q2. Document in which goals of the organization are specified

• looks for documents in which the goals of the organization are specified. The goals meant here are the objectives, other than the purely financial and budgetary objectives, that the organization has to achieve. Additionally, we ask to what extent the goals of the organization in these documents refer to measurable targets?

• The documents meant here may be published or unpublished. • Other country specific documents may be added • There could be one column added with the answer category: this document does not exist Concerning the goals (other than purely financial goals) of your organization; in which documents are these specified and do these goals refer to measurable targets?


Q 20. Kind of indicators measured

- Update 2009: since the relevance of these questions for result control, this question is from now onwards included in level 1

To make goals measurable, indicators are used. What is measured with the indicators that are used?

Q 21. Use of indicators for steering and control

- Update 2009: since the relevance of these questions for result control, this question is from now onwards included in level 1


- Update 2009: For this question, a five point scale ranging from ‘No’ to ‘yes, to a large extent’ is recommended, because such scale makes statistical analyses easier.

To what extent are the indicators used in the steering relation between the organisation and the parent ministry (as opposed to being used for internal organisational purposes)? 5 – point scale ranging from ‘ to a very large extent’ to ‘not at all’ Q3. Reporting of attainment of goals and results (other than purely financial goals) • Looks for frequency of reports in which the goal attainment of the organization is

reported. • The reports meant here are documents, other than purely internal documents. They may be

published or unpublished. These reports are meant for minister and parent ministry or at least communicated to them and accessible for them.

• One could add country-specific names of reports (like Budget proposal/annexes to the National Budget)

• Depending on the country specific situation the notion “a political (e.g. minister) or an administrative (e.g. parent ministry) senior unit” can be restricted to minister or parent ministry

How frequently reports your organization about results and achieved goals (other than purely financial goals) to the political (e.g. minister) or the administrative (e.g. parent ministry) senior unit” (one answer)?

Q4. Who evaluates results

By whom are the results or goal attainment (other than purely financial goals) of the organization evaluated (more answers possible)?


Q5. Sanctions and rewards

• Looks for sanction and reward schemes in case of good/bad results or the achievement /non-achievement of the goals or targets

• Kind of sanctions and rewards are defined in a general way, so that they are applicable for international comparison. If one adds or specifies categories more narrowly, one should be able to link new categories with the one suggested here

Are there rewards in case of good results or the achievement of the goals or targets for the organization?

Are these rewards (more answers possible):

• These two questions can be merged in one question Are there sanctions in case of bad results or the failure to achieve the goals or targets?

Are these sanctions (more answers possible):


• These two questions can be merged in one question

Q6. Regular internal or external audit, actors and focus

Update 2009: • Teams could decide not to differentiate between regular and ad hoc audits, in order to

reduce the number of questions • Teams could decide to further simplify these questions by restricting the kind of audits to

those focused on performance (content of audit is on the organizational results or performance)

Is your organization subject to a regular audit/inspection/scrutiny and who performs these regular audits (more answers possible):

On what issues is the content of these audit(s) /inspection/scrutiny predominantly focused (more answer possible): Financial

issues Performance and General organisational results

Legality and rule-compliance

Internal control systems

Others Not applicable.

Internal audit by ourselves

o o o o o o

By other actors contracted by us

o o o o o o

By other unit of o o o o o o


the executive government By the court of audit

o o o o o o

By other actors commissioned by the government

o o o o o o

Q7. Ad hoc audits and specific inquiries

Have your organization been subjected to an ad hoc (non-routine) audit or specific inquiry within the last 5 years (more answer possible):?

On what issues was the content of the ad hoc audit(s) or specific inquiry predominantly focused (more answer possible):


Structural control

Q8. Presence of governing board

• Ask for the presence of a ‘governing board’ o which is responsible for overseeing the direction and delivery of the organization’s

performance o which has decision making competences as to the organization (not a board which

only can advise the management of the organization) o Optional: one could ask for the specific focus of the decisions of the board

(budgets, strategy, operational decisions, oversight…)? o Optional: one could ask for the presence of a board with solely advisory

competences or with decision making competences as to the strategy and other aspects of the organization

Has your organization a governing board?

Alternative phrasing: Has your organization a board?

Q9. Who appoints board members?

• Indicator of structural control • Update 2009: Inclusion of this question is no longer obligatory. This variable remains

however included in the COBRA database, which enables comparison across countries • In case the organization has several boards, it could be recommended to repeat the same

questions for the different boards, or for the main boards • ‘consultation’ in this question refers to both formal and informal consultation or influence In case your organization has a (governing) board, by whom or which actor is it appointed (are the members appointed)?


Q10. Composition of board

• measures relative presence (as an indicator of influence) of different actors; enables to measure level of structural control by government/minister

• one can restrict the question to members with voting rights • one can change the word ‘respresentatives’ by ‘board members’ in the collumn heading of

the table below If there is a (governing) board in your organization, how many board members are there from the following groups, and how many of these have voting rights?

Q11. Appointment of CEO

• Update 2009: Inclusion of this question is no longer obligatory. This variable remains however included in the COBRA database, which enables comparison across countries

• In case the organization has several boards, it could be recommended to repeat the same questions for the different boards, or for the main boards

• ‘consultation’ in this question refers to both formal and informal consultation or influence By whom or what actor is the CEO of the organization appointed?


Q12. Type of contract of CEO

• Update 2009: Inclusion of this question is no longer obligatory. This variable remains however included in the COBRA database, which enables comparison across countries

On what type of contract is the CEO appointed?

Q13. Who evaluates the CEO

• Update 2009: Extra answer categories could be included which look for formal or informal consultation of specific actors by those who evaluate the CEO formally

• Another formulation would be: “to which actor is the CEO directly accountable to with regards to (the achievement of) objectives set for him personally”. This formulation is however not recommended, since it does not question to what extent there is a evaluation process.

By whom or what actor is the performance of the CEO formally and regularly evaluated with regards to (the achievement of) objectives set for him personally (more answers possible)?


Q14. Basis of accountability for CEO

- Update 2009: For this question, a five point scale ranging from ‘No’ to ‘yes, to a large extent’ is recommended, because such scale makes statistical analyses easier.

On what basis, on what ground is the CEO of the organization accountable to the oversight authority (more answers possible)?

Financial control

See Q4 under cluster 1: identification and environment


Control by formal and informal contact (element of structural control)

Q 15. Frequency of formal contact /number and focus of steering meetings

- Update 2009: This question asks for formal contact between the senior management of the organisation on the one hand and the minister/parent department on the other hand. An extra similar question could ask for such contacts between lower levels of the organization on the one hand and the minister/department on the other hand.

- Teams could also differentiate re. the content of the contacts (predominantly focused on administrative and budgetary issues, or predominantly focused on task-related issues or focused on policy issues). See Q28 level 2

How many times (approx.) does the senior management of your organisation have formal contact (in formal steering meetings with written proceedings or official agenda) with the responsible political (eg minister) and/or administrative (eg department/parent ministry) superiors?

Q 16. Frequency of informal contact

• Informal contacts: Refers to steering signals communicated from superior units to the organization that does not necessarily follow the formal steering dialogue and formal procedure. It may come as an addition to the formal steering dialogue, i.e. informal conversations, informal meetings, e-mail, phone ect.

How many times does the senior management of your organisation have informal contact (informal meetings without written proceedings, telephone calls, e-mails, …) with the responsible political and/or administrative superiors?


Q16bis. Extent of mutual trust between the organization and its parent department/minister

How would you describe the level of trust between your organization and the parent department/minister?

- Very high level of trust - Rather high level of trust - neutral - rather low level of trust - very little to no trust

Q17. Extent of influence of different actors on the direction of the organisation

• looks for the actors which have an influence on the decisions made by the organization with respect to general direction, strategy…16

• update 2009: from now onwards this question is included in level 1, and therefore obligatory

• Update 2009: For this question, a five point scale ranging from ‘No’ to ‘yes, to a large extent’ is recommended, because such scale makes statistical analyses easier.

• An alternative phrasing is: To what extent is your organizations (in one way or another) influenced by the following actors when making decisions concerning direction and strategy?

• In the list of actors, ‘parent minister’ and ‘core department/ministry’ may be merged. Which organizations influences in some way the decisions (concerning direction and strategy) made by your organization and to what extent? (please use 5 point scale)

16 Such an influence can be the consequence of instructions, regulation, participation in decision making, or by processes of consultation of these actors. Such an influence can be the consequence of control, audit, scrutiny, monitoring by these actors. An organization can anticipate or react on these actions by altering its decisions.




General questions on control

Q18. General criteria as the basis of control by minister or parent ministry

What kind of general values or criteria are central in the steering and control by the minister or department (Introduce ranking in use of criteria)?

Ex ante control

Q19. Systems of ex ante control

• In some countries there are ex ante control systems by means of prior approval requirements (or veto or nullification rights) by the involved minister with respect to the decisions made by the (board of) the organization. In some countries control agents of the minister and the minister of Finance are in the boards of state agencies without voting rights in order to exercise these systems of ex ante control (E.g. ‘the commissaire du gouvernement’ in Belgium and France). Such ex ante control systems can block or alter decisions of the organization before they are implemented.

Example from the Flemish survey: Has the responsible minister and/or the core (mother) department of your organisation:

If the responsible minister or the parent ministry has indeed such a veto right or influence, do you consider it to be focused on


Ex post control

Q 20. Kind of indicators measured

- update 2009: from now onwards, this question is included in Level 1 To make goals measurable, indicators are used. What is measured with the indicators that are used?

Q 21. Use of indicators for steering and control

- update 2009: from now onwards, this question is included in Level 1 To what extent are the indicators used in the steering relation between the organisation and the parent ministry (as opposed to being used for internal organisational purposes)?

If indicators are used (to a large or some extent) in the relation between the organisation and the parent ministry, how are they used?


Q22. Who measures

Goals and indicators are linked to results. By whom are the results of the organisations measured (more answers possible)?

Q23. Who takes initiative of reporting

Who takes initiative for reporting results/achieved goals?

Structural control

Q24. Accountability of board members

On what basis, on what ground are the members of the governing board accountable to the oversight authority (more answers possible)?


Q25. Influence through decision making bodies

• In Flemish survey linked to Q17 ex ante control To what extent does the parent minister/ministry exert influence through the organization's decision-making bodies (board or similar)?

If parent ministry exert influence to a large extent or to some extent through the organization's decision-making bodies - is such influence:

Financial control

Q 26. Type of budget

If your organisation has a budget, can you consider this budget predominantly as:

Q 27. Type of accounting system

What are the basic accounting principles for the organisation?


Questions as to control by formal and informal contact

Q 28 Focus of formal contacts/steering meetings

Q 29 Relevance of informal steering signals

C18a_1 To what extent are the following factors focussed on during these steering held between parent ministry and the organization?

To what extent are informal steering signals communicated to the organization from the parent ministry?

To what extent are informal steering signals from parent ministry emphasized in your organization?

Looking back at the past 5 years, has informal steering signals from parent ministry become more or less comprehensive in character (with regards to frequency, field of responsibilities)?


Cluster 4: Culture of the organization


No questions in level 1


Q1. self-assessment of the organizational culture

• Motivation: It is important to use the instrument for assessing culture as a single and coherent set of questions. The thirty-six items together form one instrument for assessing nine sub-dimensions of culture. Earlier research (Tepeci) and own empirical results show that there are indeed nine sub-concepts (after factor analysis, Tepeci), that show high correlations and scale reliability between and for the items that constitute one sub-dimension (own empirical results, see appendix).

• In order to retain a valid instrument and in sake for comparison, please do not eliminate items or do not change scales.

Evaluate your organization as it is now. Point for every item the number that according to you describes how characteristic the item is for your organization (thus not for you personally) on a 1 – 7 scale. Try to look at your organization from a distance.



No questions


Cluster 5: performance of the organization


Q1. Use/development of performance techniques as conditions for/indications of performance

• Motivation: These variables measure the extent to which organizations use techniques that are assumed to augment organizational performance. Conceptually we have clustered the variables into two categories– external and internal techniques. The variables (items) that constitute these categories seem to correlate highand significantly and have high internal scale reliability (see appendix).

• Update 2009: For this question, a five point scale ranging from ‘No’ to ‘yes, to a large extent’ is recommended, because such scale makes statistical analyses easier.

To what extent following things happen in your organization

Items that can be added: • performance proclamation • business process reengineering • scheduling


• … Items can be dropped


Q2. Self-assessment of general performance

How many points do you give your organisation with regard to general results of the organisation, on a 1 – 10 scale. 1 is the lowest score, 10 is the highest score.

Q3. Self-assessment of performance with respect to different criteria

• Motivation: Again the twelve items together form one single instrument. Conceptually five dimensions of performance can be identified, all of which have high and significant inter-item correlations and all of which seem reliable scales, except for the ‘stability in the environment’ perspective (see appendix). Moreover, factor analysis( see appendix) shows three dimensions of performance; of which the first dimension clusters the goal perspective items (effic, effect and qual), of which the second dimension clusters the personnel perspective items (satstaff and motiv) and of which the third dimension clusters the public performance items (respsoc, accsoc and democrat). Bearing in mind this conceptual and empirical multidimensionality of the concept of performance, we prefer to integrate all variables in the survey.

How would you assess the results of the organization on following criteria, on a 1 – 10 scale? 1 is the lowest score, 10 is the highest score


Common data in the COBRA-research:

Appendices

I. level 3 questions

II. List of definitions

III. Definitions of tasks: extra elements

IV. Methodological argumentation

For questions and comments

Dr. Koen Verhoest Bram Verschuere

Public Management Institute Katholieke Universiteit Leuven

[email protected]

[email protected]

mailto:[email protected]

Cobra – proposal for common survey questions

Appendix I. Level 3 questions for each cluster

Cluster 0: Respondent information No questions on level 3

2




Q. Application of freedom of information legislation

Does Freedom of Information legislation apply to the organization?

Yes

No

Q. influence of EU in the organizations’ set up

What was the influence of the EU in the organization's [current] set-up?

strong influence

some influence

no influence (if no influence, please proceed to question 10)

If there was EU influence, what was this due to?

EU legislation

EU funding

both

other, please specify

Q. Responsibility for day to day management – different structures

Who is responsible in the organization for the day-to-day management?

Individual topmanager

Several topmanagers

Committee of directors

3


Committee of directors and managers

Other (specify)

Q. Subsidiaries of the organization

Does the organization have subsidiary companies?

Yes No

In case of no, proceed to question E54_3.

E54_1_A In case of yes, how many subsidiary companies does the organization have?

E54_2 In case of yes, in what manner is the subsidiary company (-ies) affiliated with the organization?

Wholly owned company

Partly owned company

Both wholly owned and partly owned

E54_3 Has the organization contributed to- or on its own initiative established any foundations after the year of 1990?

Yes No

E54_3_A In case of yes, how many?

4


Cluster 2: Autonomy of the organization

Level 3: free individual questions No questions

5


Cluster 3: Steering and control of the organization


Questions as to general control Are there rules that restrict parent ministry from instruction of the organization?

Yes No

In case of no, proceed to question

In case of yes, are these rules:

Several answers possible.

Relevant for the entire organization

Relevant for parts of the organization

Concerning general instruction

Concerning detailed/case-specific instruction

Enshrined in law

Not enshrined in law

Questions as ex post control – goal setting What characterizes the organization's letter of allocation from parent ministry?


Long/comprehensive Short General Detailed

In addition to the organization's resource allocation, what other factors are communicated through the letter for allocation?


Specific and testable criteria for goal achievement

Signals of preferences

Description of purpose and activities

Professional signals

6


Production target/efficiency

Is the letter of allocation subject to changes during the budget period?

Yes, often Yes, but seldom No

Looking back at the past 5 years, what changes are most relevant with regards to the organization's letter of allocation?


Has become longer

Has become shorter

Greater focus on general lines

Greater focus on details/ singular cases

More obligatory

Less obligatory

Mainly the same

Questions as ex post control – indicators How many indicators are there in the organization's steering documents?

None <10 10 - 19 20 - 49 >50

Has the number of indicators changed during the past 5 years?

More indicators Less indicators Mainly the same

To what extent does the indicators grasp real events with regards to the activities of the organization?

To a large extent To some extent To a small/no extent

Looking back at the past 5 years, to what extent are indicators stable?

7


Mainly stable Some change Unstable

During the past 5 years, have indicators become more or less qualitative in character?

More qualitative Less qualitative Mainly the same

Questions as to ex post control - reporting Does the organization use ICT-based reporting systems for the documentation of results?

Yes No

Questions as ex post control - audit/control/regulation Who is >I>board of appeal/appeal committee for decisions made by the organization, and for

what types of decisions?


Parent ministry Economic/administrative decisions

Professional decisions

Not appellate administrative body

Independent court of appeal/ appeal board/ review board

Economic/administrative decisions

Professional decisions

Not appellate administrative body

How often does superior units or board of appeal/appeal committee reverse the decisions made by the organisation?

Parent ministry Never To a small extent/seldom

To some extent

Not relevant

Independent court of appeal/ appeal board/ review board

Never To a small extent/seldom

To some extent

Not relevant

What aspects of the organization's activities are subject to audit/control/regulation?


Administrative performance/function

Regulations on economy for governmental organizations

8


Personnel

Cost effectiveness

Level of goal achievement

The organization's enforcement of- and compliance to general rules, regulations and precepts

Budgeting and accounting

The organization's follow-up on general lines and policies provided by political authorities

Internal control systems

To what extent does the audit/control/regulation have the character of counselling?


What characterizes the organization's internal control system/mechanisms?

Well established and functioning Adequate/sufficient Inadequate/insufficient

During the past 5 years, to what extent has the organization been subjected to steering or control

from the parliament (i.e. through changes in the Government's budget proposals, Question Period, interpellation, investigation/scrutiny, public inquiries)?


During the past 5 years, has the organization been subject to:

Performance auditingYes No

Reviews/criticism from the Office of the Auditor General concerning the financial auditing

Yes No

Routine inspections from independent courts of appeal/appeal boards/review boards

Yes No

9


Non-routine inspections from independent courts of appeal/appeal boards/review boards

Yes No

Supervision/ control/ regulation from private accreditation/standardization organizations

Yes No

Have the courts, Ombud or international control bodies (i.e. ESA) during the past 5 years rewied

any cases regarding the organization's activity?

CourtsYes, one case Yes, several cases No

International supervisory bodies Yes, one case Yes, several cases No

OmbudYes, one case Yes, several cases No

In case of yes, did the decision(-s) favour the organization?

Yes, always Yes, some times No

How common are the following types of audit/control/regulation for the organization?

MutualityVery common Somewhat common Not common

Contrived randomness Very common Somewhat common Not common

CompetitionVery common Somewhat common Not common

OversightVery common Somewhat common Not common

To what extent has the organization during the past 5 years been subject to criticism from other public units, political actors or mass media due to lack of conformity/correspondence with political objectives and preferences?

10


Has not occurred

To a very little extent

To some extent

To a very large extent

What characterizes the development of regulations and precepts within the organization's field of responsibility?

The parent ministry undertake development of regulations and precepts on its own

The organization undertake development of regulations and precepts on its own

Both parent ministry and organization undertake development of regulations and precepts

Questions as to boards

Has the board/organisation established an audit committee?

yes

no

N/A

If yes, how many members does the audit committee have?

And how many members have expertise in the following?

Audit & accounting

General management

Other

Does the organisation/board publish an annual report?

yes

no

11


Does the organisation/board have a code of business conduct defining the standards of behaviour to which board members are to subscribe?

yes

no

Does the organisation/board have a code of business conduct defining the standards of behaviour to which employees of the organisation are to subscribe?

yes

no

Questions as to accountability of CEO

Is the specific role of the CEO recorded in writing?

Yes

No (if no, please proceed to question 43)

If yes, where is this written? (please select all relevant options)

Legislation

CEO contract

Other (please specify)

Is the specific accountability of the CEO recorded in writing?

Yes

No (if no, please proceed to question 45)

If yes, where is this written? (please select all relevant options)

Legislation

CEO contract

12


Other (please specify)

Questions as to financial control To what extent is there a correspondence between the budget of the organization and the accounts?

Very large Rather large Rather little Very little

Questions as to level of mutual trust • Maybe problematic because trust may not be mutual but rather in one direction (I

trust you but you may not trust me)

How can the level of mutual trust between the organization and the minister / parent ministry

be described?

Very high level

Rather high

Neutral

Rather small

Very small level

13


Cluster 4: Culture of the organization Level 3: free individual questions No questions

14


Cluster 5: performance of the organization Level 3: free individual questions

Q4. Additional question on performance techniques with a focus on quality E53_1 Are the following techniques or instruments used in the organisation?

Public reporting on the performance of the organization in yearly reports, budgets documents, operational plans, strategic plans

Yes No

Quality standards for production/service delivery Yes No

Customer surveysYes No

Quality management systems (i.e....ISO, BSC) Yes No

Internal units monitoring qualityYes No

Steering on goals and results through the Økonomiregelverket Yes No

Balanced goal steeringYes No

Value Process ManagementYes No

Activity Based Costing (ABCYes No

Economic Value Added (EVA)Yes No

Contractual steeringYes No

15


Activity Based Management (ABM)Yes No

BenchmarkingYes No

Value-based ManagementYes No

Total-quality managementYes No

Team-based managementYes No

Knowledge-based managementYes No

Service-based managementYes No

16


Appendix 2: Definitions list as part of the survey


Governmental level: The administrative level on which your organisation has responsibilities and/or performs its tasks

Core/parent department: the ministerial department with which your organisation has links for the fulfilment of your tasks. Can also be the overseeing department.

Tasks of the organization:

for definitions of tasks see part III of the appendices.

Income of the organization

Direct budget allocation by the involved oversight authority: funds from the general budget of the oversight authority that are given to the organisation

Transfers/subsidies from other governmental levels: funds from other public authorities – not oversight authority

Self generated income: refers to income that the organization during a budget period generates through it's own activities, and comes as an addition to budget allocation. Self generated income will depend on service and activity level of the organization, the extent of production and some times also demand. It does not include income that have the character of reimbursements or income that with certainty is channelled directly back to the treasury. Examples: income from sales, services, consultation ect.. Different sources are:

o Taxes and revenues: i.e. charges on administrative executive work etc. This type of income is often subject to political regulation either entirely or partially.

o Income from sales and services: i.e. income that stems from courses, seminars, selling equipment, reports and documentation, informational brochures, hiring out, production of maps and geological data etc.

o Project/commission income: This is normally of a more temperate and provisional character, it can be highly changeable and difficult to predict future income and relies on external demand. Project income stems from organizational tasks and activities that will be limited in terms of time.

17


Phase in the policy cycle

Policy preparation: designing, brainstorming, arguing for and formulating alternatives about the content of a policy

Policy-making: ultimate choice about the content of the policy

Policy implementation: process of implementing or executing the policy after it has been decided upon.

Policy evaluation: screening and evaluating the results of the policy and the policy implementation

Target group, competition and network

Territory: the territory on which your organisation performs its tasks and has its responsibilities

Public organisation: the client is a public organisation, e.g. a city, a federal agency, a court of law, etc.

Private individual: client is an individual without a public status, e.g. a citizen, a family, etc.

Private organisation: client is a private organisation, e.g. a company, a private non-profit organisation, etc.

Network: refers to cooperation- or agreements between two or more actors concerning the production/manufacturing of services/goods, or the planning and execution of tasks. The extent of formalization to the cooperation may vary between the informal and formal. Cooperation can include i.e. exchange of information, consultation, responsibilities for different stages of a production process. There should be some regularity to the cooperation (thus not an isolated event), and it may take the form of informational meetings, joint cooperative bodies, boards- or committee combinations.

Subsidiaries: organisations that have structural links with the organisation. For example organisations established by your organisation, or organisations in which your organisation has a participation or a share.

18


Cluster 2: Autonomy of the organization

Strategic management autonomy: free choice of the general principles and rules with regard to the use of resources

Strategic management autonomy concerning personnel:

o Ability to set general policy HR - Meaning that the organisation has free choice to set general principles and rules with regard to the use of resources in the HR area.

o Make decisions on general conditions: refers to the scope the organization has for making decisions on general internal conditions or policies within the organization. This could be i.e. general rules for hiring personnel, wage levels, modes of evaluating personnel, downsizing of the work force etc. It also includes the authority to announce or create new positions.

Operational management autonomy concerning personnel:

o Make decisions on individual conditions: refers to the scope the organization has for making decisions concerning individual matters, thus relevant for single individuals.

Policy autonomy: choice of policy instruments, desired output, policy effects and target groups of policy.

Target groups: groups for whom the policy essentially is addressed, e.g. seniors, youth, immigrants, etc. Groups at whom the organisation’s policy instruments are directed

Policy instruments: instruments with which the policy is implemented, e.g. financial (e.g. subsidies), communicative (e.g. training, provisions of information), legal (e.g. type of regulation, contracts)

Regulations: the set of laws, decrees, management contracts, agreements that apply to the organisations and that determine the organisation’s freedom.

19


Cluster 3: steering and control of the organization

Ex ante control

Decision-making bodies: e.g. votes in the board, board of directors, etc.

Ex post control

Qualitative: difficult to measure, , quality of services delivered

Quantitative: measurable, eg. number of clients received per hour, time for delivery of service unit, eg. level of customer satisfaction …

Quantitative output: policy output (policy achievement) in numeric terms, e.g. number of complaints per month

Societal effects: policy impact (change in situation) on society as a result of activities and achievements, e.g. decline in suicide, decline in number of smokers, etc.

Output on resources: the level of services and achievements compared to the resources used (efficiency)

Societal effects on available resources: cost effectiveness, maximal desired societal effects with minimal available resources

More freedom: more autonomy for the management (of the organisation) by e.g. delegation in policy matters or operational matters.

Less freedom: less autonomy for the management (of the organisation) by e.g. withdrawing delegation in policy matters or operational matters.

Re-structured: re-structuring of the organisation or part(s) of it, e.g. in the composition of the board, a more flexible organizational structure

Audit: the screening of the organisation, can be focused on the achievements of the organisation, financial management and policy and/or the legality of the policy of the organisation

External audit service: Review of the financial management and results of the organisation, by an external body.

Internal audit service: Review of the financial management and results of the organisation, by the organisation.

Structural control

20


CEO / Manager of the organisation: the person or persons who is/are responsible for the day-to-day management of the organisation and thus also for implementation of its functions

Audit committee: A committee with responsibility for independent review of systems of internal control and the external audit process.

Board members

o other Governmental representatives – e.g. those from local Government o representatives of employee organisations – e.g. those from trade unions o representatives of employees of the organisations – e.g. a staff representative o representatives of stakeholders, e.g. interest groups

Financial control

Input-based budget (line-item budget): allocation of resources that are needed for a certain policy implementation or work process of the organisation

Activity-based budget: resources are linked to future activities and programmes

Output-based budget: setting of the planned output of the organisation and of the resources that are needed for that.

Effect-based budget: required monetary resources are freed to realised planned or scheduled effects

Cash-based accounting: registration each time an amount leaves or enters the till.

Modified cash-based accounting: registration when debts or claims emerge, next to registration of pure cash flows

Accrual accounting: transaction-based registration, income and costs booked at the moment of realisation independent of collection or disbursement.

General control

Informal steering signals: Refers to steering signals communicated from superior units to the organization that does not necessarily follow the formal steering dialogue and formal procedure. It may come as an addition to the formal steering dialogue, i.e. informal conversations, informal meetings, e-mail, phone ect.

21


Mutuality: Formal or informal control of individuals by peer group appraisals based on egalitarian values. Cooperation is vital within this dimension of scrutiny/regulation/control.

Contrived randomness: Scrutiny/regulation/control through employing unpredictability, random selection processes, rotation arrangements, random inspection ect.

Competition: regulation/control through different forms of rivalization. Marked testing, competitive tendering, formation of a contracts etc.

Oversight: control from above or from other governmental organizations, implies strong hierarchial ties. Characterized by routine rather than unpredictability.

22


Cluster 4: culture of the organization

Focus on task accomplishment: it is very important that the task is performed well and on time.

Innovation: strong emphasis that the organisation is innovative in the fulfilment of its tasks.

Training is important: the development and the vocational training of the employees is assured by regular training and education.

Trust: trust is a value that is important, both within the organisation and in the relation with external partners.

Respect for individual rights: the rights of the individual employee are taken into consideration as much as possible

Detail orientedness: the tasks and duties of the organisation demand great accuracy and attention to detail.

Emphasis on quality of service delivery: within the organisation the idea is that customers only deserve the best.

Promotion in the organisation: there are opportunities for promotion (WITHIN the organisation)

Good financial rewards: good financial rewards is considered in the organisation as a motivator

Support of employees: it is important that employees are supported by all available means

Attention to detail: regardless of the complexity of the task, the organisation works accurately and with attention to detail.

Risk taking: the organisation does not hesitate to take calculated risks, and is given the room to do so.

Giving customers what they expect: the organisation seriously considers the wishes of the customers when performing its task.

Hard work: in general everyone works hard in the organisation

Empathy with employees: employee problems and concerns, of whatever kind, can be discussed, formally or informally

23


Integrity: integrity of the employees is a must in the organisation

Equal rewards: in the organisation equal work is rewarded with equal pay, regardless of factors such as age, gender, etc.

Honesty: honesty is expected, from the employees, customers, and others with whom there is contact.

Cooperation with colleagues: cooperation and team work among employees is stimulated in the organisation.

Valuing customers: respecting customers is an important value in the organisation.

Fair compensation: within the organisation employees who perform well earn a better compensation than less well-performing employees

Goal orientation: regardless of its formal mission, the organisation strives to mean something and certainly to fulfil its duty.

Willingness to experiment: there is room in the organisation from time to time to deviate from the prescribed path in the fulfilment of the task.

precision: the task and duty of the organisation requires precision

Advancement possibilities: employees get opportunities to advance and to enhance their position

care for employees: the organisation gives facilities to employees that are decoupled from the organisation process, e.g. child care

Accuracy: everyone works accurately in the organisation

Team spirit: team spirit is considered important and stimulated in the organisation

Result orientedness: the most important thing for the organisation are its results.

achievement-related compensation: performance related pay is used in the organisation

Creativity: creativity is important in the fulfillment of the organisation’s tasks

Keeping promises: in the organisation, in all cases and with regard to everyone we try to keep promises we made earlier

Personal career development: the personal development of the individual employee is also important for the organisation as a whole

24


Team orientedness: the task of the organisation requires a high degree of team orientedness, the organisation will not make it with individual efforts

Relations with customers: the organisation tries to broaden and deepen the relationships with clients because this is important.

Co-operation with others: cooperation with other employees is important in our organisation

25


Cluster 5: Performance of the organization

Self-assessment of performance with respect to different criteria

efficiency: the degree to which the desired outputs are achieved with the resources the organisation has.

effectiveness: the degree to which the desired societal impact and results are achieved given the objectives of the organisation

Quality of products and service delivery: e.g. number of trains that arrive in time, care for the public domain, etc.

Employee satisfaction: general well-being of employees, can be operationalised by the absenteeism, over-achievement, level of motivation, etc.

Employee motivation: can be operationalised by the efforts the employees deliver, the extra tasks the employees do, etc.

Quality of management: can be observed in organisational culture, the status of the manager by the subordinates, efficient management of the organisation.

Preserving internal cohesion: is the whole more than the sum of the parts in the organisation? Does everyone point in the same direction?

Stability of the organisation in the environment: the level to which the organisation can stand or cope with changes in the environment, can be observed in age and adaptive capacity, for instance.

Flexibility of the organisation: can the organisation adapt quickly and smoothly to new tasks and duties that are the result of external changes?

Responsiveness towards society: can the organisation adapt to or answer to real societal needs and can this be observed in service delivery?

Accountability towards society: can the organisation prove the need for using public resources?

Democratic level of service delivery: is the policy shaped in a democratic manner? Does the service delivery reach the target group?

26


Appendix 3: Definitions of tasks – extra elements1 We distinguish five different task (groups): (1) Policy formulation, development and advice (2) Regulation, scrutiny, control and inspection (3) Other kind of exercising public authority (4) Providing general public services (5) Business and industrial activities In this appendix we discuss in detail what should be understood under each category.

Task 1. Policy formulation, development and advice • Directed towards political authorities (cabinet, parliament) • Focussed on policy design, policy advice and policy evaluation • Encompasses functions like:

Policy advice Evaluation of policy and policy instruments Law preparation and commenting Development and adjustments of general frameworks for regulatory policies Active agent for policy towards government (e.g. centre of gender equality) Maintaining vital functions concerning the implementation of sectoral policies:

refer to agencies that coordinate and support implementation of policies (support function towards implementing organisations in policy field by means of information, training, lobbying…) and also to organizations that are responsible for overarching policy shaping within specific policy fields (i.e. agriculture, environment and the provision of national plans, strategies etc.)

! It is advisable to practice a very strict use of this category, or operated with a narrow understanding of the term in order to avoid “overrepresentation”. The reason for this is that all governmental organizations to some extent can be said to have activities related to policy formulation. Clarifying notes:

What with related tasks concerned with tools for policy development Information – collection and / or provision of information for public use

(national agency for statistics…): will normally be perceived as ‘general public services’, this is also mere tools – not the actual function.

1 This list of definitions is largely based on work by the team of Rokkan Center: Per Laegreid, Paul Ronness, Vidar Rolland, and Kirsten Rubecksen.

27


Research: has the quality of policy formulation, the criteria is whether or not this is the primary function (and purpose of organization) or additional task (or consequence). Normally research institutions would be perceived as ‘general public services’.

Representation: providing segments of civil society with representational and participatory opportunities (e.g. platforms for concertation with social partners…): would be included in ‘policy formulation, development and advice’.

Task 2. Regulation, scrutiny, control and inspection2 This is one of two activities that also involve the exercise of public authority. However, as defined here, regulation is a specialized and more narrow task (than other kinds of exercising public authority). It refers to the monitoring of the compliance of actors to rules, law, contract or agreements. It does not refer to making or concretizing rules or laws (cfr. Other kinds of exercising public authority). Tasks within scrutiny/regulation/control are closely connected to the follow-up of - or regulation/control in accordance with regulations, rules, law, contract or agreements. Supervisory functions may refer to qualitatively different activities (i.e. inspections, care, surveillance, control). A vital criteria is that supervisory functions are directed toward other agents or institutions outside of the organization who maintains the function. It can vary whether or not the organization with supervisory functions also have the right of enforcement. These regulatory tasks can be directed at private organizations and individuals, as well as to actors in the public sector.

Clarifying notes:

1. Does regulation encompasses different levels of activity like:

Transition of general policy and laws to more concrete rules, norms and standards where actors in policy fields have to comply to:

no, this is more related to 'other kind of exercising authority’, cf. comment on precepts below.

The application of rules, norms and standards in individual cases (e.g. allocating

building permissions based on construction laws)?:

2 Mitnicks’ definition of regulation: public administrative policing of (private) behaviour according to rules and laws

28


It depends on type of case: is normally related to 'other kinds of exercising authority'.

Controlling compliance to the rules, norms and standards:

Yes, definitively within ‘regulation, scrutiny, control and inspection’.

2. Does this category encompasses

Tribunals (different from courts) and public enquiries?

This will depend on scope and focus, is this a specialised and primary function of the organization? Tribunals are more of the category ‘other kinds of exercising public authority’, while public enquiries could be perceived as ‘regulation/scrutiny’. However, public enquiries sometimes are characterized by ad hoc ‘organizations’, and thus not included as a relevant unit of analysis.

Registration (registration of professional groups, like nurses…)

If this encompass certifying, licensing of professional groups etc. which is more ‘other kinds of exercising public authority’.

Consumer ombudsman

has the quality of ‘regulation/scrutiny/control/inspection’

Regulation towards public sector organisations (like public standards commission, controlling ethical behaviour in the public sector)

is ‘regulation/scrutiny/control/inspection’ within the public sector

29


Task 3. Other kind of exercising public authority (incl. redistributive tasks) Of course regulation is one way of exercising public authority but there are other ways. This rest category needs however to be defined. Definition: Refers to the execution of tasks according to-, on behalf of- or based in law, regulations or precepts. Such tasks are normally subordinate to rules for procedure, the Public Administration Act and the Freedom of Information Act.

Clarifying notes:

Does this category encompasses: 1. Legitimate use of violence by state and by parts of public administration: Policing,

prisons…? What about defence?

Yes. Both are classified as ‘other kinds of exercising public authority’. 2. Interests-mediating and conflict-resolving and processing in accordance to law, like

courts, justice, arbitration committees:

Yes to be classified as ‘other kinds of exercising public authority’.

3. Administration and evaluation of precepts: This involves creation, evaluation and

revision of sub-rules/precepts connected to regulative frameworks3.

To be classified as ‘other kinds of exercising public authority’.

4. Transfer of funds, like • Allocation of governmental grants and subsidies • Collection, distribution of and allocating resources (monetary/tax redistribution),

administration of governmental insurance systems 4

Yes ,both to be classified as ‘other kinds of exercising public authority’.

3 Normally the parliament decides upon the overarching regulative frameworks, general rules, laws, and lower levels create sub-rules/precepts within the specific regulation, law, rule. 4 • Redistribution as defined by Lowi : Broad groups of individuals are involved, and decisions are interrelated: have and have-nots. Redivides property. Social insurance; taxes; pensions, unemployment

30


Task 4. Providing general public services 5

Production and direct delivery of services on a non for profit basis. This could be the production of “free” collective goods, or services and goods that are partly financed by the consumer itself.

Includes Direct service provision to citizens, private organisations and public organisations

counseling or advisory tasks towards citizens, private, or public organizations informative and guidance tasks towards citizens, private, or public organizations

administrative services of different types (e.g. processing of passports…) equipment and delivery services towards citizens, private, or public organizations

training and competence building towards citizens, private, or public organizations

Clarifying notes:

1. Does this category encompasses Development of sectors: commercial development (e.g. tourism) and promotional development (e.g. culture, language…)? Yes, it can be – but a distinction must be made from more business-like

services of this type, the line can be very thin.

5 This category refers to distributive function of government as defined by Lowi : Can be disaggregated and dispensed unit by small unit, each unit more or less in isolation from other units and from any general rule. Are highly individualized decisions that only by accumulation can be called a policy. Encompasses public land and resource policies, rivers and harbours programs; defence procurement; research and development programs; labour, business and agricultural “clientele” services; primary political units is a single individual, form or corporation between which there is mutual non-interference ( each man for himself)

31


Task 5. Business and industrial activities

directed towards demand on a market and substantially financed by prices, Includes

Telecommunication services… Nationalised industries, banks, credit institutions Can be secondary tasks of meteorological institutes, rail administrations

Remaining questions:

1. Is this category equal to commercial tasks? Mainly yes, but it does not have to be a necessary criterion that the task is fully financed by i.e. prices. In addition, commercial tasks that over time doesn’t ‘pay off’ is often terminated, while the government may have strong interests in maintaining certain business or industrial services in spite of low returns.

32


Other typologies Here we list two other typologies and we should figure out how these typologies fit into the Norwegian typology. In the remaining of this document I point to some questions involved. (linkages to our categories between brackets) Bouckaert and Peters 2003

Implementation Direct service delivery (general public services) Transfer of funds (other kinds of exercising public authority)

Regulation (regulation/scrutiny) Tribunals and public enquiries ( mainly other kinds of exercising public authority) Advice and policy development (policy formulation) Information (general public services) Research (general public services) Representation (policy advice)

Van Thiel

Supervision (regulation/scrutiny) Paying benefits (other kinds of exercising public authority/general public services)6 Judging quality 7 Licensing (other kinds of exercising public authority) Making regulation (other kinds of exercising public authority) Registration (when encompassing licensing, certifying ‘other kinds or exercising public

authority’) Collecting fees (other kinds of exercising public authority) Quasi-judicature (other kinds of exercising public authority) Advise (policy formulation) Coordination (other kinds of exercising public authority/general public services)8 Research (general public services) Others…

6 Classification of paying benefits depends on some other elements: 1) what type of benefit is this? 2) is the target group wide or narrow? 3) whether the benefits are regulated by law. In earlier Norwegian classifications (Rubecksen 2003) benefits related i.e. to unemployment, social security etc., that is regulated by law/according to law, meant as general arrangements for larger groups of society, and where the public bodies concerned are able to make administrative decisions in individual cases based on law/regulations, have been classified as ‘other kinds of exercising public authority’. In other instances, activities have been classified as general public services due to having a more “service-like” character, i.e. the paying of pension benefits for government employees. 7 Judging quality can come close to a supervisory function. On the other hand most organizations have activities involving this in a larger or lesser extent, through goal and result steering systems, internal quality supervisory units etc. It may also be organized as more permanent boards or committees with particular focus of quality within a specific sector (education, health) or policy field. Judging quality may also take the form of ad hoc boards/committees, in which case this would not fulfil the criteria of relevant organizational units for analysis. The criteria for how to classify such activities should be based on the activity’s relation to law/regulations: is it carried out in accordance to law/regulations, does it involve control of quality based on criteria circumscribed in law/regulations? In this case, this points to the exercise of public authority (either as the narrower type – regulation/scrutiny, or the more general form – other kinds of exercising public authority). If the connection to law/regulations is weak or not present, it could be perceived as general public services.

33


Appendix 4: Empirical - methodological support for the importance of some questions This appendix contains some empirical support for the value of some concepts and questions, based on results of the Flemish survey of 2002-2003 (84 organisations). AUTONOMY: It is important for the conceptual coherence of the questionnaire that all questions remain in the survey. Managerial autonomy (on finances and on HRM, both strategic and operational) and policy autonomy (towards the political oversight authorities and the administrative oversight authorities) are distinct concepts. As to the Flemish data we found that correlations between the items of the different concepts show high and significant values. Items within a concept show high scale reliability and factor analysis reveals two dimensions of autonomy: managerial autonomy and policy autonomy (see appendix). STEERING: We argue that what ex post result steering is concerned all elements of the result steering cycle should be involved for conceptual reasons. Therefore, questions towards goal setting, use of indicators, measurement and evaluation of results, and sanctions and rewards are important indicators for assessing the result steering relationship between organization and oversight authority as a complete result steering cycle. Next to that, alternative ways of steering that are often underestimated in research design such as structural steering and financial steering are important to assess if we want to have a more nuanced picture of the complete steering practice. CULTURE: It is important to use the instrument for assessing culture as a single and coherent set of questions. The thirty-six items together form one instrument for assessing nine sub-dimensions of culture. Earlier research (Tepeci) and own empirical results show that there are indeed nine sub-concepts (after factor analysis, Tepeci), that show high correlations and scale reliability between and for the items that constitute one sub-dimension (own empirical results, see appendix). PERFORMANCE Q1: These variables measure the extent to which organizations use techniques that are assumed to augment organizational performance. Conceptually we have clustered the variables into three categories – internal, external and quality techniques. The variables (items) that constitute these categories seem to correlate high and significantly (except for quality techniques) and have high internal scale reliability, again except for quality techniques (see appendix). Therefore, it is important to retain all variables in the survey for reasons of this conceptual variability. PERFORMANCE Q2:

34


Again the twelve items together form one single instrument. Conceptually five dimensions of performance can be identified, all of which have high and significant inter-item correlations and all of which seem reliable scales, except for the ‘stability in the environment’ perspective (see appendix). Moreover, factor analysis( see appendix) shows three dimensions of performance; of which the first dimension clusters the goal perspective items (effic, effect and qual), of which the second dimension clusters the personnel perspective items (satstaff and motiv) and of which the third dimension clusters the public performance items (respsoc, accsoc and democrat). Bearing in mind this conceptual and empirical multidimensionality of the concept of performance, we prefer to integrate all variables in the survey.

AUTONOMY – CORRELATIONS HRM: strategic managerial autonomy

smasal smaprom smaeval smaappoi Correlation Coefficient 1,000 ,786(**) ,677(**) ,755(**)

Sig. (2-tailed) . ,000 ,000 ,000

smasal

N 79 79 79 79Correlation Coefficient ,786(**) 1,000 ,860(**) ,800(**)

Sig. (2-tailed) ,000 . ,000 ,000

smaprom

N 79 79 79 79Correlation Coefficient ,677(**) ,860(**) 1,000 ,748(**)

Sig. (2-tailed) ,000 ,000 . ,000

smaeval

N 79 79 79 79Correlation Coefficient ,755(**) ,800(**) ,748(**) 1,000

Sig. (2-tailed) ,000 ,000 ,000 .

Kendall's tau_b

smaappoi

N 79 79 79 79

HRM: operational managerial autonomy

omaadv omaprom omaeval omaappoi Correlation Coefficient 1,000 ,614(**) ,282(*) ,543(**)

Sig. (2-tailed) . ,000 ,012 ,000

omaadv


Sig. (2-tailed) ,000 . ,000 ,000

omaprom

N 81 81 81 81

Kendall's tau_b

omaeval Correlation Coefficient ,282(*) ,459(**) 1,000 ,407(**)

35


Sig. (2-tailed) ,012 ,000 . ,000 N 81 81 81 81Correlation Coefficient ,543(**) ,877(**) ,407(**) 1,000

Sig. (2-tailed) ,000 ,000 ,000 .

omaappoi

N 81 81 81 81

36


Financial managerial autonomy

maloans matariff mapartic Correlation Coefficient 1,000 ,430(**) ,567(**)

Sig. (2-tailed) . ,000 ,000

maloans

N 80 80 79 Correlation Coefficient ,430(**) 1,000 ,497(**)

Sig. (2-tailed) ,000 . ,000

matariff

N 80 80 79 Correlation Coefficient ,567(**) ,497(**) 1,000

Sig. (2-tailed) ,000 ,000 .

Kendall's tau_b

mapartic

N 79 79 79

omaperu omacoin omayears

Correlation Coefficient 1,000 ,811(**) ,402(**)

Sig. (2-tailed) . ,000 ,000

omaperu

N 79 77 77Correlation Coefficient ,811(**) 1,000 ,457(**)

Sig. (2-tailed) ,000 . ,000

omacoin

N 77 78 75Correlation Coefficient ,402(**) ,457(**) 1,000

Sig. (2-tailed) ,000 ,000 .

Kendall's tau_b

omayears

N 77 75 77

Policy autonomy

patarget patargcd painstru painstcd Correlation Coefficient 1,000 ,494(**) ,647(**) ,379(**)

Sig. (2-tailed) . ,000 ,000 ,000

patarget


Sig. (2-tailed) ,000 . ,000 ,000

patargcd


Sig. (2-tailed) ,000 ,000 . ,000

painstru


Kendall's tau_b

painstcd

Sig. (2-tailed) ,000 ,000 ,000 .

37


FACTOR ANALYSIS – AUTONOMY FIRST: Construct indexes; goal is to obtain ordinal scales of autonomy. OPA, SPA, FA, POLMIN, POLCD Reliability of these scales:

New variable Original variables Alpha SPA Smasal – Smaprom – Smaeval – Smaappoi. .93 OPA Omaadv – Omaprom – Omaeval - Omaappoi .83 FA maloans – matarif - mapartic .79

POLMIN Patarget - Painstru .79 POLCD Patargcd - Painstrucd .83

SECOND: Factor analysis with 5 variables REMARKS:

- ordinal scales, not interval - normal distribution of the variables

Component

1 2 SPA ,841 ,192 POLMIN ,300 ,812 POLCD ,080 ,920 OPA ,617 ,500 FA ,842 ,104

38


CULTURE – CORRELATIONS & RELIABILITY People oriented

respindr suppempl empathy careempl Correlation Coefficient 1,000 ,217(*) ,150 ,370(**)

Sig. (2-tailed) . ,024 ,120 ,000

respindr

N 77 76 77 75Correlation Coefficient ,217(*) 1,000 ,442(**) ,513(**)

Sig. (2-tailed) ,024 . ,000 ,000

suppempl

N 76 76 76 74Correlation Coefficient ,150 ,442(**) 1,000 ,396(**)

Sig. (2-tailed) ,120 ,000 . ,000

empathy


Sig. (2-tailed) ,000 ,000 ,000 .

Kendall's tau_b

careempl

N 75 74 75 75

Team oriented

coopcoll teamspir teamorie coopoth Correlation Coefficient 1,000 ,744(**) ,738(**) ,690(**)

Sig. (2-tailed) . ,000 ,000 ,000

coopcoll


Sig. (2-tailed) ,000 . ,000 ,000

teamspir


Sig. (2-tailed) ,000 ,000 . ,000

teamorie


Sig. (2-tailed) ,000 ,000 ,000 .

Kendall's tau_b

coopoth

N 75 76 75 77

39


Fair Compensation

goodfinrew equalrew faircomp presrelc Correlation Coefficient 1,000 ,060 ,374(**) ,194(*)

Sig. (2-tailed) . ,528 ,000 ,044

goodfinrew

N 77 75 74 75Correlation Coefficient ,060 1,000 ,419(**) ,030

Sig. (2-tailed) ,528 . ,000 ,749

equalrew


Sig. (2-tailed) ,000 ,000 . ,008

faircomp

N 74 74 75 75Correlation Coefficient ,194(*) ,030 ,250(**) 1,000

Sig. (2-tailed) ,044 ,749 ,008 .

Kendall's tau_b

presrelc

N 75 75 75 76

Customer oriented and quality services

qualsd custexp valucust custrel Correlation Coefficient 1,000 ,267(**) ,320(**) ,416(**)

Sig. (2-tailed) . ,006 ,001 ,000

qualsd


Sig. (2-tailed) ,006 . ,000 ,000

custexp


Sig. (2-tailed) ,001 ,000 . ,000

valucust


Sig. (2-tailed) ,000 ,000 ,000 .

Kendall's tau_b

custrel

N 75 77 77 77

40


Development employees

training promo advancem percadev Correlation Coefficient 1,000 ,269(**) ,189(*) ,174

Sig. (2-tailed) . ,004 ,049 ,065

training


Sig. (2-tailed) ,004 . ,000 ,000

promo

N 74 75 73 74Correlation Coefficient ,189(*) ,583(**) 1,000 ,597(**)

Sig. (2-tailed) ,049 ,000 . ,000

advancem

N 74 73 75 75Correlation Coefficient ,174 ,432(**) ,597(**) 1,000

Sig. (2-tailed) ,065 ,000 ,000 .

Kendall's tau_b

percadev

N 75 74 75 76

Detail oriented

detailor attdetai precisio accuracy Correlation Coefficient 1,000 ,591(**) ,280(**) ,216(*)

Sig. (2-tailed) . ,000 ,004 ,028

detailor


Sig. (2-tailed) ,000 . ,000 ,000

attdetai


Sig. (2-tailed) ,004 ,000 . ,000

precisio

N 74 75 77 76Correlation Coefficient ,216(*) ,357(**) ,670(**) 1,000

Sig. (2-tailed) ,028 ,000 ,000 .

Kendall's tau_b

accuracy

N 74 75 76 76

41


Honesty and ethics

trust integrit honesty keepprom Correlation Coefficient 1,000 ,441(**) ,493(**) ,346(**)

Sig. (2-tailed) . ,000 ,000 ,000

trust

N 77 75 75 75Correlation Coefficient ,441(**) 1,000 ,529(**) ,208(*)

Sig. (2-tailed) ,000 . ,000 ,034

integrit


Sig. (2-tailed) ,000 ,000 . ,000

honesty

N 75 75 76 74Correlation Coefficient ,346(**) ,208(*) ,510(**) 1,000

Sig. (2-tailed) ,000 ,034 ,000 .

Kendall's tau_b

keepprom

N 75 74 74 76

Innovation oriented

innov risktak experim creativi Correlation Coefficient 1,000 ,364(**) ,477(**) ,564(**)

Sig. (2-tailed) . ,000 ,000 ,000

innov


Sig. (2-tailed) ,000 . ,000 ,000

risktak


Sig. (2-tailed) ,000 ,000 . ,000

experim


Sig. (2-tailed) ,000 ,000 ,000 .

Kendall's tau_b

creativi

N 76 75 76 76

42


Goal oriented

taskacc workhard goalorie resorie Correlation Coefficient 1,000 ,303(**) ,399(**) ,457(**)

Sig. (2-tailed) . ,003 ,000 ,000

taskacc


Sig. (2-tailed) ,003 . ,000 ,000

workhard


Sig. (2-tailed) ,000 ,000 . ,000

goalorie


Sig. (2-tailed) ,000 ,000 ,000 .

Kendall's tau_b

resorie

N 76 76 76 77

SCALE Reliability

SCALE (index) Alpha People oriented .73 Team oriented .93 Fair compensation .61 Customer oriented .73 Development employees .76 Detail oriented .82 Honesty and ethics .79 Innovation oriented .86 Goal oriented .80 Factor analysis

Component 1 2 3 effic ,810 effec ,786 qual ,821 satstaff ,709 motiv ,658 qualmgm ,628 intcoh ,635 stabenv ,662 flexibil ,816 respsoc ,919accsoc ,931democrat ,686

PERFORMANCE Q1: correlations and scales

43


Internal techniques planyear intsteer intalloc asseres intmgmau reshrm coscalc planyear Correlation

Coefficient 1,000 ,327(**) ,308(**) ,270(*) ,295(**) ,233(*) ,399(**)

Sig. (2-tailed) . ,002 ,003 ,010 ,005 ,024 ,000 N 77 76 77 77 76 77 75intsteer Correlation

Coefficient ,327(**) 1,000 ,400(**) ,388(**) ,196 ,305(**) ,208

Sig. (2-tailed) ,002 . ,000 ,000 ,068 ,004 ,053 N 76 76 76 76 75 76 75intalloc Correlation

Coefficient ,308(**) ,400(**) 1,000 ,316(**) ,135 ,347(**) ,333(**)

Sig. (2-tailed) ,003 ,000 . ,002 ,195 ,001 ,002 N 77 76 77 77 76 77 75asseres Correlation

Coefficient ,270(*) ,388(**) ,316(**) 1,000 ,089 ,242(*) ,302(**)

Sig. (2-tailed) ,010 ,000 ,002 . ,402 ,021 ,005 N 77 76 77 77 76 77 75intmgmau Correlation

Coefficient ,295(**) ,196 ,135 ,089 1,000 ,280(**) ,345(**)

Sig. (2-tailed) ,005 ,068 ,195 ,402 . ,007 ,001 N 76 75 76 76 76 76 74reshrm Correlation

Coefficient ,233(*) ,305(**) ,347(**) ,242(*) ,280(**) 1,000 ,390(**)

Sig. (2-tailed) ,024 ,004 ,001 ,021 ,007 . ,000 N 77 76 77 77 76 77 75coscalc Correlation

Coefficient ,399(**) ,208 ,333(**) ,302(**) ,345(**) ,390(**) 1,000

Sig. (2-tailed) ,000 ,053 ,002 ,005 ,001 ,000 . N 75 75 75 75 74 75 76

External techniques

innovati extservd restruct Correlation Coefficient 1,000 ,453(**) ,503(**)

Sig. (2-tailed) . ,000 ,000

innovati


Sig. (2-tailed) ,000 . ,001

extservd


Sig. (2-tailed) ,000 ,001 .

Kendall's tau_b

restruct

N 74 77 77

Quality techniques publrep qualstan custsurv qualmgmt intunmon

Correlation Coefficient 1,000 -,008 ,069 -,154 -,095 Kendall's tau_b publrep

Sig. (2-tailed) . ,947 ,549 ,197 ,422

44


N 79 70 76 71 72Correlation Coefficient -,008 1,000 ,281(*) ,457(**) ,468(**)

Sig. (2-tailed) ,947 . ,021 ,000 ,000

qualstan

N 70 70 69 68 70Correlation Coefficient ,069 ,281(*) 1,000 ,340(**) ,236(*)

Sig. (2-tailed) ,549 ,021 . ,004 ,048

custsurv

N 76 69 78 71 71Correlation Coefficient -,154 ,457(**) ,340(**) 1,000 ,374(**)

Sig. (2-tailed) ,197 ,000 ,004 . ,002

qualmgmt

N 71 68 71 72 69Correlation Coefficient -,095 ,468(**) ,236(*) ,374(**) 1,000

Sig. (2-tailed) ,422 ,000 ,048 ,002 .

intunmon

N 72 70 71 69 72

SCALE reliability Internal techniques: .76 External techniques: .72 Quality techniques: .56

PERFORMANCE Q2: correlations and scales Goal

effic effec qual Correlation Coefficient 1,000 ,609(**) ,393(**)

Sig. (2-tailed) . ,000 ,000

effic


Sig. (2-tailed) ,000 . ,000

effec


Sig. (2-tailed) ,000 ,000 .

Kendall's tau_b

qual

N 66 66 66

Personnel

satstaff motiv Correlation Coefficient 1,000 ,565(**)

Sig. (2-tailed) . ,000

satstaff

N 68 67

Kendall's tau_b

motiv Correlation Coefficient ,565(**) 1,000

45


Sig. (2-tailed) ,000 . N 67 67

Internal processes

qualmgm intcoh Correlation Coefficient 1,000 ,488(**)


qualmgm

N 67 65Correlation Coefficient ,488(**) 1,000

Sig. (2-tailed) ,000 .

Kendall's tau_b

intcoh

N 65 66

System

stabenv flexibil Correlation Coefficient 1,000 ,462(**)


stabenv

N 67 67Correlation Coefficient ,462(**) 1,000

Sig. (2-tailed) ,000 .

Kendall's tau_b

flexibil

N 67 67

Public

respsoc accsoc democrat Correlation Coefficient 1,000 ,619(**) ,302(**)

Sig. (2-tailed) . ,000 ,003

respsoc


Sig. (2-tailed) ,000 . ,000

accsoc


Sig. (2-tailed) ,003 ,000 .

Kendall's tau_b

democrat

N 66 66 66

SCALES reliability

SCALE (index) Alpha Goal .83 Personnel .80 System .79 Stability .59 Public sector .82

46

Documents

Common data in the COBRA-research: An outlinegoals and results (other than purely financial goals) Q4. Who evaluates results Structural control Q5. Sanctions and rewards Q6. Regular