Canadian Sustainability Indicators NetworkLearning Event – June 8, 2006

Cultural Indicators

Nancy Duxbury *Creative City Network of Canada

“How do we measure culture? This is really two questions: How do we measure the magnitude of arts and culture? and How do we measure the value of arts and culture?”

(Brooks & Kushner, 2002, p. 12)

“Statistical indicators are an integral part of the ‘toolbox’ that policymakers use to understand, evaluate and communicate the importance and effectiveness of their policies and programs.”

(Madden, 2005, p. 4)

* Thank you to Diana Leung, M. Sharon Jeannotte, and John Foote for your invaluable assistance.

Overview

1. What are cultural indicators and why do they matter?

2. Who is developing cultural indicators and for what purpose?

Evolving context / Current landscape

– Contexts: 2 key tracks, and some related spheres

– General “state of play”

Snapshots of some initiatives relevant to this area

3. What unique challenges emerge when developing these indicators?

1. What are cultural indicators and why do they matter?

Definitions

• In general, indicators are defined as “bits of information that summarize the characteristics of systems or highlight what is happening in a system,” information bits that can “simplify complex phenomena” and enable a community to “gauge the general status of a system to inform action” (P. Berry, cited in City of Ottawa, 2003a, p. 3).

• “An indicator is an instrument or tool for evaluation, a yardstick to measure results and to assess realization of desired levels of performance in a sustained and objective way,” (Chapman, 2000).

• A cultural indicator is generally defined as: “a statistic that can be used to make sense of, monitor, or evaluate some aspect of culture, such as the arts, or cultural policies, programs and activities” (Madden, 2005, p. 17).

- quantitative and qualitative indicators

A few notes on the field

• Cultural indicators literature can be traced back to at least the early 1970s (Gouiedo, 1993). Yet cultural indicators – as with other social indicators – are still largely under development, particularly in their relevance to policy and program delivery.

• Interest is high as an applied field and practice – a lot of work on cultural indicators is happening, from the local to international scale. In 2005, the International Federation of Arts Councils and Culture Agencies (IFACCA) published Statistical Indicators for Arts Policy report which provides an excellent overview of the field and active initiatives internationally (Madden, 2005).

• To date, “comprehensive indicators of the cultural vitality of a community” have been difficult to find (Rettig, 2002), and a comprehensive, integrating framework has yet to form. There is no single measure that evaluates an “arts economy” comprehensively (Brooks & Kushner, 2002, p. 20).

• Sense of a general aversion to the development of aggregated indices, which can mask important details:“it conceals numerous separate dimensions of the arts and subsumes individual areas of strength and weakness. For policy-makers, this detailed information is necessary to tailor policy and management answers to particular challenges and opportunities” (Brooks & Kushner, 2002, p. 13).

Types of indicators

Even at a high level there are many ways of thinking about different types of indicators (Madden, 2005). Some examples are:

1. Cultural indicators (such as “quality of life” indicators) vs. performance indicators for the cultural sector (such as financial indicators of cultural industries) (Matarasso, 2001).

2. Cultural indicators vs. cultural policy indicators

3. Indicators about culture per se (e.g., cultural audiences, health of cultural organizations) vs. cultural components of other indicators (e.g., neighbourhood vibrancy indicators)

4. Intrinsic indicators (artistic and cultural values) vs. instrumental indicators (values such as economic and social impacts)

5. Arts vs. cultural indicators (e.g., UNESCO [1998 and 2000] cultural indicators for development focus on broader cultural phenomena and are not detailed enough to serve as a set of indicators for arts policy)

Indicator hierarchies

Applied at different levels (which may require different objectives and methods at different levels for the indicators to be effective)

(Indicatpr Hierarchies table from: Madden, 2005, p. 23)

What are cultural indicators used for? (Why do they matter?)

4 general areas of uses for cultural indicators:

1. Monitoring and evaluation- observing cultural phenomena, their changes and trends – continuous monitoring of development in a given policy area- measuring the effectiveness of policies and programs aimed at impacting cultural phenomena (e.g., setting goals; outcomes-based accountability; evaluation of program effectiveness)

2. Learning- a tool for learning, adapting, and changing ... and decision-making leading from this - guides to further investigation

3. Influencing behaviours and attitudes: “Strategic” effects- tend to affect the behaviour of institutions (could have undesirable properties)- build public confidence in cultural institutions- can stimulate public dialogue, esp. during the development process

4. Advocacy- justification of cultural policies- note: some ongoing tension between research and advocacy

(Madden, 2005, pp. 19-21)

2. Who is developing cultural indicators and for what purpose?

Evolving context / Current landscape

Contexts: 2 key tracks

Community / sustainability

3 dimensions (focusing on local scope):

1. Community QOL / Sustainability indicator projects (North America)

- growing prevalence

- culture often considered but not included due to perceived lack of data, or “silently”– this is gradually changing

2. Growing attention and involvement in cultural development/planning in communities, by local governments

3. Emergence of four-pillar model of sustainability: environment, economy, social, and culture core dimensions

- holistic perspective on community sustainability

- culture as agent of change/resiliency

- cultural lens to community planning, decision-making

- ties into integrated community sustainability planning (Canada, UK, Australia)

Cultural policy / sector

Growing pressure to document impacts (especially economic and social), tied to “making the case” and program evaluation; measurement of progress/change

Part of movement to ‘improve’ social indicators more generally, to develop better measures of progress and to meet the demands for greater accountability in government policies and programs

Influenced by related areas of work and research, including:

1. Social impacts of the arts

2. Cultural statistics programs

3. Indicator theory in general, and developments in indicator methods used in other policy spheres (social, economic, QOL, well-being), many which contain a cultural element

4. Evaluation methodology

5. Issues re evidence-based policy

Some related spheres:

Social cohesion

Cultural citizenship

Cultural rights

Immigration and settlement (and societal adaptation/integration)

Diversity – anti-racism and discrimination

Urban governance (e.g., UN-Habitat/UNESCO)

Municipal benchmarking initiatives (e.g., Ontario)

Coordination/collaboration of multiple levels of government

Recreation

Community (social) services

Economic development

Cultural tourism

General “state of play” (in process)

Individual (solo) projects > intermeshing/weaving; integrating into broader initiatives

Individual efforts > collective process/project

Informal relations > formal and informal coalition(s)

Conceptual > pilot(s)/application

Increasing recognition of the importance of location: research closest to the ground probably spawns most of the solid research on indicators to the extent that they exist *

Snapshots of some initiatives relevant to this area:

Canada

Mapping Quality of Life and the Culture of Small Cities - 5-year CURA-funded research program (2006-2011) - Thompson Rivers University lead partner to connect a variety of research projects and community partners within 4 broad themes:

- mapping points of entry- mapping community stories- mapping community and cultural assets- mapping cultural intersections and processes

Regional Vancouver Urban Observatory (RVu) - a public participation study group process to determine and develop sustainability indicators for the region. The project includes an “Arts, Culture and Community” component. The “top three” indicators developed in this area were:

1. Quantity and quality of opportunities for cultural activity, as represented by an annually updated cultural events matrix2. Percent of individuals who feel that they have adequate or better access, freedom and time for cultural and artistic activity3. Ratio of dollars spent promoting multicultural awareness/diversity and artistic endeavours relative to the dollars these

activities contribute to society

Centre for Native Policy and Research - medicine wheel model for indicator development: east is “Culture and family” Two culture indicators:- Percentage of Aboriginal people in the GVRD speaking traditional languages- Percentage of Aboriginal people in the GVRD participating in traditional activities

Snapshots – Canada (continued)

FCM Quality of Life Reporting System – Culture Working Group (municipalities) – established and beginning to work on ‘culture and arts’ area

Metropolis – 10 years of research on issues of migration and settlement – growing interest to consider cultural issues, dimensions within the Metropolis research project. The International Metropolis conference in Toronto, October 2005 featured a session on cultural indicators; a book chapter on the cultural dimensions of immigration/settlement and the development of appropriate indicators currently under development.

Department of Canadian Heritage – a number of interests, including: Metropolis cultural indicator work; cultural citizenship measures; program/policy performance measures

Creative City Network of Canada:

- Centre of Expertise on Culture and Communities re city/community impacts of cultural infrastructure and the activity it supports, measurability of impacts (beginning stages – clearinghouse, network-building)

- Intermunicipal Comparative Framework – benchmarking and documentation of municipal involvement and investment in cultural development

Individual cities, linked to and contextualized by their city plans/processes and cultural plans (e.g., City of Ottawa, City of Toronto, many others)

International

Council of Europe's Methodological Guide on the Concerted Development of Social Cohesion Indicators A massive study intended to develop an indicators set, based on the Council of Europe's definition of social cohesion as “the capacity of a society to ensure the welfare of all its members, minimising disparities.” Among the huge body of suggested indicators are two categories – Information/communication and Culture – that are directly relevant.

In the Information/communication category, indicators are suggested to answer the following questions: - Are the basic conditions in place to ensure that citizens have access to information and communication?- Are the conditions in place to ensure that citizens are well-informed?- What is the situation of the most disadvantaged in terms of information?- What are the risks of information exclusion?- Is the protection of privacy ensured?- Is consideration given to information dissemination for different cultures, ways of thinking, profession and social groups' activities, etc.?- Does information draw public attention to the least well-off and to the situation of minorities and stimulate tolerance, solidarity and mutual comprehension?- What are the  risks of attacks against the dignity and fundamental rights of people?- Are the conditions in place at information level (sic) to ensure the autonomy and personal, family and occupational development of everyone?- To what extent do citizens use available information for their personal, family and occupational development?- What is done to assist people who do not have ready access to information?- What are the risks of the dissemination of prejudicial information for personal development?- Does existing information stimulate the exercise of democracy and the full expression of citizenship?- What possibilities are available for citizens to fulfill their expectations in terms of information control, information quality and organizing their own information

networks?- What are the opportunities for the least well-off to participate in information dissemination and to have a means of communication?- What are the risks of poor information (information manipulation, excessive information, etc.) concerning the exercise of democracy and citizenship?- What are consumers' expectations and level of satisfaction in terms of information?- What is the level of citizen awareness based on the information they receive?- What values do citizens expect from the media?- What is the level of citizens' confidence in the media?- To what extent does information contribute to social links?

Then, there is a whole series of indicators to measure actions in these areas by central and local governments, media and media associations, journalists, firms and NGOs.

Council of Europe's Methodological Guide on the Concerted Development of Social Cohesion Indicators

In the Culture category, there is a similarly comprehensive set of indicators suggested to answer the following questions:

- Are the conditions in place to ensure access to culture for everyone?- What are the trends in the interest of citizens for culture and in their level of culture?- What is the situation of the most vulnerable population groups in cultural terms?- What are the risks of "acculturation"?- Are the conditions in place to ensure the preservation and promotion of cultural difference, freedom of expression and for creating links

between different cultures?- Is there cultural diversity, mutual respect between cultures, and intercultural dialogue?- What is the situation of minority cultures?- Is there a cultural renewal?- Are the conditions in place to ensure that the cultural dimension is fully integrated in the personal, family and occupational development of

citizens?- To what extent does cultural practice contribute to personal development and the creation of social links?- What is the situation of vulnerable populations with regard to the cultural practices?- What are the limits of cultural practices?- What impact does culture have on social integration and on the exercise of citizenship?- To what extent are citizens able to participate in the implementation of cultural policies?- How do citizens participate in the protections of the cultural, community and environmental heritage?- What are the risks to cultural diversity posed by the industrialization of certain cultural sectors?- What are citizens' cultural expectations and their level of satisfaction?- What are the cultural references perceived by citizens and with which they identify?- What are the shared values that culture helps to strengthen?- To what extent is culture a factor of confidence?- To what extent does culture contribute to forging social links and avoiding isolation?

Then, again, there are a series of indicators to measure actions in these areas by central and local governments, firms and NGOs.

International (continued)

UNESCO / United Cities and Local Governments - “Local policies for cultural diversity” – evaluation of local policies supporting expressions of cultural diversity, and development of related indicators

UNESCO - evaluation of municipal policies against racism and discrimination (potential tie to Urban Governance Index being developed by UN-Habitat)

OECD - economic and social impacts of culture – national basis; selected major cities (2006-07)

Hong Kong - Creativity Index (interim report - November 2004)

UK - Colin Mercer (cultural citizenship; culture and quality of life)- All local authorities are required to develop Cultural Strategies + performance assessment legislative requirements —> wider range of indicators measuring outcomes (rather than just outputs) such as human and social capital, economic, and other impacts (Mercer, 2006)- Also, Cultural Strategies must now be linked to Sustainable Community Strategies for every local authority in England. These strategies are developed through a local partnership, which draws up a Local Area Agreement. The outcomes of the agreement are evaluated by performance indicators (e.g., related to participation in cultural activities and amenities, satisfaction an/or attitude, impact, value for money). (Mercer, 2006; LGA/DCMS Cultural Pathfinder Programme Evaluation Framework, 2005)

US - e.g., Cultural Initiatives Silicon Valley; Boston Foundation Indicators Project- Project to develop inter-city comparative indicators related to key questions (Brooks & Kushner, 2002)

Australia - various initiatives: local government associations, individual cities, others- Integrated Local Area Planning

3. What unique challenges emerge when developing these indicators?

Conceptual

Definitions - Culture is something that is notoriously difficult to define. Therefore, it is difficult to reach agreement on exactly what one should be measuring.

- focus–scope (e.g., professional/non-professional; production, consumption; arts/heritage/libraries/“street culture”)

- models o cultural ecosystems o re program/policy evaluation: issues of attributing causality; isolation of effects

- measurability- statistics vs. indicators

Data availability

Since the pressure to develop cultural indicators is relatively recent, in a lot of cases the data needed to develop robust indicators simply doesn't exist because no one ever thought that it was important to collect. The corollary to this is that data collection in the cultural area (particularly collection of non-economic data) is usually low on the budget priority list of most public and private organizations.

- Development of relevant sources of reliable data – collected systematically- Better use of existing data

Use-context(s)

General purposes contextCulture is generally a "soft" area, while those who seek indicators are usually interested in "hard" measures.

Is sufficient time taken to frame the questions that researchers/policy-makers want to answer? ”Clarity at the outset is always a challenge, but in the area of culture – which has social, economic, political and environmental impacts – this challenge is much greater than it might be if one wished to measure something less complex.” *

Indicator frameworks“Frameworks are unwieldy.” — “Policy objectives are vague.”Frameworks often have too many indicators with too many “policy foci, statistical variables and stages in the cultural ‘value chain,’” leaving results too difficult to adapt to specific policy purposes (Madden, 2005, p. 9). - wish lists

Other considerations- assessing system <–> assessing/evaluating policy, actions ...- policy-relevant focus (usability value essential <–> may narrow scope/focus of interest; privilege particular

perspectives, models)- who- methodologies and practices of interpreting and presenting indicators - report cards vs. contextualized

understanding

Integration

- Conceptually with other domains- Pragmatically with other researchers

Coordination issues include:

Multiplicity of work. There has been little coordination between cultural agencies developing cultural indicators, resulting with duplicate, “doubling up” of work.

Differences in approach. “Greater coordination and sharing of work could lead to better solutions for common, or ‘generic’ problems,” and allow for cultural indicators research agenda to be “expedited” and “…a core set of cultural indicators [to] be promoted.” (Madden, 2005, p. 9)

“Improving cultural indicators is not simply about supplying better statistics and

undertaking statistical development work: it is also about understanding better

the nature of arts activities, improving the articulation of arts policies, and

being aware of the interrelationships between data and policy analysis

and the impacts that measurement can have on the arts and cultural sectors.”

(Madden, 2005, p. 13)

Reference note

All references cited in this presentation are included in the Cultural Indicators: Working Bibliography developed by the Creative City Network of Canada’s Centre of Expertise on Culture and Communities, which is available online at: http://www.creativecity.ca/cecc/bibliographies/cultural-indicators.html