Towards the ACWP Questionnaire - Australian Child Wellbeing · The Australian Child Wellbeing Project: Phase Two Report vii Table 1: Content of final field trial questionnaire Major

Towards the ACWP Questionnaire

Australian Child Wellbeing Project

Phase Two Report 17 June 2014

Petra Lietz, Elizabeth O’Grady, Mollie Tobin, Alice McEntee

and Gerry Redmond

The Australian Child Wellbeing Project: Phase Two Report ii

Background

The Australian Child Wellbeing Project was conducted by a consortium of researchers at Flinders University of South Australia, the University of New South Wales, and the Australian Council for Educational Research. It was funded by the Australian Research Council through a Linkage Grant, and supported by Australian Government Partner Organisations including the Department of Education, the Department of Social Services, the Australian Institute of Health and Welfare, and the Australian Bureau of Statistics.

The Project Steering Group was chaired by Professor George Patton, University of Melbourne, and included as members the Chief Investigators (Professor Gerry Redmond, Dr Jennifer Skattebol and Professor Peter Saunders), Partner Investigators (Prof Sabine Andresen, Professor Jonathan Bradshaw and Dr Sue Thomson), representatives of the Partner Organisations, and independent advisers: Dr Ben Edwards (AIFS); Dr Lance Emerson (ARACY) and Ms Margaret Raven (Social Policy Research Centre, University of NSW).

Further information: http://australianchildwellbeing.com.au/

Research team Flinders University: Associate Professor Gerry Redmond, Dr Helen Popple, Ms Alice McEntee and Dr Grace

Skrzypiec. University of NSW: Dr Jennifer Skattebol, Professor Peter Saunders, Dr Myra Hamilton, Ms Tammy Burnstock,

Dr Bridget Jenkins and Mr Kirk Dodd. Australian Council for Educational Research: Dr Sue Thomson, Dr Petra Lietz, Ms Elizabeth O'Grady and Ms

Mollie Tobin. University of York (UK): Professor Jonathan Bradshaw. University of Frankfurt (Germany): Professor Sabine Andresen.

Suggested citation Lietz, P., O’Grady, E., Tobin, M., McEntee, A. and Redmond, G. (2014). Towards the ACWP Questionnaire: The Australian Child Wellbeing Project Phase Two Report. Flinders University, the University of NSW and the Australian Council for Educational Research. www.australianchildwellbeing.com.au

Copyright

© Australian Council for Educational Research 2014.

This work is copyright. Apart from any use permitted under the Copyright Act 1968, no part may be reproduced by any process, nor may any other exclusive rights be exercised, without the permission of Australian Council for Educational Research, 19 Prospect Hill Road, Camberwell, Victoria 3124, 2014.

Edited by Juliet Young-Thornton and Katherine Dix.

ISBN [978-1-74286-370-2]

http://australianchildwellbeing.com.au/

http://www.australianchildwellbeing.com.au/

The Australian Child Wellbeing Project: Phase Two Report iii

List of Contents

List of contents ....................................................................................................................................... iii

List of tables ........................................................................................................................................... iv

Abbreviations .......................................................................................................................................... v

Executive summary ................................................................................................................................ vi

Chapter 1. The process of questionnaire development for the ACWP .................................................. 1

Chapter 2: The field trial questionnaire as administered in Term 1, 2014 ............................................ 6

Attachment 1: The original draft field trial questionnaire as presented to the ACWP Project

Steering Group in September 2013 of the draft field trial questionnaire ....................................... 29

Attachment 1: Appendix A – Vignettes in the ACWP ........................................................................... 60

Attachment 2: Issues raised at the PSG, 24 September 2013 and subsequent PSG telephone

conference, 7 November 2013 and research team's responses ..................................................... 62

Attachment 3: The Australian Child Wellbeing Project - Cognitive Interviews. Full report to the

Project Steering Group .................................................................................................................... 75

Attachment 3: Appendix A – Cognitive interview schedule Year 4 ...................................................... 93

Attachment 3: Appendix B – Cognitive interview schedule Year 6 .................................................... 101

Attachment 3: Appendix C – Cognitive interview schedule Year 8 .................................................... 110

Attachment 4: ACWP Field Trial Analysis plan ................................................................................... 120

References .......................................................................................................................................... 124

The Australian Child Wellbeing Project: Phase Two Report iv

List of Tables

Table 1: Content of final field trial questionnaire .......................................................................... vii

Table 2: Overview of proposed indicators for the ACWP ............................................................. viii

Table 1.1: Conceptualisation of the final field trial questionnaire ..................................................... 3

Table 1.2: Length and item allocation in ACWP .................................................................................. 4

Table 1.3: Summary of proposed indicators and comparisons in the final field trial questionnaire .. 4

Table A1.1: Questionnaire conceptualisation of the original draft field trial questionnaire .............. 29

Table A1.2: Summary of proposed indicators and comparisons in the draft field trial

questionnaire ................................................................................................................... 55

Table A3.1: Cognitive interview survey domains and indicators ........................................................ 76

Table A3.2: Unforseen sources of error and example reactive probes ............................................... 77

Table A3.3: Cognitive interviews sample by sub-group and year level ............................................... 78

Table A4.1: List of scales included in the ACWP FT survey ................................................................ 121

Table A4.2: Gender item.................................................................................................................... 123

Table A4.3: ATSI item......................................................................................................................... 123

The Australian Child Wellbeing Project: Phase Two Report v

Abbreviations

ABS Australian Bureau of Statistics

ACER Australian Council for Educational Research

ACWP Australian Child Wellbeing Project

ATSI Aboriginal and Torres Strait Islander

CALD Culturally and Linguistically Diverse

IRSD Index of Relative Socio-economic Disadvantage; SEIFA index

LBOTE Language Background Other Than English

MCEETYA The Ministerial Council on Education, Employment, Training and Youth

Affairs

PIRLS Progress in International Reading Literacy Study

PISA Programme for International Student Assessment

SEIFA Socio-Economic Indexes for Areas, Australia

SES Socio-Economic Status

TIMSS Trends in International Mathematics and Science Study

The Australian Child Wellbeing Project: Phase Two Report vi

Executive Summary

This is the Phase Two report of the Australian Child Wellbeing Project (ACWP), which is a child-centred

study that designed a major nationally representative survey of wellbeing among 8-14 year olds based

upon the perspectives of young people themselves. The survey will benchmark child wellbeing in

Australia and provide information that contributes to the development of effective services for young

people’s healthy development. For the purposes of this project, wellbeing is broadly understood as

comprising young people’s material and environmental circumstances, their relationships, and how

they think about themselves in the context of those circumstances and relationships.

This project seeks to further the understanding of how young people in general, and disadvantaged

young people in particular, understand their own wellbeing. Particular attention is given to

understanding the perspectives of young people in six groups who are often seen as experiencing high

levels of marginalisation or as having particular experiences and needs: Aboriginal and Torres Strait

Islander young people, culturally and linguistically diverse young people, young people living with

disability, young people living in regional and remote Australia, economically disadvantaged young

people, and young people living in out-of-home care.

As a result of the discussions during the development of the field trial questionnaire, the aim of the

ACWP was further clarified as follows:

The overarching aim of the ACWP is to arrive at profiles of different groups of Australian young

people in the middle years in terms of their wellbeing, with a particular focus on disadvantaged

young people and a view to international comparisons.

Two major aspects of this aim are noteworthy. Firstly, rather than developing another instrument to

measure wellbeing, young people will be grouped into clusters in terms of their perceptions of

elements of wellbeing that are canvassed in the questionnaire. Factual questions, for example about

socio-economic status, family size and gender, will be used to describe clusters. Secondly,

comparisons with international and other data sets, are aimed at relating results from the ACWP to

other surveys. Discussions with Australian jurisdictions indicate a greater interest in comparisons

between the year levels (i.e. Years 4, 6, and 8) and a lesser interest in comparisons between states

and territories.

The overall project is divided into six major phases, which include the development and conduct of a

nationally representative school based online survey among young people at school in Years 4, 6 and

8, to be carried out in Term 3, 2014.

This document reports on Phase Two of the project: Development of the wellbeing indicators. The

development of indicators was informed by the findings of the first qualitative phase of the project, a

review of literature and existing wellbeing measures, considerations related to online

implementation, two stages of feedback from the Project Steering Group, as well as cognitive

interviews with children in the target population (i.e. Years 4, 6, and 8) who responded to various

versions of the field trial questionnaire. Table 1 below summarises the content of the final field trial

questionnaire of the ACWP.

The Australian Child Wellbeing Project: Phase Two Report vii

Table 1: Content of final field trial questionnaire

Major domains Correlates / Factual Subdomains Cross-cutting themes

1.1. Family

1.2. Family possessions Paid job Organisation of household(s) Out of home care Change: -house -school -carers Family health Out of home care

1.3. Togetherness - Family cohesion - Family management Worry - Vulnerability - Harmful

8. Bullying Feeling good Learning Closeness of relationships Optimism

2.1. Friends

2.2. Number of close friends

2.3. Support and conflict

3.1. School

3.2. Absence Teacher support Parental interest

3.3 Enjoyment Pressure Success at school Outside school activities

4.1. Community/ Neighbourhood

4.2. Not applicable

4.3. Access to resources Safety

5.1. Health

5.2. Hunger Breakfast Smoking and drinking

5.3. Subjective health Mental and physical health

6.1. Money and material wellbeing

6.2. Socio-economic status Deprivation

6.3. Covered partly in family subdomain 'Vulnerability'

7.1. Self-demographics

7.2 Language background Gender Aboriginal and Torres Strait Islander Disability Aspirations Puberty

7.3 Not applicable

Table 2 below provides an overview of the proposed indicators for the ACWP. These are likely to be

further refined after the analysis of the field trial data, which will be described in the Phase Three

Report, scheduled for delivery at the end of June 2014.

The Australian Child Wellbeing Project: Phase Two Report viii

Table 2: Overview of proposed indicators for the ACWP

Major domain Subdomain/ Correlates

Indicator (Preliminary label) Comparisons

Family Factual/ correlates

Organisation of the household(s) Children's Worlds

Paid job Children's Worlds

Family possessions - Car HBSC

Family possessions - Own bedroom HBSC

Family possessions - Holiday HBSC

Family possessions - Computers HBSC

Family possessions - Dishwasher HBSC

Family possessions – Foreign Holiday HBSC

Family possessions – Bathrooms HBSC

Family possessions - Books PIRLS/TIMSS

Changed house ACWP subgroups

Changed school ACWP subgroups

Changed carers ACWP subgroups

Out of home care Children’s Worlds

Family health ACWP subgroups

Togetherness Family cohesion Children's Worlds

Family management HowRU

Worry Vulnerability - Significant other ACWP subgroups

Harmful - Significant other ACWP subgroups

Friends Factual/ correlates

Number of close friends ACWP subgroups

Support and conflict

Degree of closeness/support - closest friend

ACWP subgroups

Conflict - closest friend ACWP subgroups

School Factual/ correlates

Missed school ACWP subgroups

Teacher support MDI

Parental Interest in school PIRLS/TIMSS

Success Success at school HBSC

Enjoyment School - General satisfaction LSAC

School - Intrinsic motivation LSAC

Pressure School - Pressure HBSC

Outside school activities

Participation in outside school activities

Children’s Worlds

Community/ Neighbourhood

Resources and safety

Access to resources Children’s Society

Safety Children’s Society

Health Factual/ correlates

Hunger HBSC

Breakfast HBSC

Smoking and drinking HBSC

Subjective health Overall subjective health HBSC

Mental and physical health

Headache HBSC

Stomach-ache HBSC

Backache HBSC

Feeling low HBSC

Irritability HBSC

Feeling nervous HBSC

Difficulties getting to sleep HBSC

Dizziness HBSC

Money and material wellbeing

Factual/correlates Socio-economic status Deprivation ACWP subgroups

The Australian Child Wellbeing Project: Phase Two Report ix



Self-demographics

Factual/correlates Language background PIRLS/TIMSS

Gender PISA

Aboriginal and Torres Strait Islander PISA

Disability Youth 2012 Student Health and Wellbeing Questionnaire -New Zealand

Disability difficulties Youth 2012 Student Health and Wellbeing Questionnaire -New Zealand

Aspirations HowRU

Puberty ACWP subgroups

Cross-cutting domains

Bullying Bullying Australian Covert Bullying Prevalence Study

Bullying - location ACWp subgroups

Bullied by friend ACWP subgroups

Bullying Initiator HowRU

Feeling good Overall wellbeing Children's World’s

Importance of domains for wellbeing ACWP subgroups

Cantril ladder HBSC

Optimism Optimism for the future ACWP subgroups

Closeness of relationships

Closeness/Proximity of Relationships ACWP subgroups

Note: ACWP subgroups include comparisons by gender, disability, grade, Aboriginal and Torres Strait Islander, language background, grade, socio-economic status.

This report contains the following components:

Chapter 1: The process of questionnaire development for the ACWP.

Chapter 2: The field trial questionnaire as administered in Term 1 2014.

Attachment 1: The original draft field trial questionnaire, as presented to the ACWP Project Steering

Group in September 2013.

Attachment 2: Issues raised at the PSG, 24th September 2013 and subsequent PSG telephone

conference, 7th November 2013, as well as the research team's responses.

Attachment 3: The Australian Child Wellbeing Project - Cognitive Interviews. Full report to the Project

Steering Group, April 2014.

Attachment 4: ACWP field trial analysis plan, April 2014.

The Australian Child Wellbeing Project: Phase Two Report 1

Chapter 1. The Process of Questionnaire Development for the ACWP

This report on Phase Two of the ACWP describes the process of questionnaire development which

included the following five steps:

1. The development of a draft field trial questionnaire based on the findings of the qualitative

work in Phase One of the project, prior research and a review of existing well-being instruments

(see Attachment 1).

2. Two rounds of comments from the Project Steering Group on the draft field trial questionnaire

(see Attachment 2).

3. Cognitive interviews with young people from the target population, including interview

schedules for each year level (see Attachment 3).

4. The final field trial questionnaire (see Chapter 2).

5. Analyses to finalise the questionnaire for the main survey to be conducted in Term 4, 2014 (see

Attachment 4).

Each of these steps is discussed in turn below.

As the first step, the development of the questionnaire was informed by the interviews and focus

groups undertaken in the qualitative work in Phase One of the ACWP (see

www.australianchildwellbeing.com.au), a detailed review of existing instruments aimed at measuring

wellbeing and prior research.

The qualitative work identified the following six domains of wellbeing, and priorities assigned to these

by young people.

Family Highest priority, most frequently raised by young people and discussed as the most important element of wellbeing.

Friends Ranked as a very high priority by majority of the groups and medium for other groups. Important element is a distinction between ‘good’ and ‘bad’ friends.

School Mixed priorities, depending on group

Community/Neighbourhood Wide variety of meaning, mixed priorities

Health Less important than family, friends and school

Money and material wellbeing Least important for wellbeing

It was decided that domains and themes that were given higher priority by young people should be

given more space in the questionnaire. In addition, six themes were identified as cutting across a

number of the domains, namely bullying, guidance and rules, learning, safety, stress, and feeling good.

http://www.australianchildwellbeing.com.au/


Existing wellbeing instruments that were reviewed during this first step of questionnaire development

include the following:

Health Behaviour in School-Aged Children (HBSC)

Personal Wellbeing Index-School Children (PWI-SC)

HowRU?

Children’s Society (CS)

Children’s Worlds (CW)

Communities that Care (CC)

Growing up in Ireland

Huebner Life Satisfaction

Kidscreen

Social and Emotional Wellbeing Survey (SEWB)

Middle Years Development Instrument (MDI)

The Young Lives Study

World Vision Kinderstudy

The Longitudinal Study of Australian Children (LSAC)

Health-Related Quality of Life in children and adolescents (KINDL)

Brief Multidimensional Students' Life Satisfaction Scale (MSLSS)

Student Health and Wellbeing - New Zealand (SHWB-NZ)

In addition, questionnaires from the following were reviewed for relevant questions that would allow

international comparisons with the Years 4, 8 and 15-year-old students in up to 67 countries that form

the target populations for these assessments.

Progress in Reading Literacy (PIRLS),

Trends in International Mathematics and Science Study (TIMSS) and the

Programme for International Student Assessment (PISA)

During this first step of the development of the field trial questionnaire, the aim of the ACWP was

further refined as follows.




Two major aspects of this aim are noteworthy. Firstly, rather than developing another instrument to

measure wellbeing, young people will be grouped into clusters in terms of their perceptions of

elements of wellbeing that are canvassed in the questionnaire. Factual questions, for example about

socio-economic status, family size and gender, will be used to describe clusters. Secondly,

comparisons with international and other data sets, are aimed at relating results from the ACWP to

other surveys. Discussions with Australian jurisdictions indicate a greater interest in comparisons

between the year levels (i.e. Years 4, 6, and 8) and a lesser interest in comparisons between states

and territories.

Together, these considerations led to the development of the draft field trial questionnaire that was

presented to the ACWP Project Steering Group (PSG) in September 2013 (see Attachment 1).

Attachment 1 provides Phase One conceptualisations and policy aims for each question, details

regarding the purpose of anchoring vignettes, as well as considerations for the online implementation

of the questionnaire.

In the next step, two rounds of comments from the PSG regarding the draft questionnaire were

received and considered, one after the meeting in September 2013 and one after an additional PSG

meeting in November 2013. Comments ranged from very specific suggestions regarding question


wording, to broader considerations regarding the extent and depth with which children were to be

asked about certain aspects of wellbeing. This involved, for example, questions obtaining information

about correlates/factual of wellbeing, such as organisation of the household and aspirations, as well

as questions covering the cross-cutting themes of “feeling good” and “bullying”. In many instances,

the desire to ask questions which reach for greater depth about an aspect of wellbeing had to be

balanced against the need to keep the questionnaire a manageable length for students in Years 4, 6,

and 8. Details of the comments received and how these were taken into consideration in the

questionnaire development are provided in Attachment 2.

Another step in the questionnaire development included cognitive interviews, which were undertaken

with 22 young people from the target population in three locations, one in New South Wales and two

in South Australia. The purpose of the cognitive interviews was to observe a) respondents'

comprehension of items, b) respondents' response processes, c) any signs of fatigue, and d) general

reactions and behaviours while responding to the questionnaire. While a summary of the cognitive

interviews is given as part of Attachment 2, a full report is provided as Attachment 3.

Together, these steps led to the development of the field trial questionnaire. The final field trial

questionnaire was administered to students in Years 4, 6, and 8 in eleven schools in Victoria and New

South Wales in Term 1 2014. The content of the final field trial questionnaire is described in detail in

Chapter 2 of this report. Table 1.1 below provides an overview of the conceptualisation of the final

field trial questionnaire.

Table 1.1: Conceptualisation of the final field trial questionnaire

Major domains Correlates/ Factual Subdomains Cross-cutting themes

1.1. Family 1.2. Family possessions Paid job Organisation of household(s) Out of home care Change: house, school, carers Family health Out of home care

1.3. Togetherness - Family cohesion - Family management Worry - Vulnerability - Harmful

8. Bullying Feeling good Learning Closeness of relationships Optimism

2.1. Friends 2.2. Number of close friends 2.3. Support and conflict

3.1. School 3.2. Absence Teacher support Parental interest

3.3. Enjoyment Pressure Success at school Outside school activities


4.2. Not applicable 4.3. Access to resources Safety

5.1. Health 5.2. Hunger Breakfast Smoking and drinking

5.3. Subjective health Mental and physical health


6.2. Socio-economic status Deprivation


7.1. Self demographics

7.2. Language background Gender Aboriginal and Torres Strait Islander Disability Aspirations Puberty

7.3. Not applicable


It should be noted that, with one exception, separate questions were developed to operationalise the

concepts in Table 1.1. The exception is learning as a cross-cutting theme. Questions about learning

have been included in the following scales: “Family cohesion”, “Enjoyment at school” and “Parental

interest”.

Keeping in mind the conceptualisation and the ages of the respondents, which range from about 8 to

14 years, Table 1.2 provides information regarding questionnaire length and item allocation.

Table 1.2: Length and item allocation in ACWP

Three forms of the ACWP questionnaire with common items:

Year 4 (20 minutes)

Year 6 (30 minutes)

Year 8 (30 minutes)

Demographics/Correlates/Factual questions (5 minutes)

Unique Year 4 items Unique Year 8 items

Common items Year 4&6 Common items Year 4&6 -

- Common items Year 6&8 Common items Year 6&8

Common items Year 4&6&8 Common items Year 4&6&8 Common items Year 4&6&8

Table 1.3 provides a summary of the proposed indicators and comparisons for the final field trial

questionnaire.

Table 1.3: Summary of proposed indicators and comparisons in the final field trial questionnaire




Organisation of the household(s) Children's Worlds

Paid job Children's Worlds

Family possessions - Car HBSC

Family possessions - Own bedroom HBSC

Family possessions - Holiday HBSC

Family possessions - Computers HBSC

Family possessions - Dishwasher HBSC

Family possessions – Foreign Holiday HBSC

Family possessions – Bathrooms HBSC

Family possessions - Books PIRLS/TIMSS

Changed house ACWP subgroups

Changed school ACWP subgroups

Changed carers ACWP subgroups

Out of home care Children’s Worlds

Family health ACWP subgroups


Family management HowRU




Number of close friends ACWP subgroups



ACWP subgroups

Conflict - closest friend ACWP subgroups


Missed school ACWP subgroups

Teacher support MDI

Parental Interest in school PIRLS/TIMSS




Pressure School - Pressure HBSC






Children’s Worlds

Community/ Neighbourhood




Health Factual/ correlates

Hunger HBSC

Breakfast HBSC


Subjective health Overall subjective health HBSC


Headache HBSC

Stomach-ache HBSC

Backache HBSC

Feeling low HBSC

Irritability HBSC



Dizziness HBSC


Factual/correlates Socio-economic status Deprivation ACWP subgroups

Self-demographics

Factual/correlates Language background PIRLS/TIMSS

Gender PISA


Disability Youth 2012 Student Health and Wellbeing Questionnaire -New Zealand

Disability difficulties Youth 2012 Student Health and Wellbeing Questionnaire -New Zealand

Aspirations HowRU

Puberty ACWP subgroups


Bullying Bullying Australian Covert Bullying Prevalence Study

Bullying - location ACWp subgroups


Bullying Initiator HowRU

Feeling good Overall wellbeing Children's World’s

Importance of domains for wellbeing ACWP subgroups

Cantril ladder HBSC

Optimism Optimism for the future ACWP subgroups

Closeness of relationships

Closeness/Proximity of Relationships ACWP subgroups



Note: ACWP subgroups include comparisons by gender, disability, grade, Aboriginal and Torres Strait Islander, language background, grade, socio-economic status.

The next step in the questionnaire development will be to analyse the data collected in the field trial.

Results of the analyses will be used to guide the finalisation of the questionnaire for the main survey,

which will be conducted in Term 3 2014. Details of these analyses will be provided in the Phase Three

Report of the ACWP scheduled for delivery at the end of June 2014.


Chapter 2. The Field Trial Questionnaire as Administered in the First Term of 2014

This chapter illustrates the content of the field trial survey questionnaire as it was administered online

in the ACWP field trial to sample of Year 4, 6, and 8 students in eleven schools in New South Wales

and Victoria in Term 1, 2014. The content is the result of the development work that occurred between

the original draft field trial questionnaire (see Attachment 1) in the light of the comments received

from the PSG and the research team's responses to the feed-back (see Attachment 2) as well as

information obtained from the cognitive interviews (see Attachment 3). The content is given in the

order in which questions were shown to respondents in the online field trial administration. For each

question, information is provided about the major domain or cross-cutting theme to which it belongs

and whether it is a correlate/factual or a sub-domain of that major domain. These details reflect the

questionnaire conceptualisation given in Table 1.1 above. In addition, policy aims and intended

analyses are briefly described for each question.

Gender

Are you a girl or a boy?

□ Girl

□ Boy

Major domain: Self demographics

Correlates/Factual: Gender

Year level: 4, 6 and 8

Source: PISA

Policy Aim/Analysis: Gender will be used as a control/background variable in analyses and as a major

reporting variable to examine gender differences.

ATSI

Are you of Aboriginal or Torres Strait Islander origin?

(If you are Aboriginal and Torres Strait Islander, tick both ‘Yes’ boxes.)

□ No

□ Yes, Aboriginal

□ Yes, Torres Strait Islander


Correlates/Factual: Aboriginal and Torres Strait Islander


Source: PISA

Policy Aim/Analysis: This question will allow us to identify one of the six sub-groups of interest for

this survey, namely Aboriginal and Torres Strait Islander students.


Language background

How often do you speak English at home? (YEAR 4)

□ I always or almost always speak English at home

□ I sometimes speak English and sometimes speak another language at home

□ I never speak English at home

How often do you speak English at home? (YEAR 6 and 8)

□ Always

□ Almost always

□ Sometimes

□ Never


Correlates/Factual: Language background

Year level: 4, 6 and 8 – note the different version for year 4

Source: PIRLS/TIMSS (Year 4); TIMSS (Year 6, 8)

Policy Aim/Analysis: These questions will allow us to identify one of the six sub-groups of interest for

this survey, namely Culturally and Linguistically Diverse students. We will be able to make comparisons

with PIRLS and TIMSS data for Year 4, and with TIMSS data for Year 8. It should be noted that the use

of different measures will prevent comparisons across all three year levels within the ACWP survey.

Aspirations

What is the highest level of education you would like to finish?

□ Year 10

□ Year 11

□ Year12

□ Trade Qualification (apprenticeship)

□ TAFE Certificate (or similar)

□ University


Correlates/Factual: Aspirations


Source: TIMSS (Year 8)

Policy Aim/Analysis: While aspirations can refer to education, relationships and jobs, it is felt that

educational aspirations are easier to capture. This might necessitate an open ended question which

will then require coding.

Cantril Ladder

Here is a picture of a ladder. The top of the ladder ‘10’ is the best possible life for you and the bottom ‘0’ is the worst possible life for you. In general, where on the ladder do you feel you stand at the moment?

Select the box next to the number that best describes where you stand.

□ 10 Best possible life

□ 9

□ 8

□ 7

□ 6

□ 5

□ 4

□ 3

□ 2

□ 1

□ 0 Worst possible life


Cross-cutting theme: Feeling good


Source: Cantril (1965); HBSC

Policy Aim/Analysis: This question is a validated and accepted measure of overall wellbeing /life

satisfaction for adults (Cantril, 1965), and has been used and validated with children in the HBSC. The

Cantril Ladder is highly correlated with other measures of wellbeing, such as quality of life, subjective

health and health complaints. This question can be treated as a scale outcome variable. It could also

be treated as it is in HBSC, where two indices are created – high life satisfaction (6-10) and low

satisfaction (0-5). This question is asked at the beginning of the survey before any domain-specific

questions in order to avoid the measure of overall wellbeing being influenced by responses to more

specific questions.

Overall wellbeing and optimism for the future

How much do you agree or disagree with each of these sentences:

Strongly disagree

Disagree Neither agree nor disagree

Agree Strongly

Agree Don't know

My life is going well □ □ □ □ □ □

Overall wellbeing

My life is just right □ □ □ □ □ □

I wish I had a different kind of life

□ □ □ □ □ □

I have a good life □ □ □ □ □ □

I have what I want in life □ □ □ □ □ □

I feel positive about my future

□ □ □ □ □ □ Optimism

Cross-cutting theme: Feeling good, Optimism


Source: Overall wellbeing: Children's Worlds - Student Life Satisfaction Scale (SLSS) by Scott Huebner,

Optimism for the future: New for ACWP

Policy Aim/Analysis: This item set is included as an overall measure of perceived well-being that can

be used as a key outcome variable. Levels of wellbeing in other major domains and subdomains would

be expected to be highly correlated with this scale (see Rees et al., 2010 and Main and Bradshaw, 2012

for examples). This question is asked at the beginning of the survey before any domain-specific

questions to avoid the general question to be influenced by responses to more specific questions and

the ‘sensitising’ due to having completed a 20-30 minute survey on the topic. This item set is taken

from Huebner's Students’ Life Satisfaction Scale (Huebner, 1991). It was adapted and reduced to the

five items - as used here - and tested by Rees et al (2010) who found it to be a measure of children's

subjective wellbeing with a high level of stability and test-retest validity.

An additional item, ‘Optimism for the future’ has been added to the ‘Overall wellbeing’ scale which is

intended to be analysed separately. This new item intends to measure an additional aspect of overall

wellbeing which is theorised to be important for children’s healthy development: optimism and

resilience for the future.

Subjective health

Would you say your health is ...

□ Excellent

□ Good

□ Fair

□ Poor

Major domain: Health


Subdomain: Subjective health


Source: HBSC

Policy Aim/Analysis: This question about overall health is asked at the start of the domain, in order

for it not to be affected by responses to more specific health questions. It can be used descriptively

and also as a correlate to various subsequent questions regarding health related behaviours. This

question has been used to explore relationships between self-rated health and SES at school and

national/ international levels. This question has been asked of differing populations (elderly/young

etc) (Idler and Benyamini, 1997; Elgar et al., 2013). Analyses by Elgar et al. (2013) have linked

self-perceived health, SES and family structure.

Importance of domains for wellbeing

Here is a set of shelves. Putting something on the top shelf means it is most important to you for having a good life. Putting something on the bottom shelf means that it is least important to you for having a good life. You can put things on the same shelf if they are equally important.

Where would you put the following on this set of shelves?

Family

Friends

School

Neighbourhood/community

Health

Money/things I have

Cross-cutting theme: Feeling good


Source: New for ACWP

Policy Aim/Analysis: We already know from the qualitative work that most children rank the domains

in a similar order, with family being most important. Nonetheless, we do not want to simply measure

the relative importance, we are interested in the absolute importance of each domain. We want to

get importance onto an interval scale that allows us to measure not only which domain is more

important, but how much more important it is than another one. This might help to give relative

weight to policy importance. For example, Year 4s might still find family three times as important as

school, whereas by Year 8 young people might rate family and school about equally important for their

well-being. Knowing how young people prioritise these domains will provide important knowledge on

the sources from which they construct their wellbeing. The hypothesis is that positive experience will

matter more in the case of higher ranked domains; and also that if things are going well in a high

ranked domain, they are more likely to be going well in a lower ranked domain. Qualitative work would

suggest that we are likely to find differences across “marginalised” and “mainstream” groups in how

domains are ranked.

Organisation of household(s) – Year 4

Used as a filter for Years 6 and 8 for the following question.

Some children usually sleep in the same home each night. Other children sometimes or often sleep in different homes. Please choose which of the following sentences best describes you

□ I always sleep in the same home

□ I usually sleep in the same home, but sometimes sleep in other places (for example a friend’s or a weekend house)

□ I regularly sleep in two homes with different adults


Organisation of the household(s) – Years 6 and 8

Filtered to either present ‘First home’ only, or ‘First and Second homes’ to respondents depending

upon their answer to the previous question.

This question is about the people you live with.

Please tick all of the people who live in your home(s).

First home Second home

Mother □ Mother □

Father □ Father □

Mother’s partner □ Mother’s partner □

Father’s partner □ Father’s partner □

Grandmother □ Grandmother □

Grandfather □ Grandfather □

Brothers and sisters □ Brothers and sisters □

Other children □ Other children □

Other adults □ Other adults □

Major domain: Family

Correlates/Factual: Organisation of household(s)

Year level: Year 4, 6 and 8. Note that there is a different version for Year 4s.

Source: Children's Worlds

Policy Aim/Analysis: This question provides information about who lives with children in their homes,

and if children live in more than one home. The Year 4 question will enable comparisons with

Children’s Worlds data and will provide information in terms of living in more than one home, but not

about who is in the family. However, the Year 4 question provides sufficient detail to ascertain

whether children live in a nuclear family or in an extended family context, and if children spend

considerable time in more than one home. In order to reduce the overall length of the survey, it was

decided to only obtain information about type of home from Year 4 children, and not about the

composition of the home for Year 4 children.

The Year 6 and 8 questions will enable comparisons with data from Children’s Worlds about who lives

with children in their homes, and if children live in more than one home.

The overarching aim of including these questions in the ACWP is to provide information about whether

there are adults and/or children around the respondent, rather than to provide accurate information

about the household composition of the respondent. Information from ‘Organisation of the

household’ for Years 4,6, and 8 can be combined with information from the ‘Closeness of relationships’

question, which asks children to drop people/pets who they consider close as close into their ‘me’

circle. It is the combination of the people with whom they live, that is,their household, and whether

they feel close to someone, that is the focus of this aspect of the ACWP. Questions about family in

international surveys in education are mainly used to obtain information about migration/language

background and size of the family, as a bigger family tends to means fewer resources per child.

Paid job

How many adults that you live with have a paid job?

First Home Second Home

□ None □ None

□ One □ One

□ Two □ Two

□ More than 2 □ More than 2

□ Don’t know □ Don’t know


Correlates/Factual: Family wealth/deprivation




Policy Aim/Analysis: This question will serve as one measure of SES, by having children indicate the

number of adults in their home(s) who have paid jobs, and will be presented after the ‘Organisation

of the household(s)’ indicator. The ‘Organisation of the household(s)’ indicator will serve as a filter to

display either ‘First home’ only or both ‘First and Second homes’ depending upon respondents’

answers. Along with if children live in more than one home, it is important to have information about

the number of adults with paid jobs in the second, or other home. This is important as single parent

households have fewer resources. It is important to know if there are possible resources in the second,

or other home that can be used to support the child. Also, this measure can be used to help identify

the ACWP sub-group of interest, 'Low-SES'. This is a key variable for identifying low income

households. UK analysis shows it is highly correlated with income poverty (see Adams et al., 2012).

Closeness/proximity of relationships

Here is a circle with you at the centre.

Here are some people or things you may know:

How close are these people to you? Drag them into the circle, as close to you as you feel they are. If you don't know a person or you don't feel close to them, just leave it where it is.

Mother

Father

Stepfather

Stepmother

Foster mother

Foster father

Sister

Brother

Uncle

Aunt

Grandmother

Grandfather

Other adult

Other child

Pet

Mother's partner

Father's partner

Cross-cutting theme: Closeness of relationships


Source: Samuelson, Thernlund and Ringstroem (1996); Sturgess, Dunn, Davies (2001)

Policy Aim/Analysis: From the qualitative work, it emerged that it is important that young people

have people or pets to whom they feel close. Also, in general, having and maintaining relationships is

important for well-being. This map enables the young people to provide a picture of not only about

who is important, but also how many people are important and how close they are to the young

person. Well-being is related to the number and degree of people that young people feel close. See

Samuelson, Thernlund and Ringstroem (1996) Sturgess, Dunn, Davies (2001). Sturgess et al (2001) use

this technique to ascertain the relationship between closeness to mothers, fathers and step-fathers

on the one hand, and behavioural difficulties and closeness to friends on the other. Our analysis will

extend Sturgess et al.'s analysis by focusing on the association between relationships within the family

and relationships outside - with friends, at school; and how these vary among the 'marginalised' and

'mainstream' groups. It will also extend studies that focus on relations with parents as an indicator of

'family functioning' and its relation to other domains of well-being. It is of policy relevance to examine

Me!


how different forms of relationship within the family are associated with broader well-being and

school engagement.

To be inclusive of different family contexts, children will be able to drag a person multiple times into

rings if they have multiple brothers for or more than one father, for example.

Family cohesion

How often in the past week have you spent time doing the following things with your family?

Not at all last week

Once or twice last week

Most days last week

Every day last week

Don’t know

Talking together □ □ □ □ □

Having fun together □ □ □ □ □

Learning together □ □ □ □ □


Subdomain: Togetherness –Family cohesion


Source: Children's Worlds (modified)

Policy Aim/Analysis: These items will form a scale of family cohesion and will enable comparisons

with data from Children's Worlds. The time reference in the question stem has been reiterated in the

response options by adding 'last week' . Analysis will look at the association between the ‘Family

cohesion’ scale and other aspects of engagement and wellbeing such as the ‘Worries about family

vulnerability and harm’ scale, school engagement, overall wellbeing and the closeness of relationships,

among others.

Vignette - Family management

Below you will find descriptions of three families. Read each of the descriptions of these families, then let us know to what extent you agree with the final statement.

YES! yes no NO!

Kiara's parents encourage her to take small risks. As she shows that she can be sensible they give her more freedom. Kiara always tells her parents the truth about what she is doing. She knows her parents will always help her if she is in danger. Kiara's parents help Kiara to keep safe.

□ □ □ □

Erin's parents do not let her take any risks. They ban most things she wants to do to keep her safe. Erin lies to her parents about what she is doing. Her parents often do not know where she really is. Erin cannot ask her parents for help if she is in danger. Erin's parents help Erin to keep safe.

□ □ □ □

Anna’s parents let Anna do whatever she likes. They say she has to learn about risks herself. They never know where Anna is or what she is doing. They are too busy to help Anna if she is in danger. Anna's parents help Anna to keep safe.

□ □ □ □


Correlates/Factual: Family management

Year level: Year 8- As corresponding scale on family management is for Year 8 students only.


Policy aim/Analysis: The Vignette –Family management is intended to correct for differences in

response style and will be used for this purpose in the analyses. Further details on vignettes are

provided in Attachment 1, Section 9 and Appendix A.


Family management

These questions ask you about your family. When we ask about your mother and father we want you to think about whom you live with most of the time. This includes step-parents, foster parents or guardians.

YES! yes no NO!

This doesn’t apply to me

My parents ask me if I’ve done my homework. □ □ □ □ □

My parents would know if I didn’t come home on time □ □ □ □ □

The rules in my family are clear. □ □ □ □ □

When I am not at home, one of my parents knows where I am and who I am with.

□ □ □ □ □

My parents want me to call if I’m going to be late getting home

□ □ □ □ □

My family has clear rules about alcohol and drug use. □ □ □ □ □

If you drank some alcohol (like beer, wine, spirits or pre-mixed drinks such as Bacardi Breezers or UDL’s) without your parents’ permission, would you be caught by your parents?

□ □ □ □ □

If you carried a weapon without your parents’ permission, would you be caught by your parents?

□ □ □ □ □

If you skipped or wagged school without your parents’ permission, would you be caught by your parents?

□ □ □ □ □


Subdomain: Togetherness –Family management

Year level: Year 8

Source: HowRU

Policy Aim/Analysis: These items will form a scale of family management and will enable comparisons

with data from the HowRU survey.

Worry - Vulnerability and harm

How much do you worry that someone close to you:

Not at all A little Somewhat A lot

Will get sick? □ □ □ □

Will do something that scares me? □ □ □ □

Will get hurt? □ □ □ □

Will get arrested? □ □ □ □

Will be fighting? □ □ □ □

Won’t have enough money? □ □ □ □

Won’t have a place to live? □ □ □ □

Won’t have enough to eat? □ □ □ □

Will move away? □ □ □ □

Will hurt somebody? □ □ □ □

Will tell a lie? □ □ □ □


Subdomain: Worry –Vulnerability and Harm


Source: Adapted from Family Worries Scale (Graham-Bermann, 1993)

Policy Aim/Analysis: This question will ask children to think about 'someone in your family', instead

of specifically asking about mother/father/someone else as was presented in the original Family

Worries Scale. It is thought more important for policymakers to know that children worry about

vulnerable or harmful people in their families, but it is not as important to know if mothers are harmful

and dads are vulnerable, for example. There are questions that control for 'who you live with', and for

'people that they are close to'.


As regards out of home care children, the qualitative work shows that these children refer to their

family, not their foster carers. While we concede that it would be important to know if children worry

about vulnerable/harmful people in their foster families, this is not an evaluation of foster care

services and the number of children who are in foster homes and who identify as such is likely to be

very small in the main national survey.

In line with the PSG recommendations, we have reduced items from the vulnerability scale (Will be

worried a lot; Will lose a job; Will need help; Will be mad or angry; Will feel sad; Will feel afraid; Won’t

keep bad things from happening; Will die; Won’t listen) that are more abstract. The remaining items

in the vulnerability scale express more concrete vulnerabilities such as not having food or money.

These questions speak directly to concerns expressed by young people in the Phase 1 research. The

Family Worries Scale, on which these questions are based, is not widely used, and therefore, there are

no possibilities for direct comparisons. However, these questions, which do not deal with family

functioning or cohesiveness, will allow the examination of the prioritisation of family, and worries

about family, that are associated with material deprivation, school engagement and other measures

of wellbeing. One hypothesis to be examined is that worries about the family are important in terms

of their association with other aspects of engagement and wellbeing such as perceived family

togetherness or cohesion.

Family health

Is there anyone in your family who is seriously affected by….

You can select more than one.

□ Disability or long term illness

□ Depression or mental illness

□ Using alcohol or other drugs

□ None of these


Correlates/Factual: Family health


Source: Adapted from SHWB-NZ

Policy Aim/Analysis: This question asks whether any family member has a serious illness or issues

that may highlight the potential need for the student to care for these family members. This is likely

to be a key categorical question. There is a large literature on the relationship between parental

mental illness and drug/alcohol use and child wellbeing (and school engagement). In addition, many

marginalised young people at Phase 1 reported living with family members who were ill, or disabled,

and talked about this as influencing the way they lead their lives.

Disability

Have you had a disability for a long time (more than 6 months) (such as, hearing difficulties, visual difficulties, using a wheelchair, mental illness, learning difficulties)?

Yes □

No □

I don’t know □

Does your disability make it hard for you, or stop you... (You can select more than one.)

Doing everyday activities that other children your age can usually do (such as getting ready for school; eating, washing yourself, getting dressed or going to the toilet)

□

Talking to people, understanding what other people say or hanging out with friends □

Doing any other activity that children your age can usually do (such as sports and hobbies like football, cricket, swimming, playing games or playing a musical instrument)

□

No difficulty with any of these □



Correlates/Factual: Disability


Source: Youth 2012 Student Health and Wellbeing Questionnaire -New Zealand (question stem and

items slightly adapted for ease of understanding)

Policy Aim/Analysis: This question will identify one of six sub-groups of interest for this survey,

namely children with a disability. This item will indicate if a child self-reports as having a disability,

but will not indicate if a child has a health problem (e.g. diabetes, asthma) or a mental health issue

(depression). This item will indicate the perceived difficulties encountered, but will not provide

information about accessed or needed services (e.g. physical therapy, medication). According to the

disability expert, Pammi Raghavendra, it is possible that some young people may have difficulties

understanding the language used in the original question stem and items from the NZ Health and

Wellbeing Questionnaire. Therefore they have been revised for ease of understanding.

Socio-economic status: Deprivation

Here is a list of items that some young people of your age have. Please tell us whether you have each item on the list or whether you'd like to have it.

I have this

I don't have this but would like it

I don't have this and I don't want or need it

An iPod or other personal music player □ □ □

Some money that you can save each month, either in a bank or at home

□ □ □

The right kind of clothes to fit in with other people your age

□ □ □

My family has enough money for me to go on a school camp

□ □ □

Your own mobile phone □ □ □

<If the answer is ‘1’ to the above question, respondents will be further asked:>

Enough credit on my mobile to phone or text friends □ □ □

Major domain: Money and Material Wellbeing

Correlates/Factual: Socio-economic status: Deprivation


Source: Main, G. and Bradshaw, J. (2012) A child material deprivation index, Child Indicators Research

(items 1,2,3,5)

New for ACWP (4,6).

Policy Aim/Analysis: These items will be used to form a scale of Deprivation, in order to measure SES.

This can also be used (with the item about parent jobs) to form an indicator of SES and identify children

from 'Low-SES' backgrounds, a sub-population of interest for this study. In addition to identifying the

'economically deprived' group, this is a direct measure of child material deprivation. It is an important

measure in its own right, as an indicator of child deprivation. It is also important as an explanatory

variable for other aspects of wellbeing, for example, its relationship to engagement at school, and to

subjective wellbeing (for the latter, see Main and Bradshaw, 2012).

Family Possessions – Books

About how many books are there in your home? (Do not count magazines, newspapers or your school books.)

□ None or very few (0 - 10 books)

□ Enough to fill one shelf (11 - 25 books)

□ Enough to fill one bookcase (26 - 100 books)

□ Enough to fill two bookcases (101 - 200 books)

□ Enough to fill three or more bookcases (more than 200)



Correlates/Factual: Family possessions

Year level: 4,6 and 8

Source: PIRLS (Year 4), TIMSS (Year 8)

Policy Aim/Analysis: This question is a proxy measure of socio-economic status, and can be combined

with other socio-economic indicators, namely questions about ‘Paid job’, ‘Material Deprivation’ and

HBSC questions about social inequality to identify children from economically disadvantaged

backgrounds. These questions are used in large-scale assessments TIMSS and PIRLS, and will enable

comparisons with Australian and international data from these surveys. Data analyses from these

surveys show that there is a strong correlation between the number of books in the home and parental

education and income levels.

Car

Does your family own a car, van or truck?

No □

Yes, one □

Yes, two or more □



Year level: 6 and 8

Source: HBSC

Policy Aim/Analysis: This is the first of a set of four questions about social inequality that are asked

in the HBSC. Having these questions in the survey will enable comparative analyses of HBSC data that

control for social inequality. If analyses of the field trial data show insufficient variance on these

response options, these questions will be dropped from the main survey.

SES Deprivation – Petrol

My family has enough money to put petrol in the car, van or truck when needed.

We have this □

We don't have this but I would like it □

We don't have this and I don't want or need it □


Correlates/Factual: SES Deprivation



Policy Aim/Analysis: This item is new for the ACWP survey, and is planned to be analysed with items

from the ‘SES Deprivation’ scale. This item arose from the qualitative fieldwork, that having access to

not only a car for transportation, but to petrol when needed, may more appropriately discriminate

between levels of material deprivation in an Australian context.

Own bedroom

Do you have your own bedroom for yourself?

No □

Yes □




Year level: 6 and 8

Source: HBSC

Policy Aim/Analysis: This is the second of a set of four questions about social inequality that are asked




Dishwasher

Does your family have a dishwasher at home?

No □

Yes □



Year level: 6 and 8

Source: HBSC

Policy Aim/Analysis: This is a new question that will be included in the HBSC Family Affluence scale.

Having these questions in the survey will enable comparative analyses of HBSC data that control for

family affluence/social inequality. If analyses of the field trial data show insufficient variance on these


Holiday

During the past 12 months, how many times did you travel away on holiday with your family?

Not at all □

Once □

Twice □

More than twice □



Year level: 6 and 8

Source: HBSC

Policy Aim/Analysis: This is the third of a set of four questions about social inequality that are asked



response options, these questions will be dropped from the main survey. It should be noted that the

SES Deprivation scale contained an item about vacations. That item was removed from that scale to

avoid duplication. The reason for preferring this question is that it will provide comparative

information with the HBSC whereas the SES (Deprivation) scale provides ACWP internal comparisons

only.

Foreign Holiday

How many times did your family travel out of Australia for a holiday last year?

Not at all □

Once □

Twice □

More than twice □




Year level: 6 and 8

Source: HBSC



family affluence/social inequality.

It should be noted that the SES Deprivation scale contained an item about vacations. That item was

removed from that scale to avoid duplication. The reason for preferring this question is that it will

provide comparative information with the HBSC whereas the SES (Deprivation) scale provides ACWP

internal comparisons only. If analyses of the field trial data show insufficient variance on these


Computers

How many computers does your family own? (including laptops and tablets, NOT including game consoles and smartphones)?

None □

One □

Two □

More than two □



Year level: 6 and 8

Source: HBSC

Policy Aim/Analysis: This is the fourth of a set of four questions about social inequality that are asked




Bathrooms

How many bathrooms (room with a bath/shower or both) are in your home?

None □

One □

Two □

More than two □



Year level: 6 and 8

Source: HBSC



family affluence/social inequality. If analyses of the field trial data show insufficient variance on these



Number of close friends

How many close friends do you have?

□ None

□ One

□ Two

□ Three

□ Four

□ Five or more

Major domain: Friends

Correlates/Factual: Number of close friends

Year level: Year 4, 6 and 8

Source: Adapted from HBSC

Policy Aim/Analysis: This question has been adapted from HBSC which aims to 'determine the

proportion of adolescents that have friends of their own, the other or both genders.' We suggest to

collapse the gender categories, and ask students to indicate the number of close friends that they

have (regardless of gender) as we do not understand the rationale for separating number of friends

by gender, and the intended analysis for well-being as it seems to us that the context (having close

friends or not) is more important than gender. As in the previous version, we can know explicitly that

children have up to four friends (two males, two females), we suggest revising the response categories

to include 'four' and 'five or more'.


For the following questions, please think about your closest friend.

1 Never or

hardly ever 2 3 4

5 Always or

almost always I spend fun time with this person □ □ □ □ □

I share private thoughts and feelings with this person □ □ □ □ □

I depend on this person for help, advice, and support □ □ □ □ □

This person sticks up for me □ □ □ □ □


Subdomain: Support and conflict


Source: Waldrip (2008)

Policy Aim/Analysis: These four items will be combined to form a scale measuring the degree of

closeness or support the student feels they receive from their close friend. Friend's support or lack

thereof relate to other aspects of wellbeing, including bullying. Since we expect bullying to be linked

to marginalisation, analyses are intended to examine any relationship between friendship difficulties

and bullying.

Conflict - closest friend

Still thinking about the same closest friend: 1 Never or

hardly ever 2 3 4

5 Always or

almost always I get into fights with my friend □ □ □ □ □

My friend and I make up after a fight □ □ □ □ □

My friend bugs me or annoys me even though I ask him/her not to

□ □ □ □ □

My friend and I argue □ □ □ □ □

My friend and I disagree about many things □ □ □ □ □



Subdomain: Support and conflict


Source: Bukowski (1994)

Policy Aim/Analysis: These items have been drawn from Bukowski (1994) and have been slightly

adapted. They will be combined to form a scale that measures degree of conflict that young people

have with their closest friend. In line with recommendations from the PSG for conflict items to

measure frequency instead of degree of truthfulness, we have used the same response scale as above

in 'Degree of closeness/support'. We have revised the response categories to make them more

consistent (e.g. Never - Always). This response scale seems to be more appropriate than measuring

frequency of conflict with explicit time frames (e.g. day/week/month).

This question set on conflict with friends, together with the previous question set on support from

friends is intended to examine the quality of relationships with friends rather than frequency or type

and means of contact. As it is expected that bullying to be linked to marginalisation, it will be important

to examine whether any relationship between friendship difficulties and bullying.

General satisfaction/Intrinsic motivation

My school is a place where… Strongly Disagree Disagree Agree Strongly Agree

I feel happy. □ □ □ □

I really like to go to each day. □ □ □ □

I find that learning is a lot of fun. □ □ □ □

I feel safe and secure. □ □ □ □

I like learning. □ □ □ □

I get enjoyment from being there. □ □ □ □

The work we do is interesting. □ □ □ □

I like to ask questions in class. □ □ □ □

I like to do extra work. □ □ □ □

I enjoy what I do in class. □ □ □ □

I always try to do my best. □ □ □ □

I get excited about the work we do. □ □ □ □

Major domain: School

Subdomain: Enjoyment


Source: LSAC

Policy Aim/Analysis: These items form the sub domain enjoyment and have been drawn from LSAC.

The items will be combined to form two scales, one measuring school enjoyment with items 1 to 6

comprise the Positive Affect/General Satisfaction subscale, while the remaining items (7-12) comprise

the Intrinsic Motivation subscale.

While these two scales have been shown to be highly related to school performance, it has been

suggested that this long list of items could perhaps be replaced by a single question from the HBSC -

'How do you feel about school at present? (4 scale response - 'I like it a lot -- I don't like it at all').

Torsheim and Wold (2001) use this question to examine the relationship between school stress and

somatic problems in the HBSC. Further consideration will be given to this issue.

Teacher support

How true is each statement for you? At my school, there is a teacher or another adult …

Not at all true

A little true

Pretty much true

Very much true

who really cares about me □ □ □ □

who believes that I will be a success. □ □ □ □

who listens to me when I have something to say. □ □ □ □



Correlates/Factual: Teacher support


Source: MDI

Policy Aim/Analysis: The three items are intended to be combined to form a scale measuring teacher

support. Feeling supported by a teacher who is another adult outside the family has been shown to

be positively related to wellbeing.

Success at school

In your opinion, what does your class teacher(s) think about your school performance compared to your classmates?

Very good □

Good □

Average □

Below average □


Subdomain: Success


Source: HBSC

Policy Aim/Analysis: This question has been included to gain an indication of how the students

perceive their success at school. It is anticipated that analysis of this question could be done in

conjunction with the school's most recent reading NAPLAN score. Teacher rated success would have

four levels (very good, good, average and below average) while NAPLAN school performance has 5

levels (substantially above, above, average, below, substantially below). Together, this will result in

each student being assigned a rank from 1 to 20.

Parental interest in school

How often do the following things happen?

Every day or almost every day

Once or twice a week

Once or twice a month

Never or almost never

My parents ask me what I am learning in school.

□ □ □ □

My parents make sure that I set aside time for my homework

□ □ □ □

And often does the following happen?

At least every week

Once or twice a term

Once or twice a year


My parents talk to my teacher(s) □ □ □ □


Correlates/Factual: Parental interest


Source: PIRLS/TIMSS (items 1, 2), New for ACWP (item 3)

Policy Aim/Analysis: There was a note in the PSG meeting to add questions about parental

involvement at school. These are taken from PIRLS 2011 and have also been used in TIMSS 2011 at

both Grade 4 and 8 level - so extensive comparisons are possible. The last item, 'my parents talk to my

teachers', has been added for the ACWP to reflect the importance of home-school interaction for the

wellbeing of young people. Therefore, comparisons for Years 4 and 8 with PIRLS/TIMSS will only be

possible with the first two items.


School Pressure

How pressured do you feel by the schoolwork you have to do?

Not at all □

A little □

Some □

A lot □


Subdomain: Pressure

Year level: 6 and 8

Source: HBSC

Policy Aim/Analysis: This question is part of the school domain and is designed to measure the degree

to which students experience school work pressure.

Missed school

Last term, how many times have you missed school?

Never Hardly ever About once a week Most days Every day Don’t know

□ □ □ □ □ □

Domain: School

Correlates/Factual: Missed school


Source: Adapted from Children's Worlds

Policy Aim/Analysis: This question has been included to ascertain the extent to which the student is

absent from school. In line with findings of the qualitative phase, skipping school is considered an

indicator of the different priorities assigned to school by different disadvantaged groups. As with

Illness, there was a comment from the PSG about asking more specifically about different reasons for

truancy; however as a correlate of school performance, the reason does not matter, only the amount

of time student is absent/truant. Therefore, previous items which asked separately about absence due

to illness, and absence due to truancy, have been reduced to one question.


How often do you usually spend time doing the following activities when you are not at school?

Hardly ever or never

Less than once a week



Don't know

Taking lessons (like music, sports, dancing, languages)

□ □ □ □ □

Hanging out with friends □ □ □ □ □

Helping with housework □ □ □ □ □

Doing homework □ □ □ □ □

Watching TV or videos, or listening to music

□ □ □ □ □

Playing sports on a team □ □ □ □ □

Playing sports or doing exercise (but not on a team)

□ □ □ □ □

On the computer □ □ □ □ □

Spending time just being by myself □ □ □ □ □

Taking care of brothers or sisters or other family members

□ □ □ □ □


Subdomain: Outside school activities




Policy Aim/Analysis: Young people identified activities, fitness, sport when discussing the health

domain in the focus groups. Consideration could be given to dropping three items, namely hanging

out with friends, doing homework, and taking care of brothers and sisters as they are covered, albeit

slightly differently in the friends, school and family domains.

Bullying

These next questions ask about bullying. Bullying is when people tease, threaten, spread rumours about, hit, shove, or hurt other people over and over again. It is not bullying when 2 people of about the same strength or power argue or fight or tease each other in a friendly way.

THIS TERM how often did these things happen to you?

This did not happen to

me this term

Once or twice this

term

Every few weeks this

term

About once a

week this term

Several times a week or more this

term

Students deliberately ignored or left me out of a group to hurt me

□ □ □ □ □

I was teased in nasty ways □ □ □ □ □

I had a student tell lies about me behind my back, to make other students not like me

□ □ □ □ □

I’ve been made to feel afraid I would get hurt

□ □ □ □ □

I had secrets told about me to others behind my back, to hurt me

□ □ □ □ □

A group decided to hurt me by ganging up on me

□ □ □ □ □

Bullying – location

<If the answer is "2,3,4 or 5" to any of preceding items, respondents will be further asked:>

Where did you experience bullying?

□ Mainly at school

□ Mainly outside of school

□ About the same at school and outside of school

Cross-cutting theme: Bullying


Source: Bullying: Australian Covert Bullying Prevalence Study, Bullying –location: New for ACWP

Policy Aim/Analysis: Bullying was identified in the focus groups as an issue that affected well-being,

particularly for children with a disability. This question is taken from the Australian Covert Bullying

Prevalence Study and will enable the comparison with Australian cohorts of children. The follow-up

question about the location of bullying was newly developed to capture whether the bullying occurs

at school, where the school has more of an influence on stopping this behaviour than elsewhere.

Bullying by friend

Has somebody who you think is your friend bullied you?

□ Yes

□ No




Source: (Mishna, 2008 question)

Policy Aim/Analysis: An aspect of bullying that emerged in the focus groups was being bullied by a

friend. This question is designed to obtain descriptive information on the extent of this occurring.

Bullying - Initiator

Have you taken part in bullying another child in the last four weeks?

□ Yes

□ No



Source: HowRU

Policy Aim/Analysis: As focus groups showed that bullying does not only occur in the school context,

this question has been adapted from the HowRU survey but will still allow descriptive information to

be collated. In combination with the previous 2 questions, the survey is designed to have a satisfactory

coverage of bullying.

Access to resources

Strongly disagree

Disagree Neither disagree

or agree Agree

Strongly agree

There are places for me to go in my area □ □ □ □ □

There is nothing to do in my area □ □ □ □ □

There are lots of fun things to do where I live

□ □ □ □ □

Major domain: Community/Neighbourhood

Subdomain: Access to resources


Source: Children’s Society

Policy Aim/Analysis: These three items are used to get an indication of access to resources (or lack

thereof) in the area in which the child lives. Lack of such resources is considered to be linked negatively

to overall well-being. Policy relevance is about provision of adequate facilities in the community. Also,

some age differences are expected as while there is often ‘something to do’ for younger children (e.g.

playgrounds) there is ‘less to do’ for older children.

Safety

Strongly disagree


or agree Agree

Strongly agree

I feel safe when I am out in my local area during the day

□ □ □ □ □

I feel safe when I am out in my local area at night

□ □ □ □ □

I have plenty of freedom in the area I live in

□ □ □ □ □

Major domain: Community/Neighbourhood

Subdomain: Safety


Source: Children’s Society

Policy Aim/Analysis: These three items are used to get an indication of feeling safe in the area in

which the child lives.


A note about the health questions:

Health was seen as relatively important for well-being so receives relatively greater coverage in the

questionnaire. Whenever possible, questions have been drawn from the HBSC to enable international

comparisons. The collective element of health is covered in some questions in the family domain that

ask about worry about health of a family member or having to care for a family member with a

disability or illness.

Hunger

Some young people go to school or bed hungry because there is not enough food at home. How often does this happen to you? (Please tick one box only.)

Always □

Often □

Sometimes □

Never □


Correlates/Factual: Hunger


Source: Adapted from HBSC

Policy Aim/Analysis: A recent report has highlighted the increase in children in Australia not getting

regular meals. Hunger is an additional aspect of deprivation which is measured in the family domain.

While this item has been dropped from 2013-14 HBSC core questionnaire, it may still be an important

indicator of subjective deprivation. Additionally, it is considered to be associated with low-wellbeing

so we should correlate it with the overall well-being scale.

Breakfast

How often do you usually have breakfast (more than a glass of milk or fruit juice)?

Weekdays (Monday to Friday)

I never have breakfast during the week □

One day □

Two days □

Three days □

Four days □

Five days □


Correlates/Factual: Breakfast


Source: HBSC

Policy Aim/Analysis: This question is specifically about regular breakfast, a question often used in

surveys of school achievement as a good breakfast is the necessary basis for the physical requirements

of a school day. Additionally, it is argued that skipping breakfast is associated with other ‘health

compromising’ behaviours such as substance abuse and reduces cognitive functioning. Analysis of

Field Trial data may correlate this question with ‘Smoking and Drinking’ items in the year 8 survey.

Consideration could be given to dropping this more specific question given the item about ‘hunger’

above.


Smoking & Drinking

On how many occasions (if any) have you done the following in the last 30 days?

Never 1-2 times

3-5 times

6-9 times

10-19 times

20-39 times

40 or more

Smoked cigarettes □ □ □ □ □ □ □

Drunk alcohol □ □ □ □ □ □ □

Been drunk □ □ □ □ □ □ □


Correlates/Factual: Smoking and drinking

Year level: 6 and 8

Source: HBSC

Policy Aim/Analysis: Smoking and drinking, particularly during adolescence has been shown to have

negative impact on outcomes in later life. Descriptive analyses and international comparisons to HBSC

(Year 6 and 8) will be possible. This is a key health indicator.


In the last 6 months: how often have you had the following...?

About every day

More than once a week

About every week

About every month

Rarely or never

Headache □ □ □ □ □

Stomach-ache □ □ □ □ □

Backache □ □ □ □ □

Feeling low □ □ □ □ □

Irritability or bad temper □ □ □ □ □

Feeling nervous □ □ □ □ □

Difficulties in getting to sleep □ □ □ □ □

Feeling dizzy □ □ □ □ □


Subdomain: Mental and physical health


Source: HBSC

Policy Aim/Analysis: This question has been drawn from the HBSC where it is used as a non-clinical

measure of mental and physical health. The HBSC symptom checklist asks respondents about

subjective health complaints that are intended to measure two facets, namely psychological well-

being (d, e, f) and somatic well-being (a, b, c, g, h)of an underlying trait of psychosomatic complaints.

HBSC reports that the scale is flexible and that analyses are meaningful at single- item and at sum

score level. Qualitative semi-structured interviews with early adolescents revealed that adolescents

perceive the symptoms to be aversive physical and psychological states that interfere with daily

functional ability and well-being. Moderate to high correlations have been found between the HBSC

symptom checklist and scales measuring psychological well-being (r=-.47), depressive moods (r=.53)

and physical wellbeing (r=.42) in data from the Kidscreen study in 12 European countries.

These questions have been used to examine the relationship between physical/mental health and

socio-economic status. They appear to be well tested (see Elgar et al, 2013, Haugland et al, 2001 and

Ravens-Sieberer et al, 2008). The symptoms checklist does probably not provide sufficient details of

physical or mental illnesses that could be used by health services. However, as a high-level measure

of physical and mental health to be included in analyses of the ACWP survey, the checklist seems to

be appropriate.


Puberty

The next questions are about changes that may be happening to your body. These changes normally happen to different young people at different ages. If you do not understand a question or do not know the answer, just mark, ‘I don't know’.

Not yet started

Barely started

Definitely started

Seems complete

I don't know

Would you say that your growth in height.. □ □ □ □ □

And how about the growth of your body hair? (‘Body hair’ means hair any place other than your head, such as under your arms. Would you say that your body hair growth....

□ □ □ □ □

Have you noticed any skin changes, especially pimples?

□ □ □ □ □

(For girls only):

Have you noticed that your breasts have begun to grow?

(For boys only): □ □ □ □ □

Have you noticed a deepening of your voice? □ □ □ □ □

Have you begun to grow hair on your face? □ □ □ □ □

(For girls only): Yes No I don't know

Have you begun to menstruate? □ □ □


Correlates/Factual: Puberty

Year level: 6 and 8

Source: Carscadon (1993) linked to Tanner scale

Policy Aim/Analysis: It is theorised that a wide range of wellbeing and self-perception issues are

linked to onset of puberty, rather than to age as such. Thus when we are talking about transitions then

we might expect that perceptions of relationships, safety etc might change with pubertal

development. It is expected to find differences among the different age groups on a range of issues.

Identification of pubertal development could act as an important control variable in analysis of the

relationship between external ‘facts’ and internal ‘perceptions’.

Out of Home Care

Which of the following best describes the home you live in most of the time?

I live with members of my family □

I live in a foster home □

I live in residential care or a family group home □

I live in another type of home □


Correlates/Factual: Out of home care



Policy Aim/Analysis: This question will allow us to identify a sub-population of interest, namely

children in out of home care. Feedback from the Australian Bureau of Statistics suggested that

Australian children in out of home care will not be familiar with the term, 'I live in a children's home'.

Input from policy agreed that the terms 'residential care' and 'family group home' would be familiar

to Australian children in out of home care. It was also suggested to collapse both: 'I live in a foster

home' and 'I live in residential care or a family group home' to 'I live in out of home care'. This option


will be considered after analysis of the Field Trial data. If both responses are infrequently selected,

then it may be considered to combine items two and three.

Changed carers

Are you living with the same parents or carers that you lived with one year ago?

Yes □

No □


Correlates/Factual: Change


Source: Adapted from Children's Society

Policy Aim/Analysis: This item aims to ascertain if the student has had a major change to their living

situation over the past year which may relate to their wellbeing. Again, this captures instability, which

is closely related to material deprivation (see Redmond and Skattebol, 2013).

Changed house/school

In the past year, …

No Yes, once Yes, more than once

have you moved house? □ □ □

have you changed schools? □ □ □


Correlates/Factual: Changed house/school


Source: Adapted from Children's Worlds

Policy Aim/Analysis: Qualitative work has shown that student transience/mobility can be related to

well-being. The question was adapted from the Children's Worlds where it asks only about whether

or not a change has occurred. However, it is expected that the frequency of change would have a

stronger relationship with wellbeing, rather than knowing whether or not a student has moved. The

frequency response scale and the time frame of ‘the past year’ take into account that in South

Australia, for example, most Year 8 students will have had one move as transition from primary to

secondary school occurs between Year 7 and Year 8. However more than one move of house or school

is suggested to be negatively related to other areas of wellbeing.


Attachment 1: The Original Draft Field Trial Questionnaire as Presented to the ACWP Project Steering Group in September 2013 of the Draft Field Trial Questionnaire

This attachment contains the draft Phase 2 Report as it was presented to the PSG meeting in Sydney

on 24 September 2013.

Table A1.1 provides information about the questionnaire conceptualisation at that time and guides

the structure of this attachment.

Table A1.1: Questionnaire conceptualisation of the original draft field trial questionnaire

Major domains Correlates/ Factual Subdomains Cross-cutting themes

1.1. Family

1.2. Size and composition of family Care for family member (sick, disabled, younger) Organisation of household(s) Out of home care

1.3. Togetherness: - Family cohesion - Family conflict - Attachment to mother - Attachment to father - Opportunities for pro-social involvement - Family management Worry: - Vulnerability - Harmful

8. Bullying Guidance and rules Learning Safety Stress Feeling good

2.1. Friends

2.2. Number and quality of friends

2.3. Contact with friends Support and conflict

3.1. School

3.2. Absence Success at school

3.3 Enjoyment Pressure Relationships


4.2. Deprivation

4.3. Access to resources Safety

5.1. Health

5.2. Oral health Hunger Food and drink Breakfast Physical fighting Smoking and drinking

5.3. Subjective health and body image Outside school activities


6.2. Socio-economic status


7.1. Self demographics

7.2 Language background Gender Aboriginal and Torres Strait Islander Disability Aspirations

7.3 Not applicable


It should be noted that the content had not been finalised at that stage. The draft report stated that

final content would depend on various further considerations, including results of cognitive interviews

and taking into account the flow and visualisation in the online implementation as well as input by the

PSG and other experts.

The question order also had not been finalised. It was stated that, for example, while many of the

factual/correlates questions could be combined in one demographics section, others might be

required for branching within the questionnaire (e.g. Disability).

1.1 Family

Each group without exception consistently ranked ‘family’ as the most important domain although

definitions and perceptions of what family meant differed between individuals and groups. While the

nuclear family was the most common definition, several children included grandparents as family

members. Although we did not ask specifically about whom the young people lived with, it was clear

that not all people who they saw as ‘family’ lived in the same household. Moreover, children in the

Aboriginal and Torres Strait Islander and culturally and linguistically diverse groups (and some other

children) tended to refer to family in terms of an extended network of parents, siblings, grandparents,

aunties and uncles, and cousins. Children living in out of home care did not refer to foster carers as

family, but referred instead to biological parents and siblings. Many children across the different

groups nominated pets as being family members (or friends), and this seemed to be particularly

relevant to children with a disability and to children in out of home care. Most children talked about

the relational aspects of family life. Brothers and sisters were discussed in this context as well as

parents (and in the case of Aboriginal and Torres Strait Islander and culturally and linguistically diverse

groups, other family members too). Positive aspects (which predominated) included love, support and

fun. Negative aspects included fighting (between parents, or between siblings and parents), and

bullying.

Reciprocal care between adult and child was also a feature of children’s perception of family in several

of the groups (it was especially marked among young people in the Aboriginal and Torres Strait

Islander and disability groups) – that is, young people often saw their roles in terms of providing

support as well as receiving it. Many young people moreover saw provision of help and support to

family members as both enjoyable and integral to their wellbeing, as evidenced by one young person

who spent time with his brother who had a disability.

1.2 Family correlates

Size and composition of family

This question is about the people in your family. When you think about your family, who do you think about? (Please tick all that apply.)

Mother □

Father □

Mother’s partner □

Father’s partner □

Grandmother □

Grandfather □

Brothers and sisters □

Step-brothers and step-sisters □

Aunties and uncles □

Cousins □

Other adults □

Pets □


This question is part of the family domain and is designed to allow young people to create their own

definition of family. It is a newly developed item specifically for the ACWP. It also provides information

on the size and composition of their family. It is anticipated that this question will be asked of year 4,

6 and 8 and will take on some type of interactive format type (e.g. drag and drop). See below for some

examples.

Here are some people and things that you may know.

How close are these people to you?

Put the pictures in the circles. If you don’t know a person or thing just leave it where it

Nana Neighbour

Sister Teacher

Pets


Cousin

Brother

Auntie

Below is a picture of a family.

Who is part of your family?

Cousin Brother

Auntie

Now circle the person who is most important to you.



Care for family member (sick, disabled, younger)

This question is about the people in your family.

Is there anyone in your family who is seriously affected by…. (Please tick all that apply.)

Disability or long term illness □

Depression or mental illness □

Using alcohol or other drugs □

None of these □

This question is part of the family domain and is included as a correlate of wellbeing. It asks whether

any family member has a serious illness or issues that may highlight the potential need for the student

to care of these family members. The question was adapted from the SHWB-NZ and can be asked of

all cohorts. Due to this adaption, comparisons with the SHWB-NZ will not be possible.

Organisation of household(s)

The following three questions are part of the family domain and ask about the young person's home

situation which is considered as correlate of wellbeing. Each question has a similar purpose and has

been drawn from the Children’s Worlds survey (international data from 7-13 year olds) which will

enable international comparisons with appropriate age groups.

The following two questions seek information about the organisation of the household. For the field

trial, the intention is to include the first version with young people in year 4 and the second version

with young people in years 6 and 8.

Your home and the people you live with

Some children usually sleep in the same home each night. Other children sometimes or often sleep in different homes. Please choose which of the following sentences best describes you

(Please tick only one box.)

I always sleep in the same home □

I usually sleep in the same home, but sometimes sleep in other places (for example a friend’s or a weekend house)

□

I regularly sleep in two homes with different adults □

This question is about the people you live with

(Please tick all of the people who live in your home(s).)

If you always live in the same home, please just fill in Column A.

If you live regularly in more than one homes with different adults, please fill in Columns A and B.

Column A: First home you live regularly Column B: Another home / Another place you live regularly










The next question seeks information as to whether or not the child is in out of home care.

Which of the following best describes the home you live in most of the time?

(Please tick one box only.)



I live in a children’s home □



This question seeks information about possible major relocation in the student’s life as a correlate of

wellbeing. The question has been drawn from Children’s Worlds and comparisons with that data set

could be made. It is anticipated that this question will be asked of year 4, 6 and 8.

In the past year, …

(Please tick one box in each row.) Yes No

have you moved house? □ □

have you changed local area? □ □

have you changed schools? □ □

have you lived in another country for over a month? □ □

This question aims to ascertain if the young person has had a major change to their living situation

over the past year which may relate to their wellbeing. It is part of the family domain and has also

been drawn from Children’s Worlds of which data could be compared. All three target groups could

be asked this question.


Yes □

No □

This question seeks information for one component of a composite measure of family wealth or

deprivation. It has been drawn from Children’s Worlds and could be asked at all three grade levels and

comparisons could be made with that data set.

How many adults that you live with have a paid job? (Please tick one box only.)

None □

One □

Two □

More than 2 □

Don’t know □

1.3 Family subdomains

Togetherness - Family cohesion

This is part of the family domain; subdomain togetherness. The items have been selected from

Children’s Worlds and could be combined to form a scale of family cohesion. The question will be

asked of all grade levels in the ACWP survey to enable comparisons with the international sample of

7-13 year olds of the Children’s Worlds data.


(Please tick one box in each row.)

Not at all Once or twice Most days Every day Don’t know

Talking together □ □ □ □ □ Having fun together □ □ □ □ □ Learning together □ □ □ □ □

Togetherness - Family conflict, attachment to mother and father, opportunities for pro-social involvement, family management

This set of 21 items is included to obtain information on different aspects of family life. The items have

been taken from the HowRU? survey and, in line with that survey, are expected to form the following

sub-scales:

Family conflict: Items i, j, k

Attachment to mother: Items a, b, c


Attachment to father: Items, d, e, f

Opportunities for pro-social involvement: Items g, h, l

(Poor) family management: Items m, n, o, p, q, r, s, t, u (note: Items q-u Year 8 only)


YES! yes no NO!


a. Do you feel very close to your mother? □ □ □ □ □ b. Do you share your thoughts and feelings with your mother? □ □ □ □ □ c. Do you enjoy spending time with your mother? □ □ □ □ □ d. Do you feel very close to your father? □ □ □ □ □ e. Do you share your thoughts and feelings with your father? □ □ □ □ □ f. Do you enjoy spending time with your father? □ □ □ □ □ g. If I had a personal problem, I could ask my mum or dad for help

□ □ □ □ □

h. My parents give me lots of chances to do fun things with them □ □ □ □ □ i. We argue about the same things in my family over and over □ □ □ □ □ j. People in my family have serious arguments □ □ □ □ □ k. People in my family often insult or yell at each other □ □ □ □ □ l. My Parents ask me what I think before most family decision affecting me are made

□ □ □ □ □

m. My parents ask me if I’ve done my homework. □ □ □ □ □ n. My parents would know if I didn’t come home on time □ □ □ □ □ o. The rules in my family are clear. □ □ □ □ □ p. When I am not at home, one of my parents knows where I am and who I am with.

□ □ □ □ □

q. My parents want me to call if I’m going to be late getting home

□ □ □ □ □

r. My family has clear rules about alcohol and drug use. □ □ □ □ □ s. If you drank some alcohol (like beer, wine, spirits or pre-mixed drinks such as Bacardi Breezers or UDL’s) without your parents’ permission, would you be caught by your parents?

□ □ □ □ □

t. If you carried a weapon without your parents’ permission, would you be caught by your parents?

□ □ □ □ □

u. If you skipped or wagged school without your parents’ permission, would you be caught by your parents?

□ □ □ □ □

Worry - Vulnerability and Harmful

This set of items is part of the family domain and attempt to address the subdomain of worry. It has

been drawn from the Family Worries Scale (Graham-Bermann, 1993) which is administered by an

interviewer. It is intended to adapt this question in the online self-administered questionnaire item so

that the items are asked about the most important family member only (see last picture in section 1.2

above). It contains two subscales:

Vulnerability: Items 1, 2, 4, 7, 8, 9, 10, 11, 12, 13, 14, 15

Harmful: Items 3, 5, 6, 16, 17, 18

Item # How much do you worry that (insert family member name here):


1 Will get sick? Mother □ □ □ □

Father □ □ □ □

Someone else in my family □ □ □ □


Item # How much do you worry that (insert family member name here):


2 Will be worried a lot? Mother □ □ □ □



3 Will do something that scares me?

Mother □ □ □ □



4 Will get hurt? Mother □ □ □ □



5 Will get arrested? Mother □ □ □ □



6 Will be fighting? Mother □ □ □ □



7 Will need help? Mother □ □ □ □



8 Won’t have enough money? Mother □ □ □ □



9 Won’t have a place to live? Mother □ □ □ □



10 Won’t have enough to eat? Mother □ □ □ □



11 Will feel sad? Mother □ □ □ □



12 Will move away? Mother □ □ □ □



13 Will feel afraid? Mother □ □ □ □



14 Won’t keep bad things from happening?

Mother □ □ □ □



15 Will die? Mother □ □ □ □



16 Will hurt somebody? Mother □ □ □ □



17 Won’t listen? Mother □ □ □ □



18 Will tell a lie? Mother □ □ □ □




2.1 Friends

In the qualitative phase, the friends domain was ranked very highly by four focus groups and in the

middle range by three other focus groups. Friends were considered to be very important to

participants, but also problematic. A complex picture of friendship groups and friends emerged, with

children in most groups referring to both ‘good friends’ and ‘bad friends’, with some also

distinguishing between these two groups and ‘best friends’. Many young people mentioned having

fun with their friends. They saw ‘good friends’ as ones who stood up for you and provided trust,

closeness, respect, support, and other forms of help. ‘Bad friends’, on the other hand, could be mean

or confusing, turned their back on you, told others your secrets, talked about you behind your back,

or bullied you, even though they were in the same very broad - and sometimes quite large - friendship

group. A few young people mentioned ‘best friends’ as people with whom you could share any secrets.

Many respondents also counted their pets as friends (some also saw pets as family members)

particularly for the rural and regional, economically disadvantaged, and children with a disability

groups. Additionally, Aboriginal and Torres Strait Islander children often identified a wide range of

relatives as friends.

Most friendships were made at school and friends were generally of the same age, although age

grouping was less important for children in communities where interaction with kin was highly valued.

Friendship was seen in both passive and active terms. Some young people talked about a sense of

fulfilment from helping friends. In comparison with other groups, the friendship domain was

problematic for young people with a disability. Many of these respondents talked about struggling to

establish and maintain friendships, and expressed the hurtfulness of being recognised as different

both at school and on the street, resulting in a tendency to highly value friendships with children in

their extended family (and their pets). Similarly, for the children from culturally and linguistically

diverse backgrounds, friendships were a complicated area which sometimes overlapped with a sense

of being bullied and excluded at school, while others expressed supportive friendships by which they

formed opinions about their own identity (subjects their friends thought they were good at), which

was at times in conflict with parental guidance and expectations. For the Aboriginal and Torres Strait

Islander group, while the domain of friends was generally regarded as important, it was not given the

same importance as was the case with other groups. Relationships tied to family and cultural

participation appeared to accommodate the friends domain for Aboriginal and Torres Strait Islander

young people.

2.2 Friends correlates

Number and quality of friends

Q1 At present, how many close male and female friends do you have? (Please tick only one box in each row.)

Males Females

□ None □ None

□ One □ One

□ Two □ Two

□ Three or more □ Three or more

This question is part of the friends domain and has been drawn from the HBSC. It attempts to gain

factual information on the number of male and female friends the student has. It could be asked of all

cohorts of young people.

This question is part of the friends domain and asks further about the number of close friends within

school and out of school. This question will be asked of all three year groups except those who answer

‘none’ to the previous question. The question has been drawn from the HBSC however the wording

has been adapted slightly to more clearly distinguish between close friends that attend the same


school as the student and those that do not. Given these changes, comparisons with the HBSC data

for this item will be inappropriate.

Q2 At present, how many close friends do you have that go to the same school as you, and that don’t go to the same school as you? (Please tick only one box in each row.)

Go to my school Don’t go to my school

□ None □ None

□ One □ One

□ Two □ Two

□ Three or more □ Three or more

2.3 Friends subdomains

Contact with friends

The following two questions are part of the friends domain; subdomain contact with friends and

attempt to assess face-to-face contact with friends outside of school hours. Both have been drawn

from the HBSC with slight adjustments to the question stem for clarity and one response option has

been dropped from the first question (‘6 days’ as there are only 5 days in a school week). Due to these

changes however any comparisons with the HBSC will need to be done with caution. It is intended

that all year levels will be asked these questions.

Q3 How many days a week do you usually spend time with friends after school?

□ 0 days

□ 1 day

□ 2 days

□ 3 days

□ 4 days

□ 5 days

Q4 How many evenings per week do you usually spend with your friends?

□ 0 evenings

□ 1 evening

□ 2 evenings

□ 3 evenings

□ 4 evenings

□ 5 evenings

□ 6 evenings

□ 7 evenings

Q5 How often do you talk to your friend(s) on the phone or send them text messages or have contact through the internet?

□ Rarely or never

□ 1 or 2 days a week



□ Every day

At a recent project team meeting, it was discussed whether the above response options are

appropriate in that many young people would be expected to talk to their friends more frequently

than every day.

There was also concern that the question is dated due to advancements in technology e.g. internet on

phones. If the interest is in contact with friends that is not face-to-face, then perhaps the question

should be rephrased accordingly. It was also discussed whether the interest is in contact that occurs


outside of school hours specifically, for if this is not specified then children may record times when

they send each other text messages or emails during school hours.

Hence, the following question was suggested:

How often do you talk to your friends on the phone, via text message, email or through using social media sites outside of school hours (e.g. facebook)?

Home phone Mobile phone or text message Email or social media (e.g. facebook)

□ Never - I don’t have a home phone

□ Rarely or never (but I have a home phone)

□ 1 or 2 days a week □ 3 or 4 days a week □ 5 or 6 days a week □ Every day (once) □ Every day (2-3 times) □ More than 3 times every day

□ Never - I don’t have a mobile phone

□ Rarely or never (but I have a mobile phone)

□ 1 or 2 days a week □ 3 or 4 days a week □ 5 or 6 days a week □ Every day (once) □ Every day (2-3 times) □ More than 3 times every day

□ Never - I don’t have access to the internet outside of school

□ Rarely or never (but I have access to the internet outside of school)

□ 1 or 2 days a week □ 3 or 4 days a week □ 5 or 6 days a week □ Every day (once) □ Every day (2-3 times) □ More than 3 times very day

However, this would take up quite some response time, so ultimately it might be more effective to

just change the last response option and include additional response options as per the reworded

version above (i.e. Every day (once), Every day (2-3 times), More than 3 times every day). Any advice

from the PSG would be appreciated.


The following questions will only be asked if a respondent indicates having a close friend in response

to the first question (Q1) under the friends correlates specified above (see 2.2).

Q6 For the following questions please think about one of your close friends who goes to your school (Q6a) OR that does not go to your school (Q6b).

1 never or

hardly even 2 3 4

5 extremely

much

I spend fun time with this person □ □ □ □ □ I share private thoughts and feelings with this person □ □ □ □ □ I depend on this person for help, advice, and support □ □ □ □ □ This person sticks up for me □ □ □ □ □

This question is part of the friends domain; subdomain support and conflict and the four items are

intended to be combined to form a scale measuring the degree of closeness the student feels towards

their close friend. Two versions of this question will be asked: one that refers to a close friend that

goes to the young person’s school and another that refers to a close friend that does not go to the

young person’s school. The item has been drawn and adapted from Waldrip, Malcolm and Jensen-

Campbell (2008).

Q7 Still thinking about the same close friend who goes to your school (Q7a) OR does not go to your school (Q7b), how true is the following statement about your relationship with this friend?

1 not

at all true

2 a

little true

3 somewhat

true

4 pretty true

5 really true

I get into fights with my friend □ □ □ □ □

My friend bug me or annoy me even though I ask him/her not to □ □ □ □ □

My friend and I argue a lot □ □ □ □ □

My friend and I disagree about many things □ □ □ □ □

This set of questions have been drawn from Bukowski, Hozaand & Boivin (1994) and form part of the

friend domain; subdomain support and conflict. The items have been adapted slightly from their

original form and are intended to be combined to form a scale that measures the degree of conflict

the student has with their close friend. As for the previous question, two version of this question will

be asked: one referring to a close friend at school and one referring to a close friend out of school.


3.1 School

The domain of school evidenced a mixed evaluation in the ranking exercises, where it was generally

ranked in the middle of hierarchies (for example in the mainstream, out of home care, culturally and

linguistically diverse, and rural and regional groups) or else was ranked quite low (as was the case

among children with a disability, economically disadvantaged children and Aboriginal and Torres Strait

Islander children). Even when lowly ranked, the benefits of school were generally agreed upon: for

gaining an education (generally with a view to securing future opportunities and a satisfying job), as a

social meeting place where friendships could be built and nurtured, and for some as the place to

access sports programs and maintain physical fitness and wellbeing. A distinct variation to this

consensus concerned the perception of school made by young people with a disability. School for this

group was distinguished by safety and bullying issues rather than positive social interaction. For this

group, ‘home time’ was seen as the best part of the school day. Many young people did not see school

as the only (or even the most important) locus for learning. Many young people (especially Aboriginal

and Torres Strait Islander and culturally and linguistically diverse young people) greatly valued cultural

and other forms of learning that occurred at home or within a community, or learning associated with

the practice of sports and hobbies. Children from economically disadvantaged backgrounds discussed

school very little in terms of acquiring an education – even though the proposed benefits were

understood. They seemed to appreciate school mostly for its social qualities.

Some groups were concerned about the stress, depression and anxiety caused by what they viewed

as an excessive amount of homework or ‘extra exams’ (i.e. NAPLAN) to study for. Others (for example

in the out of home care group) appeared to be totally unstressed by homework, but expressed a clear

dislike for it. School was also viewed as the main domain where rules and guidance were confronted.

Most guidance and rules were acceptance but ‘strange’ rules relating to uniforms, showing colour,

out-of-bounds areas, etc were viewed with suspicion. School was also perceived, particularly by the

culturally and linguistically diverse group, as a place for social acclimatisation and a place to practice

English skills.

There was much discussion of teachers among the different groups. The importance of supportive

relationships between parents and teachers was a concern for some, particularly culturally and

linguistically diverse young people. In many groups, opinions of teachers were somewhat equivocal,

with discussion of both good and bad (or lazy) teachers, and sometimes ‘angry teachers’. Some young

people reported having special relationships with one or more teachers that were clearly very

important to them. Others focused on the difficulties associated with telling teachers about problems

such as bullying, not necessarily because of what the teacher would do in response, but because of

reactions by fellow students.

3.2 School correlates

Absence - Illness and truancy

In the last two months, how many times have you missed school due to illness? (Please tick only one box.)

Never □ Hardly ever □ About once a week □ Most days □ Every day □ Don’t know □

This question is part of the school domain. It is included for factual information to ascertain the extent

to which the young person is legitimately absent from school. It has been drawn from the Children’s

World’s questionnaire and may be administered to all cohorts perhaps with rewording of ‘illness’ to

‘sickness’ for the year 4’s. It is anticipated that comparisons could be made at each year level with the

Children’s World’s data which is an international study of 7-13 year olds.


In the last two full weeks of school, how many times did you skip a whole school day?

(Please tick only one box.)

None □ One of two times □ Three or four times □ Five or more times □

This question is also part of the school domain and is included for factual information on truancy.

While the previous question is about authorised absence, this question asks about unauthorised

absence from school. In line with findings of the qualitative phase, skipping school is considered an

indicator of the different priorities assigned to school by different disadvantaged groups. The question

has been taken from the PISA 2012 student questionnaire which is designed for 15 year olds.

Comparisons with results from Australia and 66 other education systems could be made with these

data for the year 8 cohort but the question could equally be administered to young people in years 4

and 6.

Success at school

In your opinion, what does your class teacher(s) think about your school performance compared to your classmates? (Please tick only one box.)

Very good □ Good □ Average □ Below average □

This question assesses the school domain; and has been included to gain an indication of how young

people perceive their success at school (subdomain). The question has been drawn from the HSBC

mandatory question set and is intended to be asked at all three grade levels. It is anticipated that

analysis of this question could be done in conjunction with the school's most recent reading NAPLAN

score. Teacher rated success would have four levels (very good, good, average and below average)

while NAPLAN school performance has five levels (substantially above, above, average, below,

substantially below). Together, this will result in each student being assigned a rank from 1 to 20.

3.3 School subdomains

Enjoyment of school

My school is a place where…

(Please tick one box in each row.) Strongly disagree

Disagree Agree Strongly

Agree

…I feel happy □ □ □ □ …I really like to go to each day. □ □ □ □ …I find that learning is a lot of fun. □ □ □ □ …I feel safe and secure. □ □ □ □ …I like learning. □ □ □ □ …I get enjoyment from being there. □ □ □ □ …The work we do is interesting. □ □ □ □ …I like to ask questions in class □ □ □ □ …I like to do extra work. □ □ □ □ …I enjoy what I do in class. □ □ □ □ …I always try to do my best. □ □ □ □ …I get excited about the work we do. □ □ □ □

The above 12 items are part of the school domain; subdomain enjoyment and have been drawn from

LSAC. The items will be combined to form two scales, one measuring school enjoyment with items 1


to 6 comprise the Positive Affect/General Satisfaction subscale, while the remaining items (7-12)

comprise the Intrinsic Motivation subscale1. Comparisons will be possible with the year 8 cohort as

the LSAC research collected data from 12-13year olds. It is anticipated that this question could be

administered to all three grade levels in the ACWP however it may be too long for children in grade 4.

Schoolwork pressure

How pressured to you feel by the schoolwork you have to do? (Please tick only one box.)

Not at all □ A little □ Some □ A lot □

This question is part of the school domain and is designed to measure the degree to which young

people experience school work pressure. It has been drawn from the HBSC and it is anticipated that

comparisons will be made with this data during analysis which has been collected from an

international sample of 11, 13 and 15 year olds. This question could be asked at every grade level

however perhaps is less relevant for year 4 student.

Relationships - Teacher support

How true is each statement for you? At my school, there is a teacher or another adult …

(Please tick one box in each row.) Not at all true

A little true

Pretty much true

Very much true

Who really cares about me □ □ □ □ Who believes that I will be a success □ □ □ □ Who listens to me when I have something to say □ □ □ □

This question is part of the school domain; subdomain relationships. The three items are intended to

be combined to form a scale measuring teacher support and the questions have been drawn from the

Middle Years Development Indicator (MDI). The question could be asked at year 4, 6 and 8 and

comparative analysis could be made at grade 4 and 6 as the MDI collected data from a Canadian year

4 sample (ages 9-12).

4.1 Community/Neighbourhood

Young people in the different groups expressed a wide range of views on the meaning of Community.

Indeed, the terms appeared to confuse some children. For the most part, the

Community/Neighbourhood domain was ranked quite low, with two significant exceptions. The

culturally and linguistically diverse and Aboriginal and Torres Strait Islander groups each ranked

Community/Neighbourhood in second position under family. For many groups who did not rank

Community/Neighbourhood highly, there was some confusion when it came to defining a

Community/Neighbourhood. In such cases the mention of ‘online communities’ helped stimulate how

Community/Neighbourhood might be conceived – as one child put it, a Community/Neighbourhood

could ‘be people you know of’, but did not necessarily have to know intimately. For others,

Community/Neighbourhood encompassed a number of different groupings, relationships, and

domains spanning family and school, the local council and sporting clubs. The culturally and

linguistically diverse group, many of whom were born overseas and spoke their own language at

home, had a considerably more global conception of Community/Neighbourhood than the other

groups. Their communities consisted of people from the ‘same culture’ and from the ‘same country’,

and their strong sense of a global Community/Neighbourhood included contextual themes of war,

peace, or cultural conflict. For the Aboriginal and Torres Strait Islander group, concepts of

1 www.growingupinaustralia.gov.au/data/rationale.html

http://www.growingupinaustralia.gov.au/data/rationale.html


Community/Neighbourhood were strongly linked with family and culture, including a wide variety of

social experiences shared with extended kinship networks. Given previous literature and the findings

of the Phase 1 work, the term ‘neighbourhood’ was adopted instead of ‘community’ as the term

‘neighbourhood’ may be easier to operationalise.

Community/Neighbourhood emerged as a theme of low importance to most groups of children in

Phase 1, with the exception of the Aboriginal and Torres Strait Islander, and Culturally and

Linguistically Diverse groups. Consequently, this domain should occupy relatively little questionnaire

space comprising only a limited number of questions about safety and access to resources in the local

area.

4.2 Community/Neighbourhood correlates

Deprivation

Deprivation is also considered important for the community/neighbourhood domain. However, this

will be assessed in the money and material wellbeing domain (see social-economic status

scale)through asking whether young people have, or desire to have “A garden at home or somewhere

nearby like a park where you can safely spend time with your friends (garden)”. Other items might be

included upon further review of relevant community/neighbourhood scales.

4.3 Community/Neighbourhood subdomains

Access to resources

Strongly disagree


or agree Agree

Strongly agreed

There are places for me to go in my area □ □ □ □ □ There is nothing to do in my area □ □ □ □ □ There are lots of fun things to do where I live □ □ □ □ □

This question has been drawn from Children’s Society and is included as a scale to measure access to

resources in the neighbourhood.

Safety

Strongly disagree


or agree Agree

Strongly agreed

I feel safe when I am out in my local area during the day

□ □ □ □ □


□ □ □ □ □

I have plenty of freedom in the area I live in □ □ □ □ □

This question has been drawn from Children’s Society and is included as a scale to measure perceived

safety in the neighbourhood.

5.1 Health

Health was seen as an important domain in all groups, though it was usually ranked behind family,

friends and school. Understanding of the term on an abstract level most commonly included the

importance of diet (related to both nutrition and weight) with a lot of discussion about ‘healthy food’

and ‘junk food’. Sport, fitness, personal hygiene and mental health were also commonly discussed.

Maintaining good levels of fitness was considered beneficial to a healthy lifestyle, from ‘playing footy’

to ‘races with friends’ and ‘dancing’. Being active was understood as a healthy pursuit. While physical


health was generally discussed within the groups and interviews, an understanding of mental health

also emerged in some of the groups, including the culturally and linguistically diverse group, the

disability group, the Aboriginal and Torres Strait Islander group and the mainstream group. In these

groups, young people talked in abstract terms about mental health issues such as depression and

anxiety standing as barriers to wellbeing and living a good life.

While health was generally seen in individual terms (young people’s own health) discussion of health

in the Aboriginal and Torres Strait Islander group also encompassed a collective dimension, where the

health of each family member was seen as important to the wellbeing of all family members. To a

greater extent than was evident in other groups, health was also seen as associated with adequate

material resources, with considerable discussion of the need for adequate nutrition for health.

In addition to the questions in the family subdomain of worry about the health of other family

members and the question about missing school due to illness in the major domain of school, the

following questions have been selected to cover much of the content that emerged from the

qualitative interviews. With the exception of the last question (i.e. "Outside school activities"),

questions were selected from the Health Behaviour in School-Aged Children (HBSC) survey as this will

allow comparisons with nationally representative data for 11 and 13yearolds from 43 countries. Data

from the HBSC's most recent cycle, in 2009/10, will be available from the 1st October 2014. This date

coincides with the start of the analyses from the ACWP main survey.

5.2 Health correlates

Oral Health

How often do you brush your teeth? (Please tick one box only.)

□ More than once a day

□ Once a day

□ At least once a week but not daily

□ Less than once a week

□ Never

This question is part of the health domain and is included as a correlate of wellbeing. It has been drawn from the HBSC and as such comparisons could be made at the higher grade levels as this data is collected from ages 11, 13 and 15. All year levels could be asked this question.

Hunger

Some young people go to school or bed hungry because there is not enough food at home. How often does this happen to you? (Please tick one box only.)

□ Always

□ Often

□ Sometimes

□ Never

This question has been included as a measure of hunger which is considered to be a correlate of

wellbeing. It forms part of the health domain and can be asked at all three grade levels. Like many of

the health related questions, it has been drawn from the HBSC of which comparisons could be made.


Food & Drink

How many times a week do you usually eat or drink...?

(Please tick one box in each row.) Never


Once a week

2-4 days a week

5-6 days a week

Once a day, every day

Every day, more than

once

Fruits □ □ □ □ □ □ □ Vegetables □ □ □ □ □ □ □ Sweets (candy or chocolate) □ □ □ □ □ □ □ Coke or other soft drinks that contain sugar

□ □ □ □ □ □ □

This question is included to gain some indication of what types of food the young person has access

to and consumes. It has been drawn from the HBSC and is part of the health domain. It is intended

that year 6 and 8 student will be asked this question.

Breakfast

How often do you usually have breakfast (more than a glass of milk or fruit juice)?

Weekdays (Monday to Friday) (Please tick one box only.)

□ I never have breakfast during the week

□ One day

□ Two days

□ Three days

□ Four days

□ Five days

This question has been included to understand the frequency with which young people have breakfast

which is considered as a correlate to wellbeing. It is part of the health domain and will be asked at all

three year levels.

Physical fight

During the past 12 months, how many times were you in a physical fight? (Please tick one box only.)

□ I have not been in a physical fight

□ 1 time

□ 2 times

□ 3 times

□ 4 times or more

This question about fighting is part of the health domain and has been included as a correlate of

wellbeing. It has been drawn from the HBSC and will be asked at year 4, 6 and 8.

Smoking & Drinking

On how many occasions (if any) have you done the following things in the last 30 days? (Please tick one box in each row.) Never

1-2 times

3-5 times

6-9 times

10-19 times

20-39 times

40 or more

Smoked cigarettes □ □ □ □ □ □ □ Drunk alcohol □ □ □ □ □ □ □ Been drunk □ □ □ □ □ □ □

This question provides an indication of student drinking and smoking health behaviour. This is

considered as a correlate of wellbeing and will be asked at the grade 6 and 8 level. It has been drawn

from the HBSC to enable comparisons.


5.3 Health subdomains

Subjective health and body image

The following three questions are part of the health domain; subdomain subjective health and body

image.

Would you say your health is....? (Please tick one box only.)

□ Excellent

□ Good

□ Fair

□ Poor

This scale has been included as a measure of overall health and is intended for Year 4, 6 and 8. This

and the following question have been drawn from the HBSC where they are used as a non-clinical

measure of mental and physical health.

In the last 6 months: how often have you had the following...? (Please tick one box in each row.) About every

day More than

once a week About every

week About every

month Rarely or

never

Headache □ □ □ □ □

Stomach-ache □ □ □ □ □

Back ache □ □ □ □ □

Feeling low □ □ □ □ □

Irritability or bad temper □ □ □ □ □

Feeling nervous □ □ □ □ □

Difficulties in getting to sleep □ □ □ □ □

Feeling dizzy □ □ □ □ □

This scale has been included as a measure of health complaints and is intended for young people in

Year 6 and 8.

Do you think your body is... (Please tick one box only.)

□ Much too thin

□ A bit too thin

□ About the right size

□ A bit too fat

□ Much too fat

This question is about body image and is intended for young people in Year 8 only.


This question has been drawn from the Children's World Survey. Its purpose is to measure the

frequency with which young people participate in activities outside of school which is a subdomain of

health. It could be asked at all three grade levels.


How often do you usually spend time doing the following activities when you are not at school?

(Please tick one box in each row.) Hardly ever or never




Don't know

Taking lessons (like music, sports, dancing, languages)

□ □ □ □ □

Hanging out with friends □ □ □ □ □ Reading for fun (not homework) □ □ □ □ □ Helping with housework □ □ □ □ □ Doing homework □ □ □ □ □ Watching TV or videos, or listening to music □ □ □ □ □ Playing sports on a team □ □ □ □ □ Playing sports or doing exercise (but not on a team)

□ □ □ □ □

On the computer □ □ □ □ □ Spending time just being by myself □ □ □ □ □ Taking care of brothers or sisters or other family members

□ □ □ □ □

6.1 Money and material wellbeing

Across all the groups, money was consistently ranked as the lowest, or closest to the lowest, domain

for a good life. Money was seen as important as an enabler for respondents to access the ‘basics’ of

life - usually identified as food, shelter and clothes. Much less emphasis was placed on the

accumulation of material items such as computer games, iPods, cars, or other ‘non-essentials’.

Of all the groups, the Aboriginal and Torres Strait Islander respondents placed money highest in their

understanding of wellbeing and a good life. This may reflect the likelihood that most of this group was

economically disadvantaged (and perhaps more so than the economically disadvantaged group that

we talked with). Aboriginal and Torres Strait Islander young people referred not only to basics such as

nutrition and shelter, but also linked the lack of it to serious consequences, including ill-health (‘get

skinny’) and crime. Aboriginal and Torres Strait Islander young people also spoke with personal

experience of the consequences for themselves of living in an economically disadvantaged

environment. For example not asking for or expecting birthday presents, or not getting a lift by car to

and from an activity because their parents could not afford the required petrol. As such, they displayed

a pragmatic concern with money as an enabler for improving their everyday wellbeing.

Additionally, across some of the groups, namely the mainstream group and the regional and rural

group, there was evidence of money being used to enact an ethical code. Respondents from both

groups spoke of using money in a charitable way in order to address wider social inequalities such as

poverty and homelessness. Redistributing money was one of the proposed ways young people

thought they would feel good about themselves, by giving to others and contributing to society.

Most surveys ask about material wellbeing by getting young people to indicate the items they or their

family possess. The lack of possessions is then frequently interpreted as "deprivation". As part of her

doctoral thesis, Gill Main examined the validity and reliability of deprivation scale used in the The

Children’s Society 2010-11 Survey and found them to be quite satisfactory.

The question proposed to measure deprivation in the ACWP contains ten items from Children’s Society

plus three items that were added to reflect results of the ACWP qualitative work. The three new items

cover the ability to go on a school camp, having credit on the mobile phone and for parents to be able

to put petrol in their car (these questions were previously proposed for inclusion in the Children’s

Society.


6.2 Money and material wellbeing correlates

Socio-economic status

I have

this

I don't have this

but would like it

I don't have this and I

don't want or need it

Some pocket money each week to spend on yourself (pocket money) □ □ □ Some money that you can save each month, either in a bank or at home (saving money)

□ □ □

A pair of designer or brand name trainers (like Nike or Vans) (trainers) □ □ □ An iPod or other personal music player (MP3 player) □ □ □ Cable/satellite TV at home (cable/satellite) □ □ □ A garden at home or somewhere nearby like a park where you can safely spend time with your friends (garden)

□ □ □

A family car for transport when you need it (car) □ □ □ The right kind of clothes to fit in with other people your age (clothes) □ □ □ At least one holiday away from home each year with your family □ □ □ Trips or days out with your family at least once a month (day trips) □ □ □ Enough credit on my mobile to phone or text friends □ □ □ My family has enough money to put petrol in the car when needed □ □ □ My family has enough money for me to go on a school camp □ □ □

Further correlates/factual questions regarding deprivation (e.g. being hungry, not having breakfast)

are included under the health domain.

6.3 Money and material wellbeing subdomains

There are no subdomains for this domain, however questions in the subdomain worry in the major

domain family contain attitudinal items regarding the extent to which young people and their families

have enough to eat, enough money and a place to live.

7.1 Self demographics

This set of items are correlates of wellbeing related to the self and include questions around family

language background, gender, Aboriginal and Torres Strait Islander status, disability and aspirations.

Information about out of home care and home location are asked as part of the factual/correlates/ in

the family domain so need not be repeated here. The following are some examples of the types of

items to be used for these correlates drawn from well-established international studies such as PISA,

TIMSS and PIRLS.

Language background

How often do you speak English at home? (Please tick one box only.)




This question is from TIMSS and PIRLS and is asked at the grade 4 level.


□ Always

□ Almost always

□ Sometimes

□ Never


This question is from TIMSS and PIRLS and will be asked at the grade 6 and 8 level.

Gender

Are you a girl or a boy? (Please tick one box only.)

□ Girl

□ Boy

This question is from TIMSS and PIRLS and is asked at the grade 4 and 8 level.

Aboriginal and Torres Strait Islander


(For persons of both Aboriginal and Torres Strait Islander origin, tick both ‘Yes’ boxes.)

No □ Yes, Aboriginal □ Yes, Torres Strait Islander □

This question is drawn from the Australian national version of the PISA student questionnaire. While

this is aimed at 15 year olds, it could be asked at all three year levels.

Disability

Have you had a disability for a long time (more than 6 months) (such as, hearing difficulties, visual difficulties, using a wheelchair, mental illness?

Yes □ No □ I don't know □ Does your disability make it hard for you, or stop you.... (You can select more than one)

Doing everyday activities that other children your age can usually do (such as getting ready for school, eating, washing yourself, getting dressed or going to the toilet)

□

Talking to people, understanding what other people say or hanging with friends □ Doing any other activity that children your age can usually do (such as sports and hobbies like football, cricket, swimming, playing games or playing a musical instrument)

□

No difficulty with any of these □

Students were first asked the question ‘Have you had a disability for a long time (more than 6 months)

(such as, hearing difficulties, visual difficulties, using a wheelchair, mental illness)?’ For students who

indicated ‘Yes’ to this question they were classified as having a disability. If they selected ‘No’ they

were coded as not having a disability.

If students selected ‘I don’t know’ they were further asked if their disability made it hard (or stopped

them) doing certain activities (SD06A01 to SD06A04). If these students confirmed that they had an

issue with any of the activities, they were defined as having a disability. In the case were a student

selected ‘I don’t know’ to the first question and they indicated having no issues with the activities,

they were defined as not having a disability.

Aspirations

How far in your education do you expect to go? (Please tick one box only.)

Finish year 10 □ Finish year 12 □ Finish TAFE training certificate, apprenticeship or traineeship □ Finish a TAFE or College Diploma (e.g. Diploma in Information Technology) □ Finish an undergraduate degree (e.g. Bachelor of Arts, BA, Bachelor of Commerce) □ Finish a postgraduate degree (e.g. Honours, Master of Arts, PhD) □ I don’t know □


This question has been drawn from the TIMSS year 8 Australian questionnaire and would allow

comparisons with the Year 8 cohort of the ACWP survey. A modified version could be created for

young people in years 4 and 6.

8.1 Bullying

Bullying was a theme which often emerged in the qualitative work with all the groups and most

individual participants. Though the majority of respondents flagged bullying as an extremely

important issue for them, discussion and experience of bullying was quite diverse and multi-faceted.

This was not only because there were multiple types of bullying identified – physical, verbal, mental

and cyber – but also because bullying could span a number of domains, including family, school,

community/ neighbourhood, and friends. However, most young people talked about bullying in a

school setting, and most identified peers as the agents of bullying.

It was noteworthy that many respondents drew a sharp distinction between ‘good friends’ and ‘bad

friends’. While ‘good friends’ were supportive and trustworthy, ‘bad friends’ were those

acquaintances who our respondents felt were fickle, untrustworthy, and occasionally ‘mean’. This

indicates that young people’s experiences of bullying are nuanced and fluid; bullying appears to occur

frequently within friendship groups (the action of ‘bad friends’), and a ‘bully’ one week may not

necessarily be a ‘bully’ the next.

Throughout the focus groups and interviews, bullying was framed in two major ways. For some of the

groups (rural and regional, mainstream, and economically disadvantaged) bullying tended to be

discussed in an abstract sense – something which they had noticed was an issue but had not directly

experienced. It is worth noting in this context the researchers’ opinion that discussion in the

economically disadvantaged group was perhaps dominated by young people who did not appear to

be economically disadvantaged. Conversely, the respondents from other groups (Aboriginal and

Torres Strait Islander, culturally and linguistically diverse, young people with disabilities, young people

in out of home care), detailed their lived experience of bullying. Indeed for some participants from

these groups, their experience of bullying was constant and ongoing.

Finally, it is worth highlighting the experience of young people with disabilities who experienced

bullying, not only in the school setting, but also in the wider community/neighbourhood and in public

places. It is also worth mentioning that while some young people talked about bullying in the home,

the researchers did not probe deeper on this issue.

The following three questions are suggested to form part of the crosscutting domain of bullying.

Thinking back over the last 12 months (since about [name of month] last year) did another child or children pick on you by…

Yes No

…shoving, pushing or hitting you? □ □ …calling you names or insulting you? □ □ …writing messages/notes? □ □ …leaving you out of games or chats? □ □

This question has been drawn from LSAC and attempts to gain an understanding of the degree to

which the young person has been bullied over the past 12months. While 12 months might be too long

a period to consider, particularly for young people in year 4, and the third item might be qualified to

say "insulting messages/notes", any adaptation would affect comparability. As it stands, the question

could be asked at each grade level and comparisons be made with LSAC data.

Has somebody who you think is your friend bullied you?

□ Yes

□ No


This question has been drawn from Mishna, Wiener and Pepler (2008) and could be asked at all three

year levels. As information on "general" bullying is asked in the first question, this question specifically

aims at the bullying by friends. From the qualitative work it emerged that bullying was not just done

by "others" who are not close but also by people who are considered friends.

Have you taken part in bullying another child recently (i.e. in the last four weeks)?

□ Yes

□ No

This question has been taken from the HowRU? survey and is concerned with whether or not young

people have acted like bullies themselves. As such behaviour is not limited to the school setting, the

question was modified so that the location of the bullying episode was not included. This makes

comparisons with the HowRU? survey inappropriate but captures the essence of the qualitative work

on this issue.

8.2 Guidance and rules

In their discussions stretching across a number of domains, young people indicated that guidance and

rules are important dimensions of a good life. The interviews and focus groups showed that

participants displayed a nuanced understanding of rules and guidance. Across all groups, respondents

were supportive of many of the rules that structured their daily lives. They overwhelmingly felt that

many rules were put in place to establish and maintain their own safety and wellbeing. Most of the

respondents spoke about guidance and rules as emanating from two basic domains: family and school.

Rules and guidance, nonetheless, could be problematic in at least two major ways. First, some

respondents talked about a mismatch of rules across different spheres and domains (counterpoising

school and home, and different parents’ houses) which could cause them distress. Second, some

respondents spoke at length about rules and guidance which they did not respect. That is, rules for

which the rationale was unreasonable, unknown or senseless. Several school rules were mentioned in

this respect. Inconsistency and incoherency were therefore seen as key issues with respect to young

people’s perceptions of guidance and rules.

Currently, questions around guidance and rules are included within the family domain. For example,

“The rules in my family are clear” and “My family has clear rules about alcohol and drug use”.

8.3 Learning

Most respondents expressed an interest in learning, both in school and in other forums, although its

importance was emphasised by some groups more than others. As noted above, some young people

(notably those in out of home care), had a somewhat ambivalent attitude towards formal learning and

homework. Others clearly enjoyed learning, while some valued it for its instrumental value (so as to

get a job in adulthood, and not to get ripped off). While school was often seen as the central site of

learning and education in young people’s lives, many participants also said that they valued learning

within their families and their communities. This was particularly true for some groups. For example,

the Aboriginal and Torres Strait Islander and culturally and linguistically diverse young people

repeatedly spoke about gaining cultural knowledge from parents and Elders which they could not get

at school.

Questions around learning have been included in the school domain.

8.4 Safety

Among some of the groups, safety and security emerged as a key concern. Safety was usually discussed

in terms of perceived threats to physical safety, sometimes from environmental dangers such as

traffic, but more often from other people. In some cases therefore, concerns with safety could also be

interrelated with issues of bullying. The rural and regional group articulated considerable concern


about the safety of their neighbourhoods. The Aboriginal and Torres Strait Islander young people also

voiced their concerns about safety however to a lesser extent. Some of the respondents from the

disability group spoke about their fears of strangers in public as well as issues of physical bullying at

school.

Questions to do with safety are include in the school domain. For example, one item “My school is a

place where… feel safe and secure.” Likewise, three items ask about safety in the

community/neighbourhood domain.

8.5 Stress

Another key theme which emerged throughout the first round of qualitative data analysis was the

impact of stress on the lives of many of our respondents. While this appeared to be a common concern

among many of the young people that we spoke to, its source varied somewhat between groups.

Stress could be influenced by a number of different factors, including concerns for other family

members’ wellbeing, and bullying. Among the mainstream and culturally and linguistically diverse

groups in particular, homework and schoolwork were repeatedly mentioned as the contributors to

stress, in combination with before and after school activities. Many young people discussed feeling

loaded with homework, and feeling pressure to do well at school (one person mentioned NAPLAN in

this respect), and felt they had little time to themselves. Some young people demonstrated

considerable resilience in employing a number of techniques to deal with stress, including spending

time alone, watching TV, talking to their parents, napping, spending time with pets, and listening to

music.

Questions to do with stress have been included in the school domain. For example, “How pressured

to you feel by the schoolwork you have to do?” Other stress is being measured by the Worry

subdomain in the family domain. A set of questions on health symptoms (headaches, stomach aches,

dizziness, etc.) is also being asked in the health domain.

8.6 Feeling good

Since young people talked about "feeling good" mostly in the context of other domains, some

discussion has occurred as to whether or not the ACWP should include a measure of social and

emotional wellbeing.

If such a measure is desired, this section describes some alternatives.

Firstly, the following two sets of questions provide a scale of social and emotional wellbeing. The

versions for young people in primary and secondary school are equated onto the same scale, so that

primary and secondary scales scores (based on all items in each survey) are directly comparable. The

items have been drawn from the ACER Social-Emotional Wellbeing Survey (SEW) for which Australian

data are available for comparison across all three year levels.


Social and emotional wellbeing (primary version for year 4 and 6)

Disagree Agree

I am a happy person. I feel safe and free from danger. I lose my temper a lot. I get along pretty well with members of my family. I could do a lot better in my schoolwork. My classmates like me. I feel lonely. I get along well with my teacher. I worry a lot about my schoolwork or what others think about me. I get into trouble a lot. I like being at school. I break things. I yell and scream at people a lot. I feel very bad for long periods of time. I clean up and try to make my classroom and home a nice place. I get nervous a lot. I am sometimes very mean to other people. I like the kind of person I am. I love to learn. When I grow up I will have a good life.

Social and emotional wellbeing (secondary version for year 8)

Strongly disagree

Disagree Agree Strongly

agree

I am a happy person. □ □ □ □ I feel safe and free from danger. □ □ □ □ I lose my temper a lot. □ □ □ □ I get along pretty well with the members of my family. □ □ □ □ I could do a lot better in my schoolwork. □ □ □ □ I get along with most of my classmates. □ □ □ □ I feel lonely. □ □ □ □ I get along with most of my teachers. □ □ □ □ I worry too much about my schoolwork or what others think of me.

□ □ □ □

I get into too much trouble. □ □ □ □ I feel like I belong in my school. □ □ □ □ I feel very stressed. □ □ □ □ I volunteer to do things to make my school and community a safer and better place to live.

□ □ □ □

I participate in many different activities inside and outside of school (e.g., clubs, sport, music, drama, community).

□ □ □ □

During the past six months, I have felt so hopeless and down almost every day for one week that I have stopped doing my usual activities.

□ □ □ □

I am sometimes quite mean to other people. □ □ □ □ I like the kind of person I am □ □ □ □ I am someone who loves to learn. □ □ □ □ I am very hopeful about my future. □ □ □ □ I am very curious about why things are the way they are. □ □ □ □ I am very enthusiastic and have lots of energy. □ □ □ □


Another option would be to use the questions included the Communities that Care survey. These items

were a subset of items from the Short Mood and Feelings Questionnaire (Angold et al., 1995), the

Short Spence Anxiety scale (Spence et al., 2003) and the PedsQl General Wellbeing Scale (Varni, et al.,

2003).

It is anticipated that further work will be conducted to identify the measure of ‘feeling good’ that is

the most appropriate given the overall aim of the ACWP.

8.7 Summary of proposed indicators and comparisons

The suggested indicators that can be calculated based on responses to the questions currently

proposed in the draft ACWP survey are summarised in Table A1.2.

Table A1.2: Summary of proposed indicators and comparisons in the draft field trial questionnaire

Domain Subdomain Indicator (Preliminary label) Comparisons


People you live with Children's Worlds

Changed house Children's Worlds

Changed school Children's Worlds

Changed carers Children's Worlds


Family conflict HowRU

Attachment to mother HowRU

Attachment to father HowRU Opportunities for pro-social involvement HowRU

(Poor) family management HowRU




Close friends - Male HBSC

Close friends - Female HBSC

Close friends - Same school ACWP subgroups

Close friends - Outside school ACWP subgroups

Contact with friends

Time spent with friends ACWP subgroups

Frequency talking to friends ACWP subgroups


Friends - support ACWP subgroups

Friends - conflict ACWP subgroups


Missed school - Illness Children's Worlds

Missed school - Truancy PISA




School - Pressure HBSC

Relationships Teacher support MDI

Community/ Neighbourhood Health




Factual/ correlates

Oral health HBSC

Hunger HBSC

Food and drink HBSC

Breakfast HBSC

Physical fight HBSC


Subjective health and body image

Overall subjective health HBSC

Headache HBSC

Stomach-ache HBSC

Back-ache HBSC

Feeling low HBSC

Irritability HBSC


Domain Subdomain Indicator (Preliminary label) Comparisons



Dizziness HBSC

Overall body image HBSC



Children's Worlds


Factual/correlates Socio-economic status (Deprivation) Children's Society

Self-demographics

Factual/correlates Language background PISA

Gender PISA


Aspirations TIMSS


Bullying General bullying LSAC


Feeling good Social and emotional wellbeing** SEWB

Mood and feelings** CATS

Spence anxiety** CATS

PedsQI General wellbeing** CATS Notes * ACWP subgroups include comparisons by gender, disability, grade, Aboriginal and Torres Strait Islander, language

background, grade, socio-economic status. ** At the time of writing, a number of alternative scales and comparison data were considered for this cross-cutting

domain.

9.1 Description of the use of anchoring vignettes in the ACWP

Differences in response styles are considered to be a serious source of bias in surveys using Likert

items. Differences in response style can occur between individuals (e.g. Austin et al., 2006; Bolt, 2009),

between groups within countries (e.g. Marin et al. 1992; Moors 2004) and between countries (e.g.

Heine et al., 2002; Johnson et al., 2008). Several types of response styles have been described (e.g.

Greenleaf, 1992; Clarke, 2000; Johnson & al., 2005; Thomas & al., 2008). All of them can make it

difficult to distinguish authentic differences from “stylistic” biases in respondent behaviour (e.g.

Moors, 2004; Van de Vijver & Poortinga, 1997; van Hemert, Poortinga & van de Vijver, 2007).

Extreme response style refers to people who tend to select the answer categories at the extreme

sides of the Likert scale (e.g. Strongly agree / Strongly Disagree; or Never / Always) rather than the

intermediate responses.

Intermediate response or modesty style refers, by contrast, to people who tend to avoid the extreme

answer categories, and to make most frequent use of the middle answer categories. A special case of

middle response style is Central response style (also called Midpoint response style), where people

tend to prefer the neutral category (e.g. Neither agree nor disagree) when provided.

Acquiescent response style refers to people who tend to agree with most or all statements,

independently of their content. Such respondents give ‘positive’ answers (e.g. Agree / Strongly Agree;

or Often / Always) even to sets of items with opposite meanings (blind agreement, sometimes called

“Yeah-saying”).

Disagreement response style refers to the opposite: individuals who select negative answers for most

of the items (e.g. Disagree / Strongly disagree; Never / Almost never), regardless of content.

Proposed explanations of differences in response styles include the assumption of frame-of-reference

effects whereby responses to attitude (or other) questions might differ systematically depending on

which frame of reference (either across countries or across sub-groups within countries) is applied.


These frames-of-reference include so-called “cultural macro values” (King et al. 2004), the Big-Fish-

Little Pond Effect” (Marsh and Hau 2003) and social desirability (Holtgraves, 2004).

Anchoring vignettes have been used successfully, initially in health research and, more recently in

education in the 2012 international PISA testing programme to adjust for differences in response style.

In principle, an anchoring vignette is developed to provide a frame of reference for the respondent's

answers to subsequent items. Ultimately, the respondent's answers to the vignettes are used as a

scoring mechanism which adjusts - at the individual respondent level - for differences in response

styles as some respondents are more inclined to agree than others, some are more inclined to use

extreme responses or sit in the middle between different response alternatives.

Desirable characteristics of a successful vignette are as follows:

There is close to universal agreement about the rank order of the three options;

Differences in respondents’ choices are reflected in their inclination to select strongly

agree rather than agree or strongly disagree rather than disagree;

Differences in respondents’ choices are also reflected in where they place the middle

option – they may rank it with one of the other options or between the other two options.

The vignettes to be included in the ACWP questionnaire need to tap the underlying construct of the

item or set of items. The example below relates to the subdomain of family conflict.

9.2 Vignette

YES! yes no NO!

The X family fix their problems by talking to each other when everyone is calm. They try to find solutions that everyone agrees with. The X family get on well together.

□ □ □ □

The Y family fix their problems by having big arguments. Kim is in charge of the Y family. If the argument goes on too long then Kim decides what to do. Everyone has to do what Kim says even if they do not agree. The Y family get on well together.

□ □ □ □

The Z family argue about the same problems every day. Everyone joins in shouting and yelling and saying mean things to each other and nothing gets fixed. The Z family get on well together.

□ □ □ □

9.3 Actual attitude scale

These questions ask you about your family. When we ask about your mother and father we want you to think about who you live with most of the time. This includes step-parents, foster parents or guardians.

YES! yes no NO!


i. We argue about the same things in my family over and over □ □ □ □ □ j. People in my family have serious arguments □ □ □ □ □ k. People in my family often insult or yell at each other □ □ □ □ □

Each student's responses to the attitude scale items are then scored taking into account his or her

responses to the vignette.

Further examples of vignettes that have been developed for the ACWP are given in Attachment 1:

Appendix A.

10.1 Online implementation considerations

Several pending and important considerations regarding the online implementation of the ACWP

questionnaire need to be kept in mind throughout the questionnaire development process as they

may introduce limitations and restrictions with regard to the interface, flow and content. A research


study imposes several considerations and constraints for the survey design, especially in regards to

data capture and subsequent analysis, compared to a commercial product.

In the online implementation, a balance has to be struck between engaging respondents, the intent

of survey items, survey administration and school Information and Communication Technology (ICT)

environments, data collection monitoring, data capture and data export, and intended analysis.

ACER's experience shows that online administration in the ICT environment of schools is challenging.

School ICT environments will most likely be less current, and not as easily updated as ICT environments

in the general population. They also frequently are less flexible in their systems due to centrally

imposed restrictions to the systems. Some of these issues are discussed in more detail below.

10.2 Online survey specification issues

Survey administration

Three web-based versions will need to be developed for years 4, 6 and 8 with shared and

unique items between the surveys.

Audio capabilities will be required to allow questions to be read out to young people if

required.

Various items types are required including Likert types, variations of drag-and-drop

word/image associations, ordered lists with images (i.e. ability to order response options),

selecting images (ability to select pre-coded ‘hot-spots’), access to survey structure

capabilities, progress bar, survey monitoring of respondents, possibility for respondents to

verbally record answers (year 4).

Delivery

Web-based delivery is required according to the supported browsers, software and

specified environments.

Provision of a software compatibility diagnostic page for initial troubleshooting with

schools during survey delivery of the Field Trial and Main Survey.

Computer based delivery by secure USB memory stick will need to be developed, in case of

school difficulties with access and administration.

A flexible delivery method will be required to allow respondents to log in and out of the

survey and complete the survey at their convenience, while saving respondent data.

The survey will need to be compatible with I-Pad delivery (e.g. items to be correctly sized

with rotated display (vertical/horizontal).

Administration

There will need to be the provision of a monitoring interface in the Assessment Master to

enable the client to monitor survey progress (e.g. not started, started, completed,

submitted) of individual respondents and to monitor aggregate progress by specified

sampling variables (e.g. school, state/territory, school sector).

The ability to import, generate and manage login credentials for respondents will be

required (e.g. student and school levels).

Differential access for respondents and schools: ability to restrict access to the survey and

invalidate log-in credentials after survey administration at the school level.


Reporting facilities

Use of standard reporting facilities at both the student school levels will be required. For

young people, a secure, generated PDF or survey link to report selected results. For

schools, reporting facilities should be able to produce aggregate, anonymous and select

results that are interactive by domain (e.g. questions about friends or school) or other

client specifications.

Data will need to be exported to specific format and coding specifications including data

format, identification and naming conventions of items, assets and response options, as

well as data capture and coding for not administered, invalid and missing data.

Support and maintenance service

Appropriate maintenance and support throughout the entire project is required.

The capabilities of children also need to be considered throughout the online interface development

process. For example the following aspects will require consideration:

Attention span

Literacy levels, particularly given the focus on disadvantaged groups which have been

shown to have reading levels of up to two years lower than their grade level

Age appropriate


Attachment 1: Appendix A – Vignettes in the ACWP

Family vignettes (Subdomain: Togetherness)

The options are ordered here from the highest to the lowest. In the assessment the options should be

randomly ordered.

1.1 Family conflict

The issue discussed is conflict management.

The underlying construct is the efficacy of conflict management and the inclusiveness of the solutions.

The X family fix their problems by talking to each other when everyone is calm. They try to find

solutions that everyone agrees with.

The X family get on well together.

The Y family fix their problems by having big arguments. Kim is in charge of the Y family. If the

argument goes on too long then Kim decides what to do. Everyone has to do what Kim says even if

they do not agree.

The Y family get on well together.

The Z family argue about the same problems every day. Everyone joins in shouting and yelling and

saying mean things to each other and nothing gets fixed.

The Z family get on well together.

1.2 Family management Year 8 only

The issue discussed is parental care and responsibility.

The underlying construct is the efficacy of parental management of the child.

X’s parents encourage her to take small risks. As she shows that she can be sensible they give her more

freedom. X always tells her parents the truth about what she is doing. She knows her parents will

always help her if she is in danger.

X’s parents help X to keep safe.

Y’s parents do not let her take any risks. They ban most things she wants to do to keep her safe. Y lies

to her parents about what she is doing. Her parents often do not know where she really is. Y cannot

ask her parents for help if she is in danger.

Y’s parents help Y to keep safe.

Z’s parents let Z do whatever he likes. They say he has to learn about risks himself. They never know

where Z is or what he is doing. They are too busy to help Z if he is in danger.

Z’s parents help Z to keep safe.

1.3A Opportunities for pro-social involvement

The issue discussed is engagement with parents.

The underlying construct is the nurturing quality of the relationship the child has with their parents in

relation to trust, inclusiveness and enjoyment.


This vignette is about inclusiveness in making and communicating decisions.

The X family have to make room for granny to come and live with them. The parents talk about how

to do this with everyone in the family before they make a decision.

The X family include everyone in important decisions.

The Y family have to make room for granny to come and live with them. The parents decide how to

do this and then tell everyone else what they have to do.

The Y family include everyone in important decisions.

The Z family have to make room for granny to come and live with them. The parents disagree. They

keep arguing and no-one in the family knows what is going to happen when granny arrives.

The Z family include everyone in important decisions.

1.3B Opportunities for pro-social involvement

The issue discussed is engagement with parents.

The underlying construct is the nurturing quality of the relationship the child has with their parents in

relation to trust, inclusiveness and enjoyment.

This vignette is about trust in relation to problems.

X’s parents always listen to X. They try to understand her point of view and encourage her to do things

that she wants to do.

X’s parents help her with her problems.

Y’s parents have clear ideas about how Y should behave and what Y should do. They only support Y if

he does what they want. They are not interested in what Y thinks.

Y’s parents help him with his problems.

Z’s parents do not think about Z much. They let Z do whatever he likes. Sometimes they get angry with

Z but the next time he does the same thing they think it is funny.

Z’s parents help Z with his problems.

1.4 Attachment to mother and father

ACER is not sure these constructs need vignettes. The underlying construct seems to simply be the

extent of closeness, sharing of feelings and enjoyment of company.

If vignettes are required, a possible example is included below.

X likes being with his mother. They both enjoy doing things together. X can talk to his mother about

anything.

X is close to his mother.

Y is never sure if his mother will be nice to him or angry with him. He has to be careful what he says

so she does not get cross. They have some good times and some bad times together.

Y is close to his mother.

Z tries to keep out of her mother’s way. Her mother is always cross. Z hardly tells her mother anything

because that just makes it worse. Z tries not to argue with her mother.

Z is close to her mother.


Attachment 2: Issues raised at the PSG, 24 September 2013 and subsequent PSG telephone conference, 7 November 2013 and research team's responses

The Project Steering Group members were presented with the draft selection of questions and input

was sought regarding:

1. Whether or not any major elements are missing from the questionnaire;

2. What might be dropped or given lesser importance given that too much material is currently

proposed;

3. Any other overarching considerations regarding the aims, implementation and conduct of the

survey.

The following section summarises the feedback obtained at the meeting, first in general and then by

domain.

Responses about how the comments and feedback have been addressed are provided under a heading

"response" after each point.

General comments and feedback

1. At a general level, questions were raised with regard to wellbeing indicators and that alignment

between what is asked and measured should be the focus. More specifically, how are indicators

defined and how do the questions provide the information needed to develop the indicators.

Response: Further work on this aspect is reflected in Table 2 of the Executive Summary.

2. The look and feel of the questionnaire will be very different with the online interface. It was agreed

that other than the meeting minutes, further feedback could hold off until this has been

developed, at least in part.

Response: Development of the online survey has proceeded and was illustrated to the PSG at the

special meeting on 7 November 2013.

3. It was agreed that the authoring of the questionnaire could proceed in line with what has been

proposed in this report, also with a view to the draft report of the cognitive labs of the

questionnaire which is due in November. At the end of October when at least one domain will

have been authored in the online tool, a PSG teleconference will be arranged and at that time

further feedback will be sought.

Response: Further feedback on the survey was sought at the meeting on 7 November 2013. Feed-

back received and corresponding responses are detailed below:

Demographics

Some reservations were expressed as to whether the question SES (number of books) is the best that

can be done to capture, presumably, educational attainment (and interest in education) of parents.

Response: There are several questions planned relating to family affluence and child deprivation, as

well as number of people with paid jobs in the respondent’s home. Number of books in the home is

still the best single measure of home possessions and is highly related to education of the parents,

behaviours conducive to learning such as reading and student achievement. Also, we do need

information that allows the identification of socio-economically disadvantaged children.


Success at school

It was queried whether the question may be difficult to answer by many students.

Response: The question asks students to rate their performance relative to the performance of their

classmates, as judged by their class teacher(s). From a very early age students can provide information

about performance of peers in their class both, absolutely in terms "who is best" but also relatively in

terms of how they perform in comparison to others. Also, the question has been used in the HBSC in

many countries and languages without any reported issues.

Aspirations

The question on this was in the October questionnaire but is missing from the current draft.

Response: This question was inadvertently dropped from the online demonstration and will be

inserted back into the Year 6/8 questionnaire. As regards whether or not this question might be too

complicated it should be noted that it has been used satisfactorily in Australia in TIMSS Year 8 level.

The field trial study will show through too many missing or empty response options if changes need

to be made to the question and/or its response options.

Bullying

There is the question of whether we need to know whether bullying is taking part in schools as well

as elsewhere. Bullying (at school or elsewhere) could be expected to lower wellbeing but bullying at

school would have a further negative impact through its effect on the school domain (i.e. the bullied

student not liking being at school). Further, knowing that bullying was taking place at school could

help explain having poor attitude to school and not being happy at school.

Response: We will insert an additional question as follow-up to the very first four statements seeking

information about the type of bullying, if any, a young person has experienced. If a young person

responds with "yes" to any of the statements, it will be followed up with the question "Has this

happened...a) at school; b) outside school; c) in both places?" We have also contacted Professor Donna

Ross to provide feedback on the current question set on bullying with a view of comparability with the

Australian Covert Bullying Prevalence Study.


At the teleconference a question was asked whether this topic had been discussed with the

researchers working on the Second Child and Adolescent Mental Health survey. Part of the reason was

because of not being sure whether the question picked up well enough some of the generally accepted

forms of poor mental health, such as constant anxiety, mood changes, and depression. George Patton

observed that the question, drawn from the HBSC, seemed fairly weak in this respect and suggested

checking what the literature had to say on how well the question worked for this purpose.

The other reason for suggesting making contact with the Mental Health survey researchers was to

ensure greater consistency between the ACWP results and those of the Mental Health survey.

Response: The HBSC symptom checklist" asks respondents about subjective health complaints that

are intended to measure two facets, namely psychological wellbeing (d, e, f) and somatic wellbeing (a,

b, c, g, h)of an underlying trait of psychosomatic complaints. HBSC reports that the scale is flexible

and that analyses are meaningful at single- item and at sum score level. Qualitative semi-structured

interviews with early adolescents revealed that adolescents perceive the symptoms to be adverse

physical and psychological states that interfere with daily functional ability and wellbeing. Moderate

to high correlations have been found between the HBSC symptom checklist and scales measuring

psychological wellbeing (r=-.47), depressive moods (r=.53) and physical wellbeing (r=.42) in data the

Kidscreen study in 12 European countries. It should be possible to carry out analogous (if not directly

comparable) studies using the ACWP survey.


International comparisons

There was a comment that some variables or indicators will be able to be compared with the HBSC or

Children’s World or other surveys. These comparisons will be able to be done often not for all three

year levels (i.e. Year 4, 6, 8) but for the particular age group that constituted the target population in

that survey. It may also be possible to compare SES internationally with the HBSC – both surveys will

contain identical questions on family affluence. This should allow international comparisons of social

gradients, particularly of subjective wellbeing and health indicators. However, it may be difficult to,

say, cross classify one indicator with another (school achievement with health, for example) and

compare them internationally, because there will not be any survey that will have the same way of

measuring both indicators as the ACWP. So, many international comparisons of interest will be

somewhat approximate.

In developing the ACWP one of three options were available:

have a really good instrument that meets Australia’s needs but is only partially

internationally comparable; or

have an instrument that is internationally comparable and meets Australia’s needs only

partially;

reach a working compromise between the two – which is where the ACWP is heading.

Response: As noted above, it is intended that some international comparisons can be carried out using

bivariate or multivariate analysis. One example is health symptoms by SES. Other international

comparative analyses will be possible at a descriptive, univariate level for particular ages or grades.

These comparisons, however, will be accurate for those topics and age groups.

However, it is important to bear in mind that the ACWP is a national rather than an international

survey where the instruments are developed jointly by participating countries and/or experts for

administration of the same, albeit translated, instruments in all participating countries. In those

instances, international comparisons on all variables and combinations of variables/scales are

theoretically possible - although not necessarily valid or appropriate.

The ACWP has been developed from the "bottom up" by starting with the views and thoughts about

wellbeing of young people who live in Australia, with a particular emphasis on young people who have

experienced some disadvantage. In this way, domains and issues for the survey were identified. Then,

wherever possible, questions that addressed these domains and issues for the ACWP were selected

from existing surveys to enable international comparisons on these aspects as Australia has not

participated so far in international surveys on the topic. As a consequence, the ACWP survey reflects

this combination in line with its overarching aim:




4. Full results of the cognitive labs will be presented to the PSG at the next official meeting in March

2014.

Response: In the interim, a preliminary summary of cognitive interviews is given below.

At the time of writing this report, three sets of cognitive interviews have been recently undertaken

with children from the groups of disadvantage of interest to the survey, specifically: Aboriginal and

Torres Strait Islander children, children from culturally and linguistically diverse backgrounds and

children from low-socioeconomic backgrounds. Details of these cognitive interviews are as follows:

1. Regional South Australia: 8 young people aged 9 to 14


2. Sydney: 6 young people aged 12-14; 3 young people aged 9-10 provided feedback regarding the

online interface

3. Adelaide: 7 children, aged 9-14

The overall aims of the cognitive interviews were:

1. To probe intended respondents' comprehension and responses to specified items of concern

or potential difficulty with the questionnaire design, which included:

Individual response options

Response scales

Clarity of instructions

Survey assumptions (e.g. about family structure)

New drag and drop items

Survey length and item relevance (e.g. with an aim to reduce survey length)

2. To detect unforeseen problems with questionnaire design by examining for intended

respondents:

Comprehension of items

Response processes

Fatigue

Behaviour

To address the first aim, explicit questions were developed for each of the three cognitive interview

locations, with slightly different versions for each age group (Years 4, 6 and 8). Administered questions

included all items from: the Family domain; demographics; overall wellbeing; socioeconomic

deprivation; disability; interactive item types (i.e. drag and drop) for ‘closeness of relationships’ and

the ‘importance of survey domains for wellbeing’. Vignettes were also administered, which have now

been excluded from the field trial survey questionnaire.

To address the second aim of cognitive interviewing, interviewers were asked to follow up on

behaviour they observed during the administration of the survey to young people such as hesitation,

skipping of answers, apparently responding too quickly etc. Interviewers followed up by using reactive

verbal probes, for example, "In your own words, can you tell me what this question is asking?” "What

do you have to do to answer this question?"

Cognitive interviews with children in Regional South Australia were administered using paper-based

surveys and with an earlier version of the draft survey. Cognitive interviews were administered online

to children in Sydney and Adelaide as the online survey delivery has been further developed. Therefore

a core set of issues were targeted across the three interviews, with additional issues targeted for

interviews in Sydney and Adelaide with the inclusion of the new item types and online administration.

Results from the Regional South Australia cognitive interviews are reported below. Results from

Sydney and Adelaide will soon be integrated, and presented in the report to the PSG meeting in March

2014. This report will present the integrated findings for the following issues:

How easy or difficult they found the question on family composition and number of adults

having a paid job.

Whether or not young people liked the new drag and drop item formats used in the "me"

item about who is close to them and the prioritising of well-being domains using a set of

shelves.

To seek suggestions for the possible reduction of statements under the SES/Deprivation

scale.

To obtain feedback on the design and features of the online survey template.


During cognitive interviews in Regional South Australia, young people were asked explicit questions

about the following topics while they worked through the survey:

1. Whether young people found the use of different response options for different questions

confusing.

Participants in the 13-14 age group reported that using the ‘YES! yes no NO! This doesn’t apply to me’

response scale to choose an answer for items was much more difficult than using the other three

response scales. Participants reported that they would have liked a ‘neutral’ option between yes and

no, and placed a tick between these two response categories. Interestingly, participants explained

that it was easier to select an answer to the vignettes using the same response scale in comparison to

items about their own family, even though the vignettes did not include a ‘This doesn’t apply to me

option’.

The other three response scales are also 4 point scales with no neutral option. These results may

suggest that participants’ difficulty in using the ‘YES! yes no NO! This doesn’t apply to me’ scale may

be associated with the response options of ‘YES!’ and ‘NO!’. These end-points on the scale may be

perceived by participants as more extreme than endpoints on the other scales: ‘Not at all – A lot’, ‘Not

at all true – Very much true’ and ‘Strongly disagree – Strongly agree’. Furthermore, these results may

suggest that it is more difficult for participants to select answers for items about their own family,

rather than vignettes or items about their school because family items may be of a more sensitive

nature.

Participants in the 9-12 age group were not consistently administered the Family items due to some

concerns about possible sensitivities for particular children. Overall, no issues or difficulties were

reported between response scales for this group for children who did complete some of the Family

items.

2. In the family domain, on the statements regarding vulnerability and harm, whether or not such

statements were appropriate and whom they were thinking about when answering these

questions. This was done as the question had been changed from the original format in which

the child was asked about a specific person (e.g. mum or dad) to "someone in your family".

Also, concerns had been expressed regarding whether some of the statements (e.g. worry that

someone dies or gets arrested) may be too sensitive to administer in a survey.

Participants in both groups reported that they thought about ‘family’ in a general or very inclusive

manner. This means that children reported not to think about one specific member of their family for

these items, or about a specific family member for individual items (e.g. Dad about getting sick,

or brother about getting hurt). These results may be related to the administration, and sample. Most

of the young people who participated in the cognitive interviews were extended family members, and

reported that they thought of their ‘family’ as ‘everyone in the room’, or all young people, parents

and community workers present at the cultural centre. Furthermore, Aboriginal young people in

general may have a more inclusive conceptualisation of family to broadly include all extended family

and community members.

3. As regards the vignettes, to check comprehension of the instructions and for any order

violations.

Participants in the 9-12 age group reported that the vignettes were confusing for them to initially

understand but that they were familiar with other item formats due to standardised testing. The

vignettes ask participants to rate how much they agree with a statement, after reading brief

hypothetical scenarios about students/families/teachers. However, other items also ask participants

to rate how much they agree or disagree with listed statements. These results suggest that perhaps

younger students may benefit from a practice vignette at the start of the online survey, to better

understand the intent of the item type. Participants reported that they were trying to find the ‘right

or correct student/family/teacher’ from among the presented options. This also suggests that for


younger students, perhaps the differences between the ‘high’ and ‘low’ options are too stark, and the

vignettes should be revised to reduce the difference between the extreme options.

Comments and feedback by domain

Family

1. The concept of family was discussed and it was apparent that it is very difficult to define and

measure. Family has a multitude of different meanings for different people and it would be

difficult for any question to capture all of the combinations and permutations of the composition

of the family.

Response: The pictorial item of "Who is part of your family" and the question "When you think

about your family" have been removed.

2. The ultimate goal is to know if children feel safe in the home regardless of who is in the home.

Response: This aspect is picked up by some items in the "Worry - Vulnerability and Harmful"

subdomain which ask about whether young people worry someone in the family who will get

hurt, arrested, involved in a fight.

3. This diversity and complexity suggests that the initial question should be rethought to reflect this.

Response: We have carefully gone through all questions in the family domain with this in mind.

Together with the information obtained from the self-demographics question will go some way

to covering the diversity and complexities of family composition.

4. As suggested in the draft field trial questionnaire for the scales of worry, vulnerability and

harmful, it is desirable to hyperlink whoever the student identifies as being closest to in the

family, to that person appearing in subsequent questions that refer to a specific family member.

Careful consideration has to be given as to which scales should be customised in this way, as it

would affect comparability for questions where comparisons with existing data sets are planned.

Response: From discussion and further reflection it was considered more important to ascertain

whether young people feel close to an adult rather than ascertaining who this adult is. The

question stem of the "Worry - Vulnerability and Harmful" therefore does not ask about a specific

family member. In the same vein, the "Me" item which asks young people to "drag & drop" people

or pets as close as they feel to themselves now asks about adults other than (step-,

foster)mother/father rather than specifically, for example, about teachers, aunties, uncles, or

neighbours.

5. An alternative could be to add a preamble explaining that young people should think about their

family and whatever and whoever that is, and respond to questions with reference to this.

Response: Please see previous response.

6. Children are quite responsive to what is considered as ‘normal’ family and so may respond to

nuclear ideals/references. If we introduce different/alternative concepts of ‘family’, if may

confuse them. However field trial can test how this goes.

Response: We have taken out the potentially confusing "Who is in your family" questions.

7. For questions that refer to a specific family member (e.g. mother/father in worry scale),

suggestion was to take out definition/name and ask about ‘family’ as children know what this is.

This would allow them to refer back to their own definition of family rather than anything

prescribed by the question.

Response: We have taken up this suggestion.


8. The group also identified challenges associated with definition of ‘family’ in terms of ‘living with’

– physical proximity and safety or ‘relationship with’, sense of belonging and closeness.

Response: The "Me" item (see response above) is aimed at obtaining information about whether

young people feel close to someone.

9. Difficulties in interpreting those people who the young person does not select as being part of

their family – e.g. do they have an aunt and not consider them as family or do they not have an

aunt. Is this even an issue?

Response: As per above, the essential aspect is that young people feel that someone is close to

them, not specifically who this person is.

10. Layout of the question will need to be considered so that it does not bias results.

Response: Layout considerations are pervasive in the online design and relate also to the

template design and question length. For example, every effort will be made to avoid

respondents having to "scroll down" a screen to answer questions as this has been shown to

increase missing data.

11. For the question: “These questions ask you about your family. When we ask about your mother

and father we want you to think about whom you live with most of the time. This includes step-

parents, foster parents or guardians.” It was suggested that item g onwards resonates more with

what came out of phase 1. Maybe items a-f are less important than the other items.

Response: The family questions have been revised to be as inclusive as possible while still be

manageable on the screen and being able to be understood by young people.

12. It was suggested that the worry scale had too many items and could perhaps be reduced,

especially after the field trial and perhaps more so for the younger groups. It was suggested that

a question on anxiety might be more relevant/pertinent than the worry scale.

Response: This scale has been reduced from 18 to 12 items. The intended factor analysis of the

field trial survey data is likely to reduce this number further.

13. It was also suggested that the items could be shorted by simply asking participants about

someone in their family rather than specifically about their mother and father. The name could

also be replaced with the person identified as the ‘closest’ family member.

Response: Please see previous response about the changes in the reference points of, for

example, the "Worry - Vulnerability and Harmful" scale.

14. It was also suggested to ask about parental/or family involvement in school.

Response: The following new question has been added to capture this aspect:

How often do the following things happen?

(Please tick one box in each row.) Every day or almost every day


Once or twice a month


My parents ask me what I am learning in school. □ □ □ □ My parents make sure that I set aside time for my homework □ □ □ □ My parents talk to my teacher □ □ □ □

These questions have been administered in PIRLS and TIMSS at both Grades 4 and 8 so will enable

comparisons with the countries participating in those studies.

15. Clarity is required in terms of output aims, and how this translates into policy and intervention.


Friends

1. It was highlighted by the project team that the aim of these questions is to capture factual

number and contact with friends, and then the student’s perspective on quality of those

friendships.

2. The question was raised around the importance of the volatility of friendship and how this might

be captured in the survey. It was suggested that this does matter and would expect those young

people experiencing this at the time of the survey to feel alone. One way around this may be to

ask about quality with reference to the previous year, but this may not be appropriate.

3. Friends was discussed as a source of both bullying and support.

4. It was suggested that the questions on friendships should aim to capture: how well supported

children feel; reciprocity in friendships; and resilience of friendships (ability to ‘make up’)

Response to the above four points: The following statements are aimed at capturing support by

(first four items) and conflict with friends (last five items). A specific time span has not been

provided in the stem as it does not seem appropriate:

For the following statements, please think about your closest friend:

1 = never or hardly ever

2 = 3 = 4 = 5 = always or

almost always

I spend fun time with this person □ □ □ □ □ I share private thoughts and feelings with this person

□ □ □ □ □

I depend on this person for help, advice, and support

□ □ □ □ □

This person sticks up for me □ □ □ □ □ I get into fights with my friend □ □ □ □ □ My friend and I make up after a fight (Appears if answer to previous question is 2, 3, 4, or 5)

□ □ □ □ □

My friend bugs me or annoys me even though I ask him/her not to

□ □ □ □ □

My friend and I argue a lot □ □ □ □ □ My friend and I disagree about many things □ □ □ □ □

5. The question on how often you have contact with your friends seems unnecessarily complex and

has a high reading load. These questions may need to be rethought. What information do we

need to capture about contact with friends?

Response: Upon further discussion it has been decided to drop the questions about time spent

with friends and frequency of talking to friends. The questions in their current form would not

provide the fine-grained information required to differentiate different qualities of friendship.

Therefore, as the questionnaire has to be reduced in length, it was decided to remove these

questions.

6. The question about conflict with friends is aimed at understanding the young person’s own sense

of their social skills and how this works at different ages. The intent is to know if young people

have the capacity to connect with their peers and if not this would provide space for

emotional/policy support to encourage such connections. The suggestion was to change ‘close

friend’ to ‘closest friend’ and to perhaps adjust the labels of the response options.

Response: The question stem has been changed to "closest friend".

7. The PSG had some discussion around social media and facebook friends which may be an

important area to keep an eye on. As the survey is using existing questions, it may miss this aspect

of friendship.


Response: Again, to get specific and differentiating information on this aspect of communication

with friends and impact on young people's feelings regarding their friends would require a suite

of detailed questions that is beyond the scope of the ACWP survey.

School

1. Some suggestions were offered to include more reasons for truancy rather than just the

frequency, for example lack of food.

Response: While reasons for truancy might be interesting they are considered not sufficiently

central to the ACWP survey. Questions about being hungry themselves or worrying about

someone else being hungry as well about having breakfast are included in the health and family

domains.

2. Comparisons with academic achievement are of high policy relevance. Some potential issues with

the use of NAPLAN data for this purpose include that, according to a member’s prior meetings

with jurisdictional bodies, in general parental permission would be required. Some states appear

to be stricter on permissions with using these data than others.

3. It was suggested that linkage could be longitudinal rather than at an individual level so that

permissions are not required.

4. It was suggested that student’s self-assessment of their achievement is often close to their actual

achievement and as such this question could be asked.

5. The Chair asked if the PSG group could help in terms of using NAPLAN data and there was

potential for such support.

Response to points 2 to 5 above: The proposed question on success at school which combines

information from students about their performance relative to their class mates as judged by

their teacher with school level NAPLAN information will result in a 20-point scale that is

considered to have sufficient variance to enable meaningful analyses.

The discussion about individual level NAPLAN data is complex as it involves ethics applications

and parental consent, jurisdictional support and questions of data matching and confidentiality.

Also, the trade-off needs to be considered between the advantage of having NAPLAN data taking

into account its error margins at the individual student level compared with the potential adverse

effects on response rates from having to:

a) obtain explicit parental consent to using NAPLAN data, and

b) ask young people questions such as name and date of birth that can uniquely identify them

for subsequent matching of NAPLAN data. (At a Project Steering Group meeting on 14 April

2014, it was agreed not incorporate NAPLAN data into the survey.)

6. It was also suggested that a question about student’s perceptions of their parental expectations

about their school success be added to the survey.

Response: Parental expectations are reflected in the young person's educational aspirations

about which a question is included in the survey (see also response regarding aspirations further

below).

Community/Neighbourhood

1. Children’s concepts are varied for this domain and therefore broad questions should be aimed

for on learning culture and connection to environment, etc.

2. It might also be desirable to include something to do with sense of civic contribution.


3. Connectedness to the local area and population, access to facilities, and perceptions of safety are

often the main focus of research according to the literature.

4. It was agreed that community/neighbourhood is a difficult construct that required lots of

prompting (in the qualitative work) and that it would be difficult to interpret anything with such

varied definitions. This idea of community/neighbourhood having a different meaning for

different people means that responses could not be properly compared.

5. It was suggested that perhaps questions about the ‘neighbourhood’ would be the best way

forward. For example, ‘Can I go outside and ride my bike?’ ‘Is there a park?’

6. Ideas around safety and belonging seem most relevant. There was discussion about the

difference between a student saying ‘I feel safe’ versus ‘it is safe’. This still needs to be considered

as to how ‘safety’ is experienced in different target groups.

7. The questionnaire for the field trial is unlikely to include a definition of community. However, a

definition could be the focus in the next round of qualitative work.

8. The idea that ‘community’ is potentially very important for the culturally and linguistically diverse

young people group was discussed. The PSG agreed that this could be an area for qualitative

engagement with this participant group as the issue could be too complex to capture in a large

scale survey. It was suggested to explore social and cultural geography literature – ‘space and

place’.

Response: The following questions have been added to the survey to capture resources and

safety in the neighbourhood:

Strongly disagree


or agree Agree

Strongly agree

There are places for me to go in my area □ □ □ □ □ There is nothing to do in my area □ □ □ □ □ There are lots of fun things to do where I live □ □ □ □ □ I feel safe when I am out in my local area during the day

□ □ □ □ □


□ □ □ □ □

I have plenty of freedom in the area I live in □ □ □ □ □

It was agreed that more in-depth aspects of community/neighbourhood and their importance for

wellbeing would be pursued further in the qualitative work in Phase 4 that accompanies the

national survey.

Health

1. It was explained that a cross-analysis with the worry scale could be done with the health items.

2. There was some discussion around the importance of puberty and that this was an important

factor for wellbeing for young people in year 8 It was recommended that a question on puberty

to be included in the year 8 questionnaire version.

Response: The following questions have been added to the survey:


The next questions are about changes that may be happening to your body. These changes normally happen to different young people at different ages. If you do not understand a question or do not know the answer, just mark, "I don't know".

(Please tick one box in each row.) Not yet started

Barely started

Definitely started

Seems complete

I don't know

Would you say that your growth in height.. □ □ □ □ □ And how about the growth of your body hair? ("Body hair" means hair any place other than your head, such as under your arms. Would you say that your body hair growth....

□ □ □ □ □

Have you noticed any skin changes, especially pimples?

□ □ □ □ □

(For boys only): Not yet started

Barely started

Definitely started

Seems complete

I don't know

Have you noticed a deepening of your voice? □ □ □ □ □ Have you begun to grow hair on your face? □ □ □ □ □ (For girls only):

Yes No I don't know

Have you begun to menstruate? □ □ □ If yes, how old were you when you started to menstruate?

(please specify)

The questions have been showed to be satisfactorily related to the Tanner pictorial scale

(Carscadon et al. 1993). To facilitate ethics approval and parental consent, it was decided to use

the word questions rather than pictorial scale's which is quite explicit in terms of the genitalia

that are depicted.

3. It was queried whether the question on health symptoms had been validated with other

instruments, or questions relating to mood?

Response: This question has been drawn from the HBSC where it is used as a non-clinical

measure of mental and physical health. The HBSC symptom checklist" asks respondents about

subjective health complaints that are intended to measure two facets, namely psychological

wellbeing (d, e, f) and somatic wellbeing (a, b, c, g, h) of an underlying trait of psychosomatic

complaints. HBSC reports that the scale is flexible and that analyses are meaningful at single –

item and at sum score level. Qualitative semi-structured interviews with early adolescents

revealed that adolescents perceive the symptoms to be adverse physical and psychological

states that interfere with daily functional ability and wellbeing. Moderate to high correlations

have been found between the HBSC symptom checklist and scales measuring psychological

wellbeing (r=-.47), depressive moods (r=.53) and physical wellbeing (r=.42) in data the Kidscreen

study in 12 European countries.

4. It was emphasized that one important issue of validity for any survey question or scale was its

predictive validity. This might be particularly important for (mental) health questions.

Response: Predictive validity is one check of the validity of a construct that, of course, is

dependent on what the construct is meant to predict. While some analyses of this type may be

possible, the main purpose of the ACWP survey is to arrive at different profiles of wellbeing of

young people in Australia.


1. It was agreed that the aim of questions about money and material things was to capture both

notions of material wellbeing and socio-economic status.


2. It was pointed out that two questions on deprivation (‘my family has enough money to put petrol

in the car’ and ‘my family has enough money for me to go on school camp’) are not well phrased

as they do not directly ask about deprivation.

Response: The SES deprivation scale was included in the cognitive interviews and young people

were asked about the least relevant items, also with a view of potentially reducing the number

of items that form the scale. Intended analyses of this scale will provide further information as to

how well the items work together to reflect the underlying construct.

3. It was reported by the research team that money was discussed by young people more in terms

or activities and options, not just ownership of ‘things', and the ability to use them.

Aspirations

1. The question about expected education seems different to what was seen as important from

phase 1. Aspiration for future might not be career related e.g. relationships.

2. It was suggested that career is often identified as the highest priority, after relationships.

3. There was a suggestion to add a question on parental education. However, there are often lots

of missing data with this question. There was some discussion around how else this information

could be obtained as it is an important policy question. For example, data could be sourced from

the MySchool website or the possibility that ABS could provide the address of where people live

(however this has implications for anonymity issues). The question about how many people in

the home that have paid jobs might be enough. There was a suggestion to add a question to the

parental consent form. This is probably not feasible for the field-trial as ethics approval has

already been granted and the project timeline has to be kept.

4. It was suggested that an extra response option (I don’t know) be added to avoid missing data.

However it was reiterated that in the past this response option has not worked too well due to

missing data. Also, the addition of any response option will change the frequencies and invalidate

comparisons with any existing data sets.

Response to points 1-4 about aspirations: Expected education has been shown to be highly

correlated with parental education. Hence, given the problems with missing data on parental

education, this question is likely to give some approximate information on this issue. As regards

aspirations about careers, this has been asked in other surveys but often in an open-ended format

which requires coding and then often does not show expected relationships (e.g. greater interest

in science as a school subject related to interest in science as a career). Hence, the amount of

effort in collecting and processing the data is unlikely to be warranted.

As regards aspirations concerning relationships, subsequent discussions indicated that this would

be too complex an area to do it justice in a survey that has to be completed by respondents in 20

or 30 minutes. to cover this topic appropriately, questions would need to ask about aspirations

concerning integration into the community, peer connections, general cooperation with others

as well as empathy.

Bullying

1. It was suggested that the definition of bullying is that it is repeated and so frequency response

option might be better.

2. It was highlighted that fighting between equals is not bullying. Follow up work through

consultation with bullying experts (e.g. Australian human rights commission or Donna Cross

(Edith Cowan University) or Ken Rigby (Adelaide University) will be undertaken so that we can ask

standard items on this. The general feeling among the PSG was that more justification is required.


Response: Please see earlier responses to issues raised about the questions on bullying. The

process of seeking feed-back on the proposed questions is currently occurring.

Feeling good

1. It was explained that this could be something with which comparisons be made with other

Australian and International surveys. Other surveys see this as the main dependant variable,

however its inclusion does not necessarily have to be a main focus of the ACWP.

2. The PSG were surprised by the response scales for the primary version of the SEW and did not

see any obvious validity of using this scale compared with other measures or scales of wellbeing.

It was agreed that further review of existing mental health/feeling good scales would be

undertaken. Part of this further work is reflected in the corresponding section of the "cross-

cutting domains" above.

Response: It is agreed that the survey should include an overall measure of perceived wellbeing

that can be used as an outcome variable for analysis if needed. This question needs be asked at

the beginning of the survey before any domain-specific questions to avoid the general question

to be influenced by responses to more specific questions and the "sensitizing" due to having

completed a 20-30 minute survey on the topic. The following statements taken from the

Children's Worlds were considered to best reflect the overall aspects of wellbeing discussed

during the qualitative work in Phase 1 of the ACWP.



Disagree Neither

agree nor disagree

Agree Strongly

Agree Don't know



I wish I had a different kind of life □ □ □ □ □ □



Anchoring vignettes

1. The discussion around the anchoring vignettes suggested that further explanations were required

about the purpose of these tools aimed at adjusting for differences in response styles between

individuals and groups.

Response: Further information on anchoring vignettes has been given above and in Appendix A.

The cognitive interviews will include the vignettes to further investigate the trade off between

obtaining information about individual response behaviour that can be used subsequently to

adjust for difference in response behaviour and the amount of reading and cognitive load

involved.


Attachment 3: The Australian Child Wellbeing Project - Cognitive Interviews. Full report to the Project Steering Group

ACER: Petra Lietz, Mollie Tobin

Flinders University: Helen Popple

University of New South Wales: Tammy Burnstock, Kirk Dodd, Bridget Jenkins, Jen Skattebol

14 April 2014

The purpose of the Australian Child Wellbeing Project is to further our understanding of how young

people in general and disadvantaged young people in particular, understand their own wellbeing.

Particular attention is given to understanding the perspectives of young people in six groups who are

often seen as experiencing high levels of marginalisation or as having particular experiences and

needs: Aboriginal young people, culturally and linguistically diverse young people, young people living

with disability, young people living in regional and remote Australia, economically disadvantaged

young people, and young people living in out of home care.

Phase 1 of the project involved a range of qualitative methods to examine wellbeing from the

perspectives of mainstream and disadvantaged young people, aged 9-14, and to better understand

possible differences between groups. The findings from Phase 1 of the project have been used to

inform the development of the wellbeing survey in Phase 2 of the project. (Please see report2 for

more information about the findings from Phase 1).

This report, as part of Phase 2, presents findings from cognitive interviews conducted with young

people from three of the six groups of interest to this project: Aboriginal young people, culturally and

linguistically diverse young people, and young people from economically disadvantaged backgrounds.

Findings from the cognitive interviews informed, in part, development of survey items and

considerations for online administration of the survey.

Rationale

There are many sources of potential error in survey data. Cognitive interviews specifically help to

evaluate surveys by examining if participants understand and respond to survey items as intended by

researchers, and are recommended to precede any survey implementation (Drennan, 2003; Jobe &

Mingay, 1989; Willis, 2005).

Survey design rules cannot address all issues that are associated with error, particularly survey

response error (Willis, 2005). Therefore, cognitive interviews are used to address potential issues

related to survey response error, and to improve the reliability and validity of large-scale survey data,

and recommendations used to inform policymaking (Desimone & Le Floch, 2004). In addition,

cognitive interviewing is important for cross-national surveys (Wildy & Clarke, 2009), and for surveys

applied across cultural groups (Solano-Flores & Li, 2009).

2 Skattebol, J, et al. (2013) Understanding children’s perspectives on wellbeing: Phase One Report for the Australian Child Wellbeing Project, Flinders University, the University of NSW and the Australian Council for Educational Research. www.australianchildwellbeing.com.au


The ACWP survey intends to make international comparisons regarding young people’s wellbeing, and

to be used across a range of culturally and linguistically diverse young people within Australia.

Examining how young people interpret and respond to the survey items will improve the survey’s

measurement of a wellbeing that is rooted in young people’s experiences and understanding.

Aims and design

In Phase 2, the principal aims of the cognitive interviews were:

1. To probe intended participants' comprehension and responses to the wording and format of

questions and responses;

2. To detect unforeseen problems with questionnaire design and administration with the target

sample by examining participants’:

Comprehension of items

Response processes

Fatigue

Behaviour

Table A3.1 provides a summary of the major domains, and associated sub-domains that were piloted

in the cognitive interviews. As the project team was simultaneously working towards a draft ACWP

survey, not all ACWP survey questions were cognitively tested. Rather, a sample of identified items

that posed potential difficulties for respondents were cognitively tested.

In summary, the survey piloted questions and scales in the ‘Family’, ‘Money and material wellbeing’,

‘Self-demographics’ and ‘Cross-cutting’ domains. . The survey also piloted anchoring vignettes.

Anchoring vignettes provide a frame of reference for the respondent's answers to subsequent items.

The main aim of using anchoring vignettes is to adjust for differences in response styles as some

participants are more inclined to agree than others, some are more inclined to use extreme responses

or sit in the middle between different response alternatives. In the pilot survey for the cognitive

interviews, vignettes were presented before items and scales associated with the sub-domains.

Table A3.1: Cognitive interview survey domains and indicators

Major domain Subdomain/Correlates Indicator (Preliminary label)

Family

Factual/correlates

Organisation of the household(s)

Paid job

Family Posessions – Books

Changed carers

Out of home care

Care for family member

Togetherness

Family cohesion

Family conflict

Family management

Worry Vulnerability - Significant other

Harmful - Significant other

Money and material wellbeing Factual/correlates Socio-economic status Deprivation

Self-demographics Factual/correlates

Language background

Gender

Aboriginal and Torres Strait Islander

Disability

Cross-cutting Feeling good

Overall wellbeing

Importance of domains for wellbeing

Closeness of relationships Closeness/Proximity of Relationships


To address the first principal aim of the cognitive interviews: ‘to probe intended participants'

comprehension and responses to the wording and format of questions and responses’, specific verbal

probes (proactive) were included in the interview schedule to follow-up with the participant regarding

identified issues with questions or questionnaire design. The interviewer sat alongside the participant

while the participant completed the online survey and followed-up with the participant regarding

identified issues by using the verbal probes in the interview schedule. The interviewer recorded the

relevant information in the interview schedule.

Proactive verbal probes were included in the cognitive interviews schedules to examine respondent

responses and potential difficulty with the following:

1. Response options of:

2. Don’t know

3. Not at all – A lot

4. Strongly disagree – Strongly Agree

5. Not at all – Every day

6. YES! yes no NO! This doesn’t apply to me

7. Interactive item formats: ‘Importance of domains’ (Bookshelf); ‘Closeness of relationships’

(drag and drop circle)

8. Question instructions: ‘Organisation of the household’; Vignettes

9. Question assumptions for ‘Vulnerability and harm’ question; ‘SES Deprivation’ question;

‘Family closeness, conflict and management’ question.

Complete cognitive interview schedules for years 4, 6, and 8 are included as appendices, and are

presented in a paper-based format, and as such, are not presented in an online format. Interviewers

used additional probes (reactive) to elicit further information from the participants if there were

unanticipated problems with questionnaire design by observing respondents during the cognitive

interview. For example, participants may hesitate, delete answers, ask questions, fill out the online

survey too quickly or skip items. Table A3.2 below presents potential sources of survey response error

and example reactive probes.

Table A3.2: Unforseen sources of error and example reactive probes

Source of error Probe

Survey instructions (unclear, conflicting)

Can you tell me what these instructions are asking you to do?

Clarity of items (general phrasing, specific words, multiple interpretations)

Can you tell me in your own words what this question is asking? What does the word ‘X’ mean to you in the question?

Assumptions (survey items have an assumed logic which may not hold true for participants)

How well does this question apply to you?

Participant knowledge (participants may not remember or have the appropriate knowledge to answer questions)

How did you come up with your answer to this question? How easy or difficult was it to answer this question?

Sensitivity (items may be of a sensitive nature for participants)

Is it okay to talk about ‘X’ in a survey? This question uses the word ‘X’, does that sound okay to you?

Response categories (may be unclear, overlap, missing illogical)

How easy or hard was it to choose an answer? Why did you choose this answer instead of a different answer?

Source: Adapted from Willis, G.B. (2005). Cognitive interviewing: A tool for improving questionnaire design. Thousand Oaks, CA: Sage Publications Inc.


Administration

Cognitive interview schedules were prepared to trial several factual questions and attitudinal scales

that had been developed for the draft ACWP survey with young people in years 4, 6 and 8. However,

age criteria were flexible in administration to accommodate for a wider range of possible participants

in the cognitive interviews, between the ages of 8 and 15 years. Table A3.3 below presents our

cognitive interview sample by year level and sub-group.

In total, 22 cognitive interviews were conducted across three sub-groups of interest for this survey. In

several instances, cognitive interviews were conducted as a group to accommodate for the unique

administration in community centres, and to accommodate possible sensitivities concerning the

nature of some survey items for participants. Asterisks have been used within cells to note instances

of group administration. For Aboriginal and Torres Strait Islander participants, years 4 and 6 were

combined and were administered as a group interviews due to the sensitive nature of some questions.

For participants in the culturally and linguistically diverse sub-group, focus group interviews were

conducted with year 4 students to focus on the unique online administration of the survey and solicit

their feedback about the online survey design. Specifically:

Can the children easily navigate the survey (next button; progress bar)?

Do the children like the template display (colours, graphics)?

Explain the planned features for the template (audio recording and a cartoon?)

What else could make the survey fun for other children to complete?

Table A3.3: Cognitive interviews sample by sub-group and year level

Year 4 Year 6 Year 8

ATSI 2* 3* 3

CALD 3* 2 4

Disability _ _ _

Low-SES 3 2 1

Out of Home Care _ _ _

Regional and remote _ _ _

TOTAL 8 7 7

The research team conducted preliminary cognitive interviews with Aboriginal young people using a

paper-based survey, as the survey had not yet been authored online. Later, online cognitive interviews

were conducted with culturally and linguistically diverse young people and young people from

economically disadvantaged backgrounds, using tablets that the research team provided.

Aboriginal and Torres Strait Islander sample

The cognitive interviews were conducted on 7 August 2013, with Aboriginal young people at an

Aboriginal cultural centre in South Australia. The Aboriginal community, including Elders, parents, and

community workers also participated in Phase 1 of the project and had developed a collaborative

relationship with the research team. Researchers de-briefed community members and children about

relevant results from Phase 1 of the project. Children who were present were offered the opportunity

to participate in the cognitive interviews as part of Phase 2. Children were given gift certificates after

completing the cognitive interviews.

A small group of approximately 8 young people in total participated in the cognitive interviews for

ages 9-12 (n=5) and ages 13-14 (n=3), some of whom were related (e.g. siblings, cousins). Due to the

nature of administration in a community centre, cognitive interviews were administered as groups.


Due to required literacy levels, children who were younger than 8 years old participated in an informal

group activity instead of the cognitive interviews. Parents and community workers also participated

in the group cognitive interviews to encourage child participation and for collaboration between the

Aboriginal cultural centre and the research team. Cognitive interviews took approximately 30 minutes

with each group. Researchers recorded participant feedback for issues outlined in the interview

schedule, as well as any other issues associated with survey design and administration.

Culturally and linguistically diverse sample

The cognitive interviews were conducted with young people from culturally and linguistically diverse

backgrounds on 19 November 2013 at a community centre in western Sydney. The centre provides

services and develops strategies to meet the needs of address the needs of local migrants and focuses

on recent arrivals, emerging communities and special needs groups including humanitarian entrants,

refugees, women, youth, children, aged and the unemployed. Five researchers conducted cognitive

interviews were conducted with nine children between the ages of 9 -15. All of the children had been

involved in Phase 1 of the project, and were familiar with the study aims. Centre staff helped to

facilitate participation in the cognitive interviews and were present while parents dropped off their

children during introductions and snacks.

Participants in Year 4 participated in a focus group discussion about the online administration of the

survey, and interacted with the online survey on tablets to provide suggestions for researchers. The

rest of the cognitive interviews were conducted one-on-one with participants and researchers.

Children were given gift certificates after completing the cognitive interviews.

Economically disadvantaged sample

The cognitive interviews were conducted with young people from economically disadvantaged

backgrounds on 25 November 2013 at a community centre in a low-socioeconomic suburb in a

southern metropolitan area of South Australia. Three researchers conducted six cognitive interviews

and an additional young person completed the survey to get an estimate of time needed to complete

the survey. Young people from the centre had participated in Phase 1 of the project, and were familiar

with the study aims. Participants were de-briefed about the findings from Phase 1 of the study.

Participants were provided with snacks during the interviews and gift certificates after completing the

cognitive interviews.

Results

Results will be aggregated across participants and presented below under each question of the

cognitive interview schedule. In the case of shared questions across year levels 4, 6, and 8, results will

be aggregated across year levels. As cognitive interviews were conducted with the Aboriginal and

Torres Strait Islander sub-group with a paper-based survey, results will also be integrated and reported

when applicable.

In summary, results from the cognitive interviews helped to inform the following revisions for the

years 4, 6, and 8 surveys:

revise the format of the ‘Closeness of relationships’ questions for online administration;

change the display of the ‘Organisation of the household’ and ‘Paid job’ questions for all

years, by using a filter question and building in online conditionals to display only the

appropriate questions for each respondent’s living situation;

remove unclear items in the ‘Vulnerability and harm’ question;

remove vignettes and ‘Family closeness/conflict’ items for years 4 and 6


remove ‘Family conflict/closeness’ items for year 8, and only retain ‘Family management’

items;

remove irrelevant items in the ‘Material deprivation’ scale;

revise the question stem and items in the ‘Disability’ questions.

1. Gender – Years 4,6, and 8



□ Girl

□ Boy

No unanticipated issues were reported for this question.

2. ATSI - Years 4,6, and 8



No □

Yes, Aboriginal □

Yes, Torres Strait Islander □

Unanticipated issues: Two respondents, one from the CALD group (Year 6) and the other from

the low-SES group (Year 4), reported initial confusion with this question and did not understand

the terms used. However, as they had no knowledge of the terms, they assumed that the correct

answer must be ‘No’.

Implications for the survey: As this question intends to identify a sub-group of interest, Aboriginal

and Torres Strait Islander participants, participants who may be confused by the question stem,

non-Aboriginal and Torres Strait Islander students, most likely will not select ‘Yes’ and the stem

should not be confusing for Aboriginal and Torres Strait Islander young people.

3. Language – Year 4

How often do you speak English at home?


□ I always or almost always speak English at home.

□ I sometimes speak English and sometimes speak another language at home.

□ I never speak English at home.

Unanticipated issues: No unanticipated issues were reported for this question, except for one

participant from the low-SES sample, who selected ‘Sometimes’ as he was ‘learning Japanese in

school and practiced at home’. However, this should not be an issue for most participants as no

other non-CALD participant selected this response. This question should be able to identify CALD

participants.

4. Language – Years 6 and 8



□ Always

□ Almost always

□ Sometimes

□ Never


Unanticipated issues: Three participants from the CALD sample explained that the frequency of

speaking English at home depends on which family members they speak to (i.e. parents = less

often and siblings = more often) therefore they had some difficulty selecting between response

options that are not ‘Always’. However, they would be identified as CALD.

Implications for the survey: Results from TIMSS and PIRLS show that these questions are able to

map population statistics for migrant status relatively well, so they should not be changed.

5. Overall well-being – Years 4,6, and 8



Disagree Neither

agree nor disagree

Agree Strongly

Agree Don't know






Interviewer: [If the child selects the response option of ‘Don’t know’ to ascertain of they really

don’t know or are selecting a ‘soft’ option then]:

‘Why did you select that answer ‘Don’t know’?’

Results: Two respondents selected the ‘Don’t know’ option. One participant (Year 4) selected

‘Don’t know’ as he said that he ‘doesn’t know how is life is going’, and genuinely did not know.

The other respondent selected ‘Don’t know’ because he ‘does not get what he wants’, and

therefore should have used ‘Disagree’.

Implications for the survey: To keep the response option of ‘Don’t know’ as one child genuinely

was not able to answer the question. The research team should examine data from the pilot

should look at the frequency of this response option.

Unanticipated issues: One participant found the order of ‘Disagree to Agree’ to be confusing and

suggested for it to be switched. However, this was explicitly probed for later respondents and

was not a reported issue. Several participants were overwhelmed by the length of the question,

even though it appeared as only the fourth screen in the survey.

Implications for the survey: Consideration should be given on how to reduce the overall length

of the survey and questions with multiple items, as they visually overwhelm respondents.

6. Importance of domains for well-being – Years 4,6, and 8



Family

Friends

School


Health

Money/things I have

Interviewer: ‘Here is a question which asks you to respond other than by ticking a box. Do you

like it? Why (not)?


Results: Almost all respondents reported that they like this question, as it was more interactive,

and seemed to require less reading and writing. However, some respondents were initially

confused as they did not understand how many domains they had to place or where they could

be placed.

Implications for the survey: It seems that perhaps students’ initial confusion is related to not

reading the question stem or carefully reading the question stem. Hopefully when the audio

feature is present in the field trials, students can listen to the question stem, thereby alleviating

the burden on respondents to read a lot of text. The analysis of pilot data should indicate if there

is an issue with the question instructions even with the available audio feature.

Interviewer: ‘Are there enough shelves? Or would you have liked fewer or more shelves?’

Results: Respondents were satisfied with the number of shelves and ordered the domains over a

wide range of shelves.

7. Closeness of relationships – Years 4,6, and 8



How close are these people to you? Drag them into the circle, as close to you as you feel they are. If you don't know a person or you don't feel close to them, just leave it where it is.

Mother

Father

Stepfather

Stepmother

Foster mother

Foster father

Sister

Brother

Uncle

Aunt

Grandmother

Grandfather

Other adult

Other child

Pet

Mother's partner

Father's partner

Interviewer: ‘Here is another question which asks you to respond other than by ticking a box.

Do you like it? Why (not)?

Results: Most all respondents liked the drag and drop feature of this question and liked the

interactive elements of the questions. However one respondent found it difficult to drag and drop

on a tablet, and another student found there to be ‘too many options’. Many respondents

dragged people into the inner ‘Me!’ ring. A few respondents wanted to differentiate between

multiple ‘aunts and uncles’ or ‘brothers and sisters’.

Implications for the survey: In order to be inclusive, all of the respondent options will remain

until after an analysis of pilot data. Respondents will not be able to drag people in to the inner

‘Me!’ ring in order to retain the 5 point scale. In order to allow respondents more freedom to

differentiate between relationships with people from the same ‘category’ (e.g. multiple brothers

or sisters), respondents will be able to drag each option up to 5 times per ring.

Me!


Interviewer: ‘At the moment, the question asks you to leave a person who you don't feel close

to where it is. How would you feel if you were told to put any person or thing you don't feel close

to in the outer most circle?

Results: Some respondents were confused about the outer rings, and asked if the rings are for

people that they do not know or do not like, even though instructions indicated to leave people

that they do not know, or are not close to, where they are on the page.

Other respondents expressed discomfort if they were to put people that they are ‘not close to’ in

the outer rings of the circle, or explained that ‘they would not have anyone to put in those rings’.

Most respondents tended to use the inner rings.

Implications for the survey: The audio feature will be available for the pilot, and hopefully should

help respondents read the question stem. Also, an analysis of pilot data should focus on the

distribution of items across the rings, to see how frequently the outer rings are used, and how

well the strength of respondent relationships correlates with overall wellbeing.

8. Organisation of the household and SES – Year 4

Some children usually sleep in the same home each night. Other children sometimes or often sleep in different homes. Please choose which of the following sentences best describes you (Please tick one box.)



□



First Home Another Home

None □ None □

One □ One □

Two □ Two □

More than 2 □ More than 2 □

Don’t know □ Don’t know □

Interviewer: ‘Can you tell me in your own words what this question is asking?’

Results: Respondents were able to paraphrase the question instructions (both ‘Organisation of

the household and ‘Paid job’), but some respondents provided implausible answers (e.g.

indicating jobs across two households when they indicated that they always sleep in the same

home), suggesting that they may not have fully understood the question stem, or the format of

the question was confusing to them.

Implications for the survey: These items should be split across screens with applied conditionals

to display the correct items according to their living situation.

9. Organisation of the household and SES – Years 6 and 8














None □ None □

One □ One □

Two □ Two □



Results: The presentation of these items was overwhelming to most respondents, which made

them hesitant to attempt the question. Respondents were confused about the instructions, and

how to decide to fill out ‘First home’ only or ‘First home’ and ‘Second home’. Most respondents

understood ‘paid job’, but some respondents were unsure if their parents/caregivers had paid

jobs themselves.

Implications for the survey: Questions should be presented separately on screens so as to not

overwhelm respondents. In order to present the correct questions according to their living

situation, it was decided to present the Year 4 ‘Organisation of the household’ question as a

preliminary filter question, in order to present ‘Organisation of the household’ and ‘Paid job’ for

Years 6 and 8 either for one home or two homes.

10. Vulnerability and Harm – Years 4,6, and 8

How much do you worry that someone in your family: Not at all A little Somewhat A lot

1. Will get sick? □ □ □ □

2. Will be worried a lot? □ □ □ □

4. Will get hurt? □ □ □ □

5. Will get arrested? □ □ □ □

6. Will be fighting? □ □ □ □

8. Won’t have enough money? □ □ □ □

9. Won’t have a place to live? □ □ □ □

10. Won’t have enough to eat? □ □ □ □

12. Will move away? □ □ □ □

15. Will die? □ □ □ □

16. Will hurt somebody? □ □ □ □

17. Won’t listen? □ □ □ □

18. Will tell a lie? □ □ □ □

Interviewer: [We want to ascertain if children think about particular members of their family when answering these questions or about their family in general].

‘Tell me, who were you thinking about when you were answering these questions?’

[Follow-up probe]

Interviewer: ‘Which person or people did you think about to answer these questions, for

example: mum, dad, grandma, brother, aunt, cousin?’

Interviewer: Is it okay to ask these types of questions in a survey?

Results: Respondents thought about a range of people. Aboriginal students mostly thought about

their entire family or community, and several respondents thought about ‘their entire family’ or

a different person for each item. Others thought about their primary caregiver solely, such as

their mother. Others thought about a vulnerable or sick family member, such as a grandparent

or distant relative who is ill.

Other respondents had difficulty understanding the hypothetical nature of the items, and

therefore had difficulty selecting an answer. Respondents selected a variety of response options.


Respondents did not report any issues regarding the appropriateness of asking these sorts of

questions in a survey.

Implications for the survey: As respondents indicated thinking about a wide range of people

when answering these questions, the research team will not specify specific people to think about

when answering these questions. However, in order to try and make sure that respondents think

of people who are in their family and close to them, instead of a distant relative who may be sick,

this question will follow the ‘Organisation of the household’ and the ‘Closeness of relationships’

questions. The research team also eliminated two items from the question to try and reduce the

survey length, and items that posed potential issues for respondents’ comprehension. The items

‘will be worried a lot’ and ‘will not listen’ were removed as they seemed to be more hypothetical

and unclear to respondents.

11. Family cohesion – Years 4,6, and 8


(Please tick one box in each row.) Not at all last week


Most days last week

Every day last week

Don’t know




Interviewer: [Participants will be using different response scales throughout the survey. We want

to ascertain if it is difficult to switch between response scales, and if particular response scales

are more difficult than others.]

‘Tell me, what do you think about answering questions with ‘Not at all – A lot’, ‘Strongly

disagree – Strongly Agree’ and ‘Not at all – Every day’?

Was it easier, harder or the same to come up with answers?

Results: Most respondents did not indicate a preference for response options and indicated that

all scales were comparatively easy to respond to. One respondent did indicate that the response

option of ‘Somewhat’ was difficult to understand.

Implications for the survey: Question and response options seem to work well so suggest to

keep as is for survey.

12. Vignette Year 4


(Please tick only one box in each row.) YES! yes no NO!

Wonder Woman’s parents always listen to Wonder Woman. They try to understand her point of view and encourage her to do things that she wants to do. Wonder Woman's parents help her with her problems.

□ □ □ □

Spider-Man’s parents have clear ideas about how Spider-Man should behave and what Spider-Man should do. They only support Spider-Man if he does what they want. They are not interested in what Spider-Man thinks. Spider-Man’s parents help him with his problems.

□ □ □ □

The Hulk’s parents do not think about Hulk much. They let Hulk do whatever he likes. Sometimes they get angry with Hulk but the next time he does the same thing they think it is funny. Hulk’s parents help Hulk with his problems.

□ □ □ □

Interviewer: Can you tell me what these instructions are asking you to do?


Results: One respondent stopped the cognitive interview because the survey had ‘too much

reading’ and it was ‘hurting [his] eyes’. Other respondents provided unclear explanations for the

question instructions, and one order violation was recorded. This means that respondents

incorrect responses for the vignettes.

Implications for the survey: In order to reduce the survey length, and because of the various

issues reported with the question, the research team decided to not administer this question to

Year 4 students.

13. Family Items – Year 4


YES! yes no NO!


i. We argue about the same things in my family over and over □ □ □ □ □

j. People in my family have serious arguments □ □ □ □ □

k. People in my family often insult or yell at each other □ □ □ □ □

a. Do you feel very close to your mother? □ □ □ □ □

b. Do you share your thoughts and feelings with your mother? □ □ □ □ □

c. Do you enjoy spending time with your mother? □ □ □ □ □

d. Do you feel very close to your father? □ □ □ □ □

e. Do you share your thoughts and feelings with your father? □ □ □ □ □

f. Do you enjoy spending time with your father? □ □ □ □ □

g. If I had a personal problem, I could ask my mum or dad for help □ □ □ □ □

h. My parents give me lots of chances to do fun things with them □ □ □ □ □

l. My Parents ask me what I think before most family decision affecting me are made

□ □ □ □ □

Tell me, what you think about answering questions with ‘YES! - NO!’?

Was it easier, harder or the same to come up with answers compared to other questions?

Results: One respondent did not understand the response option of ‘Does not apply to me’. No

other preferences reported as regards the response scale for these items for Year 4 students.

Implications for the survey: In order to reduce the survey length, as seeing a long list of items

intimidated respondents, and as a question about ‘Closeness of relationships was included’, then

the research team decided to not administer these questions to Year 4 students.

14. Vignette Year 6

Below you will find descriptions of three families. Read each of the descriptions of these families, then tick how much you agree with the final statement about each family.


The Bacon family fix their problems by having big arguments. Kim is in charge of the Bacon family. If the argument goes on too long then Kim decides what to do. Everyone has to do what Kim says even if they do not agree. The Bacon family get on well together.

□ □ □ □

The Lettuce family fix their problems by talking to each other when everyone is calm. They try to find solutions that everyone agrees with. The Lettuce family get on well together.

□ □ □ □

The Tomato family argue about the same problems every day. Everyone joins in shouting and yelling and saying mean things to each other and nothing gets fixed. The Tomato family get on well together.

□ □ □ □



Results: Respondents seemed to be overwhelmed by the amount of text presented on the screen

with this question. As such, they often did not read the question instructions. Respondents began

to ask if they ‘had to complete the question’ and ‘how many more questions’ were in the survey.

However, no order violations were recorded so respondents did have some understanding of the

question instructions.

Implications for the survey: In order to reduce the survey length, and because of the various

issues reported with the question, the research team decided to not administer this question to

Year 4 students.



YES! yes no NO!


i. We argue about the same things in my family over and over

□ □ □ □ □




b. Do you share your thoughts and feelings with your mother?

□ □ □ □ □



e. Do you share your thoughts and feelings with your father?

□ □ □ □ □


g. If I had a personal problem, I could ask my mum or dad for help

□ □ □ □ □

h. My parents give me lots of chances to do fun things with them

□ □ □ □ □


□ □ □ □ □

Tell me, what do you think about answering questions with ‘YES! - NO!’?


Results: No preferences were reported regarding differences between response scales, though

one respondent did ask what the response options of ‘YES!’ and ‘NO!’ meant. Most respondents

were overwhelmed at the length of items presented on the screen.

Implications for the survey: In order to reduce the survey length, as seeing a long list of items

intimidated respondents, and as a question about ‘Closeness of relationships was included’, then

the research team decided to not administer these questions to Year 6 students.


16. Vignette Year 8



Spider-man’s parents let Spider-man do whatever he likes. They say he has to learn about risks himself. They never know where Spider-man is or what he is doing. They are too busy to help Spider-man if he is in danger. Spider-man’s parents help Spider-man to keep safe.

□ □ □ □

Catwoman’s parents encourage her to take small risks. As she shows that she can be sensible they give her more freedom. Catwoman always tells her parents the truth about what she is doing. She knows her parents will always help her if she is in danger. Catwoman’s parents help Catwoman to keep safe.

□ □ □ □

Wonder Woman’s parents do not let her take any risks. They ban most things she wants to do to keep her safe. Wonder Woman lies to her parents about what she is doing. Her parents often do not know where she really is. Wonder Woman cannot ask her parents for help if she is in danger. Wonder Woman’s parents help Wonder Woman to keep safe.

□ □ □ □


Results: Several respondents were able to understand and explain that the question instructions

were asking “if you agree with what the paragraph says is really keeping Spiderman or Catwoman

or Wonderwoman safe”. However, one respondent recorded an order violation, and another

respondent did not understand the difference between the YES!/yes’ and ‘NO!/no’ response

options, and the word ‘extent’.

Implications for the survey: The research team decided to administer the vignette to Year 8

students, as they will have higher literacy skills than the other year levels. However, analysis of

pilot data should look for order violations, and if so, consider revising the response scale (if

revised in the Family Management items), and rewording the question stem to make instructions

clearer.




YES! yes no NO!


m. My parents ask me if I’ve done my homework. □ □ □ □ □

n. My parents would know if I didn’t come home on time □ □ □ □ □

o. The rules in my family are clear. □ □ □ □ □

p. When I am not at home, one of my parents knows where I am and who I am with.

□ □ □ □ □


□ □ □ □ □

r. My family has clear rules about alcohol and drug use. □ □ □ □ □

s. If you drank some alcohol (like beer, wine, spirits or pre-mixed drinks such as Bacardi Breezers or UDL’s) without your parents’ permission, would you be caught by your parents?

□ □ □ □ □


□ □ □ □ □


□ □ □ □ □



□ □ □ □ □




□ □ □ □ □



□ □ □ □ □


□ □ □ □ □


□ □ □ □ □




□ □ □ □ □



Results: Respondents were overwhelmed by the length of the question and number of items. In

addition, it was difficult for respondents to think about hypothetical situations and answer

accordingly. One respondent reported that it was easier to use an ‘Agree – Disagree’ scale than

this response scale.

Implications for the survey: In order to reduce the survey length, the research team decided to

only administer the ‘Family management’ items to Year 8 students, and not the ‘Family conflict

and closeness’ items. The response scale would remain the same, but an analysis of pilot data

should look at the variance of response options.


18. SES Deprivation – Years 4, 6 and 8

Here is a list of items that some young people of your age have.

Please tell us whether you have each item on the list or whether you'd like to have it.

I have this

I don't have this but

would like it

I don't have this and I don't want

or need it

Some pocket money each week to spend on yourself (pocket money)

□ □ □

Some money that you can save each month, either in a bank or at home (saving money)

□ □ □

A pair of designer or brand name trainers (like Nike or Vans) (trainers)

□ □ □

An iPod or other personal music player (MP3 player) □ □ □

Cable/satellite TV at home (cable/satellite) □ □ □

A garden at home or somewhere nearby like a park where you can safely spend time with your friends (garden)

□ □ □

A family car for transport when you need it (car) □ □ □

The right kind of clothes to fit in with other people your age (clothes)

□ □ □

At least one holiday away from home each year with your family (holiday)

□ □ □

Trips or days out with your family at least once a month (day trips)

□ □ □


My family has enough money to put petrol in the car when needed

□ □ □

My family has enough money for me to go on a school camp □ □ □

Interviewer: Did you find it easy or difficult to answer this question?

Results: Most respondents did not find this question difficult to answer or have difficulty with the

response options, but they did suggest several items for deletion. Furthermore, several

respondents were becoming increasingly fatigued at this point and were overwhelmed by a long

list of items.

Interviewer: If you had to make the question shorter, which statements would you delete?

Several uncertainties around the family car (i.e. cars are broken; family cars are parent

work cars;

Vans/trainers not commonly understood

Cable TV not commonly understood

Park/garden suggested for deletion (e.g. everyone has one; respondent isn’t allowed to

leave the house alone anyway)

Credit for mobile phone

The right kind of clothes

Implications for survey: The research team decided to remove many of the material deprivation

items from this scale in order to reduce the survey length and make the question more relevant to

students. However, the ‘family car’ item remained even though respondents had expressed some

confusion about the item. The research team felt that this item is important in an Australian

context of large distances, and an equivalent item about ‘money for public transport’ would not

appropriately target material deprivation. For the items about ‘mobile credit’ and ‘money to put

petrol in the car’ these were then conditioned in the online survey to only appear if respondents

indicate that they do indeed have these items.


19. SES No. of Books

About how many books are there in your home? (Do not count magazines, newspapers or your school books.) (Please tick only one box.)

None or very few (0 - 10 books) □

Enough to fill one shelf (11 - 25 books) □

Enough to fill one bookcase (26 - 100 books) □

Enough to fill two bookcases (101 - 200 books) □

Enough to fill three or more bookcases (more than 200) □

Unanticipated issues: This question was much more difficult for respondents to answer than

anticipated. Respondents spent a lot of time counting the number of books in their home, as

books were not on shelves but in ‘boxes’ or ‘cupboards’. Furthermore, many of the books in the

home were children’s books that belonged to younger siblings. One respondent mentioned

having digital books on an e-reader, making it more difficult to estimate the number of books.

Students selected a range of responses.

Implications for the survey: The Year 4 question will have illustrations in the online survey, to

maintain comparability with the international survey item. Therefore this visual aid will hopefully

help Year 4 students to better estimate the number of books in the home. Analysis of pilot data

for years 6 and 8 should examine the distribution of responses, and comparisons with the

international data.

20. Disability

Do you have any long-term disability (lasting 6 months or more) (e.g., sensory impaired hearing, visual impairment, in a wheelchair, learning difficulties)?

Yes □

No □

I don’t know □

Unanticipated issues: Several respondents had difficulties understanding if a condition they

experienced counted as a ‘disability’ according to the question stems. These conditions included:

diabetes, a speech impediment, and suffering a major car accident.

Implications for the survey: The research team sought the advice and feedback from a disabilities

education expert at Flinders University, who proposed clearer definitions of disability in the question

stem and items, to be better understood by children.

21. Disability II

Does this disability cause you difficulty with, or stop you doing... (you may choose as many as you need)

everyday activities that other people your age can usually do □

communicating, talking, mixing with others or socialising □

any other activity that people your age can usually do □

no difficulty with any of these □

No reported unanticipated issues.

22. Out of home care

Which of the following best describes the home you live in most of the time? (Please tick one box only.)







23. Changed carers


Yes □

No □


24. Care for family member






None of these □



Attachment 3: Appendix A – Cognitive interview schedule Year 4

Date: _______________

Interviewer name: _______________

Participant Age: _________Year level: __________Gender: ____________

1. Please ensure that you have the correct interview schedule for the participant (i.e. year 6 or 8).

2. Please ensure that the participant has provided the proper consent before starting the interview.

3. Please read the instructions below to the participant, or paraphrase the instructions. Please read the

specified verbal probes to participants after they have completed the correct item, or, follow-up with

potential problems during the survey administration with the appropriate follow-up probe from Table

1. Record information in the spaces provided.

Instructions to Participant

Thank you very much for agreeing to help us today. We are trying out a new online survey that has

been designed for young people like you, which will be taken by thousands of other kids next year.

The survey is still a draft and we are making improvements online. You get to be one of the first people

to take the survey and let us know what you like, what you don’t like, what is easy and what is difficult.

There aren’t any right or wrong answers to the survey – this isn’t a test. We just want to know what

you think about the survey so that we can make it better for next year. You will answer questions

about your family and yourself, and how you think and feel about these things. There will be different

types of questions in the survey: you may read a short paragraph or tell us how much you agree or

disagree with something.

While you take the survey, we will sit with you and we will ask you some questions and write down

your answers. Again, there are no right or wrong answers to the questions, we just want to know what

you think. It will take between 20 to 30 minutes to complete. We will not share your answers with

anyone and you can stop at any time if you don’t want to do the survey anymore.

Do you have any questions?

Okay, let’s start. I will read some instructions before you start the online survey.

Please read the instructions and each question carefully. Please answer each question as best as you

can.


For each question you will click a box with your mouse. If you need to change your answer, remove

the check by clicking on it and then select your new response.

You can go back to questions on different screens by using the progress bar at the top of the screen.

There are no right or wrong answers. The only right answer is the one that is best for you.

You can ask questions if you don’t understand something or need help. Your answers will not be

shared with anyone.

1. Gender



□ Girl

□ Boy

_____________________________________________________________

_____________________________________________________________

2. ATSI



No □

Yes, Aboriginal □


_____________________________________________________________

_____________________________________________________________

3. Language background






_____________________________________________________________

_____________________________________________________________

4. Overall well-being



Disagree Neither

agree nor disagree

Agree Strongly

Agree Don't know






Interviewer: [If the child selects the response option of ‘Don’t know’ to ascertain of they really don’t

know or are selecting a ‘soft’ option then]:



_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

5. Importance of domains for well-being



Family

Friends

School


Health

Money/things I have

Interviewer: ‘ ‘Here is a question which asks you to respond other than by ticking a box. Do you like

it? Why (not)?

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________


_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

6. Closeness of relationships



Mother Foster father Grandmother Mother's partner

Father Sister Grandfather Father's partner

Stepfather Brother Other adult

Stepmother Uncle Other child

Foster mother Aunt Pet

How close are these people or things to you? Drag them into the circle, as close to you as you feel they are. If you don't know a person or thing or you don't feel close to them, just leave it where it is.

Me!


Interviewer: ‘Here is another question which asks you to respond other than by ticking a box. Do

you like it? Why (not)?

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

Interviewer: ‘At the moment, the question asks you to leave a person who you don't feel close to

where it is. How would you feel if you were told to put any person or thing you don't feel close to

in the outer most circle?

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

7. Organisation of the household and SES

Some children usually sleep in the same home each night. Other children sometimes or often sleep in different homes. Please choose which of the following sentences best describes you (Please tick only one box.)



□




None □ None □

One □ One □

Two □ Two □




_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

8. Vulnerability and Harm











15. Will die? □ □ □ □





Interviewer: [We want to ascertain if children think about particular members of their family when

answering these questions, or about their family in general].


_____________________________________________________________

_____________________________________________________________

[Follow-up probe]:

Interviewer: ‘Which person or people did you think about to answer these questions, for example:

mum, dad, grandma, brother, aunt, cousin?’


_____________________________________________________________

_____________________________________________________________

9. Family cohesion




Most days last week

Every day last week

Don’t know




Interviewer: [Participants will be using different response scales throughout the survey. We want to

ascertain if it is difficult to switch between response scales, and if particular response scales are more

difficult than others.]

‘Tell me, what do you think about answering questions with ‘Not at all – A lot’, ‘Strongly disagree –

Strongly Agree’ and ‘Not at all – Every day’?


_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

10. Vignette Year 4 – RECORD PARTICIPANT ANSWERS BELOW



Wonder Woman’s parents always listen to Wonder Woman. They try to understand her point of view and encourage her to do things that she wants to do. Wonder Woman's parents help her with her problems.

□ □ □ □

Spider-Man’s parents have clear ideas about how Spider-Man should behave and what Spider-Man should do. They only support Spider-Man if he does what they want. They are not interested in what Spider-Man thinks. Spider-Man’s parents help him with his problems.

□ □ □ □

The Hulk’s parents do not think about Hulk much. They let Hulk do whatever he likes. Sometimes they get angry with Hulk but the next time he does the same thing they think it is funny. Hulk’s parents help Hulk with his problems.

□ □ □ □



_____________________________________________________________

_____________________________________________________________



YES! yes no NO!







□ □ □ □ □






□ □ □ □ □


□ □ □ □ □


□ □ □ □ □



_____________________________________________________________

_____________________________________________________________

12. SES Deprivation


I have this

I don't have this

but would like it

I don't have this and I

don't want or need it

Some pocket money each week to spend on yourself (pocket money) □ □ □


□ □ □

A pair of designer or brand name trainers (like Nike or Vans) (trainers) □ □ □




□ □ □


The right kind of clothes to fit in with other people your age (clothes) □ □ □


□ □ □

Trips or days out with your family at least once a month (day trips) □ □ □


My family has enough money to put petrol in the car when needed □ □ □




_____________________________________________________________

_____________________________________________________________

_____________________________________________________________


_____________________________________________________________

_____________________________________________________________

_____________________________________________________________



(Please tick only one box.) □






_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

14. Disability


Yes □

No □

I don’t know □

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

15. Disability II






_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________








_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

17. Changed carers


Yes □

No □

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________







None of these □

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

Interviewer: ‘Do you have any questions or something you want to share about the questions or

ways of answering this survey before we finish?’

‘Thank you very much for your help.’


Attachment 3: Appendix B – Cognitive interview schedule Year 6

Date: _______________





3. Please read the instructions below to the participant, or paraphrase the instructions. Please read

the specified verbal probes to participants after they have completed the correct item, or, follow-up

with potential problems during the survey administration with the appropriate follow-up probe from

Table 1. Record information in the spaces provided.


















can.




You can go back to questions on different screens by using the progress bay at the top of the screen.



shared with anyone.

1. Gender


□ Girl

□ Boy

_____________________________________________________________

_____________________________________________________________

2. ATSI



No □

Yes, Aboriginal □


_____________________________________________________________

_____________________________________________________________

3. Language background


□ Always

□ Almost always

□ Sometimes

□ Never

_____________________________________________________________

_____________________________________________________________


How much do you agree or disagree with each of these sentences: (Please tick one box in each row.)

Strongly disagree

Disagree Neither

agree nor disagree

Agree Strongly

Agree Don't know









_____________________________________________________________

_____________________________________________________________





Family

Friends

School


Health

Money/things I have

Interviewer: ‘ ‘Here is a question which asks you to respond other than by ticking a box. Do you like

it? Why (not)?

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________


_____________________________________________________________

_____________________________________________________________

_____________________________________________________________










Me!




_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________




_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________


Some children usually sleep in the same home each night. Other children sometimes or often sleep in different homes. Please choose which of the following sentences best describes you (Please tick only one box.)



□




None □ None □

One □ One □

Two □ Two □




_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

Interviewer: ‘"How easy or difficult do you find this question? ’

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________



How much do you worry that someone in your family:











15. Will die? □ □ □ □







_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

[Follow-up probe]:




_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

9. Family cohesion




Most days last week

Every day last week

Don’t know











_____________________________________________________________

_____________________________________________________________

_____________________________________________________________


Below you will find descriptions of three families. Read each of the descriptions of these families, then tick how much you agree with the final statement about each family.


The Bacon family fix their problems by having big arguments. Kim is in charge of the Bacon family. If the argument goes on too long then Kim decides what to do. Everyone has to do what Kim says even if they do not agree. The Bacon family get on well together.

□ □ □ □

The Lettuce family fix their problems by talking to each other when everyone is calm. They try to find solutions that everyone agrees with. The Lettuce family get on well together.

□ □ □ □

The Tomato family argue about the same problems every day. Everyone joins in shouting and yelling and saying mean things to each other and nothing gets fixed. The Tomato family get on well together.

□ □ □ □


_____________________________________________________________

_____________________________________________________________

_____________________________________________________________



YES! yes no NO!



□ □ □ □ □





□ □ □ □ □




□ □ □ □ □



□ □ □ □ □


□ □ □ □ □


□ □ □ □ □




_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

12. SES Deprivation

Here is a list of items that some young people of your age have.

Please tell us whether you have each item on the list or whether you'd like to have it.

I have this


would like it



□ □ □


□ □ □


□ □ □




□ □ □



□ □ □


□ □ □


□ □ □



□ □ □



_____________________________________________________________

_____________________________________________________________

_____________________________________________________________


_____________________________________________________________

_____________________________________________________________

_____________________________________________________________


About how many books are there in your home? (Do not count magazines, newspapers or your school books.) (Please tick only one box.)







_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

14. Disability


Yes □

No □

I don’t know □

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

15. Disability II






_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________







_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

17. Changed carers


Yes □

No □

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________








None of these □

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________





Attachment 3: Appendix C – Cognitive interview schedule Year 8

Date: _______________





3. Please read the instructions below to the participant, or paraphrase the instructions. Please read the

specified verbal probes to participants after they have completed the correct item, or, follow-up with

potential problems during the survey administration with the appropriate follow-up probe from Table

1. Record information in the spaces provided.


















can.




You can go back to questions on different screens by using the progress bay at the top of the screen.



shared with anyone.

1. Gender


□ Girl

□ Boy

_____________________________________________________________

_____________________________________________________________

2. ATSI



No □

Yes, Aboriginal □


_____________________________________________________________

_____________________________________________________________

3. Language


□ Always

□ Almost always

□ Sometimes

□ Never

_____________________________________________________________

_____________________________________________________________




Disagree Neither agree nor disagree

Agree Strongly Agree

Don't know









_____________________________________________________________

_____________________________________________________________

_____________________________________________________________





Family

Friends

School


Health

Money/things I have

Interviewer: ‘Here is a question which asks you to respond other than by ticking a box. Do you like

it? Why (not)?

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________


_____________________________________________________________

_____________________________________________________________

_____________________________________________________________










Me!




_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________




_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________














None □ None □

One □ One □

Two □ Two □




_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

Interviewer: ‘"How easy or difficult do you find this question? ’

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________













15. Will die? □ □ □ □







_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

[Follow-up probe]:




_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

9. Family cohesion




Most days last week

Every day last week

Don’t know











_____________________________________________________________

_____________________________________________________________

_____________________________________________________________




Spider-man’s parents let Spider-man do whatever he likes. They say he has to learn about risks himself. They never know where Spider-man is or what he is doing. They are too busy to help Spider-man if he is in danger. Spider-man’s parents help Spider-man to keep safe.

□ □ □ □

Catwoman’s parents encourage her to take small risks. As she shows that she can be sensible they give her more freedom. Catwoman always tells her parents the truth about what she is doing. She knows her parents will always help her if she is in danger. Catwoman’s parents help Catwoman to keep safe.

□ □ □ □

Wonder Woman’s parents do not let her take any risks. They ban most things she wants to do to keep her safe. Wonder Woman lies to her parents about what she is doing. Her parents often do not know where she really is. Wonder Woman cannot ask her parents for help if she is in danger. Wonder Woman’s parents help Wonder Woman to keep safe.

□ □ □ □


_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________




YES! yes no NO!


m. My parents ask me if I’ve done my homework. □ □ □ □ □

n. My parents would know if I didn’t come home on time □ □ □ □ □

o. The rules in my family are clear. □ □ □ □ □

p. When I am not at home, one of my parents knows where I am and who I am with.

□ □ □ □ □


□ □ □ □ □

r. My family has clear rules about alcohol and drug use. □ □ □ □ □

s. If you drank some alcohol (like beer, wine, spirits or pre-mixed drinks such as Bacardi Breezers or UDL’s) without your parents’ permission, would you be caught by your parents?

□ □ □ □ □


□ □ □ □ □


□ □ □ □ □



□ □ □ □ □






□ □ □ □ □


□ □ □ □ □





□ □ □ □ □



_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________


12. SES Deprivation


I have

this


would like it



□ □ □


□ □ □


□ □ □




□ □ □



□ □ □


□ □ □


□ □ □



□ □ □



_____________________________________________________________

_____________________________________________________________

_____________________________________________________________


_____________________________________________________________

_____________________________________________________________

_____________________________________________________________



(Please tick only one box.) □







_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

14. Disability


Yes □

No □

I don’t know □

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

15. Disability II

Does this disability cause you difficulty with, or stop you doing...

(you may choose as many as you need)





_____________________________________________________________

_____________________________________________________________

_____________________________________________________________







_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

17. Changed carers


Yes □

No □

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________








None of these □

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________





Attachment 4: ACWP Field Trial Analysis plan

Prepared by ACER, 8 April 2014

1. Overall Aims

The main aim of the field trial (FT) data analysis is to inform decisions regarding the questions that will

be maintained for the main survey. A secondary aim is to examine whether some of the analyses

intended to be undertaken with main survey data will be feasible and meaningful.

To achieve these aims the following will be examined:

Comprehension of the questions: High proportions of missing values, for example, may

indicate an issue with understanding or survey length.

Performance of the questions and response options: Examination of range, minimum,

maximum, mean, standard deviation is intended to reveal misfitting response options.

Psychometric properties of the scales through reliability, exploratory and confirmatory

factor analyses.

Validity of constructs: Here, correlation analyses will be employed to see whether

hypothesised relationships hold.

The characteristics of the vignette designed for the family management scale.

Online, computer-based administration considerations, for example by examining

questions for which students needed to use the scroll bar to view all items in terms of high

proportion of missing data.

The following section discusses in more detail how the analysis will address these issues. Due to the

limited sample size in the FT only some preliminary analyses regarding substantive research questions

will be possible. Hence, the main focus of the FT analysis will be on the properties and behaviour of

the measures in the survey instrument.

2. Methodology

2.1 Data Cleaning

The data will be thoroughly cleaned and checked to address any coding issues for and to prepare the

data in a systematic manner for analysis. Dubious data can influence adversely subsequent analyses,

hence the data cleaning process aims to eliminate impossible or incorrect values. While the computer

based design of the data capture should minimise these errors, data cleaning is still required to ensure

data accuracy and consistency. Further, results for items that had validation rules will need to be

checked to ensure the applicability and appropriateness of these rules.


2.2 Frequencies

For each item, frequencies of categorical questions and descriptive statistics for numerical questions

will be calculated. These statistics - minimum, maximum, mean, range, mean and % missing - will be

used to examine the behaviour and properties of each item in terms of their distribution, use of

categories (e.g. all have been used) and proportion of missing data which should not exceed ten per

cent.

2.3 Scale Analyses

The ACWP FT survey contained some groups of items that were designed to form a scale. These are

listed in the table below:

Table A4.1: List of scales included in the ACWP FT survey

Name of scale & subscales

Items # Number of factors

Overall Wellbeing WB01A01-A05 1

Vulnerability FA05A01-A11 1

Family Cohesion FA06A01-A03 1

Family Management FA08E01-E09 1

Degree of closeness/support of friend FR02A01-A04 1

Degree of conflict with close friend FR03O01-O05 1

Teacher support SC02A01-A03 1

School enjoyment & motivation Positive affect Intrinsic motivation

SC06A01-A06 SC06A07-A12

2

Outside school activities SC08A01-A10 1

Mental & Physical Health Psychological wellbeing Somatic wellbeing

HE05A04-A06 HE05A01-A03 & A07-A08

2

Deprivation MW01A01-A05 1

In order to consider the measurement of these scales, the classical item statistics for these scaled

items will be reported. These will include the following:

Scale reliabilities (Cronbach’s alpha): Generally, reliabilities of 0.80 or more are described

as high; between 0.70 and 0.80 as moderate; between 0.60 and 0.70 as low; and below

0.60 as very low.

Item-total correlations: These correlations indicate to what extent individual items

correlate with the overall scale score (for all other items). Low item-total correlations

(< 0.3) indicate items with poor scaling properties.

Numbers of students with valid and missing responses.


2.4 Factor Analysis

To further examine the structure of the scale measures, exploratory and confirmatory factor analysis

will be conducted. For example, the questions on Family Management (FA08E01-FA08E09) are

expected to form a uni-dimensional scale consisting of all nine items. Similarly four of the items asked

on Overall Wellbeing (WB01A01-WB01A04) are proposed to form a uni-dimensional scale of wellbeing

that can be used in further analyses. Results of factor analyses can also be used to shorten scales by

removing items that do not add much to the measurement of the underlying factor.

2.5 Vignette

One anchoring vignette on family management was included in the survey (FA07E01-FA07E03). This

aimed to provide a means to adjust for differences in response style as some respondents are more

inclined to agree than others, some are more inclined to use extreme responses or sit in the middle

between different response alternatives. The vignette attempted to provide a frame of reference by

getting respondents to react to three family management scenarios. To ascertain the usefulness of

the vignette for the MS, the following will be examined:

Is there close to universal agreement about the rank order of the three scenarios or are

there many ties?

Are the hypothesised correlations between the family management scale and other

constructs (e.g. family cohesion, vulnerability) stronger for the unadjusted scale or for the

scale that is adjusted by the anchoring vignette?

If these questions are answered in the affirmative, it will indicate support for inclusion of the vignette

in the MS.

2.6 Correlations

Validity checks will be done mainly by way of correlation analyses to examine whether hypothesised

relationships between scales hold. An example of a convergent validity check would be that higher

psychological wellbeing would be related to higher overall wellbeing whereas an example of a

discriminant validity check would be to see whether there is actually no relationship between student

gender and family cohesion. Another example would be whether or not the expected negative

correlations between the scales measuring Vulnerability and Harm of a Significant other (FA05A01-

FA05A11) and the Overall Welling scale will be supported by the FT data.

2.7 Administration Issues

Analysis of the FT data will also provide insights into the survey administration process. Careful

examination of frequencies of each item and comparisons with the presentation of the questions as

they appeared to respondents will provide insight into administration issues. For example, in the case

of the vulnerability scale students were required to scroll down in order to view and respond to all

questions. Here, a higher proportion of missing values for the items appearing towards the end may

indicate a need to reposition these items to the following screen for the MS.

In a similar way, missing data can be examined at the end of the survey which may indicate if the

survey length is appropriate.

Finally, feedback obtained from principals, survey administrators and students will be taken into

account in finalising the survey for the MS.


3. Field Trial analysis Results (Example)

The following section provides an example of how the results of the FT analysis will be written up.

Each question is presented along with its purpose and relevant analysis as per the methodology

section. Interpretations of these results are offered which lead into a recommendation for the

question for main survey. In other words, a short summary is provided as to whether there is sufficient

support for questions or items to be a) carried through from the FT to the MS as is, b) adapted or c)

removed.

3.1 Demographics

Gender

Gender was included as a control/background variable and is to be used as a major reporting variable

to examine gender differences.

Table A4.2: Gender item

Item # Item

SD01A01 Are you a girl or a boy? Note Response Type: Multiple Choice; Response Options: Girl (1)/Boy(2)

ATSI

This item was included to identify of one of the 6 sub-groups of interest for this survey, Aboriginal and

Torres Strait Islander Students (ATSI = '2','3', '2 AND 3').

Table A4.3: ATSI item

Item # Item


SD02A01 No

SD02A02 Yes, Aboriginal

SD02A03 Yes, Torres Strait Islander

Note Response Type: Tick all that apply; Response Options: tick (1)/no tick (2)


References

Adams, N. Carr, J., Collins, J., Johnson, G., Matejic. P. (2012). Households below average income: An analysis of

the income distribution 1994/95 –2010/11. Department of Work and Pensions, London.

Angold A., Costello E.J., Messer E.C., Pickles A., Winder F., Silver D. (1995). Development of a short

questionnaire for use in epidemiological studies of depression in children and adolescents.

International Journal of Methods in Psychiatric Research. 1995, 237-249.

Austin, E. J., Deary, I. J., and Egan, V. (2006). Individual differences in response scale use: Mixed Rasch

modelling of responses to NEO-FFI items. Personality and Individual Differences, 40, 1235-1245.

Bolt, D. M. and Johnson, T. R. (2009). Addressing score bias and differential item functioning due to individual

differences in response style. Applied Psychological Measurement, 33(5), 335-352.

Bukowski, W. M., Hoza, B., and Boivin, M. (1994). Measuring friendship quality during pre-and early

adolescence: The development and psychometric properties of the Friendship Qualities Scale. Journal

of Social and Personal Relationships, 11(3), 471-484.

Clarke III, I. (2000). Extreme response style in cross-cultural research: An empirical investigation. Journal of

Social Behavior and Personality. 15(1), 137-152.

Elgar F.J., De Clercq B., Schnohr C.W., Bird P., Pickett K.E., Torsheim T., Hofmann F., & Currie C. (2013).

Absolute and relative family affluence and psychosomatic symptoms in adolescents. Social Science

and Medicine, 91, pp. 25-31.

Graham-Bermann, S. (1996). Family worries: Assessment of interpersonal anxiety in children from violent and

nonviolent families. Journal of Clinical Child Psychology,25(3), 280-287.

Greenleaf, E. A. (1992). Improving rating-scale measures by detecting and correcting bias components in some

response styles. Journal of Marketing Research, 29(2), 176-188.

Haugland, S., Wold, B., Stevenson, J., Aaroe, L.E. and Woynarowska, B. (2001). Subjective health complaints in

adolescence. A cross-national comparison of prevalence and dimensionality. European Journal of

Public Health, 1, 4-10.

Heine, S. J., Lehman, D. R., Peng, K., and Greenholtz, J. (2002). What's wrong with cross-cultural comparisons

of subjective Likert scales: The reference-group problem. Journal of Personality and Social Psychology,

82, 903-918.

Holtgraves, T. (2004). Social desirability and self-reports: Testing models of socially desirable responding.

Personality and Social Psychology Bulletin, 30(2), 161-172.

Huebner, E.S. (1991). Initial development of the Students’ Life Satisfaction Scale. School Psychology

International 12, 231-40

Idler, E.L. & Benyamini, Y. (1997). Self-rated health and mortality: A review of twenty-seven community

studies. Journal of Health and Social Behavior 38, 21-37.

Johnson, T., Kulesar, P., LLc, I., Cho, Y. I. and Shavitt, S. (2005). The relation between culture and response

styles: Evidence from 19 countries. Journal of Cross-Psychology, 36(2), 264-277.

Johnson, T., Kulesa, P., Cho I.Y., Shavitt. S. (2008).The Relation Between Culture and Response Styles: Evidence

from 19 Countries. Paper presented at the International Conference on Survey Methods in

Multinational, Multiregional, and Multicultural Contexts (3MC), June, 2008, Berlin.

King, G., Murray, C. L., Salomon, J. A., and Tandon, A. (2004). Enhancing the validity and cross-cultural

comparability of measurement in survey research. American Political Science Review,98, 191-207.

http://www.tandfonline.com/loi/hcap19?open=25#vol_25


Main, G. and Bradshaw, J. (2012). A child material deprivation index, Child Indicators Research, 5(3), 503-521.

Marin, G., Gamba, R. J., and Marin, B. V. (1992). Extreme response style and acquiescence among Hispanics.

Journal of Cross-Cultural Psychology, 23, 498-509.

Marsh, H. and Hau, K (2003). Big-fish-little-pond effect on academic self-concept: A cross-cultural (26

countries) test of the negative effect of academically selective schools. American Psychologist, 58(5),

364-376.

Mishna, F., Wiener, J., and Pepler, D. (2008). Some of My Best Friends—Experiences of Bullying Within

Friendships. School Psychology International, 29(5), 549-573.

Moors, G. (2004). Facts and artefacts in the comparison of attitudes among ethnic minorities: A multigroup

latent class structure model with adjustment for response style behavior. European Sociological

Review, 20(4), 303-320.

Ravens-Sieberer, U, Gosch, A., Rajmil, L., Erhart, M., Bruil, J., Power, M., Duer, W., Auquier, P., Bernhard, C.,

Czemy, L., Mazur, J., Czimbalmos, A., Tountas, Y., Hagquist, C. and Kilroe, J. (2008). The KIDSCREEN-52

Quality of Life Measure for Children and Adolescents: Psychometric Results from a Cross-Cultural

Survey in 13 European Countries. Value in Health, 11(4), 645-658.

Ravens-Sieberer U., Erhart M., Torsheim T., Hetland J., Freeman J., Danielson M., Thomas C. (2008). An

international scoring system for self-reported health complaints in adolescents. European Journal of

Public Health. 18(3), 294-9.

Rees, G., Goswami, H. and Bradshaw, J. (2010). Developing an index of children's subjective well-being in

England, Summary Report. The Children's Society, London.

Skattebol, J., Hamilton, M., Skrzypiec, G., Burnstock, T., Redmond, G., Jenkins, B. and Dodd, K.. (2013).

Understanding children’s perspectives on wellbeing: Phase One Report for the Australian Child

Wellbeing Project, Flinders University, the University of NSW and the Australian Council for

Educational Research. www.australianchildwellbeing.com.au

Spence, S.H., P.M. Barrett, and C.M. Turner. (2003). Psychometric properties of the Spence Children’s Anxiety

Scale with young adolescents. Journal of Anxiety Disorders, 17(6), 605-625.

Torsheim, T. and Wold, B. (2001, School-related stress, school support, and somatic complaints, Journal of

Adolescent Research, 16(3), 293-303.

van de Vijver, F. J. R. and Poortinga, Y. H. (1997). Towards an integrated analysis of bias in cross-cultural

assessment. European Journal of Psychological Assessment, 13(1), 29-37.

van Hemert, D.A., Poortinga, Y.H. and van de Vijver, F.J.R. (2007). Emotion and culture: A meta-analysis.

Cognition and emotion, 21(5), 913-943.

Varni, J. W., Seid, M., and Kurtin, P. S. (1999). Pediatric Health-Related Quality of Life Measurement

Technology: A Guide for Health Care Decision Makers. JCOM, 6(4), 33-40.

Waldrip, A. M., Malcolm, K. T. and Jensen-Campbell, L. A. (2008). With a Little Help from Your Friends: The

Importance of High-quality Friendships on Early Adolescent Adjustment. Social Development, 17, 832–

852.

Documents

Towards the ACWP Questionnaire - Australian Child Wellbeing · The Australian Child Wellbeing Project: Phase Two Report vii Table 1: Content of final field trial questionnaire Major