Http:// Multinational study of children’s well-being and daily activities Format pre-test and instrument testing SPANISH AND BRAZILIAN

http://www.udg.edu/eridiqv

Multinational study of children’s well-being and daily activities

Format pre-test andinstrument testing

SPANISH AND BRAZILIAN REPORT

Coordinated by:Ferran Casas (ERIDIQV,

University of Girona)Developed by:

Mònica González, Sara Malo, Carme Montserrat, Dolors

Navarro, Irma Bertran, Cristina Domènec and Claudia Sarrión

(ERÍDIQV) – and by Jorge C. Sarriera, Angela Paradiso, Anelise L. Rodrigues, Tiago Calza, Rebeca Linhares, Tamiris Kreibich and Fabiane

Schultz. (GPPC: Grupo de Pesquisa em psicologia

comunitária. Universidade Federal do Rio Grande do Sul).

Pre-pilot (1)Pre-pilot (1) Goal: To analyse and discuss the Goal: To analyse and discuss the

format of the questions with the format of the questions with the children.children.

Three stages:Three stages:a)Researchers gave an explanation

of the purpose of questionnaire format testing, and asked each child to help us to design a questionnaire which would be easily understood and answered by children of their same age.

b)b) The children did some individual work The children did some individual work answering questions, talking to the researcher answering questions, talking to the researcher as often as they wished, in order to give them as often as they wished, in order to give them practical experience of answering the same 5 practical experience of answering the same 5 items (on satisfaction with school aspects) in items (on satisfaction with school aspects) in different formats, raising their awareness of different formats, raising their awareness of format differences.format differences.

c)c) Semi-directed group discussions were Semi-directed group discussions were organised with about 8 children that had done organised with about 8 children that had done the individual work. Each child had a folder the individual work. Each child had a folder with 14 different formats of the same items with 14 different formats of the same items (some formats were identical to those in the (some formats were identical to those in the individual work and some different). Children individual work and some different). Children were asked about the pros and cons of each were asked about the pros and cons of each format and to suggest the best formats for our format and to suggest the best formats for our purposes – including ideas that were not in any purposes – including ideas that were not in any of the formats on the table.of the formats on the table.

Pre-pilot (2)Pre-pilot (2)

(a) About the 5 items: (a) About the 5 items: (1) Phrasing, understanding, alternatives. (2) Capital letters or not. (3) Each item on one line or not. (4) Each item with borders/frame or not. (5) Each answer with borders/frame or not. (6) Items alternatively shaded or not.

(b) About the scale: (b) About the scale: (1) Different ranges (5, 6, 7, 10, 11 points).(2) With numbers and with neutral value (0-10, 0-7) or

without neutral value (1-6, 1-10).(3) Without numbers (1-5). (4) With faces/emoticons. (5) With numbers inside or outside frames.(6) With numbers at the top of blocks of answers, or with

numbers at each answer/line.(7) If numbers on each line, the number above or inside the

boxes.

Format aspects that were discussed Format aspects that were discussed (1)(1)

(c) Labels on the scale: (c) Labels on the scale: (1) End-labelled. (2) Labels at the ends and in a neutral point. (3) Labels for all values.

(d) Direction of the scales: (d) Direction of the scales: (1) bipolar. (2) unipolar positive. (3) unipolar negative.

(e) Contents: What is best understood? Does it have the same (e) Contents: What is best understood? Does it have the same meaning?meaning?

(1) Happy(2) Satisfied(3) Content

Format aspects that were discussed Format aspects that were discussed (2)(2)

Examples of formats (1)Examples of formats (1)

Actualment, fins a quin punt et trobes satisfet o satisfeta amb cadascuna d’aquestes coses de la teva vida?

0 = Totalment insatisfet/a

Totalment satisfet/a = 10

Amb els teus assoliments en els estudis

0 1 2 3 4 5 6 7 8 9 10

Amb les coses que has après 0 1 2 3 4 5 6 7 8 9 10 Amb les relacions amb els companys/ es de curs

0 1 2 3 4 5 6 7 8 9 10

Amb les relacions amb els professors/ es

0 1 2 3 4 5 6 7 8 9 10

Amb el centre on estudies, en general 0 1 2 3 4 5 6 7 8 9 10 Amb la teva vida d’estudiant 0 1 2 3 4 5 6 7 8 9 10

Examples of formats (2)Examples of formats (2) Actualment, fins a quin punt et trobes content o contenta amb cadascuna d’aquestes

coses de la teva vida?

AMB LES COSES QUE HAS ACONSEGUIT APRENDRE

AMB LES TEVES NOTES

AMB LES RELACIONS AMB ELS COMPANYS/ ES DE CURS

AMB LES RELACIONS AMB ELS MESTRES

AMB L’ESCOLA ON ESTUDIES, EN GENERAL

AMB LA TEVA VIDA D’ESTUDIANT

Examples of formats (3)Examples of formats (3) Actualment, fins a quin punt en trobes satisfet o satisfeta amb els teus assoliments en els

estudis?

COMPLETAMENT INSATISFET

NI SATISFET NI INSATISFET

COMPLETAMENT SATISFET

0

1

2

3

4

5

6

7

8

9

10

Actualment, fins a quin punt en trobes satisfet o satisfeta amb les coses que has après?




0

1

2

3

4

5

6

7

8

9

10

Actualment, fins a quin punt en trobes satisfet o satisfeta amb les relacions amb els companys/es de curs?




0

1

2

3

4

5

6

7

8

9

10

When children were asked to help us as When children were asked to help us as researchers, they all became impressively researchers, they all became impressively cooperative and highly motivated to do both the cooperative and highly motivated to do both the individual work and the group discussion.individual work and the group discussion.

All other conclusions from this pre-pilot always All other conclusions from this pre-pilot always have at least one exception; i.e. there is always at have at least one exception; i.e. there is always at least one child of each age who does not least one child of each age who does not understand some of the formats or questions understand some of the formats or questions raised, or at least one who claims to have a better raised, or at least one who claims to have a better understanding of a format other than the one understanding of a format other than the one most children identify as the clearest.most children identify as the clearest.

Most children of all three age groups consider Most children of all three age groups consider that any of the presented formats can be that any of the presented formats can be understood at their age. However, some formats understood at their age. However, some formats require much more time to read and understand require much more time to read and understand than others, and are therefore not as “easy”.than others, and are therefore not as “easy”.

Shading alternate lines makes questions much Shading alternate lines makes questions much easier to read without mistakes.easier to read without mistakes.

Some summarised results: (1) For all Some summarised results: (1) For all agesages

The key to the children clearly understanding the The key to the children clearly understanding the formats lies in the researchers providing explanations formats lies in the researchers providing explanations and investing time answering questions and and investing time answering questions and interacting with them. Explanations given by other interacting with them. Explanations given by other children often brought them to faster understanding children often brought them to faster understanding than those given by researchers.than those given by researchers.

Faces/emoticons are considered to be both attractive Faces/emoticons are considered to be both attractive and easy to understand. However, they are only and easy to understand. However, they are only considered appropriate for “the easy questions” - considered appropriate for “the easy questions” - “difficult” (and “serious”) questions require other “difficult” (and “serious”) questions require other formats according to 10-year-olds (Brazil) and 12-formats according to 10-year-olds (Brazil) and 12-year-olds (Spain).year-olds (Spain).

Repeating the questions is boring. Reading again and Repeating the questions is boring. Reading again and again is very time-consuming. Similar items should be again is very time-consuming. Similar items should be grouped.grouped.

A few children from each age group report that it is A few children from each age group report that it is easier to have each question on only one line, but easier to have each question on only one line, but most report to understand any question equally if it is most report to understand any question equally if it is on two lines.on two lines.

Some summarised results: (2) For all Some summarised results: (2) For all agesages

Some 8-year-olds were faster than most 12-Some 8-year-olds were faster than most 12-year-olds doing the individual work. We re-year-olds doing the individual work. We re-checked their comprehension, but it did not checked their comprehension, but it did not seem to be worse.seem to be worse.

Most prefer faces/emoticons, but not all of Most prefer faces/emoticons, but not all of them.them.

Most children report to equally understand Most children report to equally understand capital and lower-case letters. Some prefer capital and lower-case letters. Some prefer capitals (most in Brazil) and a few prefer lower-capitals (most in Brazil) and a few prefer lower-case letters. case letters.

Some children reported that “satisfaction”, Some children reported that “satisfaction”, “happy” and “content” are ”adults words” for “happy” and “content” are ”adults words” for the same meaning. “the same meaning. “ContentContent” is the most ” is the most common among childrencommon among children

Some children suggested avoiding “technical” Some children suggested avoiding “technical” words and were often active in proposing easier words and were often active in proposing easier words and phrasings.words and phrasings.

Some summarised results: For 8-Some summarised results: For 8-year-oldsyear-olds

If a scale has numbers, most 8-year-olds in If a scale has numbers, most 8-year-olds in Spain consider that a 0-10 scale is easier to Spain consider that a 0-10 scale is easier to understand than other ranges. Some say they understand than other ranges. Some say they do not have a preference, because they do not have a preference, because they understand all of them.understand all of them.

If the scale has numbers, most consider the If the scale has numbers, most consider the numbers easier to understand if they are in a numbers easier to understand if they are in a box. Some others prefer “no numbers”, but box. Some others prefer “no numbers”, but titles in the columns. But most say they have titles in the columns. But most say they have no preference and are able to understand any no preference and are able to understand any option.option.

In Spain they seem to consider bipolar scales In Spain they seem to consider bipolar scales easier to understand, but this point remains easier to understand, but this point remains unclear. In Brazil they consider unipolar scales unclear. In Brazil they consider unipolar scales easier.easier.


Most prefer faces/emoticons, but some prefer Most prefer faces/emoticons, but some prefer numbers.numbers.

Most children report to equally understand capital Most children report to equally understand capital and lower-case letters. The others prefer lower-and lower-case letters. The others prefer lower-case.case.

If a scale has numbers, in Spain most say that a 0-If a scale has numbers, in Spain most say that a 0-10 scale is the easiest to understand. Some prefer 10 scale is the easiest to understand. Some prefer other ranges and some say that all are equally other ranges and some say that all are equally understandable. In Brazil they consider labels on all understandable. In Brazil they consider labels on all numbers make it easier to understand. If the scale numbers make it easier to understand. If the scale has numbers, they consider the number easier to has numbers, they consider the number easier to understand if inside a box.understand if inside a box.

They seem to consider unipolar positive scales They seem to consider unipolar positive scales easier to understand, both in Spain and Brazil, but easier to understand, both in Spain and Brazil, but this point remains unclear.this point remains unclear.


Some children reported that “satisfaction”, Some children reported that “satisfaction”, “happy” and “content” are different words with “happy” and “content” are different words with the same meaning. However, they seem to the same meaning. However, they seem to consider that “content” is the easiest to consider that “content” is the easiest to understand both in Spain and Brazil.understand both in Spain and Brazil.

The meaning they give to “satisfaction” is The meaning they give to “satisfaction” is unclear. Most consider “to be happy” and “to be unclear. Most consider “to be happy” and “to be content” as synonymous, but they seem to content” as synonymous, but they seem to consider “content” easier to understand.consider “content” easier to understand.

Although they consider emoticons nice, some Although they consider emoticons nice, some children, particularly in Brazil, pointed out that children, particularly in Brazil, pointed out that “looking at the faces different children may “looking at the faces different children may clearly understand different things for each face”.clearly understand different things for each face”.

In Spain they insisted that colourful In Spain they insisted that colourful questionnaires are less boring.questionnaires are less boring.


Faces/emoticons are “nicer” and best for “easy Faces/emoticons are “nicer” and best for “easy questions”, but end-labelled 11-point scales are the questions”, but end-labelled 11-point scales are the second best format and they say it gives more second best format and they say it gives more possible answers for “complex” questions.possible answers for “complex” questions.

If a scale has numbers, most consider a 0-10 scale If a scale has numbers, most consider a 0-10 scale easier to understand and “more precise”, but all easier to understand and “more precise”, but all options are considered equally easy by others.options are considered equally easy by others.

Scales with a label at the middle point are considered Scales with a label at the middle point are considered “more difficult” to understand.“more difficult” to understand.

They seem to consider unipolar positive scales easier They seem to consider unipolar positive scales easier to understand, but this point remains unclear.to understand, but this point remains unclear.

They understand “satisfaction”, “happy” and They understand “satisfaction”, “happy” and “content”. “Satisfaction” is considered more “formal “content”. “Satisfaction” is considered more “formal language” or more “adult language”. The language” or more “adult language”. The differentiation they make between “happy” and differentiation they make between “happy” and “content” is unclear, a few say they are very different, “content” is unclear, a few say they are very different, others not.others not.


For 8-year-olds, scales with faces/emoticons seem For 8-year-olds, scales with faces/emoticons seem to be best. However, a few items with scales of to be best. However, a few items with scales of any range, including 0-10, can be included.any range, including 0-10, can be included.

For 10-year-olds, a mixture of faces/emoticons For 10-year-olds, a mixture of faces/emoticons and scales with numbers seems to be best. Scales and scales with numbers seems to be best. Scales should be for the questions considered more should be for the questions considered more “difficult” or “serious” by children.“difficult” or “serious” by children.

For 12-year-olds, most items included can be on a For 12-year-olds, most items included can be on a 0-10 scale. However, they say a colourful 0-10 scale. However, they say a colourful questionnaire gives a sense of it “being nice”. questionnaire gives a sense of it “being nice”. Emoticons could be included for questions they Emoticons could be included for questions they consider “easy”.consider “easy”.

Provisional conclusions regarding Provisional conclusions regarding format from the Catalan and format from the Catalan and Brazilian contextsBrazilian contexts (1) (1)

Whichever option is used, a number of children Whichever option is used, a number of children may not understand the format. Researchers must may not understand the format. Researchers must be as open as possible to dialog with the surveyed be as open as possible to dialog with the surveyed children of these ages in order to facilitate their children of these ages in order to facilitate their understanding of the format of the questionnaire understanding of the format of the questionnaire and the purpose of the research.and the purpose of the research.

Further research should be done into what makes Further research should be done into what makes questions “easy”, “difficult” and “serious”. In questions “easy”, “difficult” and “serious”. In principle, “difficult” questions seem to be those principle, “difficult” questions seem to be those that make the child think for a while.that make the child think for a while.

It is better to have grouped items, than repeating It is better to have grouped items, than repeating the same question for every item. Alternative the same question for every item. Alternative shading of the items makes them easier to follow.shading of the items makes them easier to follow.

Provisional conclusions regarding Provisional conclusions regarding format from the Catalan and Brazilian format from the Catalan and Brazilian contextscontexts (2) (2)

12-year-olds consider that end-labelled scales are 12-year-olds consider that end-labelled scales are easier to understand than scales with a label on easier to understand than scales with a label on the neutral value. This probably means that the neutral value. This probably means that unipolar scales are clearer.unipolar scales are clearer.

More testing must be done on the understanding More testing must be done on the understanding of unipolar and bipolar scales at these ages. In of unipolar and bipolar scales at these ages. In Brazil, unipolar positive scales seem to be better Brazil, unipolar positive scales seem to be better understoodunderstood

The term “content” seems to be easier to The term “content” seems to be easier to understand among 8 and 10-year-olds than understand among 8 and 10-year-olds than “happy” or “satisfied”, both in Brazil and in Spain. “happy” or “satisfied”, both in Brazil and in Spain. 12-year-olds seem to understand “satisfied” 12-year-olds seem to understand “satisfied” perfectly, although they consider it “formal adult perfectly, although they consider it “formal adult language”. However, more testing will be needed language”. However, more testing will be needed on the differences attributed to each of these on the differences attributed to each of these concepts at each age.concepts at each age.

Provisional conclusions regarding Provisional conclusions regarding format from the Catalan and Brazilian format from the Catalan and Brazilian contexts contexts (3)(3)

Decisions taken: format used with 8 y.o.Decisions taken: format used with 8 y.o.

HOW HAPPY ARE YOU WITH ...

Don’t Know

… THE HOUSE OR FLAT WHERE YOU LIVE?

… THE PEOPLE WHO LIVE WITH YOU?

… ALL OF THE PEOPLE IN YOUR FAMILY?

… YOUR RELATIONSHIP WITH YOUR FATHER?

… YOUR RELATIONSHIP WITH YOUR MOTHER?

… YOUR BROTHERS OR SISTERS?

Decisions taken: format used with 10 Decisions taken: format used with 10 y.o.y.o.

To what extent do you currently feel happy with each of the following things in your life?

0 = Not satisfied at all

Completely satisfied = 10

the house or flat where you live? 0 1 2 3 4 5 6 7 8 9 10

the people who live with you? 0 1 2 3 4 5 6 7 8 9 10

all of the people in your family? 0 1 2 3 4 5 6 7 8 9 10

your relationship with your father? 0 1 2 3 4 5 6 7 8 9 10

your relationship with your mother? 0 1 2 3 4 5 6 7 8 9 10

your brothers or sisters? 0 1 2 3 4 5 6 7 8 9 10

Decisions taken: format used with 10 Decisions taken: format used with 10 y.o.y.o.

To what extent do you currently feel satisfied with each of the following things in your life?

0 = Not satisfied at all

Completely satisfied =

10

the house or flat where you live? 0 1 2 3 4 5 6 7 8 910

the people who live with you? 0 1 2 3 4 5 6 7 8 910

all of the people in your family? 0 1 2 3 4 5 6 7 8 910

your relationship with your father? 0 1 2 3 4 5 6 7 8 910

your relationship with your mother? 0 1 2 3 4 5 6 7 8 910

your brothers or sisters? 0 1 2 3 4 5 6 7 8 910

Testing the 3 questionnaires with Testing the 3 questionnaires with the adopted format (Catalonia only) the adopted format (Catalonia only) (1):(1): No. of Questions

Items (8-year-olds)

Items (10 year-olds)

Items (12-year-olds)

You 6 6 6 6

Your home and the people you live with

2 12 12 12

Money and possessions 3 7 8 9

Your friends and other people 2 7 7 7

The neighbourhood or town where you live

2 14 17 18

School 2 9 11 11

How you use your time 7 11 15 15

How you feel about yourself 3 4 8 8

More about you 3 11 25 26

Your life 2 2 2 22

Finally 1 4 4 4

TOTAL 33 87 115 138

Appendix (More about you) 20 12 17 21

We based our questionnaires on the Israeli We based our questionnaires on the Israeli version, including many agreement/disagreement version, including many agreement/disagreement items. items.

We deleted explanations at the beginning for the We deleted explanations at the beginning for the 8 and 10-year-olds. We made explanations verbal.8 and 10-year-olds. We made explanations verbal.

We deleted many socio-demographic items, or We deleted many socio-demographic items, or placed them in the appendix. We placed some placed them in the appendix. We placed some other items in the appendix, in order to use a other items in the appendix, in order to use a short version of each Q as a basis.short version of each Q as a basis.

We added new items on satisfaction with the city We added new items on satisfaction with the city (village/neighbourhood) services and living (village/neighbourhood) services and living conditions. We also changed the phrasing of some conditions. We also changed the phrasing of some agreement/disagreement items regarding the agreement/disagreement items regarding the city, and added some others.city, and added some others.

Testing the 3 questionnaires with Testing the 3 questionnaires with the adopted format (Catalonia only) the adopted format (Catalonia only) (2):(2):

In the 12 y.o. Q we added one scale on values In the 12 y.o. Q we added one scale on values aspired to.aspired to.

In the 10 and 12 y.o. Q we added a set of items In the 10 and 12 y.o. Q we added a set of items about self-categorization as a media consumer.about self-categorization as a media consumer.

We added “I don’t know” to many items. Children We added “I don’t know” to many items. Children were told they should understand “Not were told they should understand “Not appropriate” as “I do not know”.appropriate” as “I do not know”.

Children who filled in their Q very fast were given Children who filled in their Q very fast were given an “appendix” with additional items.an “appendix” with additional items.

Most children of all ages finished their Q in about Most children of all ages finished their Q in about 40 minutes. Many had time to fill in the appendix 40 minutes. Many had time to fill in the appendix as well.as well.


The 8-year-olds’ Q provoked very few questions.The 8-year-olds’ Q provoked very few questions. Reading skills were clearly better in private Reading skills were clearly better in private

schools than in public schools.schools than in public schools. Many children complained about the inclusion of Many children complained about the inclusion of

very sensitive “personal” questions (religion, very sensitive “personal” questions (religion, being wealthy).being wealthy).

Phrasings about city/village/neighbourhood Phrasings about city/village/neighbourhood services may need adaptations according to the services may need adaptations according to the context (rural, semi-urban, big city).context (rural, semi-urban, big city).

Different language-specific issues arise according Different language-specific issues arise according to the individual cultural and social context, and to the individual cultural and social context, and these need to be discussed with children. these need to be discussed with children.


Spanish 8 and 10-year old children had problems Spanish 8 and 10-year old children had problems answering the following items:answering the following items:

Pocket money Pocket money I would like my town to be greenerI would like my town to be greener Going to sleep hungryGoing to sleep hungry Feeling secure at home (doubt: when alone or when Feeling secure at home (doubt: when alone or when

with relatives?)with relatives?) Religion or spiritualityReligion or spirituality Adults at home having a paid job.Adults at home having a paid job. Calculating time invested in different activities. Calculating time invested in different activities.

However, they can report the time each activity However, they can report the time each activity starts and finishes.starts and finishes.


At 8 years old reading skills are very varied (i.e. in At 8 years old reading skills are very varied (i.e. in Brazil most children start school and reading when Brazil most children start school and reading when they are 7 y.o.). However, there is something they are 7 y.o.). However, there is something perhaps more important: they want to give quick perhaps more important: they want to give quick answers. They avoid nuance – i.e. they quickly look answers. They avoid nuance – i.e. they quickly look for the significant part of the question and “skip” for the significant part of the question and “skip” the statement.the statement.

In general, 8-year-olds need short, concrete In general, 8-year-olds need short, concrete questions.questions.

Reading the instructions for the questionnaire is too Reading the instructions for the questionnaire is too time-consuming for 8 and many 10 y.o. In a group time-consuming for 8 and many 10 y.o. In a group administration context it is much better to give administration context it is much better to give them verbal explanations and answer all the them verbal explanations and answer all the questions they raise. Trained researchers should questions they raise. Trained researchers should also allow clarifications by peers – but only focused also allow clarifications by peers – but only focused on understanding the format and questions and on understanding the format and questions and avoiding “contamination” of the answers. avoiding “contamination” of the answers.

Testing the 3 Q: General Testing the 3 Q: General observations (1)observations (1)

There is a big gap between 10 and 12-year-There is a big gap between 10 and 12-year-olds with regard to their understanding of olds with regard to their understanding of complexity and more abstract words. complexity and more abstract words.

12 y.o. have an outstanding interest to seek 12 y.o. have an outstanding interest to seek out nuances and be precise.out nuances and be precise.

Testing the 3 Q: General Testing the 3 Q: General observations (2)observations (2)

http://www.udg.edu/eridiqv

Multinational study of children’s well-being and daily activities

Format pre-test andinstrument testing

SPANISH AND BRAZILIAN REPORT

Coordinated by:Ferran Casas (ERIDIQV,

University of Girona)Developed by:

Mònica González, Sara Malo, Carme Montserrat, Dolors

Navarro, Irma Bertran, Cristina Domènec and Claudia Sarrión

(ERÍDIQV) – and by Jorge C. Sarriera, Angela Paradiso, Anelise L. Rodrigues, Tiago Calza, Rebeca Linhares, Tamiris Kreibich and Fabiane

Schultz. (GPPC: Grupo de Pesquisa em psicologia

comunitária. Universidade Federal do Rio Grande do Sul).

Documents

Http:// Multinational study of children’s well-being and daily activities Format pre-test and instrument testing SPANISH AND BRAZILIAN