Test Information Booklet
March 2014
V. 0.2
Global Scale of English 10-58
1
Contents
Introduction ........................................................................................................ 2
Test Validation ................................................................................................... 3
Test Coverage .................................................................................................... 6
Test Questions ................................................................................................... 7
Scores and Results .......................................................................................... 13
2
Introduction
Progress: An overview
Progress is a part adaptive computer-based test of English language proficiency designed
to be used on a number of occasions through a course of study to accurately show
learners’ progress.
The Purpose of Test
The test is normally used in a learning context in conjunction with relevant materials and
formative assessment tasks, but it can be used independently. It provides detailed
information to a teacher about a group or class of learners who are studying any course,
and information about each learner in the group. The information allows the teacher to
make decisions about adapting learning material to suit the level of both group and
individual learners and providing extension activities where the group or particular
individuals are weak. It also allows the teacher to tailor the learning program to
particular learners, giving extra support and input where required.
Who is it for?
The test is designed for Adult learners who are 16 or older. Progress can be used
alongside adult or upper secondary courses. It is intended to be used with
comprehensive integrated skills courses not short or partial courses.
Why take an integrated skills test?
Some of the questions Progress uses test a single skill such as speaking or writing. When
assessing these skills we test skills such as pronunciation and fluency, the ability to
argue as well as written conventions along with grammar and vocabulary. A number of
the questions on the test are integrated skills questions. These questions test more than
one skill at the same time.
Using integrated skills questions means that Progress is a better test of a learner’s
English. In real life and in the classroom learners use more than one skill to complete
communicative tasks. To order something in a restaurant we need to listen and speak, to
take notes in a classroom we need to listen and write. Integrated skills questions test
how well learners can use the skills they have learnt and practised in the classroom and
used in real life.
3
Test Validation
Test design
Progress is designed specifically to measure progress in language proficiency. It employs
a part adaptive method. Part of the test uses an adaptive algorithm which takes a
learner’s answers to a previous question to select the most suitable question to present
next. Progress selects these items from a large item bank making each learner’s
experience different.
The adaptive nature of the test allows Progress to quickly and accurately estimate a
learner’s English proficiency. This estimate is then used to choose further questions
which are fine-tuned to the learner’s level allowing a very accurate measure of their
proficiency. By taking two or three tests over a period of study the learner, and the
teacher, can see how much progress is being made.
Test development
The questions in Progress have been developed by international teams of writers who
are very experienced in writing assessment questions. Teams are based in the UK,
Australia, the USA and Hong Kong. All questions have been tagged with a Global Scale of
English (GSE) level and linked to a ‘can do’ statement.
Once written, all questions are reviewed by the teams in the different countries.
Comments and suggestions for improvement are stored with the test questions on a
secure database. The questions then go through a further review by an expert panel and
decisions are made on the quality of the questions; which to keep and which to reject.
All questions are then thoroughly checked by Pearson staff and images and high quality
recordings are added to complete the questions before they go forward to be calibrated
in a large scale field test.
After the field testing further checks are made on item quality based on the
measurement characteristics of the questions. Questions are eliminated from the item
pool if they are too easy or too difficult, if weaker learners get them right but stronger
learners get them wrong, or if they show any bias. These checks then result in a bank of
the best quality questions. Questions are selected from this bank to go into the final
tests.
Field testing
To ensure the questions used in Progress measure accurately they have been field
tested. The Field Test took place in Asia, Europe, South America and the Middle East. It
involved almost 4500 learners in 18 different countries who spoke 13 different
languages. The Field Test used 64 tests with over-lapping questions. This meant a model
could be built linking all questions and all learners. The Field Test contained questions
from other tests which had already been calibrated to give further evidence of the
difficulty of the test questions.
The answers that the learners gave were analysed and used to establish the difficulty of
each question. This process was applied, as described above, to keep or reject
questions, to help the adaptive algorithm select questions and in the scoring of the test.
4
Alignment to the Common European Framework of Reference (CEF)
Progress is aligned to the Common European Framework of Reference (CEF) levels in a
number of ways:
Question writers are very experienced teachers and authors who were also
trained on the CEF levels.
Questions were written to specific CEF levels using ‘can do’ statements taken
from the CEF documents.
Questions from other Pearson Tests (PTE Academic, PTE General and Versant)
which were already aligned to the CEF were used in the Field Test as anchor
items. PTE Academic uses the same reporting scale as Progress so is absolutely
aligned. PTE General and Versant use similar scales which are closely related.
Many learners who took the Field Test were studying at levels which had been
previously linked to CEF levels in a comprehensive academic study.
Learners’ responses in the Field Test were independently rated by examiners
trained on the CEF levels to check they were at the right level of proficiency.
From the Field Test analysis the averaged candidate ability estimates were compared
with human ratings-based estimates of their CEFR levels to produce the conversion
equation for theta1 to the GSE. The regression line shows a very good fit to the data at
A1, A2 and B1, where the test is designed to measure, as shown in the plot below. The
B2 and C1 points are extrapolated and not an outcome from the dataset.
Theta estimates from Conquest analysis plotted to GSE scale
1 Theta is a psychometric value expressing the estimate of ability
5
Scoring
Progress uses the most up-to-date scoring technology. Progress uses an adaptive
algorithm to ensure that the questions selected for a learner are suitable for their level.
Progress also uses automated scoring, including the automated scoring of speaking and
writing items. This means scoring is objective and consistent wherever or whenever
Progress is taken. The automated scoring engines are trained using large samples of
learners’ answers which were collected in the Field Test and marked by experienced and
well-trained examiners. These scores are then further validated by using separate sets of
learners’ responses.
Because Progress is part adaptive and closely follows the learner’s language level, it is a
very accurate test. Alongside this it uses questions which have a number of score points
which means we have a large amount of information about the learner’s English
proficiency.
6
Test Coverage
The test covers all four language skills; reading, listening, speaking and writing as well
as knowledge of grammar and vocabulary.
Skills or
Knowledge
Test Focus
Reading To demonstrate reading skills, learners will be asked to:
read and understand the main points from signs, newspapers
and magazines
understand the detail of short texts
understand the detail in longer texts
Listening To demonstrate listening skills, learners will be asked to:
listen for specific information in listening texts
show understanding of meaning in context and the detail of
short dialogues
follow and understand short texts and show understanding by
writing down or repeating accurately what was said
Speaking To demonstrate speaking skills, learners will be asked to:
speak clearly using appropriate stress and intonation
pronounce words so that they can be understood
describe pictures or other visual material connecting ideas
together accurately and with a range of language
Writing To demonstrate writing skills, learners will be asked to:
describe a scene or picture accurately using appropriate
vocabulary
write a story accurately linking ideas and using appropriate
vocabulary
write a short essay giving different points of view, linking ideas
and using accurate and appropriate language
Grammar To demonstrate knowledge of grammar, learners will be asked to:
choose the right word or phrase to make an accurate sentence
understand the difference between different grammatical
tenses and other structures
put words in the right order to make grammatical sense
Vocabulary To demonstrate knowledge of vocabulary, learners will be asked to:
produce words which relate to common themes and topics such
as family, work and social situations
use appropriate words in different contexts
show an understanding of the different meaning of words and
how they relate to other words
7
Test Questions
What kinds of questions are in the test and what do they measure?
The test has a number of different question types. This gives learners a chance to
demonstrate their English skills in different ways. There are questions where learners
choose the correct option or where they write the answer into an open question. There
are questions where the learner repeats or copies what has been said as well as
questions where learners describe something or write a short essay. The questions are
similar to the questions and tasks learners will have done in the classroom as part of
their learning and so should be familiar.
Because Progress is part adaptive, different learners will see different questions and may
not be presented with all the questions described below.
Vocabulary Questions
There are three vocabulary question types. Vocabulary is also tested as part of Describe
Image, Short Essay and Read and then Write which are Integrated Skills questions.
Question What do the learners
have to do?
What is being tested?
Fill in the Table This question asks the
learner to complete a set
of vocabulary items with
appropriate words. The
words are presented as a
table of related words.
This question tests the
vocabulary knowledge of the
learner. It tests the words
the learner knows and the
accuracy of the form of the
word. It tests the learners
knowledge of word families
and related sets of words
that they may have met in
the classroom or when
learning English.
Choose the Right Word or
Phrase
This question asks the
learner to choose the
correct word to complete
a number of sentences.
The sentences are related
by a similar theme.
This question tests the
vocabulary knowledge of the
learner in a written context.
It tests the vocabulary the
learner knows and whether
they can understand the use
of the vocabulary in the
context of a sentence. It
tests the range of
vocabulary the learner
knows.
8
Complete the Dialogue This question asks the
learner to select words
from a word bank to
complete a dialogue.
This question tests the
vocabulary of the learner in
a spoken context. It tests
the vocabulary the learner
knows and whether they can
understand the use of the
vocabulary in the context of
a conversation. It tests the
range of vocabulary the
learner knows.
Grammar Questions
There are three grammar question types. Grammar is also tested as part of Short Essay,
and Read and then Write which are Integrated Skills questions.
Question What do the learners
have to do?
What is being tested?
Choose the Right Word or
Phrase
This question asks the
learner to choose the
correct word to complete a
number of sentences. The
sentences are related by a
similar theme.
This question tests the
knowledge of grammar of
the learner. It tests the
range of grammatical
knowledge as well as the
accuracy of grammar in a
written context.
Choose the Right Word or
Phrase. You may choose
more than one.
This question asks the
learner to choose from a
number of options. They
may choose one or more
than one answer. The
sentences are related by
the grammatical structure
which is being tested.
This question tests the
grammatical knowledge of
the learner. It tests words
which are related to each
other in that they have
similar meanings or
grammatical uses. It tests
grammatical knowledge in a
written context.
Drag and Drop This question asks the
learner to re-order a
sentence correctly.
This question tests the
grammatical knowledge of
the learner at sentence
level. It tests word order,
connectors and discourse
markers. It tests
grammatical knowledge in a
written context.
9
Reading Questions
There are three reading question types. Reading is also tested as part of Read and then
Speak, Read and then Write, and Listen and Read which are all Integrated Skills
questions.
Question What do the learners
have to do?
What is being tested?
Choose the Right Picture This question asks learners
to read a short text and
select the best picture to
match with the text.
This question tests the
global understanding of
short messages, notes and
short pieces of writing.
Choose the Right Word or
Phrase
This question asks learners
to read a short text and
select the best word or
phrase to complete the
text.
This question tests the
global understanding of
short messages, notes and
short pieces of writing.
Short Answer This question asks the
learner to read a longer text
and answer questions on
the text.
This question tests the
reading comprehension of
the learner. It tests specific
information included in the
text.
Listening Questions
There is one listening question type which tests only listening. Listening is also tested as
part of Listen and then Write, Listen and then Speak, and Listen and Read which are all
Integrated Skills questions.
Question What do the learners
have to do?
What is being tested?
Listen to the Conversation
and Answer
This question asks the
learner to listen to a short
conversation and then
answer a question about
the conversation.
This question tests listening
comprehension. It tests the
accuracy of the listening
comprehension of the
learner.
10
Speaking Questions
There is one speaking question type which tests speaking and grammar. Speaking is also
tested as part of Read and then Speak and Listen and then Speak which are Integrated
Skills questions.
Question What do the learners
have to do?
What is being tested?
Describe Image This question asks the
learner to look at a
photograph or picture and
describe what they see.
This question tests the
learner’s ability to speak in
an extended way linking
concepts and ideas. It tests
the accuracy of speech
including accurate
grammar, pronunciation
and stress as well as the
fluency of the speech. It
tests the use of appropriate
words to describe the
photograph or picture.
Writing Questions
There is one writing question type which tests only writing. Writing is also tested as part
of Listen and then Write and Read and then Write which are Integrated Skills questions.
Question What do the learners
have to do?
What is being tested?
Short Essay This question asks the
learner to write a short
essay in response to a
prompt.
This question tests global
writing skills. It tests
paragraph and sentence
structure, the range and
accuracy of the language
used, the ability to
structure an argument or
discussion in a written
context. It tests grammar
and vocabulary as an
essential part of writing.
11
Integrated Skills Questions
There are five questions types which measure more than one skill at the same time.
These are called Integrated Skills Questions.
Question What do the learners
have to do?
What is being tested?
Read and then Speak This question asks the
learner to read aloud a
sentence or short text.
This question tests accurate
pronunciation and how
fluent the learner is at
speaking. It tests if the
words in the text are
understood and repeated
accurately.
Listen and then Write This question asks the
learner to listen to a
sentence or short text and
write what they have heard.
This question tests listening
comprehension at the word
and sentence level. It tests
the ability to write
accurately and understand
sentence structure, word
order and connectors.
Listen and then Speak This question asks the
learner to listen to a
sentence or short text and
then repeat it.
This question tests listening
comprehension at the word
and sentence level. It tests
pronunciation and fluency.
It tests if the words heard
are understood and
repeated accurately.
Read and then Write This question asks the
learners to read a short
story or short piece of
factual text. The text then
disappears and the learner
has to reconstruct the text.
This question tests reading
comprehension. It tests the
ability to write accurately
and understand sentence
structure, word order and
connectors.
Listen and Read This question asks the
learner to read a text and
at the same time listen to
the text. The learner has to
find the differences between
the written text and the
spoken text.
This question tests reading
and listening
comprehension. It tests the
ability to recognise
individual words in a text.
12
Test Familiarity
Learners can take the sample test at any time to familiarise themselves with the
question types in the test. Teachers can also assign this test if they want to ensure
learners do this as a homework activity.
13
Scores and Results
Global Scale of English
The test result provides scores on the Global Scale of English which ranges from 10 to
90. The scores consist of an overall score, and sub scores for listening, reading,
speaking, writing, vocabulary and grammar. The test also reports Common European
Framework levels.
The Global Scale of English is a numeric, granular scale from 10 to 90 which measures
English language proficiency. It enhances the Common European Framework of
Reference (CEFR) by showing finer gradations of a learner’s level within a CEFR band,
and can therefore demonstrate smaller and more precise improvements in a learner’s
English level.
The Global Scale of English is currently used to report scores on the internationally
recognised English language test, PTE Academic™. It is empirically aligned to the CEFR,
as described in the Common European Framework of Reference for Languages: Learning,
Teaching, Assessment (Council of Europe, 2001),and correlated to other test score
scales such as TOEFL® iBT, TOEIC® and IELTS.
What do the scores mean?
The table below shows the description of the different levels on the Global Scale of
English as it is aligned to the Common European Framework levels.
Global Scale
of English
(GSE)
Common
European
Framework
Levels
Main features of a learner’s language ability at
this level taken from the (Common European
Framework of Reference for Languages:
Learning, Teaching, Assessment, Council of
Europe, 2001).
10 - 21 <A1 This level of proficiency is likened to a tourist who
may know some individual words but does not have
enough control of language to produce full sentences
and mostly communicates with words or very basic
phrases.
The words they do know may carry a lot of
communicative meaning or be effective when used
with hand gestures or when the context is very clear
(e.g. pointing to an object in a shop).
22 - 29 A1 Learners at this level can:
14
• understand and use familiar everyday expressions
and very basic phrases aimed at the
satisfaction of needs of a concrete type
• introduce themselves and others
• ask and answer questions about personal details
(e.g. where they live, people they know
and things they have)
• interact in a simple way, provided the other person
talks slowly and clearly and is prepared to help
• initiate and respond to simple statements in areas of
immediate need or on very familiar topics, rather than
relying purely on a very finite, rehearsed, lexically-
organised repertoire of situation-specific phrases
30 - 36 A2 Lower End It is at this level that learners demonstrate proficiency
mainly as social functions:
• use simple, everyday polite forms of greeting and
address
• greet people, ask how they are and react to news
• handle very short social exchanges
• ask and answer questions about what they do at
work and in free time
• make and respond to invitations, make and accept
offers
• discuss what to do, where to go and make
arrangements to meet
Also at this level are descriptors referring to getting
out and about, e.g.:
• make simple transactions in shops, post offices or
banks
• get simple information about travel, use public
transport: buses, trains and taxis
• ask for basic information, ask for and provide
everyday goods and services
• ask for and give directions and buy tickets
37 - 42 A2 Upper End At this level there is more active participation in
conversation given some assistance and certain
limitations, e.g.:
• initiate, maintain and close simple, restricted face-
to-face conversation
• understand enough to manage simple, routine
exchanges without undue effort
• make themselves understood and exchange ideas
and information on familiar topics in
predictable everyday situations, provided the other
person helps if necessary
• though they will generally have to compromise the
message and search for words but participation in
open discussion is fairly restricted
Plus significantly more ability to sustain monologues,
e.g.:
15
• express how they feel in simple terms
• give an extended description of everyday aspects of
their environment (e.g. people, places, a job or study
experience)
• describe past activities and personal experiences
• describe habits, routines, plans and arrangements
• explain what they like or dislike about something
• give short, basic descriptions of events and activities
• use simple language to describe and compare
objects and possessions
43 - 50 B1 Lower End This is a summary of what distinguishes B1 and the
change that learners make from the upper end of A2
to becoming B1. There are two main features:
The first feature here is the ability of the learner to
maintain interaction and get across what they want to
in a range of contexts, e.g.:
• generally follow the main points of extended
discussion around them, provided speech
is clearly articulated in standard dialect
• give or seek personal views and opinions in an
informal discussion with friends
• express the main point they want to make
comprehensibly
• exploit a wide range of simple language flexibly to
express much of what they want to
• maintain a conversation or discussion but may
sometimes be difficult to follow when
trying to say exactly what they would like to
• keep going comprehensibly, even though pausing
for grammatical and lexical planning
and repair is very evident, especially in longer
stretches of free production
The second feature is the ability to cope flexibly with
problems in everyday life, e.g.:
• cope with less routine situations on public transport
• enter unprepared into conversations on familiar
topics
• make a complaint
• take some initiatives in an interview/consultation
but is very dependent on interviewer in the interaction
• ask someone to clarify or elaborate what they have
just said
51 – 58 B1 Upper End At the upper end of B1 the learner is extending their
competence including a focus on the exchange of
quantities of information:
• take messages communicating enquiries, explaining
problems
• provide concrete information required in an
interview/consultation (e.g. describe
symptoms to a doctor) but does so with limited
16
precision
• explain why something is a problem
• summarise and give their opinion about a short
story, article, talk, discussion, interview
or documentary and answer further questions of
details
• carry out a prepared interview, checking and
confirming information, though they may
occasionally have to ask for repetition if the other
person’s response is rapid or extended
• describe how to do something, giving detailed
instructions
• exchange accumulated factual information on
familiar routine and non-routine matters
within their field with some confidence
17