Measuring readiness for simplified material: a test of the first 1,000 words of English - Paul Nation 1993

Measuring readiness for simplified material: a test of the first 1,000 words of English.

In Simplification: Theory and Application ed. M.L.Tickoo, RELC Anthology Series No 31, 1993, pp 193-203.

Measuring Readiness for Simplified Material: A Test of the First 1,000 Words of English

Paul Nation
Victoria University of Wellington


The first 1,000 words of English are the essential basis for simplified teaching material. This article describes the need for a test of these words and the difficulties in making one. It contains two equivalent forms of a test along with instructions on how to use it and how to apply the information gained from it.

The importance of high frequency words

Frequency studies of English have shown that the return for learning the high frequency words is very great. Generally these high frequency words are considered to be the most frequent 2,000 words (West, 1953) although some research indicates that the return for learning vocabulary drops off rather quickly after the first 1,500 words (Engels, 1968; Hwang, 1989). The return for learning is the coverage of text, spoken or written, that knowledge of the words provides. For example, Schonell et al (1956) found that the most frequent 1,000 words in spoken English provided coverage of 94% of the running words in informal conversation. Similarly, figures from the frequency count by Carroll et al (1971) indicate that the first 1,000 words of English cover 74% of written text. Note that coverage refers to running words where each recurrence of a word is counted as additional coverage. Thus, knowing the word the gives much less than 10% coverage of written text because this word occurs so frequently. Clearly the return for learning the first 1,000 words of English is very high. By comparison, the second most frequent 1,000 words of English provides coverage of only 7% of written text.

It should not be thought that the first 1,000 words is made up mainly of words like the, and, of, they, and because. These function words make up fewer than 150 of the 1,000 words.

Lists containing the first 1,000 words

There are several lists available of the most frequent words of English. These include frequency counts (Carroll et al, 1971; Francis & Kucera, 1982; Thorndike & Lorge, 1944), and combinations of various lists (Hindmarsh, 1980; Barnard & Brown, in Nation 1984). The list chosen for this test is West's General Service List of English Words (1953). The General Service List has been used as a basis for many series of graded readers, and this provides an advantage in using it for the test. This list is rather old, based on work done in the 1930s and 1940s. However it still remains the most useful one available as the relative frequency of various meanings of each word is given. When making the tests included here the words chosen were checked against the Carroll et al count to make sure that they occurred in the first 2,000 words of that count.

Difficulties in testing the first 1,000 words

There are several difficulties involved with making a test of the first 1,000 words. The first is such a test may be used with classes of learners who speak different first languages and thus translation is not a practical approach. Second, there is the likelihood that some learners will have poor reading skills and thus the test needs to be able to be given orally if necessary. These two factors resulted in the choice of a true/false format. Multiple-choice was not possible because it is impractical in an oral form. One disadvantage of true/false is the possible strong effect of guessing, although research by Ebel (1979) indicates that this is not as likely as it seems. In an attempt to overcome possible effects of guessing, three types of responses were suggested in the instructions (True, Not true, Do not understand), and each word was tested twice, once in each version of the test. Where an item is tested twice, there are four possible sets of answers, namely both correct, both wrong, the first item correct and the second wrong, the first item wrong and the second correct. There is thus only a one in four chance of correctly guessing both items testing the same word. This is the same chance as with four-item multiple choice tests. So if the teacher feels that learners are making wild guesses, both forms of the test should be given and a mark given only when both items testing a word are correct.

A third difficulty is that the contexts for the tested words must not cause too many problems for the learners. When making the test, an attempt was made to ensure that the context words were of higher frequency than the tested word. This was not always possible for some of the words and thus a few words have some context words of the same frequency. There are no items with contexts of lower frequency. Occasionally a picture was used to avoid a lower frequency word, for example:
This can keep people away from your house.



Dog is a lower frequency word than the test word keep and so a picture was used instead of saying:
A dog can keep people away from your house.
This frequency restriction on the context was the most difficult constraint to overcome when making the test.

A fourth difficulty is that most of the high frequency words have several meanings. In the test only the most frequent meaning was tested. This was found by referring to West (1953) and the COBUILD dictionary (Sinclair, 1987).

A fifth difficulty is that using true/false items where the judgement is based on general knowledge allows other factors besides vocabulary knowledge to play a part. Some items where this may occur include:
Some children call their mother Mama.
You can go by road from London to New York.
Each society has the same rules.
Some problems of this type were removed as a result of trialling the test. There is value, however, in having the words in context in that the context can help in accessing the meaning of the word as well as limiting the meaning that is being tested. The disadvantage of drawing on general knowledge is not as great as the advantage of testing in use rather than by definitions.

A sixth difficulty is the grammatical complexity of the context of the tested words. For example, several of the highest frequency items are tested in the two-clause pattern "When _____, _____". This was unavoidable. Trialling of the test helped find some items where this caused too much difficulty and these were changed.

Using the test

Usually one form of the test (40 items) should be enough to get a useful result. When the test is given orally, the learners will need to be able to see the accompanying pictures. It is probably best if the test is given orally to one learner at a time. The teacher can repeat the items to the learners as many times as is needed. If the teacher knows the learner's first language then also requiring a translation would be a useful check. It is possible to find which word is tested by comparing the two items in the two forms of the test as both forms contain the tested words in exactly the same order. The ordering is based on frequency of occurrence according to West (1953) with the most frequent word (time) occurring first.

Only content words (nouns, verbs, adjectives, adverbs) are tested. To find what proportion of the first 1,000 words is known, multiply the total score on each version of the test (40 items) by 2.5. Multiplying by 2.5 assumes that the learners already know the same proportion of function words.

Applying the results

The results of the test can be used to help diagnose areas of weakness, set learning goals and plan a vocabulary programme, measure vocabulary growth, and assign graded reading. Let us look at each of these in turn.

Diagnosis: The test can be used to help answer this question. Is the learner's poor performance in reading or listening a result of inadequate vocabulary knowledge? Some learners, particularly those for whom English is a foreign language, have difficulty understanding spoken English. This could be because they do not know enough vocabulary or simply because they have learned English through reading and have not had enough contact with spoken English. Giving the vocabulary test in its written form should help the teacher see where the problem lies. With such learners it would be interesting to give one form of the test orally and one form through reading to see what the difference was.

Similarly, learners who have had a lot of contact with spoken English may be poor at reading and doing the test orally should reveal their vocabulary knowledge.

Set learning goals: The first 1,000 words of English are essential for all learners who wish to use the language. It is thus very important that teachers know what vocabulary knowledge their learners have and are aware of how they can systematically help them to increase this knowledge. If learners do not know all of the first 1,000 words of English it is well worth ensuring that they have the opportunity to learn those that they do not know. Nation (1990) looks at this in detail over the four skills of listening, speaking, reading and writing. Ways of doing this include substantial graded reading, direct vocabulary teaching, doing vocabulary learning exercises, and systematically providing a vocabulary focus in language learning activities. If learners' vocabulary is larger than 1,000 words, the Vocabulary Levels Test (Nation, 1990) can be used.

Measure vocabulary growth: The two equivalent forms of the 1,000 word test allow the teacher to check how much learners' vocabulary has increased over several months. This use should be treated with caution as each test has only forty items and thus the confidence interval would be large if we were measuring an individual's increase in vocabulary size. When both forms of the test were administered to the same group of learners, it was found that the most difficult items in test A tested words that were also in the most difficult items in test B. These words were ancient, stream, remain, wide, and at least. It was also found that two-thirds of the learners gained scores on tests A and B that were within two marks or less of each other. Only one of the fifteen learners tested had scores which were more than four marks different.

Assign graded reading: Various series of graded readers have several stages of readers within the first 1,000 words of English. Longman Structural Readers, for example, have books written at the 300-word stage, the 500-word stage, the 750-word stage, and the 1,100-word stage. The way that the Longman series divides the words into stages does not correspond exactly to frequency (and thus to the ordering of items in the vocabulary test) but there is rough agreement. For example, the first 10 items in the vocabulary test are made up of one test word from Longman Stage 1, six from Stage 2, and three from Stage 3. Because the agreement is rough, it is better to use learners' total scores on the test to decide what stage of graded reader they should be reading. If their vocabulary score on a 40-item test is less than 10 they should be reading at Stage 1, from 11 to 20 Stage 2, from 21 to 30 Stage 3, and above 30 Stage 4. Graded reading is an excellent way of increasing vocabulary. By reading three or more readers at one stage learners are likely to meet all of the vocabulary at that stage. Having mastered the vocabulary of that stage, they can go to the next stage without needing extra preparation for the new vocabulary (Wodinsky & Nation, 1988).

The content of test items

The items in a test which is not based on a particular piece of content knowledge inevitably reveal the personality of the test maker. Looking back over the items I see that some reflect my philosophical attitudes, "We can be sure that one day we will die" (Some learners seemed convinced that this was not true.). This same sense of inevitability is revealed in "Day follows night and night follows day" and "Your child will be a girl or a boy". I also see my jaundiced attitudes to children after having raised a family, "It is easy for children to remain still" (clearly not true), "Most children go to school at night" (perhaps that should be true), "A child has a lot of power" (true or not true? Unfortunately omitted). In the earlier versions of the test there was also a strong moral tone, "It is good to keep a promise", "It is not good to try hard", "You must look to find the way". However, although the learners did not seem to have trouble with these items, colleagues convinced me that these were culture bound and not in keeping with the tone of the last part of the twentieth century. I reluctantly changed some of them. It is after all easier to change test items than it is to change colleagues. After all, "A society is made of people living together".

References

Carroll, J.B., P. Davies and B. Richman 1971. The American Heritage Word Frequency Book. New York: American Heritage Publishing Co.

Ebel, R. L. 1979. Essentials of Educational Measurement. (3rd ed.) Englewood Cliffs: Prentice Hall.

Engels, L. K. l968. The fallacy of word counts. IRAL 6,3: 2l3-23l.

Francis, W. Nelson and Kucera, Henry 1982. Frequency Analysis of English Usage. Boston: Houghton Mifflin Company.

Hindmarsh, R. l980. Cambridge English Lexicon. Cambridge: Cambridge University Press.

Hwang Kyong Ho 1989. Reading newspapers for the improvement of vocabulary and reading skills. Unpublished M.A. thesis, Victoria University of Wellington.

Nation, I.S.P. 1984. Vocabulary Lists. English Language Institute Occasional Publication No.12, Victoria University of Wellington.

Nation, I.S.P. 1990. Teaching and Learning Vocabulary. New York: Newbury House.

Schonell, F. J., I.G. Meddleton, and B.A. Shaw l956. A study of the oral vocabulary of adults. Brisbane: University of Queensland Press.

Sinclair, J. (ed.) l987. Collins Cobuild English Language Dictionary. London: Collins.

Thorndike, E. L. and I. Lorge l944. The Teacher's Word Book of 30,000 Words. Teachers College, Columbia University.

West, Michael l953. A General Service List of English Words. London: Longman, Green & Co.

Wodinsky, M. and I.S.P. Nation. 1988. Learning from graded readers. Reading in a Foreign Language 5,1: 155-161.