Coverage refers to the extent to which a test adequately samples or represents the content, skills, or abilities it is intended to measure. The quality of the coverage of a test will also affect the validity of the test and the reliability of the results.
Types of coverage include:
-
Test content coverage: How well the test items reflect the range of linguistic knowledge (e.g. vocabulary, grammar, discourse features) and skills (e.g., reading, writing, listening, speaking) that the test claims to assess. A test with poor content coverage would automatically lack content validity. Examples:
- A test that intended to assess overall grammatical competence but which mainly contains items testing verb forms, while ignoring other grammatical features such as articles, prepositions, and word order, has poor overall content coverage of grammar. A more balanced grammar test would sample a wide range of grammatical forms and structures.
Suppose a speaking test claims to assess discourse competence (the ability to organize speech beyond the sentence). If all the tasks only require short turn-taking (e.g. giving single responses in an interview format), then the test is missing coverage of other discourse features such as turn taking, coherence and cohesion across multiple turns, managing topic shifts or repairs, etc. This weak coverage directly undermines construct validity as well as content validity
-
Curriculum coverage: How well the test represents the objectives, topics, items and subskills specified by the syllabus or taught in any preceding course. Examples:
- A proficiency test is intended to certify proficiency at a specific level - eg Cambridge First evaluates the candidates' performance at CEFR B2 level. It should therefore include balanced assessment of all the items on the syllabus for that level. (See the EAQUALS/British Council Core Inventory for General English.) If a B2 proficiency test was heavily oriented to assessment of functions/subskills/language items/topics etc specified for lower levels, it would have poor curriculum coverage and therefore the results would be less reliable.
- A progress test or achievement test should provide balanced assessment of all the items taught during the course (or in the case of a progress test all those included in the course up to that point). For example, imagine a course which provided progress tests every five units. The second test should therefore provide balanced assessment of what was taught in units 6-10. If on the other hand it emphasised items from only units 9 and 10, it would lack curriculum coverage and content validity. If some learners had missed lessons during these units the results would also be unreliable, as they would do badly in comparison with learners who had been present. If, on the other hand, the test items were drawn from all five units equally, it would be more accurately apparent what they had fully assimilated during the course and what not. Again, coverage has a direct influence on result reliability.
-
Domain coverage: How comprehensively the test represents the real-life language use situations or communicative tasks that learners are expected to perform. If not, the test would lack construct validity and also predictive validity. Examples:
- Suppose a test claims to measure academic writing ability. If the test only asks students to fill in sentence-level gaps, then its domain coverage is very limited. It leaves out important aspects of academic writing such as organizing ideas, developing arguments, using academic style and technical vocabulary or maintaining coherence and cohesion across paragraphs.
If a general purpose test is administered to a Business English student as a diagnostic test, it will not include the competencies that the learner needs - eg describing a product, dealing with complaints, giving presentations, participating in meetings etc. It therefore has weak domain coverage. On the other hand, a test based on an analysis of the learner's needs would cover all of these areas (and others) which are relevant to the specific learner and the test would therefore have strong domain coverage.
