An ELT Glossary : Reliability

Definition : Reliability is the term used in testing to indicate whether the scores on a test give an accurate result - ie competent users of the language do well whilst those with less competence do less well.

There are two types of reliability :
1) Test-retest reliability (if the learner took the test twice would s/he get the same result?)

2) Mark-remark reliability (if the test was marked by two different people, would the result be the same?)

Reliability may be threatened by various factors. For example :

  • a subjective test such as an oral interview or a test involving the production of a written text may be unreliable if there are not clearly defined marking criteria which all markers adhere to. Otherwise, different markers might award/detract marks for different factors, thus threatening mark-remark reliability - ie,g. if the same oral interview was marked by two different markers, the result might be quite different because one was concentrating on accuracy while the other gave more credit for fluency..
  • the conditions under which it is taken. Eg if during a listening test there is suddenly a lot of background noise, the candidates are unlikely to achieve the same results as they would in a silent room. Test-retest reliability is therefore threatened.
  • Confusing rubrics. If the learner doesn't fully understand the instructions s/he will go "off track" in the task and therefore obtain a score which is lower than s/he would otherwise have done. Again this threatens test-retest reliability. If eg the learner has recognised the problem half way through the test, on a retake s/he would be prepared and would do better.