Reliability measurement in survey situations is more difficult and less easily executed than in observational studies.
While it is possible to observe a certain action over and over again, it is usually possible to repeat a survey only once. This leads to a test-retest method, which compares two tests to learn how reliable they are. The method consists of administering the test to the same group of individuals on two different occasions.
Then a test-retest coefficient sometimes referred to as a coefficient of stability, is determined. The reliability coefficient, in this instance, is simply the Pearson’s product-moment correlation coefficient between the scores obtained by the same persons on the two administrations of the test.
The method is illustrated by an example below.
Example of Test-retest Method
Consider a group of adolescents who were asked to name a few methods of contraceptives at a given point in time. The reported answers were recorded in numbers 0, 1,2, etc.
At a later date, the same group was asked the same questions, and their answers were recorded exactly in the same manner.
The correlation coefficient computed from these two sets of scores provides us with a measure of stability. We illustrate this in the accompanying table, and The product-moment correlation coefficient is calculated as follows:
With 8 df, the Pearson r is significant at .05 (a table value of .632 is required for r to be significant). Thus, the reliability is established at .745, an acceptable value for this type of test.
The chief drawback of this method is that if the retest is given too quickly, the first test sensitizes the respondents to the topic, and as a result, the respondent will remember the answers already given and repeat them.
This leads to biased reliability indicators in the upward direction.
Secondly, opinions may change from situational influences before the retest. In these cases, there is a downward bias in the stability scores.
This implies that longer the time interval between two successive administrations, the lower the correlation coefficient indicating poor reliability.