Document(s) 14 de 18

Steps in data analysis and report writing

* These steps need not be in the sequence in this diagram. The sequence may be adjusted according to the needs of the research teams.

** These elements are optional and may be omitted if not relevant for research teams

Module 30: DETERMINING DIFFERENCES BETWEEN GROUPS, PART II:
ANALYSIS OF PAIRED OBSERVATIONS

1. Introduction
2. Paired t-test
3. McNemar’s chi-square test

I. INTRODUCTION

This module describes the most commonly used tests for paired observations pertaining to numerical and to nominal data:

• the paired t-test for numerical data and
• McNemar’s chi-square test for nominal data.

You will remember from Module 9 and Module 26 part III, that paired or matched observations are carried out if researchers want to ensure through their study design that the relationship between two variables they are interested in is not confounded by another variable. They therefore have to sample their cases and controls in such a way that these are similar with respect to one or more potentially confounding variables.

The concept of pairing or matching subjects is illustrated by the following examples:

Example 1:

A researcher wanted to find out whether a class of students taught with audio-visual aids (AV) receive on average better grades than those who are taught without audio-visual aids. To minimise the effect of confounding variables such as social status and previous knowledge of the subjects, each student in the AV class was paired with another in the non-AV class of similar social status and knowledge level.

Example 2:

During a nutritional survey, a quality control exercise was carried out to check the agreement between two observers in measuring the children’s weight. In this instance we have paired observations as we have a set of two observations on the same child.

Example 3:

A study team compared schistosomiasis egg counts in two villages. It recognised that egg counts vary with age and sex. It decided to ensure that the samples were comparable with respect to age and sex by selecting subjects in pairs, with one member of each pair from each village, who were matched for age and sex.

II. PAIRED T-TEST

In Module 29 a comparison of sample means was performed for unpaired numerical observations on the height of delivering mothers by using the t-test. When dealing with paired (matched) observations, comparison of sample means is performed by using a modified t-test known as the paired t-test.

In the paired t-test differences between the paired observations are used instead of the original two sets of observations.

The paired t-test calculates the value of t as:

The degrees of freedom is the number of paired observations (= sample size) minus 1.

To interpret result of the study the same table of t-values is used as for the t-test for unpaired observations (see Annex 29.1).

To illustrate how the paired t-test is used, it will be performed on the results of the nutritional survey referred to in Example 2 above. The results are:

Table 30.1: Results of quality control exercise during a nutritional survey

The null hypothesis in this study is that if observers A and B measured all the children in the population from which these 20 children were sampled, there would, on average, be no difference between their measurements. In other words, the mean difference between A and B would be zero.

We can regard this set of 20 differences (the A - B column) as a sample of all differences that would have been obtained if the observers had measured the whole population.

To perform the significance test, the value of t has to be calculated and compared to the t-table value to determine if there is a statistically significant difference between the two. This indicates the probability that the results of the study occurred by chance.

The significance test is done as follows:

1. Calculate the mean difference of the measurements between A and B in the sample. This is the sum of the differences divided by the number of measurements:

2. Calculate the standard deviation of the differences (Module 27):

Standard deviation = 1.77

Calculate the standard error (Module 27):

Standard error =

3. The value of t is the mean difference divided by the standard error:

4. Refer to the table of t-values in Annex 29.1.

The degrees of freedom is the sample size (the number of pairs of observations) minus 1 which in this case is 20 – 1 = 19.

The probability from the table is < 0.05, which allows us to conclude that there is a significant difference between the observers. A and B definitely need more training and supervision on their measuring skills as the test does not show whether one or both have been inaccurate in their measurements.

III. McNEMAR’S CHI-SQUARE TEST

The McNemar’s chi-square test is used with NOMINAL data to compare PROPORTIONS of paired observations. It is important to note that the layout of the table is different from that used with unpaired samples.

Table 30.2 shows the results of a case-control study that was conducted to determine causes of a cholera outbreak in Bombay. For each cholera case confirmed in the hospital, a subject was sought of the same sex, the same age decade and the same neighbourhood.

Table 30.2: Source of drinking water by cholera patient/control pairs in the 5 days preceding illness (incorrect layout).

However, the layout of Table 30.2 is not correct, as it does not take account of the fact that cases and controls were selected as pairs.

The correct layout is presented in Table 30.3.

Table 30.3: Source of drinking water by cholera patient/control pairs in the 5 days preceding illness (correct layout)

Adapted from Baine & Mazotti et al. (1973).

How should we interpret Table 30.3?

In 12 pairs both cases and controls drew water from the shallow well and 31 pairs drew water from the tap. These 43 pairs therefore give us no information whether drawing water from shallow wells is a risk factor for getting cholera or not. However, in 30 pairs (39%) the cases draw water from shallow wells, while the controls drew water from the tap, whereas in only 3 pairs (4%) the controls draw water from the shallow wells while the cases drew water from the tap. It would seem therefore that drawing water from shallow wells was a risk factor for getting cholera.

Before we accept that conclusion, we must perform a significance test to estimate the likelihood that these results are due to chance or sampling variation only. In this case the appropriate significance test is McNemar’s chi-square test (see Table 28.1):

To check for statistical significance we use ordinary chi-square tables (Annex 29.2).

The test can be performed on the data in our example because r + s (30 + 3) is larger than 10.

The calculation of the chi-square value is as follows:

Using an a-level of 0.01, the ?² table value is equal to 6.63 (Annex 29.2). We can see that the calculated ?² value of 20.5 is larger than the table value. This means that the p-value is less than 0.01. We therefore reject the null hypothesis and conclude that drawing water from the shallow wells was a risk factor for getting cholera.

REFERENCES

See epidemiological and statistics textbooks referred to in Modules 9 and 28.

Baine WB, Mazotti M, Greco D et al. (1974) Epidemiology of cholera in Italy in 1973. Lancet ii (Dec.): 1370–1374.

Trainer’s Notes

Module 30: DETERMINING DIFFERENCES BETWEEN GROUPS,
Part II: ANALYSIS OF PAIRED OBSERVATIONS

Introduction and discussion

• If none of the teams has paired observations and if participants have little experience with statistics, this module need not be presented.
• When explaining how to calculate the t-values and ?² values, proceed step-by-step very slowly. Again, it is more important that participants understand how to do the calculations, than why they are done this way.
• Take time to ensure that everyone knows how to read Table 30.3. This may be the first table so far in which participants deal with the numbers representing pairs of observations (if they have skipped the relevant part of Module 26).
• Make sure that participants know how to use the t-table and ?² table and how to interpret the results.

Document(s) 14 de 18