This page intentionally left blank.

Designing and Conducting
Health Systems Research Projects

Volume I: Proposal Development and Fieldwork

This page intentionally left blank.

Designing and Conducting
Health Systems Research Projects

Volume I: Proposal Development and Fieldwork

Corlien M. Varkevisser
Indra Pathmanathan
Ann Brownlee

KIT Publishers, Amsterdam
International Development Research Centre

in association with
WHO Regional Office for Africa

TABLE OF CONTENTS

Foreword

vii

Preface and acknowledgements

ix

Introduction to Part I: Proposal Development and Fieldwork

xi

 

Module 1: Course orientation

1

Module 2: Introduction to health systems research

11

Module 3: Identifying and prioritising problems for research

27

Module 4: Analysis and statement of the problem

45

Module 5: Review of available literature and information

63

Module 6: Formulation of research objectives

81

Module 7: Introduction to health systems research methodology

91

Module 8: Variables

97

Module 9: Study types

117

Module 10: Data collection techniques

137

A: Overview of data collection techniques

141

B: Design of research instruments; Interview guides and interview skills

161

C: Focus Group Discussions

181

Module 11: Sampling

195

Module 12: Plan for data collection

221

Module 13: Plan for data processing and analysis

235

Module 14: Pre-testing the methodology

261

Module 15: Work plan

275

Module 16: Budget

285

Module 17: Plan for administration, monitoring and utilisation of results

295

Module 18: Finalising and reviewing the research proposal

305

Module 19: Fieldwork activities

315

Module 20: Field work report

325

Annex to the modules: Guidelines for organising short HSR courses on proposal development and fieldwork

333

About the authors

357

FOREWORD

Health Systems Research (HSR) has proved to be a useful tool for health decision makers at all levels over the past 20 years, providing them with the necessary data for informed decision making.

The Joint HSR Project for the Southern African Region based in the WHO Office in Harare and supported by WHO Geneva, the Royal Tropical Institute (KIT) in Amsterdam and the Dutch Technical Development Co-operation (DGIS), has played a crucial role in the promotion of HSR in the African region since 1987. HSR was enthusiastically embraced by many Ministries of Health and universities. In 1996, the Regional WHO Office for Sub-Saharan Africa (AFRO) assumed full responsibility for implementing HSR. Following the recommendation of Health Ministers of the Region, WHO/AFRO in 1998 included HSR as a regular programme for all its 46 member states.

The present HSR training modules, developed by an interdisciplinary, international team of practical researchers, have been highly instrumental in raising the interest for HSR. Originally designed for health managers at different levels as a tool to develop problem solving research in the Southern African Region, the modules also proved useful in Malaysia and were further elaborated by staff of the School of Public Health The 1991 combined version, published by International Development Research Centre, Canada and WHO, Geneva,* was translated in French, Spanish and Portuguese, and sections of it appeared in Arabic, Vietnamese and Chinese. In different parts of the world the modules facilitated the development and implementation of hundreds of research protocols by health staff and researchers. The HSR modules are used in the Community Health and Social Science Departments of many African, Asian and Latin American universities to train students and prepare them for their fieldwork. They are also used by Masters of Public Health courses in Europe and the USA and by international research programmes interested in applied research.

This unanticipated application of the modules in academic as well as health management circles led to the rapid exhaustion of the 1991 edition and the several subsequent reprints. Various groups of users made many useful suggestions for changes and improvements. The HSR Unit in AFRO, with agreement from IDRC, therefore decided to organise a revision of the HSR modules. An interdisciplinary group of Southern African researchers reviewed and made revisions in two workshops in 1998 and 1999. Two of the three original editors finalised the present version. IDRC took on the final responsibility for the publication, which was financed by AFRO and IDRC and published by KIT.

It is hoped that this revised version of the modules will fulfil the same need as preceding ones have done. Certainly many new and persisting health problems urgently require operational research. How to support necessary health reforms and at the same time ensure equity in access to health care for high-risk groups remains a major challenge. HSR is one of the tools we have to obtain deeper insight in these challenges and optimally focus our resources.

Dr. Rufaro R. Chatora, Director of the Division of Health Systems and Services Development (DSD), WHO/AFRO, Harare

Dr. Christina Zarowsky, Senior Health Specialist, IDRC, Ottawa

Ms. Catherine Hodgkin, Head Health Department, KIT, Royal Tropical Institute, Amsterdam


* Corlien M. Varkevisser, Indra Pathmanathan and Ann Brownlee (1991) Designing and conducting Health Systems Research projects. Part I: Proposal development and fieldword; Part II: Data analysis and report writing. Ottawa: Health Sciences Division of the International Development Research Centre (IDRC) and Geneva:Programme on Health Systems Research and Development of the World Health Organisation.

This page intentionally left blank.

PREFACE AND ACKNOWLEDGEMENTS

The present volume ‘Designing and conducting Health Systems Research’ is a thorough revision of Volume 2 of the Health Systems Training Series which the International Development Research Centre (IDRC) in Canada and WHO HQ in Geneva published in 1991 and reprinted several times under the same name. It became necessary to revise the modules, because over the years inevitable shortcomings and gaps were detected which needed to be addressed.

Health managers, for example, stressed that implementation of the research findings and recommendations were somewhat underexposed in the modules. This point is now taken care of in Module 1 by adding a fourth, implementation phase to the Health Systems Research (HSR) training cycle which initially consisted of three phases: HSR proposal development (15 days), fieldwork (roughly 6 months) and data analysis and report writing (2 weeks). The implementation of research findings and recommendations is further elaborated in Module 33. Furthermore, health managers pleaded, understandably, for shorter courses. This wish has been taken care of by stressing more explicitly in Modules 1 and 3, as well as in the Course Guidelines (annexed to Part 1 of this volume) that the proposal development phase can be shortened by having research teams select their research topic in the field before the onset of the course, preferably under guidance of a facilitator. In addition, the WHO/AFRO HSR Programme based in Harare, is at present developing modules for participatory rapid action in health research at health centre and district levels which can be carried out and integrated in the day to day activities of staff and community members.

Research staff from Community Health, Social Science and other university departments/ research institutes in Sub-Saharan Africa or other parts of the world who are using the modules had other wishes. They advocated that, in addition to the already well-emphasized problem-solving, analytical research approaches, more weight should be given to descriptive research. A descriptive diagram has therefore been added to the problem analysis diagram in Module 4. In all subsequent research steps, if relevant, the distinction between analytic and descriptive studies has been elaborated. Qualitative research methods have also been given more weight and they were more thoroughly integrated with quantitative methods in the research methodology (Modules 8-14). This applies, for example, to Modules 10 (Data collection techniques) and 11 (Sampling techniques). Furthermore, two new modules have been added to Part 2 (Data analysis and report writing) of the volume: one on Measures of association based on risk (Module 25), which used parts of Module 30 in the 1991 version, and one on the difficult issue of Confounding variables (Module 26). This need for extension was also reflected in the most recent evaluation of HSR training (1997).*

Facilitators, finally, desired more elaborate examples of crucial research and data-analysis techniques. Therefore, Module 10B (Development of research instruments) has been elaborated with a section on interview techniques with interview exercises, and Module 10C (FGDs) now contains an example of a transcribed focus group discussion with codes in the margin. To Module 13 (Plan for data analysis), an example of a full-fledged questionnaire and of a master sheet have been added, and Module 23 (Analysis of qualitative data) now provides an example of a filled-in compilation sheet. Module 5 (Literature review), has been extended with an example of a literature review.

Apart from these additions, in all modules parts that had proven to be unclear or incomplete were rewritten, and many examples and references were replaced by more recent ones or elaborated.

The present revision was initiated in a workshop held from 2-11 November 1998 in Arusha by a group of interdisciplinary researchers and managers convened by the manager of the WHO/AFRO HSR Programme (since 1992 Gabriel Mwaluko). All participants had thorough experience with the modules and with HSR: Sambe Duale, Lawrence Gakuri, Pilate Khulumani, Steve Kinoti, Gabriel Mwaluko, Jude Padayachi, Brian Pazvakavambwa, Corlien Varkevisser and Godfrey Woelk. In August 1999 a group of three people (Alasford Ngwengwe, Corlien Varkevisser and Godfrey


* Corlien M. Varkevisser, Indra Pathmanathan and Ann Brownlee (1991) Designing and conducting Health Systems Research projects. Part I: Proposal development and fieldword; Part II: Data analysis and report writing. Ottawa: Health Sciences Division of the International Development Research Centre (IDRC) and Geneva:Programme on Health Systems Research and Development of the World Health Organisation.

Woelk) made further revisions and synchronised the different texts in the WHO/AFRO/HSR office in Harare, supported by staff of the HSR office (since 1999 headed by Isabel R. Aleta, with Makhamokha Mohale and Eric Naterop as APOs). Corlien Varkevisser and Ann Brownlee finalised and edited the modules, with the blessing of Indra Pathmanathan who this time could not participate. Deborah Karugonjo (Harare) and Merel Gallée (Amsterdam) provided highly valued assistance in the production of successive computerised versions. Funds for revising and publishing the HSR modules were made available by DGIS (Dutch Development Co-operation); SARA/AED, Washington; GTZ, Germany through the GTZ MCH/FP network for Health Systems Research in Southern Africa; WHO/AFRO and by WHO HQ, Geneva. IDRC, Canada assists in subsidised distribution of the modules.

A highly varied collection of people assisted in the production of earlier versions of the HSR modules. The cradle of the modules stood in Western Africa, where in the early eighties the Project for Strengthening Health Delivery Systems (SHDS), based in Boston University, USA, at the request of AFRO developed training materials in research protocol development. SHDS followed the step-by-step approach which till today is a major key to the success of HSR courses. Modules 1-17 in this volume are heavily adapted or new versions of the original SHDS modules.* The first adaptation took place in 1988, with 12 researchers from countries that participated in the Joint HSR Project ( Omondi (Kenya), Sebatane and Makatjane (Lesotho), Chimimba and Msukwa (Malawi), Kitua and Savy (Seychelles), Tembo (Zambia) Munochiveyi, Taylor and Woelk (Zimbabwe) and Joint Project staff which also finalised the version (Corlien Varkevisser and Martien Borgdorff). These ‘green modules’* found their way to Malaysia, where Indra Pathmanathan further developed them, with assistance from Maimunah Abdul Hamid, K. Mariappan and C. Sivagnanasundram (Sri Lanka), in the course of numerous protocol development workshops. The same occurred in Southern and Eastern Africa. At the initiative of Yvo Nuyens, who fathered the Joint HSR Project in WHO Geneva, and supported by IDRC (Annette Stark), the five volumes of the Health Systems Research Training Series emerged, of which Designing and Conducting Health Systems Research Projects formed Volume 2. These ‘pink modules’, published in 1991 in Ottawa by IDRC and WHO, form a thorough merge of the ever developing Southern African and Malaysian versions. They were integrated in Harare (Corlien Varkevisser and Leon Bijlmakers), in consultation with Indra Pathmanathan, and with thorough editing support from Ann Brownlee, one of the authors of the original SHDS modules. The present HSR modules are therefore a truly global production. It is even impossible to mention every contributor, because many HSR course facilitators and participants through their questions and critical remarks inspired further changes.

With such a colourful and interactive origin it seems highly unlikely that the present reprint will be the last one. Whenever the modules are used, they will be adapted. We hope, however, that in their present form they will last for some years and will be of use to health staff as well as university students.

Dr. Corlien M. Varkevisser, Royal Tropical Institute/University of Amsterdam

Dr. Ann Brownlee, University of California, San Diego

June 2003


* Regional Assessment of Health Systems Research Training in Eastern and Southern Africa. HSR Project and SARA/AED,Harare/Washington. SS Ndeki, 1997.

*Ann Brownlee, Thomas Nchinda and Yolanda Mousseau-Gershman (1983) Health Services Research Course: How to develop proposals and design research to solve priority problems. Boston: Boston University Health Policy Institute.

* Joint World Health Organisation/Royal Tropical Institute/Dutch Technical Development Co-operation Project on Health Systems Research for the Southern African Region (1988) Health Systems Research Training Course: How to develop research proposals to solve priority health problems. Geneva: World Health Organisation. WHO/SHS/HSR/88.3.

INTRODUCTION TO PART I: Proposal Development and Fieldwork

Since Module 1 of this volume provides a thorough introduction to the course and how it is organised, here we will only provide a brief overview of where the reader can find various types of information and how the course can be used.

Part I, Proposal Development and Fieldwork, contains modules 1-20, of which the first 18 will lead the course participants through all steps that the development of their proposal requires. Modules 19 and 20 guide them through the fieldwork period and preliminary data analysis.

Each module contains detailed instructions for group work on the successive steps in the development of the proposal. At the end of each module, facilitators will find Trainer’s Notes, providing guidelines on how to present the modules and how to assist the groups in the writing of their research proposal.

After Module 20 an annex has been added with general guidelines for the planning and management of HSR workshops, the training methodology and the supervision of fieldwork. The annex includes an example of a course schedule and guidelines for budgeting an HSR course. Furthermore, an information circular for course participants and a course evaluation form have been added.

The course schedule presented applies to a full-time workshop for beginners, lasting just over two weeks. Depending on the level of the participants, the duration of the course can be shortened. The training materials can also be used in university settings, stretched out over a trimester or quarter with weekly sessions.

The most important guideline for the application of these HSR modules is BEING FLEXIBLE. The only general rule is that participants and facilitators should be conscious of the cyclical nature of the process of developing a research proposal. In many group work sessions, participants will therefore be referred back to earlier parts of the proposal they developed to make adjustments, it required. In Module 18 they are advised to review once again all sections of their proposal when writing the summary.

The managers of a course can adapt the time devoted to presentations and group work, as well as the sequence of modules, to the needs of their target groups. It is, for example, quite possible to combine Modules 4 (Statement of the problem) and 5 (Literature review). Also Modules 8 (Variables), 9 (Study type) and 10 (Data collection tools) are closely connected, and the group work for Modules 9 and 10 has already been combined. These modules form the core of any research proposal. Enough time will have to allotted to the group work for these modules and to field-testing of the data collection tools, as the quality of the tools will determine the quality of the findings.

For groups of participants that include health managers who already have extensive administrative experience it may be feasible to present modules 15 (Work plan), 16 (Budget) and 17 (Plan for project administration, monitoring and utilisation of results) as one block, entitled ‘Management of a Research Project.’*

For Part II of this volume, Data Analysis and Report Writing containing modules 21-33, the same principles apply as outlined for Part I. We have included some specific guidelines on how to use these modules in the introduction to Part II.

It is advisable to distribute Part II during the first workshop. Participants who use qualitative research techniques, such as focus group discussions, would benefit from reading Module 23 (Analysis of qualitative data), when developing their plan for data analysis (Module 13). For those using quantitative techniques, Modules 22 (Description of variables) and 24 (Cross-tabulation of quantitative data) may be useful when variables are introduced (Module 8) or when data analysis is discussed (Module 13).

Designing and Conducting Health Systems Research Projects
Part I: Proposal Development and Fieldwork

Module 1
COURSE ORIENTATION

This page intentionally left blank.

Module 1: COURSE ORIENTATION

COURSE OBJECTIVES

At the end of this course, you should be able to:

  1. Describe what HSR is and understand the contribution it can make towards solving priority problems in health care within the local context.
  2. Prepare a health systems research proposal by completing the following steps:
    • Identification, analysis and description of a research problem
    • Review of relevant literature and other available information
    • Formulation of research objectives
    • Development of an appropriate research methodology
    • Preparation of a work plan for the study
    • Identification of resources required and preparation of a budget
    • Development of a strategy for distribution and utilisation of research results
  3. Implement this proposal in your own working situation during a period of 4-6 months.
  4. Analyse and interpret the results.
  5. Prepare and present a final report of the research findings, including recommendations for solving the problem and a plan of action for their implementation.
Whom is the health systems research course aimed at?

The health systems research (HSR) course has been developed for mid- and higher- level managers, health workers and health-related staff, as well as interested researchers.

What training method is used in the HSR course?

The training method applied is based on learning by doing. Course participants will themselves develop research proposals that they will actually carry out in the field.

Each participant and trainer brings to this course his or her own experiences in applied research and in the management of health or health-related projects. Thus, the course should not be perceived as having a teacher-student orientation. It should rather provide a forum for sharing information where everyone can contribute the benefits of his or her own experience and knowledge. This sharing will add greatly to the richness and relevance of the course.

What type of projects will be developed?

Together with community leaders and other health decision-makers from the district, provincial or even national level, course participants will select priority problems in their own work situations that cannot be solved unless more information is collected. Preferably, the topics will have been selected before the training starts, although they may need more specification. In most cases, a team of course participants will carry out the planned research alongside their regular duties. Therefore, the project will have to be of modest size. For example, a maximum of 30 days for fieldwork and preliminary analysis per group member, and between 4,000 and 8,000 US$ per research project would be advisable.

How long is the course?

The course, which includes three main components, will cover a period of about 7 months, with an additional fourth component for the implementation of research results over 18-24 months. Thereafter the new activities resulting from the study are supposed to form part of the regular planning.

./img/designandco_17_la_0.jpg

Component 1. HSR proposal development

The first 2-3 week workshop will provide an introduction to HSR. If modules 1-4 are implemented in the field with guidance from a facilitator, which we recommend, the proposal development workshop can be limited to two weeks or less. Participants will work in small groups and design research proposals, step by step, on the priority problems they have selected. As each new step is introduced, new concepts and research procedures will be presented. The participants will immediately apply these in the proposals they are developing. Modules 1-18 deal with proposal development.

Component 2. Implementation of the proposal

During the following 4-6 months, the same groups of participants will implement their proposals. It is therefore important that the groups are composed in such a way that they can easily cooperate during the fieldwork. Modules 19 and 20 give guidelines for the fieldwork and for writing a short fieldwork report - including preliminary results.

Component 3. Analysis of the data and report writing

After project implementation, participants will meet again for a 2-week workshop to further analyse and interpret the data. At the end of this workshop, a research report with recommendations for action will be prepared and presented to health policy makers, health staff and communities. Modules 21-33 pertain to data analysis, report writing, dissemination of research results and preparing implementation of recommendations based on the findings.

Component 4. Development and implementation of action plans

Together with the relevant stakeholders, (policy makers, managers, staff, community members), teams will draft action plans to implement the recommendations that are agreed upon. Because many of the participants are in direct positions of managerial responsibility, and because higher-level decision-makers and community members have been involved, it is expected that action plans can be implemented soon after the studies are completed. The proposed activities will normally be integrated in the district, provincial or national health plan and be subjected to regular monitoring and evaluation.

How will the research proposal be developed?

A number of basic steps have to be taken when developing a research proposal. These steps are presented in the flowchart below.

This flowchart appears on the back of each of the pages that mark the beginning of modules 3-18. The step in the proposal development process that the module addresses is indicated by double lines around the appropriate box in the flowchart.

./img/designandco_19_la_0.jpg

NB: Development of a research process is a cyclical process. The double-headed arrows indicate that the process is never linear.

It should be stressed that designing a research proposal is not a linear but a cyclical process. Throughout the course there will therefore be opportunities to review and, when the need arises, to revise parts of the proposal that have already been drafted. When developing the research methodology, for example, the teams may find that the objectives and even the statement of the problem need to be revised to be made more specific. When finalising the work plan and budget, the teams may determine that the research design, for financial reasons, may need to be revised so the project is more modest and thus less costly.

By the end of the first part of the course, each group will have developed a research proposal with the following chapters: (For details, see Module 18.)

EXECUTIVE SUMMARY

1. INTRODUCTION
 

1.1 Background information

 

1.2 Statement of the problem

}

Literature review may be partially or fully integrated in 1.1 and 1.2

 

1.3 Literature review

2. OBJECTIVES
3. METHODOLOGY
 

3.1 Study type, variables, data collection techniques

3.2 Sampling

3.3 Plan for data collection

3.4 Plan for data processing and analysis

3.5 Ethical considerations

3.6 Pre-test

4. WORK PLAN (including description of project staff)
5. BUDGET (including explanatory note on major budget posts)
6. PLAN FOR ADMINISTRATION, MONITORING, AND UTILIZATION OF RESULTS
 

References

List of abbreviations (if applicable)

Data collection instruments (annexed)

In the second workshop for data analysis and report writing, a similar approach will be followed.

How may this set of modules be used?

The course has been organised in such a way that each module can be dealt with independently. A module includes:

Depending on the level of the groups, it may be possible to combine certain modules and to shorten or lengthen the time allocated for presentations and group work and the total workshop time. For programme managers, for example, one week may be sufficient to prepare a first draft of a research proposal. Provincial and district level staff with some research experience may need 2 weeks, whereas novices to research will need the full 2 ½ weeks.

Note:

Participants are advised to read the course materials beforehand so that they can benefit, as much as possible, from the presentations and group work. It may be extremely useful for the participants to (re)read the course material after the presentation and group work as well, especially if they have had no previous research training or experience.

Trainer’s Notes

Module 1: COURSE ORIENTATION

Timing and training methods

1-1½ hours Personal introductions of participants and facilitators (if not completed the night before)
¾ -1 hourCourse orientation
½ hourAdministrative remarks
1-2½ hours TOTAL TIME
Materials
Personal introduction of participants and facilitators

If you were unable to do the mutual introduction of participants on the evening before the course begins, have all the participants (including the facilitators) introduce themselves. Make certain everyone indicates his or her profession, major activities and research experiences and interests. This may be done by having participants interview each other in pairs and then each introduces the person he or she interviewed. Names and a summary of the interview could be put on a flipchart and stuck to the wall.

The introduction may take 1 - 1½ hours.

Course orientation
Administrative issues

Designing and Conducting Health Systems Research Projects
Part I: Proposal Development and Fieldwork

Module 2
INTRODUCTION TO
HEALTH SYSTEMS RESEARCH

This page intentionally left blank.

Module 2: INTRODUCTION TO HEALTH SYSTEMS RESEARCH

OBJECTIVES

At the end of this session, you should be able to:

  1. Describe the major characteristics of research.
  2. Describe various components of the health system as a basis for understanding HSR.
  3. Describe types of information for decision-making in the health system and the contribution various disciplines can make in providing such information.
  4. Describe the purpose, scope and characteristics of HSR.4.
  1. The development of health systems research
  2. What is health systems research?
  3. Participants in health systems research
  4. Guidelines for health systems research

I. THE DEVELOPMENT OF HEALTH SYSTEMS RESEARCH

Why did HSR develop?

By adopting of the philosophy and strategies for Health For All, politicians and health staff at all levels are committed to ensuring that all people will attain a level of health that enables them to participate actively in the social and economic life of the community in which they live.

Although research has made major contributions to health by providing knowledge of the causes of diseases and by developing the technology to cure and prevent disease and promote health, Health For All is far from being achieved.

Why is there still so much disease that could have been prevented or cured? Because health services by themselves cannot control all of the factors that influence health. Poverty and political systems which either widen or narrow the gap between rich and poor and which promote or neglect the education of girls, for example, influence the health of people. Drought and wars may bring malnutrition and disease with which the health services can hardly cope. While communicable diseases such as smallpox and, to some extent, leprosy may be gradually conquered due to improved environmental conditions and extra effort on the part of the health services, new diseases such as HIV/AIDS may appear which upset the whole health care system and society at large.

This complex of environmental factors – geographical, socio-economic, cultural, political, demographic, epidemiological – not only influences the health of people, it also affects the health services. Countries suffering from poor economics, wars and drought usually have poorly functioning health services as well.

Still, even within less favourable environments, some services function better than others. A very important factor is the quality of information on which policy makers base their decisions. Very often this information is vague or missing. Then decisions on interventions can be completely off track, which means that money is wasted. Basic questions which health policy makers need answered include, for example:

— What are the health needs of (different groups of) people, not only according to health professionals but also according to the people themselves? Can shared priorities be agreed upon?

— To what extent do the present health interventions cover these priority needs? Are the interventions acceptable to the people in terms of culture and cost, especially to the poor? Are they provided as cost-effectively as possible?

— Given the resources we have, could we cover more needs, or more people, in a more cost-effective way? Is it possible to introduce or expand cost-sharing through insurance, to reduce the risk of unexpected high costs, in particular for the economically vulnerable? Could co-operation with the private/NGO sector be improved? Could donor agencies help solve well-defined bottlenecks in the system?

— Is it possible to better control the environmental factors which influence health and health care? Can other sectors help (education, agriculture, public works/roads, etc.)? (See Figure 2.1.)

These questions cannot be answered without collecting more information through research. That is why, since the end of the 1970’s, Health Systems Research (HSR) has been developed.

Figure 2.1: Environmental and health system factors influencing attainment of Health For All

./img/designandco_28_la_0.jpg

II. WHAT IS HEALTH SYSTEMS RESEARCH?

What is research?

RESEARCH is the systematic collection, analysis and interpretation of data to answer a certain question or solve a problem.

Characteristics of research:

Health research serves two major purposes:

First, basic research is necessary to generate new knowledge and technologies to deal with major unresolved health problems. Second, applied research is necessary to identify priority problems and to design and evaluate policies and programmes that will deliver the greatest health benefits, making optimal use of available resources.

During the past two (or even three) decades there has been a rapid evolution of concepts and research approaches to support managerial aspects of health development. Many of these have been described by specific terms such as operations/operational research, health services research, health management research, applied research and decision-linked research. Each of these has made crucial contributions to the development of HSR (WHO 1990).

HEALTH SYSTEMS RESEARCH is ultimately concerned with improving the health of people and communities, by enhancing the efficiency and effectiveness of the health system as an integral part of the overall process of socio-economic development, with full involvement of all partners.

What is meant by a health system?

There are different interpretations of what a health system is. Some give a narrow definition and only consider the different levels of the public health care services as a health system (see Figure 2.2.)

Figure 2.2: Public health care system

./img/designandco_30_la_0.jpg

The inclusion of the district council, district development committee and village development committee indicates, however, that some 25 years after Alma Ata* it has been widely recognised that local administration and other sectors than the health sector alone carry responsibility for the health of the people in a village, district or region.

Many HSR researchers have a wider perception of health systems. They also include the private sector. The private sector has many possible components:

The Primary Health Care (PHC) approach has broadened the horizon of medical care providers considerably. PHC put individuals and communities in the centre of attention. Individuals providing self-care (what mothers and other relatives do to keep children and themselves healthy) and


* The WHO/UNICEF/World Council of Churches conference, which in 1978 laid the foundation for the worldwide Primary Health Care approach.

traditional/folk healers were accepted as important potential allies of health staff. So were personnel from other sectors, which could support health, for example, through the construction of roads, the improvement of education, water, sanitation, and through income generation.

Figure 2.3: A broadly defined health system

./img/designandco_31_la_0.jpg

Figure 2.3 presents the widest possible definition of a health system, including all public and private sectors/institutions which directly influence and support the health of people, embedded in the wider environmental context that was described in Figure 2.1.

This figure would take different shapes in different societies, but everywhere individuals form part of a network of family and community members who are concerned about their health. This network prescribes or advises how to prevent illness and what to do in case of ill health. In many societies, mothers and grand mothers are key figures in early childcare. They determine nutritional and hygiene practices, alert children to dangers, provide care in case of disease, and teach children the basics of self-care.

At the other end of the spectrum, a public authority is responsible for the well being of all people inhabiting its territory. Nowadays governments of states organise public health care and, to some extent, regulate private health care initiatives. Through other social services (e.g., education, social welfare), through laws and taxes and police and army, governments are supposed to assure their citizens the resources to survive and live in peace. Since time immemorial this has been the duty of rulers, although each society has developed its own ways of ensuring ‘health for all’.

When in the 1980s many countries were struck by chronic economic crises, the World Bank advocated structural adjustment programmes to reorganise the economies, which relied on market mechanisms rather than on state control with subsidies and protection. The health and educational sectors were inevitably affected and went through a series of reforms that hit the consumer hard. The World Health Organization recognised the need for health reforms, but under the condition that these would leave the goal of HEALTH FOR ALL in tact. It therefore focussed attention on fairness of the system, which should also be affordable to the poor, and at the same time stressed that the system should be responsive to the need of patients for human, respectful treatment (see Figure 2.4).

Figure 2.4: Objectives and functions of the health system

Functions the health system performs      Objectives of the system

./img/designandco_32_la_0.jpg

Health is expressed as life expectancy by the WHO, taking into account the time lived with a disability of any kind (also due to chronic disease and old age). In the highly industrialised countries of Western Europe, for example, the average life expectancy of men is 74 years, of which 6.5 years are with disability; for women it is 80.8 and 7 years, respectively. In Africa in areas most struck by AIDS, men live now on average only 45.6 years, of which 7.6 years are with disability; for women the respective means are 48 and 8 years. Responsiveness to patients’ human needs would mean respecting the patient’s dignity and autonomy and reducing the fear and shame that sickness brings with it. Fairness ideally means financial protection for everyone by payment according to financial capacity. This can best be assured by pre-payment through an insurance

system, with fees according to capacity. The insurance revenues are then pooled and costs of care paid from the pool, so that in fact the rich help to cover the treatment of the poor. Unfortunately, such a system is hard to organise in the least developed countries where rural areas harbour mainly poor, but WHO counts on international solidarity and donor agencies for contributions.

The health system comprises both public and private health services but, for the time being, no agricultural, educational or other sectors, however relevant. The first urgency is the performance of the health system, which should be as good as possible, given the available means. To reach that aim, WHO set some criteria. Ministries of Health should weigh the public health importance of proposed health actions, set priorities, and thoroughly investigate the cost-effectiveness of different possible interventions to select the highest value for the money. In terms of resources, they should strive for a balance between investments, the use made of these investments and their maintenance. For example, if staff members are highly trained but their knowledge is under-utilised, or if buildings, equipment and means of transport cannot be maintained, these investments are highly wasteful. Likewise the services don’t function well if there is no money left for consumables such as essential medicines. The patients then have to buy medicines on the private market, out of their pockets and at unnecessary high costs, which the poor cannot afford. Good oversight is required to achieve an optimal balance among the different expenses, and it is one of the aims of HSR to provide the policy makers with the relevant data.

Good oversight and ‘stewartship’ is also required to develop a fair financing system. The Ministry of Health is usually the appropriate institution to collect money from taxes and donor agencies to finance the health care system. In the 1980’s it became clear that even PHC services could never function adequately with the required coverage (health for ALL) without a contribution from the clients. User fees were introduced in countries that hitherto had provided care free of cost, but this appeared to hit the poor out of proportion despite exemption rules. Hence WHO proposes a more structural solution by introducing prepayment through insurance and pooling of resources, which is beneficial for the poor.

Although the MOH, in many developing countries, is still the principal provider of health care, if it is to achieve the most cost-effective care, it has to consider the use of the private sector and contract services out in cases where this would be cheaper. Consequently, the MOH has to set standards of care and control for deviation in the private sector as well as the public. To have oversight and control is one of the major present day challenges for Ministries of Health.

Specific questions for specific levels of service

HSR is not only of use to policy makers; at each level managers may have questions that require further research.

Health policy makers may, for example, want to know:

Managers at district/provincial level may raise questions such as:

Hospital directors may ask:

Managers at village level (village health committees) may want to know:

(Please add your own examples.)

The major objective of HSR is to provide health managers at all levels, as well as community members, with the relevant information they need to make decisions on health-related problems they are facing.

We must be aware that problems at one level of the health system are usually connected with problems or deficiencies at other levels (see Figure 2.2). HSR should address problems from the differingperspectives of all those who are, directly or indirectly, involved. Otherwise we run the risk of coming up with results which only partly explain the problem and which are therefore insufficient to solve it.

III. PARTICIPANTS IN HSR

It is evident that many issues in health are interrelated and interact with issues in other sectors, such as production, education, the condition of wells or roads, and broader environmental factors. Research in health systems must recognise this. The research skills that are required may need to come from a variety of disciplines, e.g., public health/medicine, health economics, behavioural and social sciences, and agriculture. Therefore HSR is multi-disciplinary in nature.

Even simple research that is conducted at the operational level may require research skills from different disciplines to provide sufficient and relevant information to support decision-making. Therefore, training in HSR includes relevant aspects from various research disciplines.

Researchers who work in HSR will have to work in a trans-disciplinary way, which means working together as a team throughout all phases of the research. In the process, they need to acquire a basic understanding of the concepts and approaches as well as the potential and limitations of research techniques used in sister disciplines.

HSR, however, is not the concern of scientists alone.

Who should be involved in HSR?

The participatory nature of health systems research is one of its major characteristics. To ensure that the research is relevant and appropriate, everyone directly concerned with a particular health or health care problem should be involved in the research project(s) focused on it. This may include policymakers, managers from the health and other public services involved, health care providers and the community itself. Their involvement is critical if the research activities are to make a difference:

The roles that various types of participants will play in the research project will depend on the level and complexity of the particular study as well as its area of focus. Some projects are very complex and may need expertise from several levels, sectors and disciplines. Others may focus on simpler problems and require a more modest set-up. Health personnel may even play a major role in simple studies focusing on practical problems in their own working situations, although their projects may require assistance from researchers with skills in relevant disciplines.

Note:

Because of the participatory nature of HSR, in the modules that follow we will use the term RESEARCHER to mean anyone actively involved in planning and conducting the research.

IV. GUIDELINES FOR HSR

Bearing in mind that HSR is undertaken primarily to provide information to support decision-making that can improve the functioning of the health system, we summarise by suggesting some essential guidelines for success:

  1. HSR should focus on priority problems in health care.
  2. It should be action-oriented, i.e., aimed at developing solutions.
  3. An integrated multi-disciplinary approach is required, i.e., research approaches from many disciplines are needed since health is affected by the broader context of socio-economic development.
  4. The research should be participatory in nature, involving all parties concerned (from policymakers to community members) in all stages of the project.
  5. Studies should be scheduled in such a way that results will be available when needed for key decisions; research must be timely. Otherwise, it loses its purpose.
  6. Emphasis should be placed on comparatively simple, short-term research designs that are likely to yield practical results relatively quickly. Simple but effective research designs are difficult to develop but much more likely to yield useful results when needed.
  7. The principle of cost-effectiveness is important in the selection of research projects. Program management and operational research should focus, to a large extent, on low-cost studies that can be undertaken by management and service personnel in the course of daily activities. (There is a need for larger studies as well, however, which may require outside funding and full-time research staff.)
  8. Results should be presented in formats most useful for administrators, decision-makers and the community. Each report should include:

    8. Evaluation of the research undertaken should concentrate on its ability to influence policy, improve services and ultimately lead to better health, rather than on the number of papers published.

Thus, an HSR project should not stop at finding answers to the questions posed, but include an assessment of what decisions and activities have evolved from the study.

Trainer’s Notes

Module 2: INTRODUCTION TO HEALTH SYSTEMS RESEARCH

Timing and training methods

1 hour Introduction and discussion

Adapting the presentation to the participants

It is recommended that the content and focus of the module be adapted to the level and interests of the participants. For example:

  1. Review the background of participants (e.g., primary health care, clinical medicine, research, policy-making, or community leadership).
  2. Based on this review, select suitable examples related to the background of participants to illustrate each concept.

    Remember that understanding of abstract concepts is facilitated if participants can relate them to their own experiences.

  3. The focus and scope of this module can be varied in accordance with the expected future roles of participants in the research teams.

    If participants are fairly specialised personnel from a single discipline or from just a couple of disciplines, it would be useful to focus on the multi-disciplinary aspect of HSR and the types of information that disciplines other than those represented in the workshop can provide. For example, if workshop participants are hospital managers and clinicians, it may be most useful to illustrate the uses of research input from behavioural science for HSR; if participants are behavioural scientists such as health education officers and sociologists, it may be most useful to emphasize the importance of input from management sciences, health economics and clinical epidemiology.

  4. When presenting figures, use different overhead sheets for the different components. For example, the health system (Figure 2.3) could be presented on four superimposed sheets.

    ./img/designandco_37_la_0.jpg

  5. Ask participants to give examples of topics suitable for HSR from their own working environment.
  6. Ask whether they have participated in evaluations or other regular research activities. Demystify the concept of HSR. Identify in what stages of the research they have participated and whether the participation was optimal.
  7. Try to draw out the points mentioned in the guidelines from the participants themselves. By the end of the introductory session they should be able to come up witsome of the points on their own.
Proposed pre-workshop reading (to be sent to participants before the workshop)

World Health Organisation (1990) Health systems research: Background document at the World Health Assembly. Geneva: WHO. A43/Technical Discussions/3.

Proposed additional reading (to be available in the course library)

Bobadilla JL (1998) Searching for essential health services in low- and middle-income countries. Washington DC: Inter-American Development Bank.

Joint Project on Health Systems Research for the Southern African Region (1994) Summaries of Health Systems Research Reports 1988-1993. Harare, WHO Sub-regional Office III (now AFRO).

Joint Project on Health Systems Research for the Southern African Region (1997) Health Systems Research: Does it make a difference? Update 1996 (3rd edition) WHO: Geneva.

Joint Project on Health Systems Research for the Southern African Region. Series Health Systems Research: It can make a difference.

Volume 1: Availability, provision and use of drugs (1994)

Volume 2: Factors associated with maternal mortality (1994)

Volume 3: Under utilisation of TB services in Southern Africa. Exemplary research protocol, research results and their implementation (1996)

Volume 4: Factors influencing the functioning of primary health care at village level (1996)

Murray CJL, Kreuser J, Whang W (1994) Cost-effectiveness analysis and policy choices investing in health systems. In: Murray CJL, Lopez A. Global comparative assessments in the health sector: disease burden, expenditures and intervention packages. Geneva: World Health Organization.

Scrimshaw SCM, Hurtado E (1987) Rapid assessment procedures for nutrition and primary health care. Anthropological approaches to improving program effectiveness. Tokyo: United Nations University and Los Angeles CA: Latin America Centre, University of California.

Taylor CE (1984) The uses of health systems research. World Health Organisation: Geneva. Public Health Paper 78.

World Health Organization (2000) The World Health Report. Health systems: improving performance. Geneva: WHO.

World Health Organisation (1988) Health systems research in action: case studies from Botswana, Columbia, Indonesia, Malaysia, the Netherlands, Norway, and the United States of America. Geneva: WHO.

World Health Organization, Programme on Health Systems Research and Development (1991) From research to decision making. Geneva: WHO.

This page intentionally left blank.

Designing and Conducting Health Systems Research Projects
Part I: Proposal Development and Fieldwork

Module 3
IDENTIFYING AND PRIORITISING
TOPICS FOR RESEARCH

./img/designandco_41_la_0.jpg

NB: Development of a research process is a cyclical process. The double-headed arrows indicate that the process is never linear.

Module 3: IDENTIFYING AND PRIORITISING TOPICS FOR RESEARCH

OBJECTIVES

At the end of this session you should be able to:

  1. Identify criteria for selecting health-related problems to be given priority in research
  2. Use a group consensus technique to set priorities for research, applying the selected criteria on a number of research topics
  3. Select an appropriate subject for a research proposal that will be developed by your group during the course.
  1. Identification of topic
  2. Criteria for prioritising topics for research
  3. Nominal group technique

Note:

If topics have been selected before the workshop, either by health managers who asked for the study or by the participants together with their health managers and community leaders, recapitulate part I of Module 3, then go straight to Module 4, even if the teams have already done some problem analysis in the field. If the participant teams need to re-examine the research topics they selected before the workshop, section II of Module 3 can be used together with Module 4.

I. IDENTIFICATION OF TOPIC

In the previous module, a number of research questions were presented that may be posed at the various levels of the health system.

These questions can be placed in three broad categories, depending on the type of information sought:

  1. Description of the health situation, required for planning interventions

    Planners need to know, for example, the magnitude and distribution of health needs in a population as well as of services; the risk factors for certain problems and people’s awareness; the utilization patterns and cost-effectiveness of available and potential other interventions, in order to formulate adequate policies and adapt or plan interventions.

  2. Information required to evaluate ongoing interventions, for example with respect to:

    to assess progress and the need for adjustment on a routine basis.

  3. Information required to define problem situations in interventions in any of the fields mentioned under 2, and to analyse possible causes in order to find solutions. These causes may include lack of or inequitable distribution of resources, vague policies, and any environmental factors affecting needs, interventions and resources (See Figure 2.1).

Although research in support of planning and evaluation (categories 1 and 2 mentioned above) is an important focus for HSR, the modules will emphasise problem-solving research, because health managers are frequently confronted with problems of this type. It is assumed, however, that research skills acquired in the present course will be of use in the broader field of planning and evaluation as well. In Modele 4, moreover, all three types will be treated.

Whether a problem situation requires research depends on three conditions:*

  1. There should be a perceived difference or discrepancy between what exists and the ideal or planned situation;
  2. The reason(s) for this difference should be unclear (so that it makes sense to develop research questions); and
  3. There should be more than one possible answer to a question or more than one solution to the problem.

* This paragraph has been adapted from Fisher et al. (1983)

For example:

Problem situation:

In District X (pop. 145,000) sanitary conditions are poor (5% of households have latrines) and diseases connected with poor sanitation, such as hepatitis, gastro-enteritis and worms, are very common. The Ministry of Health has therefore initiated a sanitation project that aims at increasing the percentage of households with latrines by 15% each year. The project provides materials, and the population should provide labour. Two years later, less than half of the target has been reached.

Discrepancy:

35% of the households should have latrines, but only 15% do have them.

Research question:

What factors can explain this difference?

Possible answers:

  1. Service-related factors, such as forgetting to adequately inform and involve the population, bottlenecks in the supply of materials, differences in training and effectiveness of sanitary staff, lack of co-operation between sectors.
  2. Population-related factors, such as situations where community members lack understanding of the relationship between disease and sanitation or have other problems, for example due to poverty, which they consider more important.
  3. Physical factors/ecosystems, such as hard soil, or land subjected to frequent flooding.

II. CRITERIA FOR PRIORITISING TOPICS FOR RESEARCH

Because HSR is intended to provide information for decision-making to improve health care, the selection and analysis of the research topic should involve those who are responsible for the health status of the community. This would include managers in the health- and health-related services, health care workers and community leaders, as well as researchers.

Each topic that is proposed for research has to be judged according to certain guidelines or criteria. There may be several ideas to choose from. Before deciding on a research topic, each proposed topic must be compared with all other options. The guidelines or criteria discussed on the following page can help in this process:

Criteria for selecting a research topic:
  1. Relevance
  2. Avoidance of duplication
  3. Urgency of data needed (timeliness)
  4. Political acceptability of study
  5. Feasibility of study
  6. Applicability of results
  7. Ethical acceptability
1. Relevance

The topic you choose should be a priority problem. Questions to be asked include:

Try to think of serious health problems that affect a great number of people or of the most serious problems that are faced by managers in the area of your work.

Also, consider the question of who perceives the problem as important. Health managers, health staff and community members may each look at the same problem from different perspectives. Community members, for example, may give a higher priority to economic concerns than to certain public health problems. To ensure full participation of all parties concerned, it is advisable to define the problem in such a way that all have an interest in solving it. Even within villages, opinions may differ on how important a problem is. It is therefore obligatory to discuss the problem with community leaders, as well as peripheral villagers, males as well as females, rich and poor, exploring their perceptions of the problem.

Note:

If you do not consider a topic relevant, it is not worthwhile to continue rating it. In that case you should drop it from your list.

2. Avoidance of duplication

Before you decide to carry out a study, it is important that you find out whether the suggested topic has been investigated before, either within the proposed study area or in another area with similar conditions. If the topic has been researched, the results should be reviewed to explore whether major questions that deserve further investigation remain unanswered. If not, another topic should be chosen.

Note:

Also, consider carefully whether you can find answers to the problem in already available, unpublished information or just by using your common sense. If so, you should drop the topic from your list.

3. Urgency of data needed (timeliness)

How urgently are the results needed for making a decision or developing interventions at various levels (from community to policy)? Consider which research should be done first and which can be done later.

4. Political acceptability

In general it is advisable to research a topic that has the interest and support of the local/national authorities. This will increase the chance that the results of the study will be implemented. Under certain circumstances, however, you may feel that a study is required to show that the government’s policy needs adjustment. If so, you should make an extra effort to involve the policy-makers concerned at an early stage, in order to limit the chances for confrontation later.

5. Feasibility

Look at the project you are proposing and consider the complexity of the problem and the resources you will require carrying out your study. Thought should be given first to manpower, time, equipment and money that are locally available.

In situations where the local resources necessary to carry out the project are not sufficient, you might consider resources available at the national level; for example, in research units, research councils or local universities. Finally, explore the possibility of obtaining technical and financial assistance from external sources.

6. Applicability of possible results/recommendations

Is it likely that the recommendations from the study will be applied? This will depend not only on the management capability within the team and the blessing of the authorities but also on the availability of resources for implementing the recommendations. Likewise, the opinion of the potential clients and of responsible staff will influence the implementation of recommendations.

7. Ethical acceptability

We should always consider the possibility that we may inflict harm on others while carrying out research. Therefore, review the study you are proposing and consider important ethical issues such as:

These criteria can be measured by the following rating scales:

SCALES FOR RATING RESEARCH TOPICS

Relevance

  1. = Not relevant
  2. = Relevant
  3. = Very relevant

Avoidance of duplication

  1. = Sufficient information already available
  2. = Some information available but major issues not covered
  3. = No sound information available on which to base problem-solving

Urgency

  1. = Information not urgently needed
  2. = Information could be used right away but a delay of some months would be acceptable
  3. = Data very urgently needed for decision-making

Political acceptability

  1. = Topic not acceptable to high level policymakers
  2. = Topic more or less acceptable
  3. = Topic fully acceptable

Feasibility

  1. = Study not feasible, considering available resources
  2. = Study feasible, considering available resources
  3. = Study very feasible, considering available resources

Applicability

  1. = No chance of recommendations being implemented
  2. = Some chance of recommendations being implemented
  3. = Good chance of recommendations being implemented

Ethical acceptability

  1. = Major ethical problems
  2. = Minor ethical problems
  3. = No ethical problems

In order to assist a group in selecting and rating different research topics, we will use the nominal group technique (NGT)

III. NOMINAL GROUP TECHNIQUE

The nominal group technique (NGT) is a group discussion technique that is useful when one wants to obtain a consensus from a group on a topic where decision-making can be usefully guided by the perceptions and opinions of the various group members. The sequence of the group discussion is usually as follows: individual expression, followed by ‘voting’, followed by discussion, and another round of ‘voting’ followed by discussion etc. The group discussion comes to an end when the results of the last vote are not appreciably different from the last-but-one vote.

Steps in applying the nominal group technique*

Participants (between 6-10, all familiar with the content area being explored) are assembled in a quiet room. They are seated in a U-shaped setting so that all participants can see the display (board, flipchart or overhead). The moderator is a non-participant (in our case a course facilitator) who explains and then guides the participants through the process. The steps of the NGT process are summarised below:

  1. Individual listing of ideas on paper.

    In HSR courses, participants of a team each write one, at most two priority problems for research on a piece of paper. This is done in complete silence to prevent the group from becoming judgmental about the ideas too soon. The sheets are collected.

  2. Display of lists produced, followed by discussion.

    The facilitator takes each sheet of paper and displays all sheets on the board so that all team members can see them. The leader requests the members to briefly explain their ideas and why they suggested them.

    No comments are made by the group at this time, but as the ideas for research topics are presented the rest of the group should study them and see whether they understand what the ideas are and why they are important. If clarification is needed, this is done after all ideas have been presented. Then the participants may attempt to combine overlapping ideas (for example, two almost similar research topics could be combined into one).

  3. Voting and ranking.

    After the ideas have been clarified the facilitator asks the participants to select a certain number of ideas on the display (for example, five) that they consider most important, write them on a sheet of paper and rate them. The rating system used can vary but should be fixed in advance: for instance, 5 for the most important idea, 4 for the next most important, etc. The sheets of paper are then collected. For ranking the proposed research topics we would use the criteria and scales presented in the previous section.

  4. Summarising the results.

    The facilitator writes each individual rating on the display, next to the idea. All scores are added, resulting in a total score for each idea. The ideas are then ranked, according to the score they received.


    * Some of the ideas in this process are adapted from Williamson JW et al. (1981) Health Accounting for Quality Assurance. American Occupational Therapy Association.

  5. Discussion of the results.

    The results of the first vote are discussed in plenary. All members are urged to contribute. The facilitator may wish to select two types of ideas for clarification: those with high votes and those with divergent votes (i.e. high as well as low weights). A few new ideas may be developed in this discussion. Also, it may be possible to identify a few ‘sleeper’ ideas among those given low votes. Sometimes such ideas may get a high vote when group members understand why the idea was brought forward.

  6. Second vote and re-ranking.

    Participants are now asked to vote a second time and the whole process of ranking and discussion is repeated. Voting stops when the results from two consecutive votes do not yield a marked difference. The ranking of the revised final scores gives the order of importance of the ideas as perceived by the group.

Advantages of the NGT

The NGT (or a modified version of the NGT) is particularly useful during the research process in health systems research to:

REFERENCES

Fisher A, Laing J and Stoeckel J (1983) Handbook for Family Planning Operations Design. New York: The Population Council.

For more reading about the nominal group technique the following literature is recommended:

Abramson JH (1990) Survey Methods in Community Medicine. Epidemiological Studies, Programme Evaluation, Clinical Trials. London: Churchill Livingstone/ Longman Group Ltd (4th ed.): 188-190.

Delbecq AL, Van de Ven AH, Gustafson DH (1975) Techniques for program planning: A guide to nominal group and Delphi processes. Glenview, Illinois: Scott, Foreman.

Van de Ven AH, Delbecq AL (1972) The Nominal Group Technique. Am Jrnl of Public Health 62:337.

EXERCISE: The Chobe district health team (DHT), selecting a research project

(To be carried out in plenary, ½ hour, if this is the first discussion of possible research topics)

Introduction to the exercise

The Chobe DHT, responsible for the health of a population of 525,000, including 313,000 in Chobe town, has to choose between two important study topics:

Possibility 1

The first possibility is a study that aims to contribute to the development of alternative health financing for low-income households. The study will compare the effectiveness, feasibility and acceptability of two models of community health insurance in a rural community.

In Chobe district, almost 90% of families depend on the informal sector for their livelihood. Day-to-day expenditure is met with great difficulty, and illness affects the household budget in two ways. Not only does the household have to spend on medical treatment, but illness also might reduce the ability of the household to earn during the period of illness. In Chobe, distance and travel time limit the access to public sector clinics and hospitals. Furthermore, public sector outpatient and hospital services suffer from poor quality and resource constraints. Hence most people use private sector health services. There is one private hospital in Chobe town. Nearly three fourth (70%) of the health care expenditure is met privately through out of pocket expenses by the patients. The poor avoid use of health services unless there is very serious illness. When such illness does occur, it frequently has a catastrophic effect on family finance. The concepts of insurance & risk sharing are still alien to the rural population. Health insurance, either social or private, has remained out of the reach of the low-income households due to unaffordable premiums. Recently, community-based initiatives have been designed aiming at providing alternate more equitable financing, particularly for hospitalization for the poor. However, there is limited experience in implementing such schemes, and several questions have arisen about the feasibility, acceptability, sustainability and effectiveness of such schemes.

It is proposed to test two models of community health financing in Chobe district, using the foundation of women’s self help groups (SHG). During the past five years, the Department of Community Medicine has provided technical assistance for women’s empowerment in Chobe by facilitating SHG. Village women are motivated to form small groups of 15-20 women and contribute a monthly fixed sum to a common fund. This fund is managed entirely by the group itself and women undertake income-generating activities with the help of loans from these funds. Most of the SHG now have reserve savings. It is proposed to introduce health insurance by motivating members to make a monthly contribution towards a common reserve fund for health insurance. SHG that have been in existence for more than three years would be invited to select one of two models of health insurance.

The first model would use the reserve fund of the SHG to subsidize cost of hospitalization in the Chobe Private Hospital. Each SHG will have to purchase from the Hospital three Health Insurance cards annually, at a predetermined cost. On producing the Health Insurance card at the time of hospitalization, a patient will have to pay only 25% of the total hospital bill. The group will reimburse this 25% from the common reserve fund of the group. The group itself will decide the reimbursement limit. The second model would purchase health insurance policies for its members from a state sponsored insurance scheme. The State Scheme provides insurance policies with an annual individual premium and a family premium. The Self Help group would pay the premiums from the reserve fund and recover the cost from members in easy instalments. The insurance policy would cover reimbursement of hospitalization expenses in any public or private hospital or clinic for illness/disease or injury sustained up to a predetermined sum per person per year.

The Community Medicine Department would undertake a study to assess:

The study would take 18 months. The SHGs will be given the necessary support to develop guidelines to safeguard against the risks such as moral hazards, adverse selection, over-usage and fraud, which can derail the scheme. Concurrent with the study, a mass education programme of the community will be undertaken to render education and disseminate information regarding the benefits of health insurance. Base-line and end-line surveys together with focus group discussions will be conducted to obtain data on the research questions.

EXERCISE 1 (continued)

Possibility 2

The second possibility is to examine the reasons for the increase of the number of reported suicides in the district. This increase is alarming, not only in Chobe, but in the whole country. The victims are predominantly (85%) adolescents and young adults between 14-24 years and the number of reported suicides tripled between 1997 and 1999. 128 cases were reported in these two years nationally, of which 35% were in Chobe town (exploratory study carried out in 1999).

The number is highest in urban areas. The Minister of Health and Social Welfare expressed her concern and advocated for studies to get more insight in the reasons and possible remedies. From literature and first hand impressions, it appears that poverty and disintegration of families are major underlying factors to suicide. There is a high rate of migration of men to the mines in the neighbouring country, and women have to struggle to raise their children alone, often with minimal economic support from their husbands. There is also a significant rural-urban migration within the country (Census 1996). This migration erodes the extended family ties and also the cohesion within neighbourhoods. Yet in towns, social networks are still poorer, and for newly arriving youngsters often inexistent. Social services do not cope with the needs of the rapidly expanding urban population. Under these circumstances, young people may become easy victims of alcohol or drug use and unsafe sexual relationship, in an effort to please their peers or to stay alive. Adolescents may be suffering from all kinds of stress. In an effort to belong to and be accepted by their peers, there is extensive pressure on them to perform well in school, to obtain well-paying jobs, to take responsibility for their parents and younger siblings, and, for girls, to find solid partners. The reality of what they can expect to achieve is often very different, for boys as well as girls, which causes stress. For girls there is an additional risk of becoming pregnant against their will as they have little power to refuse men with more means and a higher status who approach them. Many feel forced to search for a (usually unsafe) abortion or risk the anger of their parents because of being expelled from schools. Then there is also the risk of STDs and HIV infection, of which youngsters are only vaguely aware, so that the shock – particularly for young girls – may be high when they become victims of such diseases.

Proposed Study: The district health team proposes to carry out a comparative study focusing on adolescents who attempted suicide and adolescents who did not (same age, sex, residence and workplace/school). Also close relatives of adolescents who committed or attempted suicide could be interviewed, as well as community members, teachers, youth leaders, and church leaders. The main objective of the study would be to get insight in the reasons for suicide and possibilities to prevent it and support adolescents in need.

Directions

Rate the two proposals in small groups, using the form on the following page, and prepare to defend your first choice in plenary. (When rating the topics on the criteria, you can either refer to the ‘Scales for rating research topics’ presented right before this exercise or use the summary scales at the bottom of the rating sheet).

EXERCISE (continued)

./img/designandco_52_la_0.jpg

Rating scale: 1=low, 2=medium, 3=high

GROUP WORK (Approximately 2¼ hours if this is the first discussion)

Meet in your working groups to list and rank the research topics that you want to consider for the research proposal you will develop, as a team, during the course.

  1. Choose a reporter who will present in plenary the topics you have considered and your final choice.
  2. a. If this is the first discussion of possible research topics, it is suggested that each group member write one or two topics on a piece of paper. Then all the topics can be listed on a flipchart and briefly discussed to eliminate duplications. Omit proposals that are obviously less relevant or too difficult to carry out. Ideally you should select no more than five to six topics for individual rating.

    b. If a pre-selection has already been done in the field, and/or different possibilities for research topics have emerged during problem analysis in Module 4, consider the two or three topics you have to choose from.
  3. Each group member should then rate the selected proposals individually, using the scoring sheet on the following page. Then for each proposal the scores of the groups members for each criterion should be tallied on a flipchart and the total scores calculated. Discuss marked differences in individual ratings as these may be due to different interpretations of the criteria.
  4. Then thoroughly review the (two) proposals that received the highest scores. At this point it is important to take into account, which proposed study your group could most realistically carry out within the coming 4-6 months. Ideally, all group members should be able to participate actively and benefit directly from the results.
  5. Finally, select the topic for your upcoming research project and prepare a brief presentation for the other members of your course. Present the flipchart with the scores and provide reasons for your final choice.
  6. Carefully document the arguments supporting your first choice and keep them for use in later sessions.

GROUP WORK (continued)

./img/designandco_54_la_0.jpg

Rating scale: 1=low, 2=medium, 3=high

Note:

If participants have been asked to make a preliminary selection of their research topics in the field, Module 3 should be sent to participants and relevant managers at least 6-8 weeks before the course starts. Preferably a facilitator/trainer should be present as well to provide technical support during the selection process. Otherwise a set of guidelines could be prepared to assist the participants and their managers in the selection process. It is best to ask each group to come with at least two potential research problems, in case one of the topics, on further analysis, proves infeasible.

Trainer’s Notes

Module 3: IDENTIFYING AND PRIORITISING TOPICS FOR RESEARCH

Timing and teaching methods

40 minutes Introduction and discussion
50 minutes Exercise: Chobe district including explanation of the nominal group technique (NTG) method
2 hours Group work
1 hour Group reporting (15 minutes per group)
4½ hours TOTAL TIME
Materials
Introduction and discussion

Discuss the process of problem identification (Part I of the module), criteria for prioritising topics for research (Part II) and the Nominal Group Technique (Part III). Be sure you are thoroughly familiar with the concepts but let the criteria as well as the definitions come, as much as possible, from the group. (Before describing the criteria listed in the module, ask the participants to brainstorm, suggesting what criteria they think are most important to consider when selecting a research topic).

Exercise: The Chobe district health team (DHT) - selecting a research project

Note:

There are no right or wrong answers to the exercise. Either proposal may receive priority for different reasons.

Group work

Note:

During this first group work session it is not important which facilitator works with which group. Once the topics have been selected, final assignments of facilitators to specific groups may be made after considering the facilitators’ familiarity with the topics chosen.

This page intentionally left blank.

Designing and Conducting Health Systems Research Projects
Part I: Proposal Development and Fieldwork

Module 4
ANALYSIS AND STATEMENT OF THE PROBLEM

./img/designandco_59_la_0.jpg

NB: Development of a research process is a cyclical process. The double-headed arrows indicate that the process is never linear.

Module 4: ANALYSIS AND STATEMENT OF THE PROBLEM

OBJECTIVES

At the end of this session you should be able to:

  1. Analyse a selected problem and the factors influencing it.
  2. Prepare the statement of the problem for the research proposal you will be developing during the course.
  1. Analysing the problem
  2. Deciding on the scope and focus of the research
  3. Formulating the problem statement

I. ANALYSING THE PROBLEM

In HSR the researcher is often required to do research on a problem with which (s)he is not very familiar. Health workers and managers or community members may be much more familiar with it. But even they may never have given critical attention to the various aspects of the problem.

A systematic analysis of the problem, completed jointly by the researchers, health workers, managers and community representatives, is a very crucial step in designing the research because it:

  1. enables those concerned to pool their knowledge of the problem,
  2. clarifies the problem and the possible factors that may be contributing to it, and
  3. facilitates decisions concerning the focus and scope of the research.

Note:

In a workshop setting, it may be impossible to obtain input from all concerned. The opinion of people who cannot be consulted (e.g. local health staff or community members) should be solicited prior to and immediately after the workshop, before finalising the research proposal.

Steps in analysing the problem

Step 1: Clarify the viewpoints of managers, health care workers and researchers in relation to the problem.

Areas of concern within the health system are often expressed in broad or vague terms by managers and health care workers. For example,

‘Complications of unsafe abortions need more attention’

‘Teenage abortion is a problem’

During initial discussions with managers and health care workers who are involved in the problem area, clarify the issues by listing all the problems in the area of concern, as they perceive them.

Remember that a problem exists when there is a discrepancy between ‘what is’ and ‘what should be’ (see Module 3). Therefore, the perceived problems should be worded in such a way as to illustrate this discrepancy.

For example, health care managers and workers may determine that the general concern about the complications from unsafe abortions among teenagers includes the following problems:

Step 2: Further specify and describe the core problem.

You should then try to identify the core problem and quantify it. Looking at the example discussed in Step 1, you may decide that the core problem is:

You should now attempt to describe more elaborately:

Step 3: Analyse the problem.

After identifying the core problem you should:

It is helpful to visualise these interrelationships in the form of a DIAGRAM. The basic principles of constructing such a diagram are illustrated below:

Figure 4.1: Elements of a problem analysis diagram

./img/designandco_62_la_0.jpg

The relationships between contributing factors and the problem can be indicated by arrows, either one-way (for cause-effect relationships) or two-way arrows (for mutual relationships). The core problem can be identified by drawing a double line around it.

Analysis of the problem involves several sub-steps.

Step 3.1: Write down the core problem(s) as defined in Step 2 in the centre of a blackboard or flipchart.

Step 3.2: Brainstorm on possible causes or factors contributing to the problem.

It is important that the viewpoints of managers, health care workers and/or researchers brought up during Step 1 are all included. Discuss the relationships between the different factors and the problem.

If desired, participants may use separate cards or pieces of paper on which to write possible contributing factors. The cards may be pinned or taped around the core problem on the board or flip chart and moved, revised, or eliminated as necessary, during the development of the diagram.

The initial diagram of the problem of complications from unsafe abortions among teenagers might look like this:

Figure 4.2: Initial problem diagram – Complications from unsafe abortion among teenagers

./img/designandco_63_la_0.jpg

Note:

that many of the perceived problems mentioned in step 1 are related to each other in a cause-effect relationship (e.g. poor management of complications from abortion and high complication rate from abortions), or in a mutual relationship (stigma on induced abortion and secrecy surrounding induced abortion).

As you can see, this initial diagram suggests that further development of the analysis could proceed in at least two directions, i.e., analysis of factors related to:

These sets of community and service factors will appear in many HSR studies. In reality they usually prove to be closely intertwined. Poor management of abortion is, for example, influenced by the negative attitudes of staff towards abortion that, in turn, is influenced by community attitudes and perceptions.

Step 3.3: Identify further contributing factors.

Extend the problem analysis diagram further by identifying additional factors that could have contributed to or aggravated the problem. It may be possible to identify several ‘generations’ of predisposing factors, by asking ‘but why’.*

Let us take another example: High defaulter rate among tuberculosis (TB) patients (Figure 4.3).

One of the factors contributing to low patient compliance may be (and often is, see Joint HSR Project, 1996) low understanding by patients of the treatment requirements. Further asking ‘but why’ may lead to the assumption that patients are not systematically counselled on the ‘what’ and ‘why’ of treatment, which in turn may be influenced by a number of important weaknesses in the training of staff which need to be addressed.

It is desirable to continue identifying underlying contributing factors until you reach basic factors that need to be modified to solve the problem, and that can be modified within the existing context. This will facilitate the formulation of research projects that can provide useful information for decision-making. The process of continued analysis by asking ‘but why’ will necessitate several revisions or extensions of the initial analysis diagram. The final version should encompass all the critical factors that may be contributing to the problem which will be studied.


* See Barnett and Abbatt (1994) for further illustrations of the ‘but why’ method.

Figure 4.3: Identifying several ‘generations’ of predisposing factors causing high defaulter rate among TB patients

./img/designandco_65_la_0.jpg

Step 3.4 Attempt to organise related factors together into larger categories, and develop your final draft of the diagram.

This final step in organising the diagram will help you not to overlook important factors and will make it easier to develop the data collection tools in a systematic way.

For example, the revised diagram focusing on the ‘high defaulter rate’ among tuberculosis patients may group contributing factors into three main categories:

For our TB example, we may categorise the contributing factors to defaulting into these three main groups (see Figure 4.4):

Socio-cultural factors, such as
Service factors, such as:
Disease-related factors, such as:

Note that diagrams are easier to read and work with if the factors that directly contribute to a problem are highlighted (see Figure 4.4).

Figure 4.4: Revised problem analysis diagram of factors contributing to high defaulter rate among TB patients

./img/designandco_67_la_0.jpg

Note:

If the research will focus on a description of a situation or health problem (size, distribution) or an evaluation (see Module 3, p. 1), it may not be appropriate to make an analytical diagram looking for causes of a problem. Here the problem is lack of information.

For example, we may need information on knowledge, attitudes and practices (KAP) of teenagers with respect to bilharzia in order to develop adequate health-education materials for schools. This is a descriptive study, for which we can make a different diagram, listing knowledge, attitudes and practices that we want to explore in the study. We can, however, go one step further and also list the factors that may have contributed to the generation of the teenagers’ KAP.

Figure 4.5: Knowledge, attitudes and practices with regard to bilharzia and its treatment and prevention (Example of a descriptive diagram)

./img/designandco_68_la_0.jpg

If it is our aim to evaluate interventions or programmes, we evaluate in general against the objectives: to what extent have we reached them? Is the intervention the most cost-effective? Is it still appropriate (covering a priority need?) etc. (see Module 3).

Evaluation can also be applied to the health system as a whole. The WHO, for example, proposes that the health system should cover the whole population with fair (financially equitable) and the best attainable level of services, delivered in a consumer-friendly way. In that case we take diagram 2.4 in Module 2 as our point of departure.

II. DECIDING ON THE FOCUS AND SCOPE OF THE RESEARCH

After detailed analysis of a problem, it is important to reconsider the focus and scope of the research. Several issues are particularly important to consider, including:

  1. Usefulness of the information. Would the information that will be collected on this problem help improve health and health care? Who would use the findings related to the factors in the diagram that would be studied? How would the findings be used?
  2. Feasibility. Is it feasible to analyse all the factors related to the problem in the 4-6 months available for research?
  3. Duplication. Is some of the information related to factors in the diagram already available? What aspects of the problem need further research?

Reconsider your problem diagram with these issues in mind. If your problem is complex and has many possible contributing factors, identify and demarcate the boundaries of possible smaller research topics. If there is more than one possible topic, use the selection criteria and ranking method that were described in Module 3 to assist you in your final decision concerning the focus and scope of your research.

Note of caution:

The dissection of the diagram into different parts and selection of one part for research is not advised if insufficient insight exists into the nature, relative weight and interrelationships of the various factors contributing to the problem. You would risk concentrating on marginal factors and coming up with marginal solutions. It is, for example, inadvisable to concentrate only on community factors or only on service factors to explain under-utilisation of services if you don’t know how these factors are interrelated and where the main problem is. An exploratory study would then be indicated, limited rather in the number of informants than the number of factors included in the study (see Modules 9, 1 0 and 1 1).

III. FORMULATING THE PROBLEM STATEMENT

The first major section in a research proposal is the ‘statement of the problem’.

Why is it important to state and define the problem well?

Because you will find that a clear statement of the problem:

What information should be included in the statement of the problem?
  1. A brief description of socio-economic and cultural characteristics and an overview of health status and the health-care system in the country/district in as far as these are relevant to the problem. Include a few illustrative statistics, if available, to help describe the context in which the problem occurs.
  2. A concise description of the nature of the problem (the discrepancy between what is and what should be) and of the size, distribution and severity of the problem (who is affected, where, since when, and what are the consequences for those affected and for the services). For a descriptive or evaluation study you will elaborate the different components of the problem.
  3. An analysis of the major factors that may influence the problem and a discussion of why certain factors need more investigation if the problem is to be fully understood.
  4. A brief description of any solutions to the problem that have been tried in the past, how well they have worked, and why further research is needed (justification for your study).
  5. A description of the type of information expected to result from the project and how this information will be used to help solve the problem.
  6. If necessary, a short list of definitions of crucial concepts used in the statement of the problem.

A list of abbreviations may be annexed to the proposal, but each abbreviation also has to be written out in full when introduced in the text for the first time.

GROUP WORK

  1. Select a reporter who will present the statement of the problem in plenary.
  2. Discuss comments you received in the previous plenary session on the choice of your topic and revise your topic, if necessary.
  3. Make an analysis diagram of the most important components of the problem or the most important factors that you think are influencing it. Use a blackboard or a flip chart and, if possible, separate cards for each factor. (See part 1 of this module for details on the steps in this process.) After making your initial diagram, try to rearrange the factors identified into broader categories.
  4. Decide whether it is feasible to explore all the factors in your problem analysis diagram. In case of doubt, consider two possibilities:
  5. a. All factors seem important and interrelated; you could not easily split the diagram up in possible sub-studies. Just continue. We will come back to possible ways of increasing the feasibility of the study when discussing Module 9 (Study type), 1 0 (Data-collection techniques) and Module 11 (Sampling).
  6. b. The diagram is so complex that several studies would be necessary to cover it. If so, demarcate the boundaries of possible projects and use the criteria and ranking system in Module 3 to select one of the sub-problems as the focus for your project.
    Note: Groups that selected their general problem areas before coming to the course may want to spend an hour or two at this point to systematically rank and choose between possible topics within their general problem areas, using the instructions in Module 3, part II.
  7. Prepare a first draft of 2-3 pages of the statement of the problem for the topic you have selected in your group.
    • First prepare an outline covering items 2 through 5 in the list presented just before this group work session.
    • Then prepare one or two paragraphs of ‘background information’ (item 1) which places the problem in its context and will be used as the introduction to the statement of the problem.
    • Finally, define crucial terms and explain abbreviations, if necessary.
  8. Identify information you need now, from the literature or from key informants, to help you focus your study and to further develop your statement of the problem. Ask course facilitators to give you assistance, if necessary. Much, if not all, of your literature review (module 5) may be included in your statement of the problem.
  9. Present your flipcharts with the problem analysis diagram and outline of your problem statement in plenary. Justify the need for your study (¼ hour per group). See the guidelines for the discussion of the presentations on the following page.
  10. Keep all materials presented, as well as your notes on the comments received in the plenary session, for use during further development of your proposal. Have the first draft of your statement of the problem computerised.

Guidelines for the plenary session following group work: presentation and discussion of problem statements

Each group should present its problem analysis diagram and problem statement in the plenary session. Facilitators and members of other groups should comment and provide suggestions for improvement.

Guidelines for presentation

Guidelines for discussion

As each team presents its work, consider the following issues for discussion:

Recommended reading:

Joint Project on Health Systems Research for Southern Africa, Harare, Zimbabwe, Series ‘Health Systems Research - It can make a difference’:

Vol. 1: Availability, provision and use of drugs (1994)
Vol. 2: Factors associated with maternal mortality (1994)
Vol. 3: Under-utilisation of TB services in Southern Africa (1996)
Vol. 4: Factors influencing the functioning of Primary Health Care at village level (1996)

All these volumes contain exemplary research protocols with elaborate ‘Statements of the Problem’ and results of studies carried out in HSR Projects in Southern Africa.

Barnett, Liz and Fred Abbatt (1994) District action research and education. London: The MacMillan Press Ltd.

Trainer’s Notes

Module 4: ANALYSIS AND STATEMENT OF PROBLEM

Timing and teaching methods

1 hour Introduction and discussion
3-5 hours Group work
1 hourPlenary
5-7 hoursTOTAL TIME (partly in the evening)
Materials
Introduction

This module is designed to assist health staff, researchers, and managers to work together to analyse the problem and prepare a problem statement.

It would be beneficial to include health service managers and community leaders who are concerned with the problem and who are likely to use the findings as resource persons during the group work. Resource persons could include, for example, the district medical officer, the hospital manager, the co-ordinator of the Tuberculosis Programme, the mayor or traditional leaders, etc.

Presentation and discussion

It is extremely important that participants understand the principle of making a diagram, with a problem in the centre and contributing factors grouped around the problem. When you give examples, get them to participate actively, identifying what the core problem is and in what direction the arrows should point.

With respect to presenting the diagrams, you might advise the groups to work with different overhead sheets on top of each other.

The final TB diagram, for example, could consist of 4 sheets:

Give due weight to descriptive studies which have no core problem or contributing factors, but simply consist of different, connected, components (e.g. knowledge, attitudes and practice studies).

Group work

This is the first time during the workshop that the participants will work in small groups. Therefore the group may experience the initial difficulties of establishing group-dynamics. Furthermore, many participants may never have had the experience of systematically analysing a problem before.

The topic for the first group work in the workshop is particularly complex. Adequate selection of a problem is crucial for subsequent development of the project. In order to ensure that the group work is productive, the workshop facilitator could assume the role of chairperson for this group work session.

In all subsequent modules, the participants should select their own group leader and reporter for each group work session.

Suggestions concerning various steps in the group work process are given below:

1. Listing viewpoints

2. Constructing a problem analysis diagram

3. Narrowing the focus of the project

4. Writing the statement of the problem

Make sure that each group keeps a copy of the final version of the problem analysis diagram. It will serve as the basis for subsequently formulating specific objectives and constructing variables.

Designing and Conducting Health Systems Research Projects
Part I: Proposal Development and Fieldwork

Module 5
REVIEW OF AVAILABLE LITERATURE
AND INFORMATION

./img/designandco_77_la_0.jpg

NB: Development of a research process is a cyclical process. The double-headed arrows indicate that the process is never linear.

Module 5: REVIEW OF AVAILABLE LITERATURE AND INFORMATION

OBJECTIVES

At the end of this session you should be able to:

  1. Describe the reasons for reviewing available literature and other information during the preparation of a research protocol.
  2. Identify the resources that are available for carrying out such a review.
  3. Prepare index cards, computer entries or notes that summarise important information obtained from literature or interviews with key informants.
  4. Prepare a review of literature and other information pertaining to your research topic that will adequately describe the context of your study and strengthen your statement of the problem.
Why is it important to review already available information when preparing a research proposal?
What are the possible sources of information?
Where can we find these different sources?

Different sources of information can be consulted and reviewed at various levels of the administrative system within your country and internationally.

ADMINISTRATIVE LEVELS        EXAMPLES OF RESOURCES

./img/designandco_79_la_0.jpg

./img/designandco_80_la_0.jpg

You need to develop a strategy to gain access to each source and to obtain information in the most productive manner. Your strategy may vary according to where you work and the topic under study. It may include the following steps:

Some agencies will assist with your literature search if requested by telephone or in writing. The request, however, should be very specific. Otherwise you will receive a long list of references, most of which will be not be relevant to your topic. If you are requesting a computerised search it is useful to suggest key words that can be used in locating the relevant references.

Note:

Facilitators should be able to provide specific information regarding national and international facilities to assist you with the search for literature.

References that are identified:

Information on an index card should be organised in such a way that you can easily find all data you will need for your report:

For an article the following information should be noted:

Author(s)’ Surname followed by initials. Title of article. Name of Journal. Year, Volume, (number): page numbers of article.

Example:

Louria DB. Emerging- and re-emerging infections: The societal variables. International Journal of Infectious Disease. 1996, 1(2):59-62.

For a book the following information should be noted:

Author(s)’ Surname followed by initials. Title of book. Place: Publisher, year, Edition

Example:

Abramson JH. Survey methods in community medicine. Edinburgh: Churchill Livingstone, 1990, 4th ed.

For a chapter in a book, the reference can include:

Author(s) of chapter (Surname(s) followed by initials). Chapter title. In: Editor(s) of book, (Surname(s) followed by initials). (eds). Title of book. Place: Publisher, year: page numbers of chapter.

Example:

Todd J and Barongo L. Epidemiological methods. In: Ng’weshemi J, Boerma T, Bennett J and Schapink D (eds). HIV prevention and AIDS care in Africa; A district level approach. Amsterdam: KIT Press, 1997: 51-68.

The formats suggested above have been adopted as standard by over 300 biomedical journals and are referred to as the Vancouver System. In other journals and books it is common to put the year, between brackets, straight after the name of the author(s). This is called the Harvard System. There are more systems in use for referencing to literature. Always carefully look what system is used in the journal you are submitting an article to and follow it systematically.

At present many journals use as few punctuation marks as possible. We therefore have minimised punctuation marks in the examples above. In Harvard style, this looks as follows:

Abramson JH (1990) 4th ed. Survey methods in community medicine. Edinburgh: Churchill Livingstone.

Further, the index card or computer entry (one for each reference) could contain quotations and information such as:

Note:

Index cards or computer entries can also be used to summarise information obtained from other sources, such as informal discussions, reports of local health statistics, and internal reports. If you don’t use cards or computers, write the information retrieved from your literature in a systematic way on A4 sheets of paper.

How do you write a review of literature?

There are a number of steps you should take when preparing a review of available literature and information:

Where do you put which information?

Clearly, you will use some literature when describing the local context (country, region) or your problem. Note that all facts you mention need a source, except some general and well-known statements.

Also, for the description of the selected problem, you will use all available raw, grey or published literature you can obtain, well quoted (see below). You may use literature from other countries or regions to illustrate your point. If these sources are many you could have a separate section on international literature (see Annex 5.4). More complex studies using theoretical models should have a separate section or chapter discussing these models, which could come after the ‘Statement of the Problem’ section.

Note:

When drafting your ‘Background’ section or the ‘Statement of the Problem’, you will usually not describe your sources one by one. Instead, you will write a coherent discussion in your own words, using all relevant literature linked to each other. It is possible to cite several sources for one statement you make. (See Annex 5.4)

Referencing

You always need to reference all the literature that you refer to in your review. When you use the Vancouver system, you will use consecutive numbers in the text to indicate your references. At the end of your paper or chapter (of a book) you will then list your references in that order, using the format described above. In your research proposal the references will come before the annexes (see Modules 1 or 18).

Alternatively, you can use the Harvard system and refer to the references more fully in the text, putting the surname of the author, year of publication and number(s) of page(s) referred to between brackets, e.g., (Shiva 1998:15-17). If this system of citation is used, the references at the end of the proposal should be listed in alphabetical order (see Annex 5.5).

The Harvard author/date system of referencing seems easier, as you can change the order of paragraphs without consequences for your referral system. However at present, computers have programmes that change the numbers of your references automatically if you reshuffle the text while using the Vancouver system.

Possible bias

Bias in the literature or in a review of the literature is a distortion of the available information in such a way that it reflects opinions or conclusions, which do not represent the real situation.

It is useful to be aware of various types of bias. This will help you to be critical of the existing literature. If you have reservations about certain references or if you find conflicting opinions in the literature, discuss these openly and critically. Such a critical attitude may also help you avoid biases in your own study. Common types of bias in literature include:

Ethical considerations

The types of bias mentioned above would put the scientific integrity of the responsible researcher in question. Moreover, careless presentation and interpretation of data may put readers who want to use the study’s findings on the wrong track. This may have serious consequences, in terms of time and money spent on HSR and it may even lead to wrong decisions affecting people’s health. A similarly serious act, for which a researcher can be taken to court, is the presentation of research results or scientific publications from other writers without quoting the author. Therefore, appropriate referencing procedures should always be followed in research proposals as well as in research reports.

Introduction to group work

For this group work session you will choose a group chairperson. In the sessions that follow you will always have a group chairperson as well as a recorder.

The functions of a chairperson are to:

The major function of the recorder is, to take care that the flip charts or overhead sheets to be presented in plenary:

Recorders may change each session, but the leadership role should remain with one group member, for efficiency’s sake. This is especially important during the last week, when the final draft of the research proposal is being prepared.

GROUP WORK (2 hours)

  1. Select a chairperson and recorder. Read their functions aloud and discuss whether you agree with these functions.
  2. Outline the topics for which you need information that will be included in the ‘Background’ or ‘Statement of the Problem’ sections of your proposal, making use of your problem analysis diagram.
  3. Search through the documents (books, articles, and bibliographies) available in the course library. List the most useful references you can find on your topic. Brainstorm on where to find additional literature.
  4. Summarise the most important information from the references. Place this information on index cards, A4 sheets of paper or in computer entries. Divide this work among group members (and make sure it continues after having drafted the research proposal.)
  5. Decide whether, apart from the background and statement of the problem sections based on literature, you need a separate section on international literature as introduction to or following the statement of the problem. You may have a section on relevant theory. Write coherent narratives based on the information you collected; analyse and comment on the contributions from various sources, rather than simply reporting on their content. A list of the references used should be presented straight after the text of your research proposal.

REFERENCES

Gibaldi J (1995) MLA Handbook for Writers of Research Papers. New York: Modern Language Association of America.

Jen Tsi Yang et al. (1996) An outline of Scientific Writing: For Researchers with English As a Foreign Language. Singapore: World Scientific Publishing. www.amazon.com/exec/obidos (through internet, Sept. 2000).

Lindsay D (1996) Guide to Scientific Writing. Australia: Addison & Wesley. (paperback)

./img/designandco_85_la_0.jpg

./img/designandco_86_la_0.jpg

vd Geest S, Whyte SR eds. (1991) The context of Medicines in Developing Countries. Studies in Pharmaceutical Anthropology. Amsterdam: Het Spinhuis Publishers. (2nd edition)

— The study consists of two parts:

(1) A description of the transaction of medicine (production, selling, consumption). The roles of drug company salesmen, pharmacists, street vendors and ‘traditional’ practitioners in selling commercial drugs are examined.

(2) A description of the meaning of medicines to its users, e.g., perceived efficacy related to cost, colour, taste, packaging.

— The methodologies used are a combination of survey techniques (to identify where people obtain what drugs, how often etc.) and qualitative techniques (participant observation and in-depth interviews about what people actually use when, how and why).

— Studies cover a wide range of countries in Latin America, Africa, Southern and SE Asia, and even Ancient Europe and Medieval China.

Reverse side of index card:

— Points emphasised in the publication:

— Observations

— New, interesting field; good mix of methodologies

— For own HSR study, use pp 131-149,199-216,199-326

Part of the literature review in support of the study Factors contributing to defaulting from out-patient treatment among tuberculosis patients who registered in Masvingo Province, Zimbabwe’.*

It is interesting that studies quantifying problems in case-finding and case-holding in TB are far more numerous than studies identifying contributing factors, and further, that knowledge, attitudes and practices of patients often receive more attention than those of staff. In areas as different as India and Honduras the same factors appeared responsible for reluctance of patients to come forward for treatment: lack of knowledge of early symptoms, fear of stigma attached to the disease, or a combination thereof.20 Yet, a study in Japan showed that he average ‘doctor’s delay’ (time between first visit to a doctor and actual diagnosis) always surpassed the ‘patient’s delay’ (time between reported onset of symptoms and first visit to a doctor). For patients who reported early with complaints, within two months after onset of symptoms, the delays in diagnosis were relatively most extensive.21 Aluoch22 found the same for Kenya.

Also with respect to patients compliance with treatment we have to consider the multiple contribution of patients, community as well as services. In particular in turbulent times, for example when the TB services are abruptly integrated in the general health system, as happened in Botswana in the end of the 1970s, defaulter rates as high as 75% have been reported.23

Other studies in South Africa,24 India,25 Papua New Guinea26 and Malaysia27 as well as the Botswana study23 concentrate more on patient and community factors, sometimes in interaction with service factors. Poverty, mobility (migrant labour), poor accessibility of TB services, lack of support from relatives, peers or employers, socio-cultural factors (conflicting perceptions of causes and treatment preferences; fear of stigma) and illness factors (low severity of disease at diagnosis, duration of symptoms) emerge as good predictors of poor patient compliance. Very interesting is, however, the shared emphasis on the poor information patients receive on their disease. Roy found in his study in Malaysia27 that even among patients who were hospitalised for two or three months before starting out-patient treatment, 70% did not know they were suffering from an infectious disease and 80% did not know how long they had to stay in hospital or how long the estimated total duration of their treatment would be. He states that most of the medical and paramedical staff have neither the training nor the interest to give health education, a task for which they have not been primarily employed.

In the field of health education, encouraging experiments have been carried out in some of the countries mentioned above. Papua New Guinea, where patients defaulted en mass both during in- and out-patients treatment, health education was turned into a participatory exercise. Patients not only were explained the basic facts of TB in understandable terms but also listened through the stethoscope to the crackles in their chests and were shown X-rays of affected lungs compared to healthy lungs. As in Malaysia, more experienced patients were involved in health education, besides the health staff; they became the peer educators of newly diagnosed victims of TB and their relatives. The percentage of non-complying out-patients dropped from 50 to 20 within half a year.26

The literature underlines the urgency of the planned research in Masvingo Province where the AIDS and TB epidemics seem to fuel each other. It also emphasises the necessity to study the problem of low patient compliance with treatment from different sides: that of the services, the patient and the community.


* In: Joint Project on Health Systems Research (1996) HSR: It can make a difference. Vol. 3: Under-utilisation of tuberculosis services in Southern Africa. Harare: WHO/AFRO.

References and notes

20. Westaway MS. Knowledge and attitudes about tuberculosis of black hospitalised TB patients. Tubercle, 1990, 71: 55-59.

21. Aoki M, Mori T, Shimao T. Studies on factors influencing patient’s, doctor’s and total delay of tuberculosis case-detection in Japan. Bulletin of the International Union Against Tuberculosis, 1990, 60(3-4):128-130.

22. Aluoch JA. A survey to assess passive case-finding in pulmonary tuberculosis in Kenya. East African Medical Journal, 1983, 60(6):360-365.

23. Varkevisser CM. TB Control in Botswana; Problem identification. Amsterdam: Royal Tropical Institute, 1977:42-53; 24-41.

24. Bell J, Jach D. Tuberculosis patient compliance in the Western Cape, 1984. South African Medical Journal, 1988, 73:31-33.

25. Barnhoorn F, Adriaanse H. Insearch of factors responsible for non-compliance among tuberculosis patients in Wardha District, India. Social Science and Medicine, 1992, 34(3): 291-306.

26. Garner P, Hill G. Brainwashing in Tuberculosis management. Papua New Guinea Medical Journal, 1985, 28:291-293.

27. Roy RN. Systematic health education of tuberculosis patients and of the population. Bulletin of the International Union Against Tuberculosis, 1990, 60(3-4):33-34.

It is interesting that studies quantifying problems in case-finding and case-holding in TB are far more numerous than studies identifying contributing factors, and further, that knowledge, attitudes and practices of patients often receive more attention than those of staff. In areas as different as India and Honduras the same factors appeared responsible for reluctance of patients to come forward for treatment: lack of knowledge of early symptoms, fear of stigma attached to the disease, or a combination thereof (Westaway 1990). Yet, a study in Japan showed that he average ‘doctor’s delay’ (time between first visit to a doctor and actual diagnosis) always surpassed the ‘patient’s delay’ (time between reported onset of symptoms and first visit to a doctor). For patients who reported early with complaints, within two months after onset of symptoms, the delays in diagnosis were relatively most extensive (Aoki et al 1990). Aluoch (1983) found the same for Kenya.

Also with respect to patients compliance with treatment we have to consider the multiple contribution of patients, community as well as services. In particular in turbulent times, for example when the TB services are abruptly integrated in the general health system, as happened in Botswana in the end of the 1970s, defaulter rates as high as 75% have been reported (Varkevisser 1977).

Other studies in South Africa, (Bell and Jach 1984) India (Barnhorn and Adriaanse 1992), Papua New Guinea (Garner and Hill 1985) and Malaysia (Roy 1985) as well as the Botswana study (Varkevisser 1977) concentrate more on patient and community factors, sometimes in interaction with service factors. Poverty, mobility (migrant labour), poor accessibility of TB services, lack of support from relatives, peers or employers, socio-cultural factors (conflicting perceptions of causes and treatment preferences; fear of stigma) and illness factors (low severity of disease at diagnosis, duration of symptoms) emerge as good predictors of poor patient compliance. (etc., etc.)

REFERENCES

Aluoch JA (1983) A survey to assess passive case-finding in pulmonary tuberculosis in Kenya. East African Medical Journal 60(6):360-365.

Aoki M, Mori T, Shimao T (1990) Studies on factors influencing patient’s, doctor’s and total delay of tuberculosis case-detection in Japan. Bulletin of the International Union Against Tuberculosis 60(3-4): 128-130.

Barnhoorn F, Adriaanse H (1992) In search of factors responsible for non-compliance among tuberculosis patients in Wardha District, India. Social Science and Medicine 34(3):291-306.

Bell J, Jach D (1988) Tuberculosis patient compliance in the Western Cape, 1984. South African Medical Journal 73:31-33.

Garner P, Hill G (1985) Brainwashing in Tuberculosis management. Papua New Guinea Medical Journal 28: 291-293.

Roy RN (1990) Systematic health education of tuberculosis patients and of the population. Bulletin of the International Union Against Tuberculosis 60(3-4):33-34.

Varkevisser CM (1977) TB Control in Botswana; Problem identification. Amsterdam: Royal Tropical Institute:42-53, 24-41.

Westaway MS (1990) Knowledge and attitudes about tuberculosis of black hospitalised TB patients. Tubercle 71:55-59.

Trainer’s Notes

Module 5: REVIEW OF AVAILABLE LITERATURE AND INFORMATION

Timing and teaching methods

¾ hour

Introduction and discussion+

3+ hours+Group work (but literature will be read till the proposal is in its final version, and then till the study has been implemented and the report is finalised)
3¾ hours TOTAL TIME
Materials

Be sure that a course library is ready for use. Prior to the workshop, facilitators should look through their own resources to find any relevant articles they may have for each research topic.

Introduction and discussion

Discuss why and how to do a review of the literature. Have participants suggest answers to the questions, but provide additional information when necessary.

Group work

This page intentionally left blank.

Designing and Conducting Health Systems Research Projects
Part I: Proposal Development and Fieldwork

Module 6
FORMULATION OF RESEARCH OBJECTIVES

./img/designandco_95_la_0.jpg

NB: Development of a research process is a cyclical process. The double-headed arrows indicate that the process is never linear.

Module 6: FORMULATION OF RESEARCH OBJECTIVES

OBJECTIVES

At the end of this session you should be able to:

  1. State the reasons for writing objectives for your research project.
  2. Define and describe the difference between general and specific objectives.
  3. Define the characteristics of research objectives.
  4. Prepare research objectives in an appropriate format for the project you are developing.
  5. Develop further research questions, and research hypotheses, if appropriate for your study.
  1. Objectives
  2. Hypotheses
  3. Title of the study

I. RESEARCH OBJECTIVES

The OBJECTIVES of a research project summarise what is to be achieved by the study.

Objectives should be closely related to the statement of the problem. For example, if the problem identified is low utilisation of child welfare clinics, the general objective of the study could be to identify the reasons for this low utilisation, in order to find solutions.

The general objective of a study states what researchers expect to achieve by the study in general terms.

It is possible (and advisable) to break down a general objective into smaller, logically connected parts. These are normally referred to as specific objectives.

Specific objectives should systematically address the various aspects of the problem as defined under ‘Statement of the Problem’ (Module 4) and the key factors that are assumed to influence or cause the problem. They should specify what you will do in your study, where and for what purpose.

A study into the cost and quality of home-based care for HIV/AIDS patients and their communities in Zimbabwe, developed at an HSR workshop, for example, had as its general objective:

To explore to what extent community home-based care (CHBC) projects in Zimbabwe provide adequate, affordable and sustainable care of good quality to people with HIV/AIDS, and to identify ways in which these services can be improved.

It was split up in the following specific objectives:

  1. To identify the full range of economic, psychosocial, health/nursing care and other needs of patients and their families affected by AIDS.
  2. To determine the extent to which formal and informal support systems address these needs from the viewpoint of service providers as well as patients.
  3. To determine the economic costs of CHBC to the patient and family as well as to the formal CHBC programmes themselves.
  4. To relate the calculated costs to the quality of care provided to the patient by the family and to the family/patient by the CHBC programme.
  5. To determine how improved CHBC and informal support networks can contribute to the needs of persons with AIDS and other chronically and terminally ill patients.
  6. To use the findings to make recommendations on the improvement of CHBC to home care providers, donors and other concerned organisations, including government.

The first specific objective usually focuses on quantifying or specifying the problem.

This is necessary in many studies, especially when a problem has been defined (but not quantified) for which subsequently the major causes have to be identified. Often use can be made of available statistics or of the health information system. In the study on the high defaulter rate of TB patients, this rate should first be established, using the records, and only then would the contributing factors to defaulting be analysed.

In the example given, the needs of AIDS patients and their relatives for care and support have been defined in the first objective. The objectives which follow concentrate on adequacy, cost and quality of care provided whereas the last two objectives specify possible improvements with respect to CHBC, and to whom the results and recommendations of the study will be fed back.

Note:

It may be helpful to use the diagram as a point of departure and check whether the problem and all major, directly contributing factors (analytic study) or major components (descriptive or evaluation study) have been covered by the objectives. An objective indicating how the results will be used should be included in every operational study, either as part of the general objective or as a specific objective.

Why should research objectives be developed?

The formulation of objectives will help you to:

Properly formulated, specific objectives will facilitate the development of your research methodology and will help to orient the collection, analysis, interpretation and utilisation of data.

How should you state your objectives?

Take care that the objectives of your study:

Keep in mind that when the project is evaluated, the results will be compared to the objectives. If the objectives have not been spelled out clearly, the project cannot be evaluated.

Using the previous example on cost and quality of CHBC, we may develop more specific research questions for the different objectives, such as:

II. HYPOTHESES

Based on your experience with the study problem, it might be possible to develop explanations for the problem, which can then be tested. If so, you can formulate hypotheses in addition to the study objectives.

A HYPOTHESIS is a prediction of a relationship between one or more factors and the problem under study that can be tested.

In our example concerning the cost and quality of HBC in Zimbabwe it would have been possible to formulate and test the following hypotheses:

  1. The role of first-line relatives in the provision of care to AIDS patients is more substantial in rural than in urban areas.
  2. The silence and stigma surrounding AIDS makes the formation of self-help groups of AIDS patients and their relatives next to impossible, which in turn maintains the high level of stigma on HIV/AIDS.

Note:

Policy makers and field staff usually feel the need for research because they do NOT have enough insight into the causes of a certain problem. Therefore, most HSR proposals present the specific objectives in the form of open statements (as given in the examples earlier) instead of focusing the study on a limited number of hypotheses.

III. TITLE OF THE STUDY

Now you can finalise the title of your study. The title should be in line with your general objective. Make sure that it is specific enough to tell the reader what your study is about and where it will be calculated.

NOT:     ‘A study on community home-based care’

BUT:     ‘A study on cost and quality of community home-based care for HIV/AIDS patients and their communities in Zimbabwe’

You might also consider fancier titles:

‘Do We Care? A study on cost and quality of CHBC for HIV/AIDS patients in Zimbabwe’*

Another example could be:

‘WORKSHOPS: Blessings or Burdens? A study of the workshops held in 1999 in Province Y - Their utility and consequences for daily working activities of health staff’


* The study with this title, used as an example in the present module, was carried out by G Woelk, H Jackson, R Kerkhoven, K Hansen, N Manjonjori, P Maramba, J Mutambirwa, E Ndimande and E Vera. It was published in December 1997 by the Department of Community Medicine, University of Zimbabwe, the Southern African AIDS Information Dissemination Service (SAFAIDS) and the National AIDS Control Programme, Ministry of Health, Harare, Zimbabwe.

GROUP WORK (2 hours)

  1. Choose a chairperson and a recorder.
  2. Hang up the flip charts that you used to present your statement of the problem so they are visible to all group members. Incorporate useful suggestions for changes that were made when you presented them in plenary. Then, use the analysis diagram as a starting point for formulating objectives, focusing, for example, on:
    • Further quantifying and specifying the problem, if required;
    • Exploring the key factors or major groups of factors that, in your opinion, might influence or cause the problem; and/or
    • Any other major research activities you propose.
  3. Prepare a general objective and specific objectives for the research proposal you are developing.
  4. After formulating your objectives ask yourself the following questions:
    • Do the objectives deal with all aspects of the research problem in a logical and coherent way?
    • Are the objectives clearly phrased?
    • Are the objectives defined in operational terms that can be measured? Are they realistic?
    • Do they indicate where the study will be conducted?
    • Do they include the development of recommendations for how the research results will be used to solve the problem?
  5. Prepare a flip chart with your objectives for use in the exercise and in the plenary discussion. Add on the title of your study and revise it, if necessary, to match the objectives

EXERCISE: Assessing the objectives of another group (½ hour)

Assess the research objectives formulated by another team using the criteria mentioned above. Compare them with the group’s statement of the problem and the title of the study.

Trainer’s Notes

Module 6: FORMULATION OF RESEARCH OBJECTIVES

Timing and teaching methods

½ hour Introduction and discussion
2 hours Group work
½ hour Exercise: Assessing the objectives of another group
1 hour Presentation by each group, followed by comments by the group that did the exercise and general discussion
1 hour Adjustments
5 hours TOTAL TIME
Introduction and discussion
Group work

Be sure to provide sufficient time for the groups to formulate good objectives for their chosen projects. As groups work from their analysis diagram, they may discover that changes are necessary (additions, regrouping, or dropping of factors). It is recommended that the diagram be displayed on a flipchart rather than on an overhead sheet with photocopies for individual group members, so it will be easier to focus the group’s attention on it. The flip chart with the diagram can also be used in Module 8 (Variables).

EXERCISE: Assessing the objectives of another group

Hold an exercise in which groups evaluate the objectives prepared by another group, using the criteria set out on the exercise sheet.

Plenary session

Have each group present their analysis diagram and the objectives they have developed. Immediately following each presentation, ask the group that analysed the objectives during the exercise to comment. Then open up the discussion to the rest of the class. (Allow 15 minutes per topic.)

Each group should also present the title of its research project.

It is important that each group receive clear feedback on the quality of the objectives they have developed, as well as practical suggestions for improvement. When providing feedback, ask yourself:

  1. Do the objectives cover all parts of the analysis diagram, in a logical order?
  2. Do the objectives really measure what the group wants them to?
  3. If the objectives were met, would the study provide the results needed to solve the problem posed in the statement of the problem?
  4. Are the objectives feasible? If too ambitious, could the scope of the study be reduced?
  5. Is the title specific enough and does it cover the objectives?
Adjustments

Facilitators in past courses have found it useful to provide a second group work session in which participants can finalise their objectives, analysis diagram and title of the research project, after they have received feedback during the plenary session.

Designing and Conducting Health Systems Research Projects
Part I: Proposal Development and Fieldwork

Module 7
INTRODUCTION TO
HEALTH SYSTEMS RESEARCH METHODOLOGY

./img/designandco_105_la_0.jpg

NB: Development of a research process is a cyclical process. The double-headed arrows indicate that the process is never linear.

Module 7: INTRODUCTION TO HEALTH SYSTEMS RESEARCH METHODOLOGY

OBJECTIVES

At the end of the session you should be able to:

  1. Identify the pertinent questions to consider when developing the methodology for your research proposal.
  2. Describe the components that should be dealt with in the methodology section of your research proposal.

In the previous modules, you:

Now you must decide exactly how you are going to achieve your stated objectives: i.e., what new data you need in order to shed light on the problem you have selected and how you are going to collect and process this data. The questions in the flow chart on the next page cover the major issues that must be examined as you develop your research design. These issues will be dealt with in Modules 8, 9, 1 0, 1 1, 1 2, 1 3 and 1 4.

./img/designandco_107_la_0.jpg

Note:The steps are interrelated. The process is usually cyclical in nature. After completing a step, you should review previous steps to ensure consistency in your proposal.

Trainer’s Notes

Module 7: INTRODUCTION TO HEALTH SYSTEMS RESEARCH METHODOLOGY

Timing and teaching methods

15 min. Introduction and discussion
15 min. TOTAL TIME

Guidelines for trainers

This page intentionally left blank.

Designing and Conducting Health Systems Research Projects
Part I: Proposal Development and Fieldwork

Module 8
VARIABLES

./img/designandco_111_la_0.jpg

NB: Development of a research process is a cyclical process. The double-headed arrows indicate that the process is never linear.

Module 8: VARIABLES

OBJECTIVES

At the end of the session you should be able to:

  1. Define what variables are and describe why their selection is important in research.
  2. State the difference between numerical and categorical variables and define the types of scales of measurement.
  3. Discuss the difference between dependent and independent variables and how they are used in research designs.
  4. Identify the variables that will be measured in the research project you are designing and develop operational definitions with indicators for those variables that cannot be measured directly.
  5. List the variables that you hope to identify and describe during your planned study but that cannot be measured at this time (qualitative data).
  1. Introduction
  2. Formulating variables
  3. Identifying indicators in qualitative studies
  4. Causes and associations; confounders

I. INTRODUCTION

In Module 4 we analysed the problem we wanted to investigate. The problem itself and all the factors that might influence it were presented in a diagram, which then served as the basis for the formulation of research objectives. Now we have come to a stage where we must ask ourselves the question:

‘What information are we going to collect in our study to meet our objectives?’

To find these associations between problems and contributing factors, it is essential that we carefully define the problem itself, as well as each of the factors identified when analysing the problem in Module 4. We do this by formulating variables.

II. FORMULATING VARIABLES

What is a variable?

A VARIABLE is a characteristic of a person, object or phenomenon which can take on different values. These may be in the form of numbers (e.g., age) or non-numerical characteristics (e.g., sex).

A simple example of a variable in the form of numbers is ‘a person’s age’. The variable ‘age’ can take on different values since a person can be 20 years old, 35 years old and so on. Other examples of variables are:

Because the values of all these variables are expressed in numbers, we call them NUMERICAL VARIABLES.

Some variables may also be expressed in categories. For example, the variable sex has two districts categories, groups, male and female. Other examples are:

Table 8.1: Examples of categorical variables

./img/designandco_114_la_0.jpg

Since these variables are expressed in categories, we call them CATEGORICAL VARIABLES.

Further breakdown of numerical and categorical variables (optional)

Numerical variables can either be continuous or discrete.

  1. Continuous. With this type of data, one can develop more and more accurate measurements depending on the instrument used, e.g.:
  2. Discrete. These are variables in which numbers can only have full values, e.g.:

Categorical variables, on the other hand, can either be ordinal or nominal.

  1. Ordinal variables. These are grouped variables that are ordered or ranked in increasing or decreasing order:

    For example:


    High income (above $300 per month);
    Middle income ($100-$300 per month); and
    Low income (less than $100 per month).
    Other examples are:
     
    Disability: no disability, partial disability, serious or total disability
    Seriousness of a disease: severe, moderate, mild
    Agreement with a statement: fully agree, partially agree, fully disagree
    Fear of leprosy: will not share food with a patient; will not enter the house of a patient; will not allow patient to live in the community.

    Note:

    Fear of leprosy is an attitude, and attitudes are often scaled (you make them into ordinal variables).

    It is obvious that the definition of what we would call high (income) or far (distance) will vary from country to country and from region to region. If a researcher has little idea about the distribution of a certain variable in a population (for example, if you don’t know whether 30%, 50%, or 95% are below the poverty line of $100 per month), it is advisable to categorise numerical data only after the pre-test, or even after data collection (see Module 13).

  2. Nominal variables. The groups in these variables do not have an order or ranking in them.

    For example:

     
    Sex:male, female
    Main food crops:maize, millet, rice, etc.
    Religion:Christian, Moslem, Hindu, Buddhism, etc.

For examples of scales of measurement, see Annex 8.1. We will come back to these distinctions in Module 22, as continuous, discrete, ordinal and nominal data require different statistical tests.

EXERCISE 1:

Look at your problem analysis diagram and give examples of numerical (continuous and discrete) and categorical (ordinal and nominal) variables.

Factors rephrased as variables

When looking at your problem analysis diagram you will notice that most of what we called ‘factors’ are in fact variables which have negative values. We phrased the contributing factors negatively on purpose (e.g., lack of knowledge) as it is much easier to visualise these factors in the negative. However, in reality not everyone with good knowledge of TB treatment is a regular attender and not everyone with poor knowledge absconds from treatment. As we conduct our study we will try to determine to what extent these contributing factors play a role. Therefore we have to formulate them in a neutral way, so that they can take on positive as well as negative values. The table below presents examples of negatively phrased ‘factors’ and how they can be rephrased as neutral ‘variables’.

Table 8.2: Factors rephrased as variables

./img/designandco_116_la_0.jpg

Operationalising variables by choosing appropriate indicators

Note that the different values of many of the variables presented up to now can easily be determined. However, for some variables it is sometimes not possible to find meaningful categories unless the variables are made operational with one or more precise INDICATORS. Operationalising variables means that you make them ‘measurable’:

For example:

Note:

When defining variables on the basis of the problem analysis diagram, it is important to realise which variables are measurable as such and which ones need indicators. Once appropriate indicators have been identified we know exactly what information we are looking for. This makes the collection of data as well as the analysis more focused and efficient.

Defining variables and indicators of variables

To ensure that everyone (the researcher, the data collectors, and eventually, the reader of the research report) understands exactly what has been measured and to ensure that there will be consistency in the measurement, it is necessary to clearly define the variables (and indicators of variables). For example, to define the indicator ‘waiting time’ it is necessary to decide what will be considered the starting point of the ‘waiting period’ e.g., is it when the patient enters the front door, or when he has been registered and obtained his card?

Annex 8.2 gives examples of common variables with different possible choices for indicators.

III. IDENTIFYING INDICATORS IN QUALITATIVE STUDIES

Certain variables cannot be defined with indicators before the study, because the information to do this is lacking. The purpose of the study may be to find this information.

For example, policy makers in Nepal would like to eliminate leprosy. They have noticed that fewer women report for leprosy treatment than men and would like to know whether stigma keeps women from reporting for treatment and/or whether the services have to be more sensitive to the needs of women for privacy at diagnosis.

We define stigma as an undesirable differentness that disqualifies a person from full social acceptance (Goffman: 1963). However, we cannot fill in more precisely in what way men and women are discriminated against, as that has still to be studied. Some indicators for stigma could be the divorce rate of male and female patients, or the degree of isolation of the patient by the healthy spouse or by the community, but how the severity of this isolation should be measured is still unknown. Possibilities includd, for example, whether patients and spouses still share a house, share food, share one bed? Do community members still accept leprosy patients as village leaders, do they welcome patients to attend village meetings, and, if so, do they still drink beer or eat together, and do they ask patients to bring their own cups?

Note: that in many qualitative studies the researcher is not primarily interested in measuring variables, but rather in identifying variables or clusters of variables that help explain a problem or reasons for success. In that case, the researcher will often try to find indicators that make the variables measurable.

One could state that in exploratory, qualitative studies we study themes, such as stigma, to understand better how patients suffer from stigma and how they cope with it. We also discover contributing factors to stigma: in some societies women are more vulnerable to stigma than men; adolescents are more vulnerable than adults who have settled economically and socially; patients with deformities are always more vulnerable to stigma than those without visible signs.

By better understanding the problem of stigma we can now give an operational definition of the strength of stigma on a scale. This enables us to measure through a quantitative study the degree of stigma male and female patients suffer from, and the most important contributing factors to stigma. (See Figure 8.1)

Figure 8.1: Relationship between qualitative and quantitative studies in understanding and measuring problems

./img/designandco_118_la_0.jpg

IV. CAUSES AND ASSOCIATIONS; CONFOUNDING

Dependent and independent variables

Because in health systems research you often look for causal explanations, it is important to make a distinction between dependent and independent variables.

The variable that is used to describe or measure the problem under study is called the DEPENDENT variable.

The variables that are used to describe or measure the factors that are assumed to cause or at least to influence the problem are called the INDEPENDENT variables.

For example, in a study of the relationship between smoking and lung cancer, ‘suffering from lung cancer’ (with the values yes, no) would be the dependent variable and ‘smoking’ (varying from not smoking to smoking more than three packets a day) the independent variable.

Whether a variable is dependent or independent is determined by the statement of the problem and the objectives of the study. It is therefore important when designing an analytical study to clearly state which variable is the dependent and which are the independent ones. Note that if a researcher investigates why people smoke, ‘smoking’ is the dependent variable, and ‘pressure from peers to smoke’ could be an independent variable. In the lung cancer study ‘ smoking’ was the independent variable.

EXERCISE 2:

Look at your analysis diagram and see if you can give an example of a dependent variable and one or two independent variables in your own study.

Although in everyday language we may speak of possible CAUSES of problems, in scientific language we prefer to speak of ASSOCIATIONS between variables, unless a causal relationship can be proven. If we find an association between smoking and cancer, we can conclude that smoking causes cancer only if we can both demonstrate that the cancer was developed after the patient started smoking and that there are no other factors that could have caused both the cancer and the habit of smoking. Nervous people, for example, may both smoke more and suffer more from cancer than persons who are not nervous.

A variable that is associated with the problem and with a possible cause of the problem is a potential CONFOUNDING VARIABLE.

A confounding variable may either strengthen or weaken the apparent relationship between the problem and a possible cause.

./img/designandco_119_la_0.jpg

Therefore, in order to give a true picture of cause and effect, possible confounding variables must be considered, either at planning stage or while doing data analysis.

For example:

A relationship is shown between bottle-feeding and diarrhea in under-twos. However, mother’s education may be related to bottle-feeding as well as to diarrhea.

./img/designandco_120_la_0.jpg

Mother’s education is therefore a potential confounding variable. In order to give a true picture of the relationship between bottle-feeding and diarrhea of under-twos, the influence of mother’s education should be controlled. This could either be addressed in the research design, e.g., by selecting only mothers with a specific level of education, or it could be taken into account during the analysis of the findings by analysing the relation between bottle-feeding and diarrhea separately for mothers with different levels of education.

Background variables

In almost every study, BACKGROUND VARIABLES, such as age, sex, educational level, socio-economic status, marital status and religion, should be considered. These background variables are often related to a number of independent variables, so that they influence the problem indirectly (hence they are called background variables). Only background variables important to

the study should be measured. Background variables are notorious ‘confounders’.

Note 1:

If you do a purely descriptive study, for example an inventory of knowledge, attitudes and practices related to bilharzia (schistosomiasis) or AIDS, you do not need to differentiate between dependent and independent variables, as there are no causal relationships between variables. In this type of study you may simply concentrate on variables and give operational definitions, with indicators if needed, to measure knowledge, attitudes and practices (see Module 4 figure 4.5).

Note 2:

In evaluation studies, however, it is particularly important that we prepare good operational definitions, because here we want to compare and measure results at the beginning of the project phase and in the middle or at the end.

According to the WHO definition of health as an outcome of the health system (see Module 2, Figure 2.4) we can, for example, measure the improvement in the health of a population by comparing the estimated life expectance at birth and time lived with a disability over the past ten years (provided the epidemiological and other environmental factors did not change). Increased fairness of the health system could be measured by the percentage out-of-pocket spending on health by the poor (living on 1 US$ or less a day) of the total health expenditure, comparing, say, the past ten years.

Responsiveness to patients’ need for human treatment is more difficult to measure, but a number of indicators could be developed, using the concepts: respect for patients (not humiliating or demeaning them); confidentiality with regard to a patient’s diagnosis and treatment, providing patients with essential information, so that they can participate in choices about their own health and treatment, and client-orientedness in the services offered (prompt attention, clean premises) (WHO 2000: 32). It is interesting that one can not only make comparisons within one country over time, but also between countries.

Note 3:

When you select the variables for your study, it is important to review your objectives, as well as your problem analysis diagram. When you review your objectives you may find that you need to consider some new factors not originally included in your problem analysis diagram. On the other hand, you may discover that your objectives are too vague and can be revised and clarified, now that you have identified your variables

You should continue to adjust your problem analysis diagram, variables and objectives until they are all in line with each other.

REFERENCES

Abramson JH (1990, 4th ed.) Survey Methods in Community Medicine. London: Churchill-Livingstone. (In particular Chapters 9 and 10)

Moser CA, Kalton G (1979) Survey Methods in Social Investigation. Hants, UK: Gower Publishing Company: 220-224.

World Health Organization (2000) The WORLD HEALTH REPORT 2000. Health Systems: Improving Performance. Geneva: WHO.

EXERCISE 3: Identification of variables in research (to be carried out in plenary, ½ hour)

Look at the following descriptions of research problems and then answer the questions that follow.

Problem 1

A health researcher believes that in a certain region anaemia, malaria and malnutrition are serious problems among adult males and, in particular, among farmers. He therefore wishes to study the prevalence of these diseases among adult males of various ages, family size, occupations and educational backgrounds in order to determine how serious a problem these diseases are for this population.

Questions:

Problem 2

A district medical officer (DMO) receives a complaint from the community that village health workers (VHWs) often run out of chloroquine. In preliminary investigations this shortage of chloroquine is confirmed. VHWs get their drugs at monthly meetings at the health centre. The DMO decides to investigate why the supply of drugs to VHWs is unsatisfactory.

Questions:

Problem 3

Occasionally, a research project is carried out without considering some of the important variables. This may result in deceptive findings or an unclear relationship between independent and dependent variables.

In a study concerning prevalence of bilharzia (schistosomiasis) in the adult population of a village community, a researcher found that being a farmer was a risk factor for developing bilharzia. He was however not convinced that it was being a farmer that made these people more likely to develop bilharzia.

Question:

Are there any variables whose inclusion in the study might ensure that the researcher could show how much being a farmer actually contributed to a person developing schistosomiasis? Are there farmers who did not get bilharzia? Which variables might help explain why some farmers got bilharzia and others did not?

GROUP WORK (2½ hours)

  1. Using the diagram of factors that possibly influence the problem you are studying (the diagram that you prepared for the statement of the problem), identify for each of these factors the variables that will be included in your study:
    • What is/are your dependent variable(s)? (List them.)
    • What are your independent variables? (List them.)
    • Which of the variables can be ‘measured’ as they are?
    ./img/designandco_123_la_0.jpgNB: This distinction cannot be made in a purely descriptive study.
    • Choose appropriate indicators for the variables that are not measurable as they are and/or formulate appropriate definitions for these variables/indicators.
    • State whether you have identified themes that need further exploration during your study before you can define the concepts adequately.

    Use the table below for your work.

    ./img/designandco_123_la_1.jpg

  2. In the table we have included a column to state which objective is covered by your variables. You may discover that some objectives are not well covered by your variables (probably because your analysis diagram and objectives are not yet completely in line with each other). In that case, you need to rethink whether the objectives are indeed important for your study, and, if so, develop variables to measure them. You may discover that your objectives are too vague when compared to the type of data (or to the variables) you would like to collect. If so, you should make your objectives more specific.

    Before you finish you should review, as a group, your problem analysis diagram, objectives and variables, and make any adjustments needed so they are all in line with each other.

./img/designandco_124_la_0.jpg

./img/designandco_125_la_0.jpg


* Adapted from Abramson (1990)

Trainer’s Notes

Module 8: VARIABLES

Timing and teaching methods

1 hourIntroduction and discussion (including first exercise)
½ hourExercise: Identification of variables in research (and discussion of answers)
2 hoursGroup work
1 hour Plenary
4½ hours TOTAL TIME
Introduction and discussion
Exercise: Examples of dependent and independent variables
Exercise: Identification of variables in research
Group work

(The following answers are by no means exhaustive)

Problem 1:

./img/designandco_128_la_0.jpg

Problem 2:

Dependent variable:

Indicator for availability of chloroquine:

‘Short of chloroquine’ should be defined in relation to the time since the date of the last drug supply and ideally also in relation to the size of the population.

For example:

If the number of tablets in stock is measured for all VHWs two weeks after the date of the last meeting at the health centre where drugs were supplied, one could say that any VHW who does not have enough drugs to treat 1% of the population for malaria is short of drugs. Since an adult needs 10 tablets for a full course, this would mean that a VHW should have at least 50 tablets available, if the village has a population of 500.

An alternative definition could be having no tablets in stock two weeks after the last date of supply.

Independent variables:

Problem 3:

Important independent variables that could be taken into account include:

Closer study revealed that schistosomiasis was present among 70% of the young farmers between 20-25 years of age, while it was almost entirely absent in farmers older than 50 years of age. It turned out that younger farmers tended to have farms much further away from the village where the land was more fertile, and they had to cross a river where they bathed on their way home in the evening. The older farmers, on the other hand, had always had their farms close to the village and obtained water from wells.

Designing and Conducting Health Systems Research Projects
Part I: Proposal Development and Fieldwork

Module 9
STUDY TYPES

./img/designandco_131_la_0.jpg

NB: Development of a research process is a cyclical process. The double-headed arrows indicate that the process is never linear.

Module 9: STUDY TYPES

OBJECTIVES

At the end of the session, you should be able to:

  1. Describe the study types most used in HSR.
  2. Define the uses and limitations of each study type.
  3. Describe how the study design can influence the validity and reliability of the study results.
  4. Identify the most appropriate study design for the research proposal you are developing.
  1. Introduction
  2. Overview of study types
  3. Deriving valid and reliable conclusions

I. INTRODUCTION

Depending on the existing state of knowledge about a problem that is being studied, different types of questions may be asked which require different study designs. Some examples are given in the following table:

Table 9.1: Research questions and study types

./img/designandco_133_la_0.jpg

The type of study chosen depends on:

When investigating health management problems, such as overcrowding in a hospital out-patient department or shortage of drugs at PHC level, a good description of the problem and identification of major contributing factors often provides enough information to take action.

When exploring more complicated management or health problems, we usually want to go further and determine the extent to which one or several independent variables contribute to the problem (for example, the contribution of low-fibre diet to cancer of the large intestine). For these types of problems more rigorous analytical or experimental studies will have to be conducted before we decide on appropriate interventions.

II. OVERVIEW OF STUDY TYPES

Several classifications of study types are possible, depending on what research strategies are used. The table below categorises studies, based on the combination of research strategies they use, including:

  1. Non-intervention studies in which the researcher just observes and analyses researchable objects or situations but does not intervene; and
  2. Intervention studies in which the researcher manipulates objects or situations and measures the outcome of his manipulations (e.g., by implementing intensive health education and measuring the improvement in immunisation rates.)
NON-INTERVENTION STUDIES

We will first concentrate on non-intervention studies and their use in health systems research. We will discuss:

1. Exploratory studies

An EXPLORATORY STUDY is a small-scale study of relatively short duration, which is carried out when little is known about a situation or a problem. It may include description as well as comparison.

For example:

A national Acquired Immunodeficiency Syndrome (AIDS) Control Programme wishes to establish counselling services for Human Immunodeficiency Virus (HIV) positive and AIDS patients, but lacks information on specific needs patients have for support. To explore these needs, a number of in-depth interviews are held with various categories of patients (males, females, married, single) and with some counsellors working on a programme that is already under way.

When doing exploratory studies we describe the needs of various categories of patients and the possibilities for action. We may want to go further and try to explain the differences we observe (e.g., in the needs of male and female AIDS patients) or to identify causes of problems. Then we will need to compare groups.

Note:

Comparison is a fundamental research strategy to identify variables that help explain why one group of persons or objects differs from another.

In HSR, small-scale studies that compare extreme groups are very useful for detecting management problems. We could, for example, compare:

Exploratory studies gain in explanatory value if we approach the problem from different angles at the same time. This is called triangulation. In a study that is looking for causes of the low percentage of supervised deliveries, it may be very useful to include observations and interviews with health staff in the maternity centres that should serve the mothers in question and interviews with their supervisors, as well as with the mothers themselves. In this manner, information from different independent sources can be cross-checked.

For some management problems such a ‘rapid appraisal’ may provide sufficient information to take action. Otherwise, a larger, more rigorous comparative study will have to be developed to test differences between groups with respect to various independent variables.

Note:

If the problem and its contributing factors are not well defined (see Module 8 group work) it is always advisable to do an exploratory study before embarking on a large-scale descriptive or comparative study.


* Such small-scale studies may be called exploratory case studies if they lead to plausible assumptions about the causes of the problem and explanatory case studies if they provide sufficient explanations to take action (Yin, 1984).

2. Descriptive studies

A DESCRIPTIVE STUDY involves describing the characteristics of a particular situation, event or case.

Descriptive studies can be carried out on a small or larger scale.

(1) Small scale, descriptive case studies

Descriptive case studies describe in-depth the characteristics of one or a limited number of ‘cases’. A case may be, for example, a patient, a health centre, or a village. Such a study can provide quite useful insight into a problem. Case studies are common in social sciences, management sciences, and clinical medicine. For example, in clinical medicine the characteristics of a hitherto unrecognised illness may be documented as a case study. This is often the first step toward building up a clinical picture of that illness.

However, if one wishes to test whether the findings pertain to a larger population, a more extensive, cross-sectional survey has to be designed.

(2) Large scale, cross-sectional surveys

Cross-sectional surveys aim at describing and quantifying the distribution of certain variables in a study population at one point of time. They may cover, for example:

Cross-sectional surveys cover a selected sample of the population. If a cross-sectional study covers the total population it is called a census.

A cross-sectional survey may be repeated in order to measure changes over time in the characteristics that were studied. The surveys may be very large, with hundreds or even thousands of study units. In these cases only a limited number of variables will usually be included, in order to avoid problems with analysis and report writing. If cross-sectional surveys are smaller they can be more complex. They may include all the elements just mentioned. Small surveys can reveal interesting associations between certain variables, such as between having tuberculosis and socio-economic status, sex, and ways of coping.

Researchers often go further and will combine a description of the study population with a comparison of a number of groups within that population (see below). Such combinations are very common, and thus the distinctions between descriptive and comparative studies are sometimes quite fuzzy.

3. Comparative or analytical studies

An ANALYTICAL STUDY attempts to establish causes or risk factors for certain problems. This is done by comparing two or more groups, some of which have or develop the problem and some of which have not.

Three commonly used types of analytical studies will be discussed here:

Figure 9.1: Types of analytical studies

./img/designandco_137_la_0.jpg

(1) Cross-sectional comparative studies

Many cross-sectional surveys focus on describing as well as comparing groups.

For example, a survey on malnutrition may wish to establish:

The researcher will not only describe these variables but, by comparing malnourished and well-nourished children, he will try to determine which socio-economic, behavioural and other independent variables may have contributed to malnutrition.

In any comparative study, one has to watch out for CONFOUNDING or INTERVENING variables. (Please look at Module 8 for examples and discussion, as well as to the next page, and Module 26).

(2) Case-control studies

In a CASE-CONTROL STUDY the investigator compares one group among whom the problem that he wishes to investigate is present (e.g. malnutrition) and another group called a control or comparison group, where the problem is absent, in order to find out what factors have contributed to the problem.

Figure 9.2: Diagram of a case-control study*

./img/designandco_138_la_0.jpg

For example, in a study of the causes of neonatal death, the investigator will first select the ‘cases’ (children who died within the first month of life) and ‘controls’ (children who survived their first month of life). (S)he then interviews their mothers to compare the history of these two groups of children, to determine whether certain risk factors are more prevalent among the children who died than among those who survived.

Note:

Controls should come from the same ‘source’ population. For example, in a hospital case-control study where cases are being sought in the hospital, cases should normally be selected from patients attending at the same hospital. If controls are selected from another hospital, they might not be from the same source population because the referral pathways may be different, and therefore they would not really be comparable to the cases.

As with a cross-sectional comparative study, the researcher has to control for CONFOUNDING VARIABLES. In case-control studies, this may be done to some extent beforehand, by MATCHING the groups for expected confounding variables. Matching means taking care that the cases and controls are similar with respect to the distribution of one or more potentially confounding variables. However, we cannot then look at the effect of the matched variable as a risk factor because we have made the cases and controls exactly the same with respect to that variable.

For example, in a study on causes of malnutrition in children-3 years you may match the welland the malnourished on age, because this factor may influence many other variables influencing your problem (e.g. time of weaning, time of teething, which are both related to diarrhoea and consequently to malnutrition). But you will not match them on economic status of parents, as you do want to know whether poverty influences malnutrition.

In general you will only match for strong confounders (such as age) that you cannot properly control by stratification during data analysis, unless you double or treble the number of informants. You also match for potentially confounding variables such as location/source of origin, as these can influence many other potential confounders (e.g. ethnic group, religion, economic status), some of which you may not even expect beforehand.


* Adapted from WW Holland et al., eds. (1985) Oxford Textbook of Public Health, Volume 3: Investigative Methods in Public Health. Oxford: Oxford University Press.

(3) Cohort studies

In a COHORT STUDY, a group of individuals that is exposed to a risk factor (study group) is compared to a group of individuals not exposed to the risk factor (control group). The researcher follows both groups over time and compares the occurrence of the problem that he expects to be related to the risk factor in the two groups to determine whether a greater proportion of those with the risk factor are indeed affected.

A well-known example of a cohort study is the study by Doll and Hill (1950) of smokers and non-smokers that was conducted among doctors to determine the importance of smoking as a risk factor for developing lung-cancer.

A study may start with one large cohort. After the study starts, the researchers determine who is exposed to the risk factor (e.g., smoking) and who is not, and follow the two groups over time to determine whether the study group (of smokers) develops a higher prevalence of lung cancer than the control group. If it is not advisable to select one cohort (for example, because only few people are affected by the risk factor, which necessitates a very large sample), two cohorts may be chosen, one in which the risk factor is present (study group) and one in which it is absent (control group). In all other respects the two groups should be as alike as possible.

The control group should be selected at the same time as the study group, and both should be followed with the same intensity.

Figure 9.3: Diagram of a cohort study

./img/designandco_139_la_0.jpg

Uses and limitations of different types of analytical studies

You may use any of the three types of analytical studies (cross-sectional comparison, case-control or cohort) to investigate possible causes of a problem.

For example, if you assume there is a causal relationship between the use of a certain water source and the incidence of diarrhea among children under five in a village with different water sources:

Cross-sectional comparative studies and case-control studies are usually preferred to cohort studies for financial and practical reasons. However, cohort studies are stronger in establishing causal relationships because confounding variables are to a large extent eliminated. If the study is well designed, the ‘confounders’ are equally distributed among the cases and controls. Experimental studies have the same advantage as cohort studies.

Cross-sectional comparative studies and case-control studies are relatively quick and inexpensive to undertake. With cross-sectional comparative studies, however, the number of stratifications one can make is limited by the size of the study. The problem with case-control studies is sometimes the difficulty of making a precise selection of a control group which is comparable to the study group on one or two specific variables (e.g., well- and malnourished children of the same sex and age, in months).

Cohort studies are a relatively sure way to establish causal relationships. However, they take longer than case-control studies and are labour intensive, and therefore expensive. The major problems are usually related to the identification of all cases in a study population, especially if the problem has a low incidence. Further, the following up all persons included in the study over a number of years may be impossible because of population movement.

INTERVENTION STUDIES

In intervention studies, the researcher manipulates a situation and measures the effects of this manipulation. Usually (but not always) two groups are compared, one group in which the intervention takes place (e.g. treatment with a certain drug) and another group that remains ‘untouched’ (e.g. treatment with a placebo).

The two categories of intervention studies are:

1. Experimental studies

An experimental design is a study design that gives the most reliable proof for causation.

In an EXPERIMENTAL STUDY, individuals are randomly allocated to at least two groups. One group is subject to an intervention, or experiment, while the other group(s) is not. The outcome of the intervention (effect of the intervention on the dependent variable/problem) is obtained by comparing the two groups.

Figure 9.4: Diagram of an experimental study

./img/designandco_141_la_0.jpg

Note:

The strength of experimental studies is that by randomisation the researcher eliminates the effect of confounding variables through the equal distribution of confounders (both known and unknown) in the experimental and control groups.

A number of experimental study designs have been developed. These are widely used in laboratory settings and in clinical settings. For ethical reasons, the opportunities for experiments involving human subjects are restricted. However, randomised control trials of new drugs are common.

For example, a researcher plans to study the effect of a new drug. (The drug has already been tested extensively on animals and has been approved for trial use.) He plans to include 300 patients in the study who are currently receiving the standard treatment for the same condition for which the new drug has been designed. He explains the study to the patients asking their consent to be divided into two groups on a random basis. One group will receive the experimental drug while the other group will continue to receive the standard treatment. He makes sure that the medications are disguised and labelled in such a manner that neither the research assistant administering them nor the patient know which drug is used. (This is called a ‘double blind’ experiment.)

At community level, where HSR is frequently undertaken, we experience not only ethical but also practical problems in carrying out experimental studies. In real life settings, it is often impossible to assign persons at random to two groups, or to maintain a control group. Therefore, experimental research designs may have to be replaced by quasi-experimental designs.

2. Quasi-experimental studies*

In a QUASI-EXPERIMENTAL STUDY, one characteristic of a true experiment is missing, either randomisation or the use of a separate control group. A quasi-experimental study, however, always includes the manipulation of an independent variable which is the intervention.

One of the most common quasi-experimental designs uses two (or more) groups, one of which serves as a control group in which no intervention takes place. Both groups are observed before as well as after the intervention, to test if the intervention has made any difference. (This quasi-experimental design is called the ‘non-equivalent control group design’ because the subjects in the two groups (study and control groups) have not been randomly assigned.)

Figure 9.5: Diagram of a quasi-experimental design with two groups

./img/designandco_142_la_0.jpg


* For a more detailed explanation of experimental and quasi-experimental designs and their advantages and disadvantages, one excellent reference is Campbell and Stanley’s Experimental and Quasi-Experimental Designs for Research (1963).

Example of a quasi-experimental study:

A researcher plans to study the effects of health education on the level of participation of a village population in an immunisation campaign. She decides to select one village in which health education sessions on immunisation will be given and another village which will not receive health education and serves as a control. The immunisation campaign will be carried out in the same manner in both villages. A survey will then be undertaken to determine if the immunisation coverage in the village where health education was introduced before the campaign is significantly different from the coverage in the ‘control’ village which did not receive health education. (Note: The study is quasi-experimental because the subjects were not assigned to the control or experimental groups on a random basis).

Another type of design that is often chosen because it is quite easy to set up uses only one group in which an intervention is carried out. The situation is analysed before and after the intervention to test if there is any difference in the observed problem. This is called a ‘BEFORE-AFTER’ study. This design is considered a ‘pre-experimental’ design rather than a ‘quasi-experimental’ design because it involves neither randomisation nor the use of a control group.

Figure 9.6: Diagram of a before-after study

./img/designandco_143_la_0.jpg

Example of a ‘before-after’, pre-experimental study:

The out-patient clinic of hospital X is extremely crowded. Waiting times of over 5 hours for patients before they are attended to are not uncommon. The hospital management has a study carried out to analyse the bottlenecks and implements most of the recommendations made. Three months later, another study is done to check to what extent the bottlenecks have been solved and where further action is necessary.

This design is often used for management problems that pertain to one single unit (hospital, school, village). However, if the problems occur at a larger scale or if they might be influenced by other factors apart from the intervention during the trial, it is highly recommended that the design include both a study and a control group.

In the trial with health education on immunisation, for example, it would have been quite risky to work without a control group. Outside events (such as a health education campaign on immunisation by radio or other mass media) might have led to improved knowledge on immunisation in both the study group and the control group. (NB: The immunisation campaign by radio provides a so-called ‘rival explanation’ for your results.) If you had had just a study group and no control you might have concluded erroneously that all of the increase was due to your own intervention.

III. DERIVING VALID AND RELIABLE CONCLUSIONS

Whatever research design is selected, a primary concern is that the conclusions of the study be VALID and RELIABLE.

What are validity and reliability in research findings?

Validity means that your scientific observations actually measure what they intend to measure (your conclusions are true).

Reliability means that someone else using the same method in the same circumstances should be able to obtain the same findings (your findings are repeatable).

Reliability (repeatability) refers to the possibility to replicate (repeat) the observations and is related to the precision of the instrument used for scientific observations. Validity refers to the soundness of the observations and to the accurateness of the data collected by the research method/instrument.

Figure 9.7: Validity and reliability; graphic presentation of possible combinations

./img/designandco_144_la_0.jpg

For example:

Four different teams of researchers set out to determine the body weights of three children whose true body weights were 10 kg, 15 kg and 20 kg respectively and obtained following four sets of results.

Team 1

./img/designandco_145_la_0.jpg

The first set of results is not valid because the results do not represent the true body weights. They are not reliable because they are sometimes too high and sometimes too low, and the relative difference from the true body weight varies from child to child.

Team 2

./img/designandco_145_la_1.jpg

The second set of results is not valid because the results again do not represent the true body weights. However, they are reliable because the results are too high by the same proportion (10%) for every child.

Team 3

./img/designandco_145_la_2.jpg

The third set of results is fairly valid because the results are almost representing the true body weight. They are not reliable because two weights are too high and one is too low and the proportion by which they differ from the true body weight is different for each child.

Team 4

./img/designandco_145_la_3.jpg

The fourth set of results is both valid and reliable because the results are the same as the true body weights, and these results have been obtained for every child.

Note:

that it is possible to implement a research instrument with precision and yet obtain invalid responses! For example, a door-to-door survey on sexual behaviour of informants may give the same type of answers throughout and therefore seem reliable. But the chance that people are concealing their true sexual behaviour is high, so that the validity may be low.

How to deal with threats to validity and reliability

At various stages of the research validity and reliability could be threatened:

Examples of threats to validity:

1. Confounding factors

Example:

You might find that children who have had pre-school education subsequently perform better in primary school. Can you conclude that pre-school education leads to better school performance?

Rival or alternate explanations include:

Education and income are therefore confounding factors.

2. History

Unexpected factors beyond your control might have produced the same effect as the intervention you were studying, thereby making it impossible for you to know whether it was your intervention that produced the impact.

Example:

A well-known example is when a certain agency that had designed a health education program for early detection of breast cancer designed a study to test the effectiveness of the program by studying the increase in the proportion of women who reported doing self-examination of breasts. However, while the study was in progress the President’s wife developed breast cancer and she appeared widely on mass media to advise women on early detection of breast cancer.

3. Differential subject loss in various groups

The type of subjects who drop out of your study or control groups may be related to some of the characteristics you are studying.

Example:

You are studying the effectiveness of a ‘weight watchers’ program by comparing the average weight loss in the ‘weight watchers’ group with that of a control group. However, a number of women in the ‘weight watchers’ group found the program too demanding and have dropped out.

4. Selectivity (or bias) in assigning subjects to various groups

Example:

You intend to study whether a programme on ‘how to stop smoking’ will be effective in helping the smokers in your hypertension clinic. Therefore you invite those who would like to attend to register themselves. You plan to compare the percentage who stop smoking among those who attend the programme with those who do not. However, it is likely that those who register themselves are those who are strongly motivated to stop smoking while those who are not motivated do not join the program (Also see Module 11).

Strategies to deal with threats to validity
  1. Triangulation. Approaching a research problem from different angles (e.g., by selecting complementary study populations or using different research techniques at the same time) (see Modules 9 and 1 0).
  2. Control group. Observing a control group who is not exposed to the risk factor or intervention reduces threats due to unexpected and confounding factors.
  3. Appropriate sampling procedures and assignment of subjects to research groups. This reduces threats due to selectivity (see Module 11).
  4. Before and after measurements. This allows us to assess whether there has been selectivity as well as differential loss of subjects. If there has been an inevitable loss of subjects, it may enable assessment of the dropouts to determine whether they had peculiar characteristics that distinguished them from those who did not drop out (see this module).
  5. Unobtrusive methods of data collection and allowing adaptation time for subjects to get used to being observed or interviewed (see Module 10B).
  6. Careful design and pre-testing of instruments, stressing the participation of health managers, staff and community members, reduce bias due to instrumentation (see Modules 10 and 14). Training of interviewers and standardisation of interview techniques and tools such as questionnaires are also important in reducing this bias.
  7. Knowledge of the environment events enables the researcher to be sensitive to external events that could affect validity (i.e., history). In case of an expatriate researcher, local key informants can contribute a lot to the validity of the study.
  8. Stratification and matching for confounding variables during the analysis of the results (see Module 26 on confounding).
Selection of study design

In selecting the design of the study, you have to consider the type of information you want to obtain and devise strategies to enable you to obtain that information.

The selection of an appropriate research design depends on:

REFERENCES

Abramson JH (1990, 4th ed.) Survey Methods in Community Medicine. London: Churchill-Livingstone.

Beaglehole R, Bonita R, Kjellström T (1993) Basic Epidemiology. Geneva: World Health Organization.

Campbell DT, Stanley JC (1963) Experimental and Quasi-experimental Designs for Research. Chicago: Rand McNally.

Doll R and Hill AB (1950) Smoking and carcinoma of the lung: Preliminary report. Br. Med. J. 2:739-748.

Gordis L (1996) Epidemiology. Philadelphia USA: W.B. Saunders Company.

Holland W, Detels R, and Knox G (ed.) (1984-85) Oxford Textbook of Public Health, Volumes 1-4. Oxford: Oxford University Press.

Katzenellenbogen JM, Joubert G, Abdool Karim SS (1997) Epidemiology; A manual for South Africa. Capetown: Oxford University Press.

Kidder LH, Judd CM (1987) Research Methods in Social Relations. Hong Kong: CBS Publishing Japan Ltd.

Moser CA, Kalton G (1989, 2nd ed.) Survey Methods in Social Investigation. Hants, UK: Gower Publishing Company.

Patton MQ (1990, 2nd ed.) Qualitative Evaluation and Research Methods. Newbury Park, USA: Sage Publications.

Rose J, Barber DJP (1989) Epidemiologie for the uninitiated. Plymouth GB: Latimer Trend & Co/ British Medical Association.

Yin RK (1984) Case Study Research; Design and Methods. London, UK: Sage Publications.

Vaughan JP, Morrow RH (1989) Manual of Epidemiology for District Health Management. Geneva: World Health Organization.

Trainer’s Notes

Module 9: STUDY TYPE

Timing and teaching methods

1-1½ hour Introduction and discussion
1-1½ hours TOTAL TIME
Introduction and discussion

It is helpful if participants read this module the evening before the presentation, so they are more familiar with the topic.

The goal of the module is to give participants an understanding of the major issues involved in choosing different research strategies rather than having them learn the various possible study types by heart.

Present Table 9.1 at the beginning of the module to illustrate basic questions that lead to the choice of different study types without getting into the details concerning each type. Then proceed with the detailed discussion of each study type. Repeat the presentation of Table 9.1 at the end of the module, summarising the different study types that are possible.

It should be stressed that unless all variables to be investigated are clearly defined, small-scale studies are preferable to large-scale studies. A combination of study types (triangulation) can be considered if some variables still have to be explored (e.g., by open-ended questions) whereas other well-defined variables need to be measured on a larger scale (e.g., degree of utilisation of services).

Note:

Try to give examples of different study types in the fields in which the participants are interested. Shorten the presentation, especially of part III, if participants will not be engaged in analytical or quasi-experimental studies.

It is advisable to proceed with the presentation of Module 10A (Overview of data-collection techniques) before participants do their group work to choose the type(s) of study they will use for their research projects. After Module 10A, the participants can be asked to do an exercise which involves selecting a study type as well as data collection techniques for certain problems.

Note:

The group work session on selection of study types is combined with group work on selection of data collection techniques. It comes at the end of Module 10A.

Designing and Conducting Health Systems Research Projects
Part I: Proposal Development and Fieldwork

Module 10
DATA COLLECTION TECHNIQUES

A: OVERVIEW OF DATA COLLECTION TECHNIQUES

B: DESIGN OF RESEARCH INSTRUMENTS; INTERVIEW GUIDES AND INTERVIEW SKILLS

C: FOCUS GROUP DISCUSSIONS

./img/designandco_151_la_0.jpg

NB: Development of a research process is a cyclical process. The double-headed arrows indicate that the process is never linear.

Introduction

Data collection is a crucial stage in the planning and implementation of a study. If the data collection has been superficial, biased or incomplete, data analysis becomes difficult, and the research report will be of poor quality.

Therefore, we should concentrate all possible efforts on developing appropriate tools, and should test them several times.

Depending on the type of study, different data-collection techniques may be used. In HSR studies we usually combine a number of different techniques and look at problems from different perspectives (triangulation).

To give full weight to data collection we have split the module into three parts:

— An overview of different data collection techniques

— Guidelines on how to develop interview guides or questionnaires, including guidelines on how to interview

— An introduction on how to conduct Focus Group Discussions with an exercise in FGD, which should be carried out in any protocol development workshop

All parts are highly relevant for young researchers who still need to gain experience in conducting research.

This page intentionally left blank.

Designing and Conducting Health Systems Research Projects
Part 1: Proposal Development and Fieldwork

Module 10A
OVERVIEW OF DATA COLLECTION TECHNIQUES

This page intentionally left blank.

Module 10A: OVERVIEW OF DATA COLLECTION TECHNIQUES

OBJECTIVES

At the end of this session you should be able to:

  1. Describe various data collection techniques and state their uses and limitations.
  2. Advantageously use a combination of different data collection techniques.
  3. Identify various sources of bias in data collection and ways of preventing bias.
  4. Identify ethical issues involved in the implementation of research and ways of ensuring that your research informants or subjects are not harmed by your study.
  1. Overview of data collection techniques
  2. The importance of combining different data collection techniques
  3. Bias in information collection
  4. Ethical considerations

I. OVERVIEW OF DATA COLLECTION TECHNIQUES

Data-collection techniques allow us to systematically collect information about our objects of study (people, objects, phenomena) and about the settings in which they occur.

In the collection of data we have to be systematic. If data are collected haphazardly, it will be difficult to answer our research questions in a conclusive way.

Example:

During a nutrition survey three different weighing scales were used in three villages. The researchers did not record which scales were used in which village. After completion of the survey it was discovered that the scales were not standardised and indicated different weights when weighing the same child. It was therefore impossible to conclude in which village malnutrition was most prevalent.

Various data collection techniques can be used such as:

1. Using available information

Usually there is a large amount of data that has already been collected by others, although it may not necessarily have been analysed or published. Locating these sources and retrieving the information is a good starting point in any data collection effort.

For example, analysis of the information routinely collected by health facilities can be very useful for identifying problems in certain interventions or in flows of drug supply, or for identifying increases in the incidence of certain diseases.

Analysis of health information system data, census data, unpublished reports and publications in archives and libraries or in offices at the various levels of health and health-related services, may be a study in itself. Usually, however, it forms part of a study in which other data collection techniques are also used.

The use of key informants is another important technique to gain access to available information. Key informants could be knowledgeable community leaders or health staff at various levels and one or two informative members of the target group (e.g., adolescents on their sexual behaviour). They can be involved in various stages of the research, from the statement of the problem to analysis of the data and development of recommendations. Other sources of available data are newspapers and published case histories, e.g., patients suffering from serious diseases, or their relatives, telling their experiences and how they cope.*


* For example, Noerine Kaleeba (1991) We miss you all: AIDS in the family. Harare: Women and AIDS Support Network.

Note:

In order to retrieve the data from available sources, the researcher will have to design an instrument such as a checklist or compilation sheet. In designing such instruments, it is important to inspect the layout of the source documents from which the data is to be extracted. For health information system (HIS) data, for example, the data compilation sheet should be designed in such a way that the items of data can be transferred in the order in which the items appear in the source document. This will save time and reduce error.

The advantage of using existing data is that collection is inexpensive. However, it is sometimes difficult to gain access to the records or reports required, and the data may not always be complete and precise enough, or too disorganised.

2. Observing

OBSERVATION is a technique that involves systematically selecting, watching and recording behaviour and characteristics of living beings, objects or phenomena.

Observation of human behaviour is a much-used data collection technique. It can be undertaken in different ways:

Observations can be open (e.g., ‘shadowing’ a health worker with his/her permission during routine activities) or concealed (e.g., ‘mystery clients’ trying to obtain antibiotics without medical prescription). They may serve different purposes. Observations can give additional, more accurate information on behaviour of people than interviews or questionnaires. They can also check on the information collected through interviews especially on sensitive topics such as alcohol or drug use, or stigmatising diseases. For example, whether community members share drinks or food with patients suffering from feared diseases (leprosy, TB, AIDS) are essential observations in a study on stigma.

Observations of human behaviour can form part of any type of study, but as they are time consuming they are most often used in small-scale studies.

Observations can also be made on objects. For example, the presence or absence of a latrine and its state of cleanliness may be observed. Here observation would be the major research technique.

If observations are made using a defined scale they may be called measurements. Measurements usually require additional tools. For example, in nutritional surveillance we measure weight and height by using weighing scales and a measuring board. We use thermometers for measuring body temperature.

3. Interviewing

An INTERVIEW is a data-collection technique that involves oral questioning of respondents, either individually or as a group.

Answers to the questions posed during an interview can be recorded by writing them down (either during the interview itself or immediately after the interview) or by tape-recording the responses, or by a combination of both.

Interviews can be conducted with varying degrees of flexibility. The two extremes, high and low degree of flexibility, are described below:

• High degree of flexibility:

For example:

When studying sensitive issues such as teenage pregnancy and abortions, the investigator may use a list of topics rather than fixed questions. These may, e.g., include how teenagers started sexual intercourse, the responsibility girls and their partners take to prevent pregnancy (if at all), and the actions they take in the event of unwanted pregnancies. The investigator should have an additional list of topics ready when the respondent falls silent, (e.g., when asked about abortion methods used, who made the decision and who paid). The sequence of topics should be determined by the flow of discussion. It is often possible to come back to a topic discussed earlier in a later stage of the interview.

The unstructured or loosely structured method of asking questions can be used for interviewing individuals as well as groups of key informants. (For details concerning focus group discussions (FGDs), see Module 10C.)

A flexible method of interviewing is useful if a researcher has as yet little understanding of the problem or situation he is investigating, or if the topic is sensitive. It is frequently applied in exploratory studies. The instrument used may be called an interview guide or interview schedule.*

• Low degree of flexibility:

Less flexible methods of interviewing are useful when the researcher is relatively knowledgeable about expected answers or when the number of respondents being interviewed is relatively large. Then questionnaires may be used with a fixed list of questions in a standard sequence, which have mainly fixed or pre-categorised answers.

For example:

After a number of observations on the (hygienic) behaviour of women drawing water at a well and some key informant interviews on the use and maintenance of the wells, one may conduct a larger survey on water use and satisfaction with the quantity and quality of the water.


* Though in principle one may speak of loosely structured questionnaires, in practice the term questionnaire appears to be so hooked to tools with pre-categorised answers that we have decided to use the term interview guide for loosely structured tools. However, in reality there is often a mixture of open and pre-categorised answers (see Module 10B). In that case we will still use the term questionnaire.

4. Administering written questionnaires

A WRITTEN QUESTIONNAIRE (also referred to as self-administered questionnaire) is a data collection tool in which written questions are presented that are to be answered by the respondents in written form.

A written questionnaire can be administered in different ways, such as by:

The questions can be either open-ended or closed (with pre-categorised answers). (See Module 10B for details concerning design of interview guides and questionnaires.)

5. Focus group discussions (FGD)

A focus group discussion allows a group of 8 - 12 informants to freely discuss a certain subject with the guidance of a facilitator or reporter. (See Module 10C for a discussion of this technique.)

6. Projective techniques

When a researcher uses projective techniques, (s)he asks an informant to react to some kind of visual or verbal stimulus.

For example: An informant may be provided with a rough outline of the body and be asked to draw her or his perception of the conception or onset of an illness.

Another example of a projective technique is the presentation of a hypothetical question or an incomplete sentence or case/study to an informant (‘story with a gap’). A researcher may ask the informant to complete in writing sentences such as:

— If I were to discover that my neighbour had TB, I would . . .;

— If my wife were to propose that I use condoms, I would . . .

Or (s)he may ask the informant: Suppose your child suffered from diarrhoea, what would you do?

Such techniques can easily be combined with semi-structured interviews or written questionnaires. They are also very useful in FGDs to get people’s opinion on sensitive issues.

7. Mapping and scaling

Mapping is a valuable technique for visually displaying relationships and resources.

In a water supply project, for example, mapping is invaluable. It can be used to present the placement of wells, distance of the homes from the wells, other water systems, etc. It gives researchers a good overview of the physical situation and may help to highlight relationships hitherto unrecognised.

Mapping a community is also very useful and often indispensable as a pre-stage to sampling.

Scaling is a technique that allows researchers through their respondents to categorise certain variables that they would not be able to rank themselves.

For example, they may ask their informant(s) to bring certain types of herbal medicine and ask them to arrange these into piles according to their usefulness. The informants would then be asked to explain the logic of their ranking.

Mapping and scaling may be used as participatory techniques in rapid appraisals or situation analyses. In a separate volume on participatory action research, more such techniques will be presented. (Also see the literature list at end of this module.)

Rapid appraisal techniques and participatory research are approaches often used in health systems research.

Differentiation between data collection techniques and data collection tools

To avoid confusion in the use of terms, the following table points out the distinction between techniques and tools applied in data collection.

Table 10A.1: Data collection techniques and tools

./img/designandco_161_la_0.jpg

Advantages and disadvantages of various data collection techniques

Table 10A.2 summarises the advantages and disadvantages of various data collection techniques.

Table 10A.2: Advantages and disadvantages of various data collection techniques

./img/designandco_162_la_0.jpg

II. IMPORTANCE OF COMBINING DIFFERENT DATA COLLECTION TECHNIQUES

When discussing different data collection techniques and their advantages and disadvantages, it becomes clear that they can complement each other. A skilful use of a combination of different techniques can reduce the chance of bias (see below) and will give a more comprehensive understanding of the topic under study.

Researchers often use a combination of flexible and less flexible research techniques.

Flexible techniques, such as

are also called QUALITATIVE research techniques. They produce qualitative data that is often recorded in narrative form.

QUALITATIVE RESEARCH TECHNIQUES involve the identification and exploration of a number of often mutually related variables that give INSIGHT in human behaviour (motivations, opinions, attitudes), in the nature and causes of certain problems and in the consequences of the problems for those affected. ‘Why’, ‘What’ and ‘How’ are important questions.

Structured questionnaires that enable the researcher to quantify pre- or post-categorised answers to questions are an example of QUANTITATIVE research techniques. The answers to questions can be counted and expressed numerically.

QUANTITATIVE RESEARCH TECHNIQUES are used to QUANTIFY the size, distribution, and association of certain variables in a study population. ‘How many?’ ‘How often?’ and ‘How significant?’ are important questions.

Both qualitative and quantitative research techniques are often used within a single study.

For example:

It has been observed in country X that children between 1 and 2-1/2 years, who have already started to eat independently, have unsatisfactory food intake once they fall ill. A study could be designed to address this problem, containing the following stages:

In this example, the first, qualitative part of the study would be used to focus the survey on the most relevant issues (mothers’ feeding behaviours and reasons for these behaviours) and to help phrase the questions in an optimal way in order to obtain the information that is needed.

The second, quantitative part of the study would be used to find out what proportion of the mothers follow various practices and the reasons for their behaviours and whether certain categories of children (e.g., the younger ones or children from specific socio-economic categories) are more at risk than others.

The third, qualitative part of the study would provide feedback on the major findings of the survey. Do the conclusions make sense to women in the study area? Have certain aspects been overlooked when interpreting the data? What remedial action is feasible to improve practices related to feeding sick children?

It is also common to collect qualitative and quantitative data in a single questionnaire. Researchers collecting both types of data have to take care that they:

III. BIAS IN INFORMATION COLLECTION

BIAS in information collection is a distortion in the collected data so that it does not represent reality.

Possible sources of bias during data collection:

1. Defective instruments, such as:

These sources of bias can be prevented by carefully planning the data collection process and by pre-testing the data collection tools.

2. Observer bias:

Observer bias can easily occur when conducting observations or utilising loosely structured group- or individual interviews. There is a risk that the data collector will only see or hear things in which (s)he is interested or will miss information that is critical to the research.

Observation protocols and guidelines for conducting loosely structured interviews should be prepared, and training and practice should be provided to data collectors in using both these tools. Moreover it is highly recommended that data collectors work in pairs when using flexible research techniques and discuss and interpret the data immediately after collecting it. Another possibility - commonly used by anthropologists - is using a tape recorder and transcribing the tape word by word.

3. Effect of the interview on the informant:

This is a possible factor in all interview situations. The informant may mistrust the intention of the interview and dodge certain questions or give misleading answers. For example: in a survey on alcoholism you ask school children: ‘Does your father sometimes get drunk?’ Many will probably deny that he does, even if it is true. Such bias can be reduced by adequately introducing the purpose of the study to informants, by phrasing questions on sensitive issues in a positive way, by taking sufficient time for the interview, and by assuring informants that the data collected will be confidential (see Module 10B).

It is also important to be careful in the selection of interviewers. In a study soliciting the reasons for the low utilisation of local health services, for example, one should not ask health workers from the health centres concerned to interview the population. Their use as interviewers would certainly influence the results of the study.

4. Information bias:

Sometimes the information itself has weaknesses. Medical records may have many blanks or be unreadable. This tells something about the quality of the data and has to be recorded. For example, in a TB defaulter study the percentage of defaulters with an incomplete or missing address should be calculated.

Another common information bias is due to gaps in people’s memory; this is called memory or recall bias. A mother may not remember all details of her child’s last diarrhoea episode and of the treatment she gave two or three months afterwards. For such common diseases it is advisable to limit the period of recall, asking, for example, ‘Has your child had diarrhoea over the past two weeks?’

Note:

All these potential biases will threaten the validity and reliability of your study. By being aware of them it is possible, to a certain extent, to prevent them. If the researcher does not fully succeed, it is important to report honestly in what ways the data may be biased.

Module 10B and Module 12, Annex 1 provide more information on how you can prevent biases in your research.

IV. ETHICAL CONSIDERATIONS

As we develop our data collection techniques, we need to consider whether our research procedures are likely to cause any physical or emotional harm. Harm may be caused, for example, by:

Several methods for dealing with these issues may be recommended:

If sensitive questions are asked, for example, about family planning or sexual practices, or about opinions of patients on the health services provided, it may be advisable to omit names and addresses from the questionnaires.

REFERENCES

Abramson JH (1990, 4th ed.) Survey Methods in Community Medicine. London: Churchill- Livingstone. (In particular Chapter 6)

Moser CA, Kalton G (1989, 2nd ed.) Survey Methods in Social Investigation. Hants, UK: Gower Publishing Company. (In particular Chapters 10-13)

Patton MQ (1990, 2nd ed.) Qualitative Evaluation and Research Methods. Newbury Park, USA: Sage Publications. (In particular Chapters 17-29)

Pretty JN, Guyt I, Thompson J, Scones I (1995) Participatory Learning & Action. A Trainer’s Guide. London: International Institute for environment and Development (IIED) (In particular Chapters 4 and 5 on semi-structured interviewing, diagramming, ranking and scoring.)

EXERCISE: Selection of study types and data collection techniques (in plenary)

Five health management problems for which studies must be developed are described below. For each problem you are asked to state:

  1. You suspect that a large proportion of women in your region (pop. 1,000,000) is anaemic, in particular women at childbearing age. You would like to determine how big the problem is, and whether women perceive it as a problem. Furthermore you would like to know whether and how women themselves could contribute to improving their anaemic condition.
  2. A district health team evaluated its malaria spraying programme by looking at available records and reports. It did not find significant flaws in the functioning of the services in different divisions and villages. Nevertheless, the incidence of malaria and mosquito counts show peaks in certain villages that the DHT cannot explain. It wants to find out if there is something wrong with the services.
  3. A community survey in your region (pop. 2,000,000) indicates that 12% of the adults (15–60) in the capital are HIV positive, as compared to 7% in road side settlements and 2,5% in the rural area (where 80% of the population live). You want to step-by-step introduce an intensive STD programme in all 11 hospitals and 180 health centres, hoping to decrease the HIV incidence. You would like to evaluate whether the STD programme has an effect. Apart from the three main questions stated above: How would you organise the evaluation time-wise? Are any ethics involved? What could be two important biases in the study?
  4. You are a midwife who has just been appointed as head of a maternity unit in a district hospital. You suspect that the number of low birth-weight babies is increasing, and you would like to know more about the physical and socio-economic conditions of the mothers to see if remedial actions should be taken. The clinic records at present are not complete enough to draw conclusions and you have neither the time nor the money to do a district-wide community survey.
  5. There are long queues (waiting times), at the out-patient department of your district hospital. You are concerned about this and you would like to find out to what extent the problem may be related to the organisation and management of the department and whether certain bottlenecks can be identified. In a later stage of the research you would like to try to eliminate some of the bottlenecks and see whether there is improvement.

GROUP WORK: Selection of study type and data collection techniques

  1. Decide what type(s) of study you will apply in your own research proposal.
    • Make your choice on the basis of your research objectives and the variables you would like to include in the study. (Hang objective and variables on a wall or flip chart board so the whole group can see them during the current group work session.) Review pages 1-2 of Module 9, to assist you in your choice of study type(s).
  2. Determine what data-collection techniques you will use for each variable in your study.
    • Display the table that was prepared during the group work on selection of variables. For each variable determine the source of data and method(s) of data collection.
    • For some variables it may be necessary to collect additional data in order to be able to define the variable and determine the scale of measurement.
      Example:Factor:Inadequate knowledge of patients about TB treatment
       Variable:Knowledge of patients
       Scale of measurement:Percentage or number of items of advice that are recounted by patients

      To determine the ‘items’ that should be used in the variable, it may be necessary to do a focus group discussion (FGD) with the staff treating TB patients and/or with a panel of experts on TB. Subsequently the results of the FGD could be used to develop questionnaires for interviews with patients. Hence the methods of data collection would be a FGD (to develop knowledge indicators) and face-to-face interviews with patients.

    • Display the results in the following table:

      ./img/designandco_168_la_0.jpg

      * These items were developed during the group work session for Module 8.

      * Preserve this table for presentation in plenary and for use in subsequent group work.

  3. Summarise which data collection techniques you will use and which groups or records will form the sources of your data for each tool.
  4. Decide whether there are any ethical problems with the type of study or data collection tools you propose.

Trainer’s Notes

Module 10A: OVERVIEW OF DATA COLLECTION TECHNIQUES

Timing and teaching methods

1 hour Introduction to data collection techniques and discussion
1 hour Exercise: Selection of study types and data collection techniques
1 hour Group work: Selection of study types and data collection techniques
1 hour Plenary
4 hours TOTAL TIME
Introduction and discussion
Exercise: Selection of study types and data collection techniques

(An answer sheet for the exercise is presented in Annex 10A.1, on the next page.)

Group work: Selection of study type and data collection techniques

./img/designandco_171_la_0.jpg


* See Grosskurth H, Mosha F, Todd J, Mwijarubi E, Klokke A, Senkoro K, Mayaud P et al (1995) Impact of improved treatment of sexually transmitted diseases on HIV infection in rural Tanzania: randomised controlled trial. The Lancet 346: 530-536.

This delay of two years would form an ethical problem as soon as the preliminary results would indicate that the extra input in STD treatment in the intervention HCs indeed reduces the risk of HIV infection. If that is the case, experiments are sometimes stopped in order to allow the control group to profit from the intervention as well.

One important bias could be that STD patients in control villages would have heard about the available treatment for STD in the experimental HCs and go there for treatment. To be able to control this potential bias, the research team analysed the treatment registers in the experimental HCs for the location of all STD patients who received treatment and deducted patients from the control villages (which appeared to be exceptions).

Another bias may be caused by intensified health education on STD symptoms and treatment over radio and TV, which could influence the treatment seeking behaviour of members of the experimental as well as of the control group.

./img/designandco_172_la_0.jpg

This page intentionally left blank.

Designing and Conducting Health Systems Research Projects
Part I: Proposal Development and Fieldwork

Module 10B
DESIGN OF RESEARCH INSTRUMENTS;
INTERVIEW GUIDES AND INTERVIEW SKILLS

Module 10B: DESIGN OF RESEARCH INSTRUMENTS; INTERVIEW GUIDES AND INTERVIEW SKILLS

OBJECTIVES

At the end of this session you should be able to:

  1. Distinguish between various stages in questionnaire design.
  2. Demonstrate appropriate techniques for wording questions and designing interview guides, questionnaires and checklists to ensure maximum quality of responses.
  3. Identify appropriate data-collection techniques for your study.
  4. Prepare your data-collection tools, taking care that you cover all important variables.
  5. Effectively carry out an interview and train research assistants in interview skills.
  1. Introduction
  2. Types of questions
  3. Steps in designing a questionnaire/interview guide
  4. Checklists
  5. Interview skills

I. INTRODUCTION

The quality of research depends to a large extent on the quality of the data collection tools. Interviewing and administering questionnaires are probably the most commonly used research techniques. Therefore designing good ‘questioning tools’ forms an important and time-consuming phase in the development of most research proposals.

Once the decision has been made to use these tools, the following questions should be considered before designing them:

II. TYPES OF QUESTIONS

In this module we will concentrate on interactive (usually face-to-face) interviews. Before examining the steps in designing a questionnaire, we need to review the types of questions used in interviews. Depending on how questions are asked and recorded we can distinguish two major possibilities:

1. Completely open-ended questions

OPEN-ENDED QUESTIONS permit free responses which should be recorded in the respondents’ own words.

Such questions are useful for obtaining in-depth information on:

For example:

‘What is your opinion on the services provided in the ANC?’ (Explain why.)

‘What do you think are the reasons some adolescents in this area start using drugs?’

‘What would you do if you noticed that your daughter (school girl) had a relationship with a teacher?’

The answers to these questions are written down as closely as possible in the words of the respondents.

Advantages of completely open-ended questions

Risks of completely open-ended questions

Suggestions to improve use of completely open-ended questions

2. Partially categorised questions

In interviews questions are often asked as open-ended questions, but to facilitate recording and analysis, some answers can already be pre-categorised.

For example:

‘How did you become a member of the Village Health Committee?’

./img/designandco_178_la_0.jpg

In this case the first four categories of answers are known, but there may be other possibilities. Therefore there is a category ‘other’ where other answers can be recorded. During the analysis, these responses can still be further categorised.

For open-ended questions, more than one answer is usually allowed. The interviewers will have to be trained to wait for additional answers

Advantages of pre-categorised answers

Risks of pre-categorised answers

Suggestions to minimise risks associated with pre-categorised answers

Clear guidelines have to be provided to interviewers on important issues. For example:

Note:

There may be open-ended questions for which all possible categories of responses are known, e.g., some health practices (family planning methods: currently using, ever used, and never used). These may be fully categorised. However, these questions, when used in HSR, will always be followed by other questions asking for elaboration on reasons and conditions for use or non-use of the practice.

3. Closed questions

CLOSED QUESTIONS have a list of possible options or answers from which the respondents must choose.

Closed questions are most commonly used for background variables such as age, marital status or education, although in the case of age and education you may also take the exact values and categorise them during data analysis (see Module 13).

Closed questions may be used to get the respondents to express their opinions or attitudes by choosing rating points on a scale.

For example:

What is your opinion on the following statement:

‘Women who have induced abortion should be severely punished.’

./img/designandco_179_la_0.jpg

Closed questions may also be used if one is only interested in certain aspects of an issue and does not want to waste time obtaining more information than one needs.

For example, a researcher who is only interested in the sources of protein in a family diet may ask:

‘Did you eat any of the following foods yesterday?’ (Circle yes if at least one item in each set of items is eaten.)

./img/designandco_179_la_1.jpg

Using attitude scales in face-to-face interviews with literate respondents is most objectively carried out if the various options for each answer are provided on different cards. The respondents can be asked to put the cards in the order preferred by them while making their choice. If the researcher only reads the options, the respondents might not consider all options equally and the scale will not accurately measure the attitudes.

Advantages of closed questions:

Risks of closed questions:

Suggestions to minimise risk associated with using closed questions:

Note:

Sometimes it is useful, especially in small-scale studies, to use pictures or drawings when asking certain questions in order to get the discussion going. In the case of illiterates, a questionnaire may even consist exclusively of pictures (See Annex 1).

III. STEPS IN DESIGNING A QUESTIONNAIRE/ INTERVIEW GUIDE*

Designing a good questionnaire always takes several drafts. In the first draft we should concentrate on the content. In the second, we should look critically at the formulation and sequencing of the questions. Then we should scrutinise the format of the questionnaire. Finally we should do a test-run to check whether the questionnaire gives us the information we require and whether interviewers as well as respondents feel at ease with it. Usually the questionnaire will need some further adaptation before we can use it for actual data collection.


* For the sake of simplicity we take semi-structured questionnaires used (containing completely open as well as partially pre-categorised and closed questions) in face-to-face interviews as an example. The same steps apply to designing other ‘questioning tools’.

Step 1: Content

Take your objectives and variables as a starting point.

Decide what questions will be needed to measure or (in the case of qualitative studies) to define your variables and reach your objectives.

When developing the questionnaire, you should reconsider the variables you have chosen and, if necessary, add, drop or change some. You may even change some of your objectives at this stage.

Step 2: Formulating questions

Formulate one or more questions that will provide the information needed for each variable.

Take care that questions are specific and precise enough so that different respondents don’t interpret them differently. For example, a question such as: ‘Where do community members usually seek treatment when they are sick?’ cannot be asked in such a general way because each respondent may have something different in mind when answering the question:

The question therefore, as a rule, has to be broken up into different parts and made so specific that all informants focus on the same thing. For example, one could:

Check whether each question measures one thing at a time.

For example: the question, ‘Do you think that the war situation leads to mental problems that require treatment by health staff?’ brings three topics, which should be split up in

Avoid leading questions.

A question is leading if it suggests a certain answer. For example: the question, ‘Do you think that people have to give bribes at hospital X to be seen by a doctor?’ hardly leaves room for ‘no’ or for other options. A better question would be: ‘Have you recently visited hospital X?’ This would be followed by a series of other probing questions such as, ‘By whom were you seen?’ ‘What were the complaints?’ ‘How much were you asked to pay?’ ‘Are the fees fixed?’ ‘Do they include an examination by the doctor if the condition of a patient is serious?’ ‘Do all patients have equal access to a doctor in case of need?’ ‘Was this what you expected?’

Sometimes, a question is leading because it presupposes a certain condition. For example: ‘What action did you take the last time your child had diarrhoea?’ presupposes the child has had diarrhoea. A better set of questions would be: ‘Has your child ever had diarrhoea?’ (If yes:) ‘When was the last time?’ ‘Did you do anything to treat it?’ (If yes:) ‘What?’

Avoid words with double or vaguely defined meanings or that are emotionally laden. Concepts such as dirty (clinics), lazy (patients), or unhealthy (food), for example, should be omitted.

Ask sensitive questions in a socially acceptable way:

Questions relating to abortion, sexual practices of adolescents, or AIDS and mental illness in the family are usually sensitive. Such questions should be formulated in such a way that the question does not judge or embarrass the respondent. Making the subject more socially acceptable and then asking a question about the respondent’s own preference or practice could lead to a more accurate response. For example: ‘Many teenagers have had abortions for unwanted pregnancies. Do you know girls who had this problem? Have you ever had an abortion?’

Another way to deal with sensitive questions (as indicated by the respondent’s hesitation in answering the question) is by asking the question indirectly. For example you could ask: ‘If your friend was considering abortion for her daughter who became pregnant while in school, what would you advise her?’

A common weakness in questionnaires is the inappropriate transformation of research questions into interview questions. You cannot ask informants: ‘Would a woman’s educational status influence her health?’ That is exactly what you have to find out by relating individual women’s level of education to a number of health conditions.

Note:

Ask the questions to yourself or to a friend and check whether the answers you get are the type of responses you want.

Step 3: Sequencing the questions

Design your interview schedule or questionnaire to be ‘informant friendly’.

Step 4: Formatting the questionnaire

When you finalise your questionnaire, be sure that:

If you use a computer, the right margin of the page should be reserved for boxes intended for computer codes. (Consult an experienced facilitator when designing your questionnaire.)

Your questionnaire should not only be INFORMANT - but also RESEARCHER FRIENDLY!

Step 5: Translation

If interviews will be conducted in one or more local languages, the questionnaire should be translated in order to standardise the way questions will be asked.

After having it translated you should have it retranslated into the original language by a different person. You can then compare the two versions for differences and make decisions concerning the final phrasing of difficult concepts.

Self-administered (written) questionnaires

All steps discussed above apply to written questionnaires as well as to guides/questionnaires used in interviews. For written questionnaires, however, clear guidelines will have to be added on how the answers to questions should be filled in.

Self-administered questionnaires are most commonly used in large-scale surveys using predominantly pre-categorised answers among literate study populations.

As a response rate of 50% or less to written questionnaires is not exceptional, these tools will rarely be used in small-scale studies. In exploratory studies which require intensive interaction with informants in order to gain better insight in an issue, self-administered questionnaires would, more over, be inadequate tools.

However, written questionnaires may sometimes be useful in small-scale studies on sensitive topics.

Even then, they are usually combined with other tools:

For example, during a FGD on sensitive issues such as sexual behaviour, a facilitator cannot always be sure that all participants dare to speak freely. A short, written questionnaire with open questions may then be used to explore remaining sensitive questions or worries of participants without revealing their identities.

Sometimes, a self-administered questionnaire seems the only possibility:

For example, on one of the smaller islands in the Indian Ocean, the shortage of nurses was acute and public complaints about the functioning of health facilities abounded. A relatively high number of nurses had been leaving the profession over the past ten years and the vacancies could not all be filled as less and less young people were choosing nursing as a career. The research team that investigated the problem (all MOH staff) appreciated the sensitivity of the topic. Aware that everyone on the island knew each other, it was decided to use a flexible self-administered questionnaire which members of the research team personally delivered at each health facility, stressing the importance of filling it in. In that way they hoped to ensure anonymity as well as increase participation in the study. (Still, the response rate was only slightly over 50%.)

IV. CHECKLISTS

Checklists can be used to systematically observe human behaviour or the condition of specific equipment (e.g., fridge, expiring dates of medicines, or completeness of records). Observations can be relatively open or can be predetermined comparisons of reality against fixed standards. Sometimes the aim is systematic content analysis (e.g., newspaper articles, health information system). The objectives of the study determine the content of a checklist. A checklist includes all the items or points that must be considered during an observation in the field, or when extracting data from existing records.

An observation checklist for rating the cleanliness and use of Blair/Ventilated Improved Pit Latrines (VIP) in a study on hygiene practices is presented below, as an example:

./img/designandco_184_la_0.jpg

V. INTERVIEW SKILLS

1. The interviewer-informant relationship and interview conditions

An interviewer needs to have the skills of a detective. (S)he should carefully, step-by-step, delve for the truth. Here the comparison stops, because the truth researchers are looking for, in general, has nothing to do with criminality. To the contrary, our respondents are possible partners who with their information help solve a shared problem, or at least help to better understand why people behave as they do.

To turn an informant into a partner, a researcher has to invest in the relationship. First of all, (s)he has to be clear to the informant about the purpose of the interview and the study. Enough information should be given to raise the interest of informants and to enable them to judge whether they would like to participate or not. Consent has to be obtained before the interview. (See Module 10A.) On the other hand, not too many details should be given about what will be asked and why. Otherwise, the researcher runs the risk informants become selective in what they tell, and conceal information in order to ‘help’ or please him/her (interviewer bias).

When sensitive topics are being explored anonymity should be ensured, for example, by not including a name in the interview notes. If a second interview would be useful, one may ask a name and address, but make sure this will never appear in the report.

Further, before the research starts, the possibility of involving informants in the discussion of the results and recommendations should be considered and discussed with informants. Feedback sessions with informants are usually rewarding for both informants and researchers. If it is impossible to organise such sessions for all participants, an interviewer can at least summarise the main results of the interview at the end and check if this is what the informant meant to say. Usually the informant will react, elaborate on the responses already given, or sometimes withdraw a statement if it seems too personal.

Partnership between interviewer and informant implies that the interviewer will try to minimise the social distance between him/herself and the informant. Interviewers should try to blend in the environment. In order to do so:

For example: A research team headed by a Regional Medical Officer (RMO) carried out a study into reasons for low utilisation of delivery services in a district in Tanzania. When interviewing Traditional Birth Attendants (TBAs), the team discovered that TBAs delivered services in case of highly complicated deliveries which could be dangerous to the mothers. The RMO was surprised about the practices of the TBAs and asked many details about what they did with respect to various complications, but did not show his surprise. After finalising the study he organised a training course for TBAs in which he took their level of knowledge as a point of departure and explained why some practices were dangerous and could better be replaced by safer alternatives.

The environment should also be supportive of the interview situation. Anything that disturbs (noise, other people listening, a formal surrounding) should if possible be avoided.

For example: A hospital-ward, with nurses running in and out, may not be a good place to ask a patient’s treatment history including traditional treatment, or his/her opinion about the treatment in the hospital. An adolescent home is not an appropriate environment for an interview with adolescents about their sexual behaviour, even if the parents are absent. A quiet place outside would be better.

If it appears impossible to prevent others from joining the interview, even after having made clear that one would like to speak undisturbed with the informant, there are different possibilities.

For example:

Tape recording may be an enabling or a disturbing environmental factor in the interview. If the informant accepts the argument of the interviewer that (s)he will not be able to remember everything that is said nor write it down, it is an asset. Playing with it at the beginning of the interview and letting informants hear themselves may help to overcome hesitation among informants who have never experienced the use of a tape recorder. Informants should be asked for their consent before use, and be assured confidentiality. If they hesitate or refuse, the only solution is to take notes. This is in any case advisable, as the tape-recorder may not always function correctly. Usually, informants forget quickly that the recorder is on, but if they appear disturbed despite their consent that it be used, it should be stopped.

Note:

2. The interviewer’s tasks

Apart from the introduction to the interview, which includes building a partnership that needs to last throughout the interview and even beyond, the interviewer has a number of other tasks:

— Posing questions

— Evaluating answers and probing for elaboration or more precision in case of a superficial or invalid answer

— Noting down answers

— Leading the discussion, but at the same time encouraging the informant to give spontaneous information relevant to the topic by letting him/her talk

Unless the interview is carried out in pairs, the interviewer will have to carry out all these tasks at the same time.

(1) The introduction to the interview should be written as the interview guide is being developed, even if there is only one researcher, but certainly if a research team conducts the interviews. This will ensure consistency, which prevents bias due to interviewer variation.

We have already paid much attention to
(2) the art of posing questions in the preceding parts of this module. Not yet covered is the issue of what to do when an answer is unsatisfactory. Clearly, the quality of an interview will be determined by the skills of the interviewer in
(3) evaluating answers and diagnosing what went wrong and why, so that corrective action can be taken on the spot.

An answer can be unsatisfactory in many ways:

Note that when something goes ‘wrong’ in an interview the interviewer should always take the blame. (S)he is the one who ‘did not ask the question clearly enough’, or who ‘did not understand the answer’, and so is asking the question again. The interviewer is the one who receives a ‘present’ in the form of information and has to show by word and body language that (s)he appreciates this highly. Nodding, humming, or repeating the last few words may all be perceived as encouragement. A silence from the side of the interviewer may also work as encouragement for the informant to continue.

(4) Noting down answers should never go at the cost of the cost of the eye contact with the informant, even if the information collected is not sensitive. Scribbles in a small notebook are preferable over more extensive notes, provided they are elaborated straight after each interview. It is advisable to take brief notes even when a tape recorder is used, just in case the recording fails. Elaborating notes after an interview takes more time than the interview itself, and the full transcription of a taped interview may take between 5-10 times the duration of the interview, and often longer.

(5) Keeping control over the interview without imposing oneself is a skill each researcher has to learn. Even though an interviewer may encourage the provision of (often valuable) spontaneous information, some spontaneous information may distract from the main issues. The interviewer has to take care that there is not only a good beginning but also a satisfactory end to the interview. In between, all topics should have been adequately covered. Otherwise the information provided by different informants will not be comparable or, in case of key informants, not be comprehensive. Therefore an interviewer will also sometimes have to stop informants who go enthusiastically off track. This can be done politely: ‘Thank you, this is interesting, but do you mind if I go back to the previous (or next) question? This (specify) is not yet fully clear to me.’

At the end, the interviewer should not only summarise the interview, which may lead to a valuable new discussion, but also respond to questions that came up during the interview, give advice (if necessary or asked for) and give an opportunity for further questions of the informant. Such ‘after-interview’ discussions and questions should always be recorded, like all spontaneous information, because discussions can shed light on complicated, not yet fully clear issues from many preceding interviews. (Remember the researcher’s function of detective!)

3. Training the research team/assistants

Those who for the first time conduct a face-to-face interview will need training in the application of all the advice provided above. Good quality interview data form the heart of an HSR study. Obtaining such data presupposes more than appropriate knowledge and skills; also specific attitudes (e.g., not imposing one self) and insight (e.g., appreciating signs of discomfort in an informant, and recognising evading or, to the contrary, highly illuminating answers) are required.

The learning process therefore preferably consists of reading and listening/observing, as well as doing. ‘Doing’ may involve a written exercise, a role-play, or actual field practice during a pre-test. The following exercise may be helpful:

EXERCISE: Interview training

  1. Written exercise: Write introductions for one of the interviews your team will be developing for your study. Then meet in small groups and critique each other’s introductions.
  2. Role-play: Six participants will be asked to participate in a role-play. Three will act as interviewers - staff members from a health centre with a high percentage of unsupervised deliveries who wish to find out the reasons for the low coverage. The other three will play the roles of mothers who have delivered in the last six months. Each of the mothers will be given a particular role to play by the facilitators. The rest of the group will know what the roles are, but the interviewers will have to find out while they are interviewing and try to cope as best they can.

    While the three interviews are taking place, observe the interactions closely. Following the interviews a brief discussion will be held focusing on the attitudes of the interviewers and their skills. We will discuss what has been done well in each of the interviews, and suggestions for how the process could be improved so as to gain more accurate and complete information.

  3. Real interview: As in 2, but now on a topic relevant to all participants. (For example, what made them join medical training.)

GROUP WORK (4 hours or more)

  1. Prepare your data-collection tools (instruments), taking care that you cover all important variables. Refer back to the table that your group prepared during the group work session at the end of Module 10A, which specifies the methods of data collection you must use. (You might divide up the work, assigning different members of the group to design the various data collection instruments required.)
  2. Take care that you have an optimal mix between open-ended and pre-categorised questions.
  3. If methods are needed other than those presented in Modules 10A and 10B, refer to Module 10C on FGD or other text-books on (participatory) methods.
  4. Discuss the possibilities for bias, which may occur when using the data-collection tools. Try to avoid bias as much as possible.

EXERCISE: Review of data collection tools

  1. Review in detail data collection tools of one other research team in relation to their objectives and variables and prepare suggestions for improving them. Be prepared to present your comments in plenary.
  2. If there is time, review the data collection tools of the other research teams in the course as well.

REFERENCES

Briggs CL (1992) Learning how to ask. A sociolinguistic appraisal of the interview in social science research. (In particular chapter 3, analysis of communicative blunders) Cambridge: Cambridge University Press. (5th edition)

Lee RM (1993) Doing Research on sensitive topics. London: Sage Publications.

Pretty JN, Guyt I, Thompson J, Scones IL (1995) Participatory Learning & Action. A Trainer’s Guide. London: International Institute for environment and Development (IIED) (In particular Chapters 4 and 5 on semi-structured interviewing, diagramming, ranking and scoring.)

Rubin HJ, Rubin IS (1995) Qualitative Interviewing. The art of hearing data. Thousand Oaks: Sage.

Seidman I (1998) Interviewing as qualitative research. A guide for researchers in education and the social sciences. New York: Teachers College Press. (In particular chapters 5-7.)

Spradley JP (1979) The ethnographic interview. New York: Holt, Rinehart and Winston.

./img/designandco_191_la_0.jpg


* This questionnaire which was made for use by non-literate health workers was shared by Dr. Peter Lamptey.

Trainer’s Notes

Module 10B: DESIGN OF RESEARCH INSTRUMENTS; INTERVIEW GUIDES AND INTERVIEW SKILLS

Timing and teaching methods

1 hour Introduction to designing questionnaires and interview skills
45 minutes Exercise: Interview training
4 hours + Group work
2 hours Exercise: Comment on the data collection tools of other groups
2 hours Plenary on data collection tools
2 hours Revision of data collection tools
11 hours+ TOTAL TIME
Introduction and discussion
Exercise: Interview training
  1. Written exercise: Ask each participant to write an introduction for one of the interviews his or her team will be developing for its study. Then ask each team to meet in a small group and critique each other’s introductions.
  2. Role-play: Ask six participants to volunteer to participate in the role play - three as interviewers (health care staff working in an area with a high percentage of unsupervised deliveries) and three as mothers who have delivered in the past 6 months. Announce that the task of the three interviewers is to interview the mothers to determine the reasons for the low percentage of deliveries supervised by the health centres and possible ways to increase coverage. Ask the three interviewers to go out of the room and each separately jot down ideas on the questions he or she might ask the mothers in five minutes interviews. While the interviewers are doing this, discuss the roles of the three mothers with the participants who have volunteered to play them and the rest of the class. (One mother may have given birth at home and honestly will tell why; another may have given birth at home and is hesitant to disclose why; and the third may have given birth in the hospital, but is not an easy talker; the interviewer will have to draw the information out of her to get a complete picture.)

    Once the ‘stage is set’, ask each of the interviewers to interview one of the mothers in turn, while the rest of the class watches. When the three interviews are completed, lead a discussion on the attitudes of the interviewers, their skills, what has been done well and suggestions for improving the interview. The ‘mothers’ can be asked what their feeling were about the interviews and what the interviewers could have done to obtain more accurate and complete information.

    Another possibility is to take a ‘real life’ situation and let three pairs of participants interview each other on a topic of common interest. Discuss this first with the participants and ask for their suggestions for a topic.

Group work
Exercise: Review of data collection tools

Note:

When groups are preparing their data collection tools they should have ideas concerning which tools they would like to pre-test and where. At this point of time, the course management team should therefore start organising the pre-test.

Designing and Conducting Health Systems Research Projects
Part I: Proposal Development and Fieldwork

Module 10C
FOCUS GROUP DISCUSSIONS

Module 10C: FOCUS GROUP DISCUSSION

OBJECTIVES

At the end of the session you should be able to:

  1. Identify the purpose, uses and limitations of the Focus Group Discussion (FGD) as a method of data collection in research.
  2. Conduct a FGD, analyse the data and report on the results.
  1. Characteristics and uses of focus group discussions
  2. How to conduct a focus group discussion
  3. Analysis of results
  4. Report writing

I. CHARACTERISTICS AND USES OF FOCUS GROUP DISCUSSIONS

A FOCUS GROUP DISCUSSION (FGD) is a group discussion of approximately 6 - 12 persons guided by a facilitator, during which group members talk freely and spontaneously about a certain topic.

A FGD is a qualitative method. Its purpose is to obtain in-depth information on concepts, perceptions and ideas of a group. A FGD aims to be more than a question-answer interaction. The idea is that group members discuss the topic among themselves, with guidance from the facilitator.

FGD techniques can, for example, be used to:

  1. Focus research and develop relevant research hypotheses by exploring in greater depth the problem to be investigated and its possible causes.

    For example:

    A district health officer had noticed that there were an unusually large number of cases of malnutrition of children under 5 reported from one area in her district. Because she had little idea of why there might be more malnutrition in that area she decided to organise three focus group discussions (one with leaders, one with mothers from the area and one with health staff from the area). She hoped to identify potential causes of the problem through the FGDs and then develop a more intensive study, if necessary.

  2. Formulate appropriate questions for more structured, larger scale surveys.

    For example:

    In planning a study of the incidence of childhood diarrhoea and feeding practices, a focus group discussion showed that in the community under study, children below the age of 1 year were not perceived as having ‘bouts of diarrhoea’ but merely ‘having loose stools’ that were associated with milestones such as sitting up, crawling, and teething. In the questionnaire that was developed after the FGD the concept ‘diarrhoea’ was therefore carefully described, using the community’s notions and terms.

  3. Help understand and solve unexpected problems in interventions.

    For example:

    In District X, the recent national (polio) immunisation days (NID) showed widely different coverage’s per village (50-90%) and in a number of villages a marked decrease in coverage was observed compared to last year. Eight FGD were held with mothers, two in town, three in rural villages with a marked decrease in NID coverage and three in villages with a high coverage throughout. It appeared that overall, the concept NID had raised confusion. Most people believed that this mass campaign strengthened the children’s immunity against any (childhood) disease, including malaria and Respiratory Tract Infections. In the villages with a low NID coverage there had been a high incidence of malaria in children immediately after the previous NID campaign and several children died. Mothers therefore believed that the NID campaign was useless.*


    * This is an adapted version of an (as yet unpublished) study carried out in Bushenyi District, Uganda, by Nuwaha et al.

  4. Develop appropriate messages for health education programmes and later evaluate the messages for clarity.

    For example:

    A rural health clinic wanted to develop a health education programme focused on weaning problems most often encountered by mothers in the surrounding villages and what to do about them. The focus group discussion could be used for exploring relevant local concepts as well as for testing drafts when developing the messages. The messages should be developed and tested in different socio-economic groups of mothers, as weaning practices may differ with income, means of subsistence and education of the mothers. Also ethnic differences may have to be taken into account.

  5. Explore controversial topics.

    For example:

    Sexual behaviour is a controversial topic in the sense that males and females judge sexual relations and sexuality often from very different perspectives. Sexual education has to take this difference into account. Through FGDs, first with females, then with males, and then with a mixed group to confront both sexes with the different outcomes of the separate discussions (listed on flip charts) it becomes easier to bring these differences in the open. Especially for teenagers, who may have many stereotypes about the other sex or be reluctant to discuss the topic openly (particularly girls), such a ‘multi-stage’ approach is useful.

Strengths and limitations

Implementation of FGDs is an iterative process; each focus group discussion builds on the previous one, with a slightly elaborated or better-focused set of themes for discussion. Provided the groups have been well chosen, in terms of composition and number (see below), FGDs can be a powerful research tool which provides valuable spontaneous information in a short period of time and at relatively low cost.

FGD should not be used for quantitative purposes, such as the testing of hypotheses or the generalisation of findings for larger areas, which would require more elaborate surveys.

However, FGDs can profitably complement such surveys or other, qualitative techniques. Depending on the topic, it may be risky to use FGDs as a single tool. In group discussions, people tend to centre their opinions on the most common ones, on ‘social norms’. In reality, opinions and behaviour may be more diverse. Therefore it is advisable to combine FGDs with at least some key informant and in-depth interviews. Explicitly soliciting other views during FGDs should be routine as well.

In case of very sensitive topics, such as sexual behaviour or coping with HIV/AIDS, FGDs may also have their limitations, as group members may hesitate to air their feelings and experiences freely. One possible remedy is the selection of participants who do not know each other (e.g., selection of children from different schools in FGDs about adolescent sexual behaviour), while assuring absolute confidentiality.

It may also help to alternate the FGD with other methods, for example, to precede it by a self-developed role play on sexual behaviour, or to administer a written questionnaire immediately after the FGD with open questions on sexual behaviour in which the participants can anonymously state all their questions and problems. This worked in Tanzania and Nepal.*


* The Tanzania-Netherlands Support Programme on AIDS, Mwanza Region, Tanzania (1990-2000+) and the Family Planning Association of Nepal Project on adolescent health in five districts of Nepal (1999-2003). The adolescent health section of WHO/HQ has developed a Narrative Research Method which is very well suited to help adolescents develop narratives and role plays about their interpretations of sexuality which can profitably precede the single sex and mixed FGDs: World Health Organization (1992) A story of the sexual experience of young people in eleven African countries; The Narrative Research Method. Geneva: WHO; World Health Organization (1993) The narrative research method; Studying behaviour patterns of young people by young people. A guide to its use, Geneva: WHO.

Another way to ensure confidentiality in a FGD on a sensitive topic is giving participants an option to introduce themselves under any name they would like to use (not necessarily their own). Further, before the discussion, it should be stressed that they may bring up experiences of friends and brothers/sisters as well as their own, and that it is not necessary to bring painful personal experiences in the open.*

II. HOW TO CONDUCT A FOCUS GROUP DISCUSSION

Determine the purpose

A FGD can be regarded as a mini-study. It therefore requires one or two clear objectives. (See Module 6.) These objectives will guide the research team in the formulation of discussion questions.

Situation analysis

Any FGD requires good knowledge of local conditions. Communities are seldom or never homogeneous. There are always differences between community members, for example in education, political power, gender, economic status and ethnic group. These differences will be reflected in their perceptions of the problems they suffer from and possible solutions. A researcher must be aware of these differences, otherwise (s)he may miss important groups of participants or obtain a hotchpotch of information. Similarly, (s)he must know which key persons or organisations could be good entry points for the selection of participants in the FGDs (e.g.: women’s groups, parent associations, youth clubs, etc.). If a FGD forms part of a bigger study, or project, it may be easy to define target groups for the discussions. Otherwise, the first task of the researcher(s) will be to explore the area and identify possible target groups. Interviews with some key informants and a rudimentary situation analysis are then indispensable. The situation analysis should preferably be carried out in a participatory way, with representatives of the study population on which the FGD focuses.

For example:

In an intervention study on sexual health among out-of-school youth in an urban area, the researcher first planned some interviews with key informants. He selected the leaders of a political youth club and of a Christian youth club and some teachers, with whom he thoroughly discussed his research topic. Through them he came in contact with youth of different backgrounds. He let each of the three groups, separated into boys and girls, draw maps of the town and asked them to mark places which they thought riskful in terms of sexual behaviour (easy contacts, unprotected sex). The drawings formed a good basis for further FGDs but also helped him to identify wider networks of adolescents at risk who had to be included in the study.

Points to be considered when preparing the FGD

Recruitment of participants:

Conducting the session

One of the members of the research team should act as ‘facilitator’ or ‘moderator’ for the focus group discussion. One should serve as ‘recorder’. The facilitator should preferably be as close as possible to the participants in their characteristics (same sex, roughly same age, etc.).

Functions of the facilitator

The facilitator should NOT act as an expert on the topic. His or her role is to stimulate and support discussion.

Functions of the recorder

The recorder should keep a record of the content of the discussion as well as emotional reactions and important aspects of group interaction. Assessment of the emotional tone of the meeting and the group process will enable you to judge the validity of the information collected during the FGD.

Items to be recorded include:

It is highly recommended that a tape-recorder be used to assist in capturing information. Even if a tape-recorder is used, notes should be taken as well, in case the machine malfunctions and so that information will be available immediately after the session for discussion.

If there is no reliable tape-recorder available, it is advisable to have two recorders.

A supplementary role for the recorder could be to assist the facilitator (if necessary) by drawing his or her attention to:

If necessary, the recorder could also help resolve conflict situations within the group that the facilitator finds difficult to handle on her own.

Number and duration of sessions

III. PROCESSING AND ANALYSIS OF RESULTS

IV. REPORT WRITING

Start with a description of the purpose of the FGDs, the selection and composition of the groups of FGD participants and a commentary on the group process, so the reader can assess the validity of the reported findings.

Present your findings, following your list of topics and guided by the objective(s) of your FGD.

Include quotations whenever possible as illustrations, particularly for key statements.

EXERCISE (3 hours total)

Conducting an FGD (75 minutes)

Participants working in groups of 6-12 conduct an FGD among themselves.

First let each group select a facilitator and a reporter

NB: It may be instructive to let the facilitator and reporter prepare the discussion guide for their group together with members of another group. Then the facilitator and reporter could come back to their own group with the guide when the FGD is to start. This resembles the real situation, where FGD members do not know which questions will be asked.

Analysis of data (30 minutes)

The reporter and facilitator analyse the notes and prepare the report.

Plenary (75 minutes)

The plenary sessions may include the following steps for each group:

  1. The recorder presents the report of the FGD of his or her group.
  2. Recorders may then ask for comments and reactions from members of the group.
  3. A discussion can be held concerning the effects of the role-played by the facilitator, the group process, and the skills of the recorder on the validity of the report of the FGD.
  4. If different groups discussed the same topic, the plenary can try to identify the different perspectives from which each group approached the topic.

If the group is big enough, one or more participants may act as observers and comment on the group process. Otherwise workshop facilitators can take that role. Sociograms are useful tools to record the flow of the discussion. (See Annex 10C.3)

REFERENCES:

Debus M, Porter Novelli (1986) Handbook for Excellence in Focus Group Research, Washington: Academy for Educational Development, Healthcom.

Helitzer-Alland, Makhambera Mano, Wangel AM (1994) Obtaining sensitive information: The need for more than Focus Groups. Reproductive Health Matters 3:75-82

Khan ME, Anker M, Patel BC, Barge S, Sadhwani H, Kohle R (1991) The Use of Focus Groups in Social and Behavioural Research: Some Methodological Issues. World Health Statistical Quarterly, 4 4:145-149.

Krueger RA (1988) Focus Groups. A practical guide for applied research. Newbury Park: Sage Publications. (In particular pp. 51-71 and 91-105.)

Morgan DL (1993) Successful Focus Groups: Advancing the State of the Art. California: Sage Publications Inc.

Powell RA and Simple HM (1996) Methodology Matters V: Focus Groups. International Journal for Quality in Health Care 8 (5):499-504.

Annex 10C.1: Example of a FGD on reasons why nurses leave the profession; discussion guide

  1. When did the leaving of nurses become a problem to you? When did it start?
  2. What, do you think, were (are?) the major reasons why they left the profession?
  3. What problems do you experience due to this ‘exodus’ of nurses?
  4. How did the Ministry of Health react?
  5. Did you take any initiatives to alert the Ministry to your problems?
  6. Is it only because so many nurses have left that you experience problems in your work, or are other factors playing a role as well?
  7. Do you have any concrete suggestions to the Ministry to improve your working situation?

./img/designandco_205_la_0.jpg

./img/designandco_206_la_6.jpg

./img/designandco_206_la_7.jpg

Remarks

Nurse A appears most talkative (and most angry). Nurse D is least talkative but important in the discussion as she tries to find a balance.

Trainer’s Notes

Module 10C: FOCUS GROUP DISCUSSIONS

Timing and teaching methods

½ hour Presentation on FGD
3 hours Exercise: Focus group discussions
3½ hours TOTAL TIME
Introduction and discussion
Exercise: Focus group discussion

Designing and Conducting Health Systems Research Projects
Part I: Proposal Development and Fieldwork

Module 11
SAMPLING

./img/designandco_209_la_0.jpg

NB: Development of a research process is a cyclical process. The double-headed arrows indicate that the process is never linear.

Module 11: SAMPLING

OBJECTIVES

At the end of the session you should be able to:

  1. Identify and define the population(s) to be studied.
  2. Identify and describe common methods of sampling.
  3. Discuss problems of bias that should be avoided when selecting a sample.
  4. List the issues to consider when deciding on sample size.
  5. Decide on the sampling method(s) and sample size(s) most appropriate for the research design you are developing.
  1. Introduction
  2. Sampling procedures
    1. Sampling methods for qualitative data
    2. Sampling methods for quantitative data
    3. Bias in sampling
    4. Ethical considerations
  3. Sample size

I. INTRODUCTION

WHAT is sampling?

SAMPLING is the process of selecting a number of study units from a defined study population.

Some studies involve only small numbers of people and thus all of them can be included. Often, however, research focuses on such a large population that, for practical reasons, it is only possible to include some of its members in the investigation. We then have to draw a SAMPLE from the total population.

In such cases we must consider the following questions:

The study population has to be clearly defined (for example, according to age, sex, and residence.) Otherwise we cannot do the sampling. Apart from persons, a study population may consist of villages, institutions, records, etc.

Each study population consists of STUDY UNITS. The way we define our study population and our study unit depends on the problem we want to investigate and on the objectives of the study.

For example:

./img/designandco_211_la_0.jpg

Representativeness

If researchers want to draw conclusions which are valid for the whole study population, which requires a quantitative study design, they should take care to draw a sample in such a way that it is representative of that population.

A REPRESENTATIVE SAMPLE has all the important characteristics of the population from which it is drawn.

For example:

If you intend to interview 100 mothers in order to obtain a complete picture of the weaning practices in District X you would have to select these mothers from a representative sample of villages. It would be unwise to select them from only one or two villages as this might give you a distorted (biased) picture. It would also be unwise to only interview mothers who attend the under-fives clinic, as those who do not attend this clinic may wean their children differently.

When using qualitative research approaches, however, representativeness of the sample is NOT a primary concern. In exploratory studies which aim at getting a rough impression of how certain variables manifest themselves in a study population or at identifying and exploring thus far unknown variables, you may try to select study units which give you the richest possible information: you go for INFORMATION-RICH cases!

For example:

Key informants should never be chosen at random, but purposively from among those who have the best possible knowledge, experience or overview with respect to topic of your study. Moreover they should be willing to share this information with you.

II. SAMPLING METHODS

As the rationale for the use of specific sampling methods in qualitative study designs is very different from the rationale underlying sampling methods in quantitative studies, we will discuss them separately.

Purposeful sampling strategies for qualitative studies

Qualitative research methods are typically used when focusing on a limited number of informants, whom we select strategically so that their in-depth information will give optimal insight into an issue about which little is known. This is called purposeful sampling. There are several possible strategies from which a researcher can choose. Often different strategies are combined, depending on the topic under study, the type of information wanted and the resources of the investigator(s).

This section owes much to Michael Quinn Patton who in his book Qualitative Evaluation and Research Methods (1990: 169-186) discusses a wide range of purposeful sampling techniques.

(1) Extreme case sampling

We have already discussed this type of sampling several times in Modules 9 and 10. Selection of extreme cases, such as good or very poor compliers to treatment, is a powerful and rapid strategy to identify contributing factors to poor compliance. In the same way, selection of well nourished children of the same age will help to identify contributing factors for malnutrition, and systematic comparison of a poorly and a well functioning district health team will give insight into factors that may contribute to the satisfactory functioning of DHTs.

Also ‘thick’ (elaborate) description of single, deviant cases may be useful. In that way AIDS was discovered in California (USA) as a newly emerging disease.

(2) Maximum variation sampling

If a researcher wants to obtain as complete as possible insight in a certain issue in all its variations, maximum variation sampling will be used.

For example, the stigma (see Module 8 Part III) of leprosy, or TB, HIV, epilepsy, is considered a complicating factor in the control of these diseases. In order to obtain insight in how stigma manifests itself in different cultures in males and females, in rural and urban areas, in well-to-do and poor patients, or in educated and illiterate ones, a researcher has to take care that all these groups are included in the sample. To assess whether social distance influences stigma, one could also interview blood relatives (parents or children), spouses, friends, near neighbours of patients and more distant community members. For leprosy and TB patients, it is useful to interview patients on treatment as well as patients declared cured, to assess if any reversal of stigma is experienced when patients’ conditions improve. If a researcher is interested in specific groups and interviews a fixed number per group, this type of sampling is also called quota sampling.

Note:

Purposeful sampling should not be haphazard. Care should be taken that for different categories of informants, selection rules are developed to prevent the researcher from sampling according to personal preference.

In the study on stigma it would, for example, be possible to select the categories of patients one would like to focus on (male/ female, on treatment and released from treatment) in a systematic way from patient registers in a number of selected clinics. One could e.g., take all, or every second or third patient in the desired categories. The clinics may have been selected purposefully (town/rural, specific ethnic areas, poor/rich areas) but if there is choice the researcher may make lists for each category and select at random (see next section) the desired number of clinics. For identifying social distance variation, selection rules will be much more pragmatic: spouses, relatives and neighbours who are available will be interviewed in a careful, indirect way in order not to hurt the patient’s interests. Still, however interesting data maximum variation sampling may generate, highlighting different factors and different perspectives, it does not provide representative data for the total population.

(3) Homogeneous sampling

Sometimes a researcher would like to have specific information about one particular group only, for example, a group that, for unclear reasons, is more at risk than others:

In country S, death registers indicate that suicide among adolescents is on the increase at an alarming rate. Within that group twice as many boys as girls commit suicide. Researchers may therefore want to concentrate on the boys to identify what factors may be contributing to these suicides, conducting in-depth interviews with parents, other close relatives, teachers and friends of a number of boys who committed suicide.

In focus group discussions (FGDs), we usually select homogeneous groups because participants discuss more freely when they are amongst people of similar social status. (See Module10C.)

(4) Typical case sampling

It is sometimes illustrative to describe in-depth some cases which are ‘typical’ for the group one is interested in. For example, one may describe a ‘typical’ family in a rural village in country A, or a ‘typical’ young school leaver who migrates from the rural areas to town in search of work, or ‘typical’ health problems of miners or malnourished children.

Such descriptions are merely illustrative; they cannot be generalised for the whole group. Typical examples can either be selected with co-operation of key informants who know the study population well, or from a survey that helps to identify the normal distribution and the modus of the characteristics we are interested in.

(5) Critical case sampling

Critical cases are those who ‘can make the difference’ with respect to an intervention you want to introduce or to evaluate.

For example, you have developed a local weaning food that, you hope, is affordable to all mothers. Before propagating it at a larger scale through MCH clinics you first interview and observe some low-income mothers as ‘test cases’. If they manage to produce and use it, this will indicate that it is affordable to the whole group.

(6) Snowball or chain sampling

This approach is particularly suitable for locating key informants or critical cases. You start with one or two information-rich key informants and ask them if they know persons who know a lot about your topic of interest. If a particular person is recommended to you by two or three different people you can be quite sure that he or she will be a valuable key informant.

The same approach can be used if an in-depth interview leads to discoveries, which seem rewarding to follow-up by a number of interviews with an additional group of informants.

For example, in an exploratory study on coping behaviour among AIDS orphans it seemed that child-headed households managed by girls survived better than those managed by boys. The researcher then interviewed more adolescent boys and girls heading households, to see whether this gender difference in ability to cope was real, and how it could be explained.

Patton (1990: 179) labels this kind of additional sampling during the study opportunistic sampling.

Flexible sampling procedures, steered by the data one collects (in relation to the objectives) forms a major opportunity for qualitative researchers to optimally exploit the field situation and explore ‘in-depth’ interesting issues which present themselves. It is exactly the opposite of the random sampling techniques discussed in the next section of this module, which are used in quantitative research to ensure representativeness of the sample for the total population. Still, if qualitative researchers can choose from a group of seemingly similar informants they will also sample at random (see example 2).

Note:

Purposeful sampling is NOT the same as convenience sampling.* CONVENIENCE SAMPLING is a method in which for convenience sake the study units that happen to be available at the time of data collection are selected in the sample. This may happen at the beginning of a study when researchers are merely orienting themselves, or, when there are many similar informants and the researchers do not (yet) have a preference for specific categories. When there seems no other choice (no one else available for an interview) researchers may also sample conveniently.


* In the earlier version of this module, following epidemiological tradition, all ‘non-random’ sampling methods were categorised under the headings of ‘convenience’ or ‘quota’ sampling. This injustice towards purposeful sampling techniques has been corrected in the present version. In HSR, purposeful and random sampling techniques are used equally.

2. Random sampling strategies to collect quantitative data

If the aim of a study is to measure variables distributed in a population (e.g., diseases) or to test hypotheses about which factors are contributing significantly to a certain problem, we have to be sure that we can generalise the findings obtained from a sample to the total study population. Then, purposeful sampling methods are inadequate, and probability- or random sampling methods have to be used.

PROBABILITY SAMPLING involves using random selection procedures to ensure that each unit of the sample is chosen on the basis of chance. All units of the study population should have an equal, or at least a known chance of being included in the sample.

Probability sampling requires that a listing of all study units exists or can be compiled. This listing is called the sampling frame.

The following probability sampling methods will be discussed:

(1) Simple random sampling

This is the simplest form of probability sampling. To select a simple random sample you need to:

For example, a simple random sample of 50 students is to be selected from a school of 250 students. Using a list of all 250 students, each student is given a number (1 to 250), and these numbers are written on small pieces of paper. All the 250 papers are put in a box, after which the box is shaken vigorously, to ensure randomisation. Then, 50 papers are taken out of the box, and the numbers are recorded. The students belonging to these numbers will constitute the sample.

(2) Systematic sampling

In SYSTEMATIC SAMPLING individuals are chosen at regular intervals (for example every fifth) from the sampling frame. Ideally we randomly select a number to tell us where to start selecting individuals from the list.

For example, a systematic sample is to be selected from 1200 students of a school. The sample size selected is 100. The sampling fraction is:

./img/designandco_216_la_0.jpg

The sampling interval is therefore 12.

The number of the first student to be included in the sample is chosen randomly, for example by blindly picking one out of twelve pieces of paper, numbered 1 to 12. If number 6 is picked, then every twelfth student will be included in the sample, starting with student number 6, until 100 students are selected: the numbers selected would be 6, 18, 30, 42, etc.

Systematic sampling is usually less time consuming and easier to perform than simple random sampling. However, there is a risk of bias, as the sampling interval may coincide with a systematic variation in the sampling frame. For instance, if we want to select a random sample of days on which to count clinic attendance, systematic sampling with a sampling interval of 7 days would be inappropriate, as all study days would fall on the same day of the week (e.g., Tuesdays only, which might be a market day).

(3) Stratified sampling

The simple random sampling method described above has as disadvantage that small groups in which the researcher is interested may hardly appear in the sample.

If it is important that the sample includes representative study units of small groups with specific characteristics (for example, residents from urban and rural areas, or different religious or ethnic groups), then the sampling frame must be divided into groups, or STRATA, according to these characteristics. Random or systematic samples of a pre-determined size will then have to be obtained from each group (stratum). This is called STRATIFIED SAMPLING.

Stratified sampling is only possible when we know what proportion of the study population belongs to each group we are interested in.

An advantage of stratified sampling is that we can take a relatively large sample from a small group in our study population. This allows us to get a sample that is big enough to enable us to draw valid conclusions about a relatively small group without having to collect an unnecessarily large (and hence expensive) sample of the other, larger groups. However, in doing so, we are using unequal sampling fractions and it is important to correct for this when generalising our findings to the whole study population.

For example, a survey is conducted on household water supply in a district comprising 20,000 households, of which 20% are urban and 80% rural. It is suspected that in urban areas the access to safe water sources is much more satisfactory. A decision is made to include 100 urban households (out of 4000, which gives a 1 in 40 sample) and 200 rural households (out of 16000, which gives a 1 in 80 sample). Because we know the sampling fraction for both strata, the access to safe water for all the district households can be calculated after the study (by multiplying the findings for the urban households by 40 and those for the rural households by 80, and then calculating statistics for the total sample).

(4) Cluster sampling

It may be difficult or impossible to take a simple random sample of the units of the study population at random, because a complete sampling frame does not exist. Logistical difficulties may also discourage random sampling techniques (e.g., interviewing people who are scattered over a large area may be too time-consuming). However, when a list of groupings of study units is available (e.g., villages or schools) or can be easily compiled, a number of these groupings can be randomly selected.

The selection of groups of study units (clusters) instead of the selection of study units individually is called CLUSTER SAMPLING.

Clusters are often geographic units (e.g., districts, villages) or organisational units (e.g., clinics, training groups).

For example, in a study of the knowledge, attitudes and practices (KAP) related to family planning in rural communities of a region, a list is made of all the villages. Using this list, a random sample of villages is chosen and all study units in the selected villages are interviewed.

(5) Multi-stage sampling

In very large and diverse populations sampling may be done in two or more stages. This is often the case in community-based studies, in which people are to be interviewed from different villages, and the villages have to be chosen from different areas. This type of sampling is frequently used in HSR.

For example, in a study of utilisation of pit latrines in a district 150 homesteads are to be visited for interviews with family members as well as for observations on types and cleanliness of latrines. The district is composed of 6 wards and each ward has between 6 and 9 villages.

The following four-stage sampling procedure could be performed*:

  1. Select 3 wards out of the 6 by simple random sampling.
  2. For each ward select 5 villages by simple random sampling (15 villages in total).
  3. For each village select 10 households. Since simply choosing households in the centre of the village would produce a biased sample, the following sampling procedure is proposed:
  4. Decide beforehand whom to interview (for example the head of the household, if present, or the oldest adult who lives there and who is available).

* This is an adaptation of the method developed by the EPI division in WHO Geneva to measure EPI coverage in districts.

A MULTI-STAGE SAMPLING procedure is carried out in phases and it usually involves more than one sampling method.

The main advantages of cluster and multi-stage sampling are that:

The main disadvantage of this type of sampling is that:

Compared to simple random sampling, there is a larger probability that the final sample will not be representative of the total study population. The likelihood of the sample not being representative depends mainly on the number of clusters that is selected in the first stage. The larger the number of clusters, the greater is the likelihood that the sample will be representative. Further, the sampling units at community level should be selected randomly (avoid convenience sampling!).

3. Bias in sampling

BIAS in sampling is a systematic error in sampling procedures, which leads to a distortion in the results of the study.

Module 10 discussed how the use of faulty data collection tools would lead to biased results. Bias can also be introduced as a consequence of improper sampling procedures, which result in the sample not being representative of the study population.

For example, a study was conducted to determine the health needs of a rural population in order to plan primary health care activities. However, a nomadic tribe, which represented one third of the total population, was left out of the study. As a result the study did not give an accurate picture of the health needs of the total population.

There are several possible sources of bias that may arise when sampling. The most well known source is non-response.

Non-response can occur in any interview situation, but it is mostly encountered in large-scale surveys with self-administered questionnaires. Respondents may refuse or forget to fill in the questionnaire. The problem lies in the fact that non-respondents in a sample may exhibit characteristics that differ systematically from the characteristics of respondents.

There are several ways to deal with this problem and reduce the possibility of bias:

Note:

The bigger the non-response rate, the more necessary it becomes to take remedial action. It is important in any study to mention the non-response rate and to honestly discuss whether and how the non-response might have influenced the results.

Other sources of bias in sampling may be less obvious, but at least as serious:

4. Ethical considerations

If the recommendations from a study will be implemented in the entire study population, one has the ethical obligation to draw a sample from this population in a representative way. If during the research new evidence suggests that the sample was not representative, this should be mentioned in any publication concerning the study, and care must be taken not to draw conclusions or make recommendations that are not justified.

GROUP WORK, PART I (2 hours)

  1. Develop in your working group:
    • a definition of your (different) study population(s);
    • a definition of your (different) study units (people, clinics, records, etc);
    • appropriate sampling procedures for your study, taking into account whether you use qualitative and/or quantitative research methods. State how you will try to avoid possible bias.
  2. Prepare a summary on a flipchart for use in the exercise ‘Commenting on each others sampling procedures’ and in the plenary discussion (after group work on sample size).

III. SAMPLE SIZE

Having decided how to select our sample, we now have to determine our sample size.

1. Sample size in qualitative studies

There are no fixed rules for sample size in qualitative research. The size of the sample depends on WHAT you try to find out, and from what different informants or perspectives you try to find that out.

For example, if you want to explore how you can involve mothers in your HC catchment area more effectively in early detection and treatment of pneumonia, you might decide to conduct some FGDs to assess mothers’ knowledge, attitudes and practices with respect to pneumonia. You could start with two FGDs among lowly educated mothers and two among mothers with more education (who usually are of higher socio-economic status). If the different data sets reconfirm each other you may stop at this point and start a small scale intervention; otherwise you conduct one or two FGDs more till you reach the point of redundancy∗: no new data comes up any more.

If your research objective is more complex e.g., attitudes of males and females towards family planning, and has policy implications for a larger area, your sample will be bigger. You might start with four FGDs, two among males and two among females, subdivided according to socio-economic status. Among male participants you could then select 5-10 users (or spouses of female users) for in-depth interviews on RH history of the couple and reasons for use and non-use of FP. For women, the same procedures can be followed. If possible, you may interview some couples, first separately, then together. Depending on ethnic differences and urban-town differences in attitudes and practices, the ‘clusters’ of FGDs and in-depth interviews can be expanded.

In exploratory studies, the sample size is therefore estimated beforehand as precisely as possible, but not determined.

Patton (1990: 183-186) stresses that richness of the data and analytical capability of the researcher determine the validity and meaningfulness of qualitative data more than sample size. Still, sampling procedures and sample size should always be carefully explained in order to avoid the allusion of haphazardness. Careful analysis of different complementing data sets can result in some plausible generalisations but without ‘proving’ them in a mathematical sense.


* Lincoln and Guba, in Patton (1990: 185)

2. Sample size in quantitative studies

For quantitative studies, calculations can be made which indicate the desirable sample size. The principles of such calculations will be discussed below.

It is a widespread belief among researchers that the bigger the sample, the better the study becomes. This is not necessarily true. In general it is much better to increase the accuracy and richness of data collection (for example by improving the training of interviewers or by better pre-testing of the data collection tools) than to increase sample size after a certain point. Also, it is better to make extra efforts to get a representative sample rather than to get a very large sample.

The following general rules may help to determine the desirable sample size of any given study:

For example, after conducting FGDs and in-depth interviews in the study on attitudes of men and women towards family planning (see Section III.1 of this module) you might decide to conduct a bigger survey. If your exploratory study revealed that age and education appear to be important factors determining FP use, you will be interested in comparing FP use in groups with different levels of education and of different ages. If you split each of these variables up in three categories, and you select four categories of informants (male users/ spouses of female users; female users; male non-users, female non-users) you would have 12 cells in each table. In order to obtain 5-10 answers per cell you would require 60-120 informants in each research area.

As other variables may have more categories, you may attempt to select 120 informants in rural areas and 120 in urban areas. However, as FP use may not be equally distributed in a population (e.g., 25% users, 75% non-users) your sample will have to be bigger than 120 in order to obtain the desired sample size. HOW big it should be can be calculated. Still, the desirable sample size can not always be achieved for lack of resources such as time, manpower and money. This constraint applies to quantitative as well as qualitative studies.

Therefore the eventual sample size is usually a compromise between what is DESIRABLE and what is FEASIBLE.

Sample size calculations

In quantitative studies, researchers will perform sample size calculations before embarking on the project to find the desirable sample size. The formulae for calculating a desired sample size are listed in Annex 11.2. They are divided into two categories, depending on whether the study:

The formulae can only be used if you have a rough idea about the outcome of the study, which is not always the case. It is always advisable to call upon a statistician or an experienced researcher who can help you in choosing and using the appropriate formulae.

We will look at a few examples to highlight some important issues.

(1) Descriptive studies with one group

Example

In a descriptive study in a certain village we want to measure, with a certain precision, the proportion of children aged 12-23 months who are immunised against measles, using a simple random sample. The following steps should be taken:

  1. Estimate how big the proportion might be (say 80%).
  2. Choose the margin of error you will allow in the estimate of the proportion (say ± 10%). This means that, if the survey reveals that indeed 80% of the children have been vaccinated, this proportion will probably be between 70 and 90% in the whole study population from which the sample was drawn.
  3. Choose the level of confidence at which you want to be able to state that the vaccination coverage in the whole population is indeed between 70 and 90%. You can never be 100% sure. Do you want to be 95% sure? or 99%? A commonly used confidence level in HSR studies is 95%.

    The formula for calculating the sample size for the single proportion expressed as a percentage is presented in Annex 11.2 (1.3).

However, at present it is usually not necessary to calculate the desirable sample size by hand. There are computer programmes to assist us (for example Epi Info). In Annex 11.3 two tables are presented, which will help you to calculate the desired sample size in the most common quantitative HSR studies. Both take a confidence level of 95% (P<0.05) as point of departure. In the measles example, you have to go down the first column till you find the 80% for the estimated vaccination coverage. If you allow a margin of error of 10 + or – the estimated 80%, you look in that column and find a desired sample size of 64. However, if you would allow a margin of error of only + or - 5%, the sample size would increase to 256, and if you would like to be 95% sure that the measles vaccination coverage is between 79 and 81%, you would need to check 6400 children. The smaller your margin of error (also called confidence interval) the bigger your sample size needs to be. If you would like to increase your confidence level from 95 to 99%, the sample would have to increase even more. But then researchers of course start to look at feasibility: do we really need that precision? Most HSR studies will be satisfied with the 95% confidence level.

Note also that in general you need more precision (or a smaller margin of error) if the estimated proportion is very small. This may be the case, for example, for the proportion of HIV* women or the maternal mortality rate in a population.

Table 11.1: Required sample size for studies of HIV prevalence in pregnant women

./img/designandco_222_la_0.jpg

The table shows that in district A, where HIV is less prevalent, a smaller margin of error is desired and, therefore, the required sample is larger. (Annex 11.4 explains how these sample sizes were calculated).

Table (a) in Annex 11.3 will, however, help you again to identify the required sample sizes in a simple way. The first line of table (a) gives you the required sample sizes for an estimated prevalence of 1%. You would need a sample of 1584 women to be 95% sure that HIV prevalence in District A would be between 0.5 and 1.5% (so 1% plus or minus 0.5%). For district B, with 10% estimated HIV prevalence, and a margin of error plus or minus 5%, the required sample size would be 144.

(2) Comparing two groups for a significant difference

In comparative studies one usually wants to demonstrate that there is a significant difference between two groups. In this type of study the sample size depends primarily on the estimated size of the difference between the two groups that are compared. The larger the difference, the smaller will be the sample that is needed to show this difference. Second, the sample size depends on how large we want the probability to be that we indeed will find a significant difference.

The larger the sample size, the larger will be the probability of finding a significant difference. In the case of many variables, the one with the smallest estimated difference between the groups should be used as the basis for calculating the sample size, as it requires the largest sample. Still, researchers will discuss whether that variable is important enough to maintain if the sample would become too big to handle. They may also decide to measure only some variables in a large sample while measuring variables that require a smaller sample in a sub-sample to save costs and ensure the feasibility of the study.

Example:

In a study, a comparison will be made between the feeding patterns of well-nourished and malnourished children of 12 to 17 months. It is expected that of the well-nourished children 90% are breastfed whereas of the malnourished children approximately 50% are breastfed. The sample size in each group of children needs to be at least 15 to show a significant difference.

However, if 90% of the well-nourished children and 80% of the malnourished children were breast fed, the sample size would need to be at least 175 in each group to show a significant difference. (Annex 11.4 explains how these sample sizes can be calculated).

Table (b) in Annex 11.3 will again help you to identify the required sample size in a simple way. Compare 90% in the column with 50% in the row (or vice versa) and you will find a required sample size of 22. In this case a slightly different formula was used as the one presented in presented in Annex 11.4. Also the calculated sample size for the second example would be bigger (262) when using the table.

Note that it may be useful to conduct sample size calculations for each of the objectives of the study. These calculations may reveal, for instance, that some but not all objectives can be met. Or they may indicate that some variables need only to be measured on a sub-sample.

GROUP WORK, PART II (1 hour)

  1. Determine the sample size requirements for the study population(s) defined in the previous group work session. Consider the issues discussed in the module when establishing the desirable sample size (s). Use the tables in Annex 11.3 if sample size calculations have to be made.
  2. Determine the feasible sample size after taking into account available time, manpower, transport and money.

    If there is a large discrepancy between the desirable and the feasible sample size you should look for a compromise and, if necessary, adjust the objectives of your study.

  3. Put a summary of your group’s work on flipchart for use in the exercise below and in the plenary discussion that will follow.
  4. It will be easier for you to develop a realistic Plan for Data Collection (Module 12) as well as the Budget for your project (Module 16) if you already know as precisely as possible where and from whom data are going to be collected. Therefore it is important that you select the sample for your study immediately after the plenary session on sampling. If you choose a multi-stage sampling strategy you may find at this stage that it is only possible to partially draw the sample.

EXERCISE (½ hour)

  1. Examine the definitions of study population and study units, the sampling procedures and the proposed sample size developed by another group.
  2. Identify possible sources of bias in sampling and suggest improvements.
  3. Put your comments on flipchart for presentation in plenary.

REFERENCES

All epidemiological and social science research handbooks mentioned in the references in Module 9 are dealing with sampling procedures and sample size. Moreover you can consult:

Swinscow TDV, Revised by MJ Campbell (1998) Statistics at Square One. London: BMJ Publishing Group. (9th ed.)

Campbell MJ, Machin D (1993) Medical Statistics: A Common Sense Approach. Clichester: John Wiley. (2nd ed.)

Annex 11.1: How to use a random number table*

  1. First, decide how large a number you need. Next, count if it is a one, two or larger digit number. For example, if your sampling frame consists of 10 units, you must choose from numbers 1-10, (inclusive). You must use two digits to ensure that 10 has an equal chance of being included.

    You also use two digits for a sampling frame consisting of 0-99 units.

    If, however, your sampling frame has 0-999 units, then you obviously need to choose from three digits. In this case, you take an extra digit from the table to make up the required three digits. For example, the number in columns 10,11, row 27: 43, would become 431; going down, the next numbers would be 107, 365 etc.

    You would do the same if you needed a four-digit number, for a sampling frame 0-9999 units. In our example of the number on columns 10, 11, 12, row 27 of the table: 431, this would now become 4316, the next down 1075, and so on.

  2. Decide beforehand whether you are going to go across the page to the right, → down the page ↓, across the page to the left, ← or up the page. ↑
  3. Without looking at the table, and using a pencil, pen, stick, or even your finger, pin-point a number.
  4. If this number is within the range you need, take it. If not, continue to the next number in the direction you chose before-hand, (across, up or down the page), until you find a number that is within the range you need.

    For example if you need a number between 0-50 and you began at column 21, 22, row 21 you get 74 which is obviously too big. So you could go down (having decided beforehand to go down) to 97, also too big, to 42, which is acceptable, and select it.


*The random number table on the following page has been taken from Hill AB (1977) A Short Textbook of Medical Statistics. London: Hodder and Stoughton, 1977:306-7.

Random sampling numbers

./img/designandco_226_la_0.jpg

Annex 11.2: Formulae for calculating sample size*

The formulae for calculating required sample size are divided in two categories:

  1. For studies trying to measure one variable with a certain precision.
  2. For studies seeking to demonstrate a significant difference between two groups.
1. Measuring one variable

In the formulae below the following abbreviations are used:

n - sample size

s - standard deviation

e - required size of standard error
(in the text of the module the term ‘margin of error’ is used for ±2 times the size of the standard error if a precision of 95% is required)

r - rate

p - percentage

1.1 Single mean

In a study the mean weight of newborn babies will be determined. The mean weight is expected to be 3000 grams. Weights are approximately normally distributed and 95% of the birth weights are probably between 2000 and 4000 grams; therefore the standard deviation would be 500 grams. The desired 95% confidence interval is 2950 to 3050 grams, so the standard error would be 25 grams. The required sample size would be:

./img/designandco_227_la_0.jpg

1.2 Single rate

The maternal mortality rate in a country is expected to be 70 per 10,000 live births. A survey is planned to determine the maternal mortality rate with a 95% confidence interval of 60 to 80 per 10,000 live births. The standard error would therefore be 5/10,000. The required sample size would be:

./img/designandco_227_la_1.jpg

1.3 Single proportion

The proportion of nurses leaving the health services within three years of graduation is estimated to be 30%. A study that aims to find causes for this, also aims to determine the percentage leaving the service with a confidence interval of 25% to 35%. The standard error would therefore be 2.5%. The required sample size would be:

./img/designandco_227_la_2.jpg


* Modified from Kirkwood B (1988) Essentials of Medical Statistics. Oxford: Blackwell Scientific Publications, 1988.

1.4 Difference between two means (sample size in each group)

The difference of the mean birth weights in district A and B will be determined. In district A the mean is expected to be 3000 grams with a standard deviation of 500 grams (as in 1.1). In district B the mean is expected to be 3200 grams with a standard deviation of 500 grams. The difference in mean birth weight between districts A and B is therefore expected to be 200 grams. The desired 95% confidence interval of this difference is 100 to 300 grams, giving a standard error of the difference of 50 grams. The required sample size would be:

./img/designandco_228_la_0.jpg

1.5 Difference between two rates (sample size in each group)

The difference in maternal mortality rates between urban and rural areas will be determined. In the rural areas the maternal mortality rate is expected to be 100 per 10,000 and in the urban areas 50 per 10,000 live births. The difference is therefore 50 per 10,000 live births. The desired 95% confidence interval is 30 to 70 per 10,000 live births giving a standard error of the difference of 10/10,000. The required sample size would be:

./img/designandco_228_la_1.jpg

1.6 Difference between two proportions (sample size in each group)

The difference in the proportion of nurses leaving the service is determined between two regions. In one region 30% of the nurses are estimated to leave the service within three years of graduation, in the other region 15%, giving a difference of 15%. The desired 95% confidence interval for this difference is 5% to 25%, giving a standard error of 5%. The sample size in each group would be:

./img/designandco_228_la_2.jpg

2. Significant difference between two groups

In the formulae below the following abbreviations are used:

n, samples size

s, standard deviation

e, required size of standard error

r, rate

p, percentage

u, one-sided percentage point of the normal distribution, corresponding to 100% - the power. The power is the probability of finding a significant result. (e.g. if the power is 75%, u = 0.67).

v, percentage point of the normal distribution, corresponding to the (two-sided) significance level. (e.g. if the significant level is 5% (as usual), v = 1.96.)

2.1 Comparison of two means (sample size in each group)

The birth weights in district A and B will be compared. In district A the mean birth weight is expected to be 3000 grams with a standard deviation of 500 grams. In district B the mean is expected to be 3200 grams with a standard deviation of 500 grams (see 1.4). The required sample size to demonstrate (with a likelihood of 90%) a significant difference between the mean birth weights in district A and B would be:

./img/designandco_229_la_0.jpg

2.2 Comparison of two rates (sample size in each group)

The maternal mortality rates in urban and rural areas will be compared. In the rural areas the maternal mortality rate is expected to be 100 per 10,000 and in the urban areas 50 per 10,000 live births (compare to 1.5). The required sample size to show (with a likelihood of 90%) a significant difference between the maternal mortality in the urban and rural areas would be:

./img/designandco_229_la_1.jpg

2.3 Comparison of two proportions (sample size in each group)

The proportion of nurses leaving the health service is compared between two regions. In one region 30% of nurses is estimated to leave the service within three years of graduation, in the other region it is probably 15%.

The required sample size to show with a 90% likelihood that the percentage of nurses is different in these two regions would be:

./img/designandco_229_la_2.jpg

(a) Sample size for measuring proportions in one group
Percentage 95% confidence level (+ or – the percentage of allowed error in column heading)

./img/designandco_230_la_0.jpg

The percentages in the column heading are 2 x standard error (e in formula)

(b) Sample size for comparison of proportions in two groups

./img/designandco_230_la_1.jpg

       u = 1.28 Power = 90%

       v = 1.96 Significance: P<0.05

(u+v)2= 10.5

Annex 11.4: Explanation of sample size calculations given in the text

1. Prevalence of HIV (p.13)

District A: the estimated HIV+ proportion is 1% = 0.01. As the 95% confidence interval is the proportion ± 2 x the standard error, the standard error is 0.25% = 0.0025.

./img/designandco_231_la_0.jpg

District B: the estimated HIV proportion is 10% = 0.1 Procedures followed are the same as above

2. Feeding patterns in malnourished and well-nourished children (p.14)

Formula used: no. 2.3 in Annex 11.2

./img/designandco_231_la_1.jpg

If the power is 75%, u = 0.67 and (u + v)2 = 6.9;
if the power is 90%, u = 1.28 and (u + v)2 = 10.5.
(The power is the probability of finding a significant result).

If the power is increased from 75% to 90%, the sample size is increased 10.5/6.9 (i.e. 1.5 times.)

Trainer’s Notes

Module 11: SAMPLING

Timing and teaching methods

The topic on sampling has two major components (sampling procedures and sample size), which preferably should be presented in two separate sessions. These sessions will require 6½ hours in total.

Materials

Introduction to Sampling Procedures (Part II of Module 11)

Timing and teaching methods

1 hour Introduction and discussion
2 hoursGroup work
Introduction and discussion
Group work, Part I

Introduction to Sample Size (Part III of Module 11)

Timing and teaching methods

½ hour Introduction and discussion
1 hour Group work
½ hour Exercise
1-½ hours Group reports in plenary
Introduction and discussion
Group work, Part II
Exercise
Plenary

Designing and Conducting Health Systems Research Projects
Part I: Proposal Development and Fieldwork

Module 12:
PLAN FOR DATA COLLECTION

./img/designandco_235_la_0.jpg

NB: Development of a research process is a cyclical process. The double-headed arrows indicate that the process is never linear.

Module 12: PLAN FOR DATA COLLECTION

OBJECTIVES

At the end of this session, you should be able to:

  1. Identify and discuss the most important points to be considered when starting to plan for data collection.
  2. Determine what resources are available and needed to carry out your study.
  3. Describe typical problems that may arise during data collection and how they may be solved.
  4. Prepare a plan for data collection for the research proposal you are developing.
  1. Introduction
  2. Stages in the data collection process

I. INTRODUCTION

Where are we now in the development of our research proposal?

Look again at the flowchart in Module 7 that introduces research design. We have just finished four crucial theoretical sessions, in which we have defined:

Now we enter a new phase in the development of our research methodology: planning our fieldwork. We have to plan concretely how we will collect the data we need (Modules 12 and 1 5), how we will analyse it (Module 13), and how we can test the most crucial parts of our methodology (Module 14). Finally, we will have to develop a work plan to budget the resources necessary to carry out the study (Module 16).

A PLAN FOR DATA COLLECTION can be made in two steps:

  1. Listing the tasks that have to be carried out and who should be involved, making a rough estimate of the time needed for the different parts of the study, and identifying the most appropriate period in which to carry out the research (the present module).
  2. Actually scheduling the different activities that have to be carried out each week in a work plan (Module 15).

Before the proposal development workshop is finished, a pre-test of the data collection and data analysis procedures should be made. The advantages of conducting a pre-test before we finalise our proposal is that we can draft the work plan and budget based on more realistic estimates, as well as revise the data collection tools before we submit the proposal for approval. However, further pre-testing and planning of the research have to be done in the field (Module 14).

Why should you develop a plan for data collection?

A plan for data collection should be developed so that:

It is likely that while developing a plan for data collection you will identify problems (such as limited manpower), which will require modification of the proposal. Such modifications might include adjustment of the sample size or extension of the period for data collection.

II. STAGES IN THE DATA COLLECTION PROCESS

What are the main stages in the data collection process?

Three main stages can be distinguished:

Stage 1: Permission to proceed

Stage 2: Data collection

Stage 3: Data handling

Stage 1: PERMISSION TO PROCEED

Consent must be obtained from the relevant authorities, individuals and the community in which the project is to be carried out. This may involve organising meetings at national or provincial level, at district and at village level. For clinical studies this may also involve obtaining written informed consent.

Most likely the principal investigator will be responsible for obtaining permission to proceed at the various levels. The Health Research Unit in the Ministry of Health or the institution organising the course may assist in obtaining permission from the national level.

Note:

In many countries research proposals have to be screened for scientific and ethical integrity by national research councils. However, proposals developed during workshops may be exempted from this procedure if the research is considered as a training exercise and the research council is assured that the course facilitators and Health Research Unit have screened the methodology during the workshop.

Stage 2: DATA COLLECTION

When collecting our data, we have to consider:

1. Logistics of data collection

WHO will collect WHAT data?

When allocating tasks for data collection, it is recommended that you first list them. Then you may identify who could best implement each of the tasks. If it is clear beforehand that your research team will not be able to carry out the entire study by itself, you might plan to look for research assistants to assist in relatively simple but time-consuming tasks.

For example, in a study into the effects of improvements in delivery care on utilisation of these services the following task division could be proposed:

./img/designandco_239_la_0.jpg

HOW LONG will it take to collect the data for each component of the study?

Step 1: Consider:

Step 2: Calculate the number of interviews that can be carried out per person per day (e.g., 4)

Step 3: Calculate the number of days needed to carry out the interviews. For example:

Step 4: Calculate the time needed for the other parts of the study, (for example, 10 days)

Step 5: Determine how much time you can devote to the study. Since the research team usually consists of very busy people, it is unlikely that team members can spend more than 30 working days on the entire study:

Note:

The recruitment of research assistants for data collection may, on the one hand, relieve the research team, but, on the other hand, the training and supervision of research assistants require time (see Annex 12.1). The team has to carefully weigh advantages and disadvantages. If none of the team members has previous research experience, they might prefer designing a study, which they can carry out themselves, without or with only minimal assistance.

If research assistants are required, consider to what extent local health workers can be used. They have the advantage of knowing the local situation. They should never be involved, however, in conducting interviews to evaluate the performance of their own health facility. Local staff from related services (teachers, community development) or students might help out. Sometimes village health workers or community members can collect certain parts of the data.

Note:

It is always advisable to slightly overestimate the period needed for data collection to allow for unforeseen delays.

In WHAT SEQUENCE should data be collected?

In general, it is advisable to start with analysis of data that is already available. This is essential if the sample of respondents is to be selected from the records. Another rule of thumb is that qualitative research techniques (such as key informant interviews, focus group discussions) that are devised to focus the content of questionnaires for interviewing larger groups of informants should be carried out before finalisation of these questionnaires. FGDs designed to provide feedback on issues raised in larger surveys, should, logically, be conducted after preliminary analysis of the questionnaires.

To use time and transport efficiently, data to be drawn from different sources but in one locality should be collected at the same time. (For example, interviews with health staff in a Health Centre, observations of equipment available in the Health Centre and interviews with mothers living around the Health Centre should be scheduled together.)

WHEN should the data be collected?

The type of data to be collected and the demands of the project will determine the actual time needed for the data to be collected. Consideration should be given to:

Note:

The field visit to obtain consent from local authorities for the research may also be used to obtain necessary details about the best period for data collection and availability of local resources (research assistants, transport), if required.

2. Ensuring quality

It is extremely important that the data we collect are of good quality, that is, reliable and valid. Otherwise we will come up with false or misleading conclusions.

In the previous modules possible sources of data distortion (bias) have been discussed which we should try to prevent. They include:

There are a number of possible measures that can be taken to prevent and to partly correct such distortions. Remember: prevention is FAR better than cure!! Cure is usually surgery: you may have to cut out the bad parts of your data or, at best, devise crutches.

Measures to help ensure good quality of data:

Stage 3: DATA HANDLING

Once the data have been collected and checked for completeness and accuracy, a clear procedure should be developed for handling and storing them:

GROUP WORK (1¾ hours)

Make a plan for data collection, considering the points below:

  1. Permission to proceed (10 minutes)
    • Which organisations or individuals should be approached to obtain permission to proceed with the research project?
    • Who will ask for permission? When? What procedures will be followed?
  2. Data Collection (1-¼ hours)
    • List the different components of your study and the number of interviews, observations or measurements required.
    • Calculate for each component how many interviews or observations one person can do per day.
    • Decide if you need extra assistance, considering the fact that you, as a research team, will probably not be able to spend more than approximately 20 working days per person in the field and 5 days per person for preparation of the fieldwork.
    • If you need research assistants: For which components of the research? How many research assistants? Who would be the right persons to assist you and for how many days will you need them?
    • How will you train them? (place, timing, content, duration, trainers)
    • How will you ensure their supervision?
    • How will the quality of the data be checked and by whom?
  3. Data handling (5 minutes)
    • How will the questionnaires/checklists be numbered?
    • How will the data be stored and who has the final responsibility for storing the data?
  4. Ethical considerations (15 minutes)

    Take care that your data collection process is ethical in all respects:

    • How have you planned to obtain informed consent from your informants? Are there any categories of informants that need special consideration (e.g., children, sick persons, mentally disabled individuals)?
    • Are certain parts of the research focused on sensitive issues? How will you handle problems that may arise?
    • Do certain parts of your research require extra attention to assure confidentially? How will you handle the issue?
  5. Summarise the outcome of your group work on a flip chart. Record the details of your discussions so that you can use them for the development of your work plan (Module 15).

Annex 12.1: Training research assistants (as well as other research team members with little research experience)

1. Interviewers’ tasks

During the fieldwork interviewers (or research assistants) may work independently or together with one of the researchers. If they go out independently they may have to carry out the following tasks:

It is imperative that the researchers train their research assistants so they can carry out their tasks accurately and correctly, according to the procedures developed by the researchers. Research assistants should not be left to develop their own procedures; otherwise bias will be almost certain to result.

The training of the research team may take 2 to 3 days. The first day may be devoted to theory, followed by 1 or 2 days of practical training, depending on the local circumstances and the nature of the study.

2. Theoretical training

Research assistants must be thoroughly familiar with the objectives of the research project and the methodology. Therefore, it is recommended that they be provided with copies of the research protocol and that the most relevant sections be discussed thoroughly, including:

It is important at this stage that the research assistants get an ample opportunity to ask questions.

Then a more in-depth discussion should follow concerning the data collection tools (interview FGD guides, questionnaires and possibly checklists) that are to be used by the research-team members. For each and every question all should know WHY the information is required.

Research assistants should be taught basic interview techniques, such as

Clear instructions should be given as well, concerning to what extent an interviewer is allowed to alter the phrasing of questions, if it seems necessary, and whether he/she should probe for answers. For questions which have pre-categorised answers it should be made clear whether the possible answers should be mentioned by the interviewer during the interview or not. (Usually they are not to be mentioned.) There should be no misunderstandings concerning how to record answers and observations.

Finally, explanations should be given concerning how an interviewer should introduce him or herself to the interviewee, what to say concerning the purpose of the study, how to ask for consent, and how to close the interview.

3. Practical training

Practical training in interviewing is essential. This may be provided in two stages. (See Module 10B part V, interview exercise.)

Firstly, role-plays can be performed, during which one trainee assumes the role of the interviewer and another plays the interviewee. Other trainees and the trainers (researchers) should observe carefully what happens and give constructive feedback right after the role-play. Then roles can be changed, until each trainee has had a chance to practice each type of interview at least once.

Secondly, a field test should be conducted, which will serve two purposes: the training of the entire research team including research assistants, and a (further) test of the data collection tools. A test of the tools is essential if a previous field-test resulted in important changes and/or if the questionnaire(s) were translated into a local language after the first field test. If the research assistants will be involved in the proper phrasing of questions this will definitely strengthen their interest and commitment.

The field test is best carried out in groups of 2 to 3 persons (see Module 14), with each team including at least one trainer and one trainee. Note that after the field test constructive critique of each interview should be made, starting from the moment the interviewee was approached up to the farewell.

4. Supervision of research assistants

Even if research assistants are used, responsibility for the research remains with the research team as a whole. To guarantee the quality of the data collected it is important to supervise the research assistants’ performance, especially at the beginning of the data collection period. If they are going out into the field independently, plans could be made to accompany them on selected visits and/or to question a small sample of the interviewees as a ‘control group’ concerning key aspects of the interview.

As a further quality control check it is important that an interviewer’s name (or interviewer’s code) appears standard on each questionnaire/checklist so that it is possible to ask for clarification if certain information is not clear.

Trainer’s Notes

Module 12: PLAN FOR DATA COLLECTION

Timing and teaching methods

¾ hour Introduction and discussion
1¾ hours Group work
1 hour Plenary
3½ hours TOTAL TIME
Introduction and discussion
Group work

Designing and Conducting Health Systems Research Projects
Part I: Proposal Development and Fieldwork

Module 13
PLAN FOR DATA PROCESSING
AND ANALYSIS

./img/designandco_249_la_0.jpg

NB: Development of a research process is a cyclical process. The double-headed arrows indicate that the process is never linear.

MODULE 13: PLAN FOR DATA PROCESSING AND ANALYSIS

OBJECTIVES

At the end of this session, you should be able to:

  1. Identify important issues related to sorting, quality control, and processing of data.
  2. Describe how data can be best be analysed and interpreted based on the objectives and variables of the study.
  3. Prepare a plan for the processing and analysis of data (including data master sheets and dummy tables) for the research proposal you are developing.
  1. Introduction
  2. Sorting data
  3. Performing quality checks
  4. Data processing - qualitative data
  5. Data analysis - quantitative data
  6. Processing and analysis of qualitative data

I. INTRODUCTION

Data processing and analysis should start in the field, with checking for completeness of the data and performing quality control checks, while sorting the data by instrument used and by group of informants. Data of small samples may even be processed and analysed as soon as it is collected.

Why is it necessary to prepare a plan for processing and analysis of data?

Such a plan helps the researcher assure that at the end of the study:

The plan for data processing and analysis must be made after careful consideration of the objectives of the study as well as of the tools developed to meet the objectives.

The procedures for the analysis of data collected through qualitative and quantitative techniques are quite different.

Preparation of a plan for data processing and analysis will provide you with better insight into the feasibility of the analysis to be performed as well as the resources that are required. It also provides an important review of the appropriateness of the data collection tools for collecting the data you need. That is why you have to plan for data analysis before the pre-test (Module 14). When you process and analyse the data you collect during the pre-test you will spot gaps and overlaps which require changes in the data collection tools before it is too late!

What should the plan include?

When making a plan for data processing and analysis the following issues should be considered:

II. SORTING DATA

An appropriate system for sorting the data is important for facilitating subsequent processing and analysis.

If you have different study populations (for example village health workers, village health committees and the general population), you obviously would number the questionnaires separately.

In a comparative study it is best to sort the data right after collection into the two or three groups that you will be comparing during data analysis.

For example, in a study concerning the reasons for low acceptance of family planning services, users and non-users would be basic categories; in a study of the reasons why nurses object to being posted in rural areas, rural and urban nurses would be basic categories; in a case-control study obviously the cases are to be compared with the controls.

It is useful to number the questionnaires belonging to each of these categories separately right after they are sorted.

For example, the questionnaires administered to users of family planning services could be numbered U1, U2, U3, etc., and those for the non-users N1, N2, N3, etc.

In a cross-sectional survey it may also be useful to sort the data into two or more groups, depending on possible sub-groups you would like to compare.

III. PERFORMING QUALITY CONTROL CHECKS

Usually the data have already been checked in the field to ensure that all the information has been properly collected and recorded. Before and during data processing, however, the information should be checked again for completeness and internal consistency.

If a questionnaire has not been filled in completely you will have MISSING DATA for some of your variables. If there are many missing data in a particular questionnaire, you may decide to exclude the whole questionnaire from further analysis.

Note:

A decision to exclude data of doubtful quality is ethically correct and it testifies to the scientific integrity of the researcher. You should keep track of any questions you had to exclude because of incompleteness or inconsistency in the answers, and discuss it in your final report.

For computer data analysis, quality control checks of data must also include a verification of how the data has been transformed into codes and subsequently entered into the computer. The same applies if data are entered into master sheets (see next page).

IV. DATA PROCESSING – quantitative data

Decide whether to process and analyse the data from questionnaires:

Data processing in both cases involves:

1. Categorising

Decisions have to be made concerning how to categorise responses.

For categorical variables that are investigated through closed questions or observation (for example, observation of the presence or absence of latrines in homesteads), the categories have been decided upon beforehand.

In interviews the answers to open-ended questions (for example, ‘Why do you visit the health centre?’) can be pre-categorised to a certain extent, depending on the knowledge of possible answers that may be given. However, there should always be a category called ‘Others, specify . . .’, which can only be categorised afterwards.

These responses should be listed and placed in categories that are a logical continuation of the categories you already have. Answers that are difficult or impossible to categorise may be put in a separate residual category called ‘others’, but this category should not contain more than 5% of the answers obtained.

For numerical variables, the data are often better collected without any pre-categorisation. If you do not exactly know the range and the dispersion of the different values of these variables when you collect your sample (e.g., home-clinic distance for out-patients, or income), decisions concerning how to categorise and code the data at the time you develop your tools may be premature. If you notice during data analysis that your categories had been wrongly chosen you cannot reclassify the data anymore.

For example, in a study into utilisation of health services, the research team wanted to establish whether income was related to utilisation. They had pre-coded income into three categories. When they analysed the data they discovered that over 80% fell in the lowest income category. In hindsight they would have preferred a five point scale in order to distinguish different grades of poverty, but as the raw data had not been recorded it was impossible to reclassify the data, and the variable was almost useless.

2. Coding

If the data will be entered in a computer for subsequent processing and analysis, it is essential to develop a CODING SYSTEM.

For computer analysis, each category of a variable can be coded with a letter, group of letters or word, or be given a number. For example, the answer ‘yes’ may be coded as ‘Y’ or 1; ‘no’ as ‘N’ or 2 and ‘no response’ or ‘unknown’ as ‘U’ or 9.

The codes should be entered on the questionnaires (or checklists) themselves. When finalising your questionnaire, for each question you should insert a box for the code in the right margin of the page. These boxes should not be used by the interviewer. They are only filled in afterwards during data processing. Take care that you have as many boxes as the number of digits in each code.

If analysis is done by hand using data master sheets, it is useful to code your data as well (see section 3 below)

Coding conventions

Common responses should have the same code in each question, as this minimises mistakes by coders.

For example:

Yes (or positive response) code - Y or 1
No (or negative response) code - N or 2
Don’t know code - D or 8
No response/unknown code - U or 9

Codes for open-ended questions (in questionnaires) can be done only after examining a sample of (say 20) questionnaires. You may group similar types of responses into single categories, so as to limit their number to at most 6 or 7. If there are too many categories it is difficult to analyse the data. (For details, see section V part 2 of this module and Module 23.)

Finally it should be borne in mind that the personnel responsible for computer analysis should be consulted very early in the study, i.e., as soon as the questionnaire and dummy tables are finalised. In fact the research team needs to work closely with the computer analyst or statistician throughout the design and the implementation of the study.

2. Summarising the data in data master sheets, manual compilation, or compilation by computer

(1) Data master sheets

If data are processed by hand, it is often most efficient to summarise the raw research data in a so-called DATA MASTER SHEET, to facilitate data analysis. On a data master sheet all the answers of individual respondents are entered by hand.

To illustrate the use of master sheets, we will give an example of a rapid appraisal carried out by students of a nursing school about the smoking habits of the inhabitants of their town. Smoking was perceived as a big problem, and the study had been designed in order to develop an anti-smoking campaign. The (24) students divided the map of their town in 24 roughly equal parts. Each student put herself in the centre of the part of town designed to her, and interviewed six persons of 15 years and above, three males and three females, so 144 in total. (Another group of students conducted FGDs observations and individual interviews in schools to obtain insight into the onset of smoking behaviour.) The questionnaire had only 17 questions (see Annex 13.1), of which 9 were asked of everyone, 4 exclusively to smokers and 4 exclusively to non-smokers. It was therefore decided to process the data by hand, divided in two groups: smokers and non-smokers, which were again subdivided in males and females. For each of the four groups, master sheets were prepared, on which all the answers of individual respondents could be recorded.

Master sheets can be made in different ways. For short simple questionnaires you may put all possible answers for each question in headers at the top of the sheet and then list or tick the answers of the informants one by one in the appropriate columns.

For example, the straightforward answers of the smoking questionnaire for male smokers could be processed as follows (see Table 13.1):

Table 13.1: Master sheet for smokers (males)

./img/designandco_255_la_0.jpg

./img/designandco_255_la_1.jpg

Note that for age and number of cigarettes smoked both the raw data and the categories have been entered. This makes it easier to control for coding mistakes and allows for calculating averages. There are 31 male smokers; if there are less than 31 answers, there must be some non-responders (NR), as happened in Q9, or a mistake was made. If you work with two persons, one reading and one writing, the risk of mistakes will be reduced, as you can discuss the answers and control for mistakes while filling in the data.

Even this limited data suggest that male smokers start usually when they are teenagers, that the informants on average smoke one package of cigarettes a day and that attempts to stop appear to increase with age.

Some answers, however, require more elaborate coding and have more categories. For example, Q4 on education and Q5 on occupation could be summarised as follows (see Table 13.1, continued):

Table 13.1 (continued): Master sheet for smokers (males)

./img/designandco_256_la_0.jpg

./img/designandco_256_la_1.jpg

It will be clear that including all possible categories for one question in the headers of the master sheet may take a lot of space. If a questionnaire has only 13 questions, that does not matter; with 64 questions it would mean that you would need several sheets to include all answers. As the idea of master sheets is to have an easy overview of all data, you may then look for another solution, and enter the different codes for one question in one column instead of having different columns of which you tick one. For education, there would in this way only be four columns: none, highest level reached in years, highest type of school reached, and still in school; for occupation only two: self and head household.

./img/designandco_256_la_2.jpg

This data reveals that Smoker 1 is, like No. 4, still dependent on his father, though No. 1 earns some money. This means the money for cigarettes is strongly limited. Numbers 2 and 3 are financially independent, and smoke more heavily, though 3 has tried to reduce or stop many times.

Finally, there are open questions such as Q8, ‘Why do you smoke?’ or Q10, ‘Why not?’, or Q17, soliciting suggestions on how the students could best approach smokers in a campaign to discourage them from smoking. Coding and analysis of such qualitative data will be dealt with in section V of this module.

Note:

In any small-scale study processed by hand in which groups will be compared, a different master sheet should be made for each of those groups, e.g., good and poor compliers to treatment. As gender is an important cross-cutting theme, it is usually also advisable to subdivide males and females within each of the groups that are being compared.

See Annex 13.2 for example of full master sheet.

(2) Compilation by hand (without using master sheets)

When the sample is small (say less than 30) and the collected data is limited, it might be more efficient to do the compilation manually.

Certain procedures will help ensure accuracy and speed.

  1. If only one person is doing the compilation use manual sorting. If a team of 2 persons work together use either manual sorting or tally counting.

    Manual sorting can be used only if data on each subject is on a different sheet of paper/ entered in a separate questionnaire.

  2. To do manual sorting the basic procedure is to:
  3. To do tally counting the basic procedure is:
  4. After doing either manual or tally counting, check the total number of subjects/responses in each question to make sure that there has been no omission or double count.

Note:

One can tally in two ways, ./img/designandco_258_la_0.jpg. The latter way is preferable as it reduces the or possibility of error.

It should be noted that hand tallying is often used in combination with master sheet analysis when the relationship between two or three variables needs to be established, or details analysed. (For example, the questionnaire forms of non-smokers whose close relatives or co-workers are smoking (Q11 and 12) and who feel disturbed by the smoke (Q13) may be selected to analyse more in detail by tallying the (health) problems these non-smokers are experiencing.

Note:

Researchers often assume that hand compilation is merely ‘common sense’ and do not train their staff in the correct procedure. Subsequently many hours of work are wasted in trying to detect the source of errors due to double counts, wrong categorisation, and omissions.

(3) Computer compilation

Before you decide to use a computer, you have to be sure that it will save time or that the quality of the analysis will benefit from it. Note that feeding data into a computer costs time and money. The computer should not be used if your sample is small and the data is mainly generated by open questions (qualitative data), unless there is a resource person who is competent in using a program for qualitative data analysis such as Qualitan or SPSS. The larger the sample, the more beneficial in general the use of a computer will be.

Computer compilation consists of the following steps:

  1. Choosing an appropriate computer program
  2. Data entry
  3. Verification or validation of the data
  4. Programming (if necessary)
  5. Computer outputs/prints

i. Choosing an appropriate computer program

A number of computer programs are available on the market that can be used to process and analyse research data. The most widely used programs are:

If you intend to use a computer, you may ask advice from an experienced person concerning which program is the most appropriate for your type of data. Note that Epi Info may be freely used and copied. All the other programs have copyrights.

ii. Data entry

To enter data into the computer you have to develop a data entry format, depending on the program you are using. However, it is possible to enter data using dBase (which is relatively good for data entry) and do the analysis in LOTUS 1-2-3 or SPSS.

After deciding on a data entry format, the information on the data collection instrument will have to be coded (e.g., Male: M or 1, Female: F or 2). During data entry, the information relating to each subject in the study is keyed into the computer in the form of the relevant code (e.g., if the first subject (identified as 001) is a male (code 1) aged 25, the data could be keyed in as 001125).

Note that data entry can be done through the private sector, which may be fast and not too expensive. Health office staff who are not accustomed to this work tend to be slow and make many errors in entry.

iii. Verification

During data entry, mistakes will definitely creep in. The computer can print out the data exactly as it has been entered, so the printout can be checked visually for obvious errors, (e.g., exceptionally long or short lines, blanks that should not be there, alphabetic codes where numbers are expected, obviously wrong codes).

Example:

If possible, computer verification should be built in. This involves giving the appropriate commands to identify errors.

Example:

The computer can be instructed to identify and print out all subjects where the ‘sex’ column has a code different from 1 (F) or 2 (M).

iv. Programming

If you use computer personnel to analyse your data, it is important to communicate effectively with them. Do not leave the analysis to the computer specialist! You as a researcher should tell the computer personnel:

A certain amount of basic knowledge of computer programming is needed to give the appropriate commands.

v. Computer outputs

The computer can do all kinds of analysis and the results can be printed. It is important to decide whether each of the tables, graphs, and statistical tests that can be produced makes sense and should be used in your report. That is why we PLAN the data analysis BEFOREHAND! (See section V.)

V. DATA ANALYSIS – quantitative data

Analysis of quantitative data involves the production and interpretation of frequencies, tables, graphs, etc., that describe the data.

1. Frequency counts

From the data master sheets, simple tables can be made with frequency counts for each variable. A frequency count is an enumeration of how often a certain measurement or a certain answer to a specific question occurs.

For example,

Smokers51
Non-smokers93
Total 144

If numbers are large enough it is better to calculate the frequency distribution in percentages (relative frequencies): 51/144 x 100 = 35% are smokers and 93/144 x 100 = 65% non-smokers. This makes it easier to compare groups than when only absolute numbers are given. In other words, percentages standardise the data.

It is usually necessary to summarise the data from numerical variables by dividing them into categories. This process may include the following steps:

(1) Inspect all the figures: What is their range? (The range is the difference between the largest and the smallest measurement.)

(2) Divide the range into three to five categories. You can either aim at having a reasonable number in each category (e.g. 0-2 km, 3-4 km, 5-9 km, 10+ km for home-clinic distance) or you can define the categories in such a way that they are each equal in size (e.g., 20-29 years, 30-39 years, 40-49 years, etc.). Sometimes one looks actively for a ‘critical’ value when making different categories. For example, in a study relating family income to prevalence of diarrhoea over a certain period, there appeared to be no statistical relation when income was arbitrarily subdivided into four categories. When the average income was calculated, however, this appeared to be a critical value. The children in families with an income above average had had significantly less diarrhoea than the children in families with an income below average.

(3) Construct a table indicating how data are grouped and count the number of observations in each group.

2. Cross-tabulations

Further analysis of the data usually requires the combination of information on two or more variables in order to describe the problem or to arrive at possible explanations for it.

For this purpose it is necessary to design CROSS-TABULATIONS.

Depending on the objectives and the type of study, two major kinds of cross-tabulations may be required:

A descriptive cross-tabulation would, for example, relate smoking behaviour to sex or occupational background:

Table 13.2: Smoking by sex

./img/designandco_261_la_0.jpg

The males appear to be smoking more (43%) than females (28%).

An analytic cross-tabulation serves to investigate if there is a relationship between smoking (independent variable) and persistent cough, or chest complaints (dependent variables/problems).

Table 13.3: Smoking in relation to persistent cough over the past 2 weeks

./img/designandco_261_la_1.jpg

Of the informants with a cough, the majority (77%) is smoking, whereas among those without a cough, only one third (33%) are smokers. The expected relationship between smoking and chest problems seems therefore confirmed.

When the plan for data analysis is being developed the data, of course, is not yet available. However, in order to visualise how the data can be organised and summarised it is useful at this stage to construct so-called DUMMY cross-tabulations.

A DUMMY TABLE contains all elements of a real table, except that the cells are still empty.

In a research proposal dummy tables should be prepared to describe the study population in order to show the crucial relationships between variables.

For the study on smoking behaviour, for example, you would have to prepare a number of descriptive dummy tables (describing characteristics of smokers and non-smokers, their behaviour and attitudes towards smoking), such as Table 13.2 but without numbers or percentages. Further you would make two analytic dummy tables, one on the relationship between smoking and persistent cough (see Table 13.3) and one on the relationship between smoking and long-term chest problems.

Some practical hints when constructing tables:

To further analyse and interpret the data, certain calculations or statistical procedures must usually be completed. Especially in large cross-sectional surveys and in comparative studies, statistical procedures are necessary if the data is to be adequately interpreted. Statistical tests should, for example, indicate whether the gender differences in smoking behaviour are true differences or due to chance. When conducting such studies it is advisable to consult a person with statistical knowledge from the start in order that:

Some elementary statistical procedures will be taught in the second workshop after field work is completed. An elementary knowledge of statistics will help you better understand the whole process of data analysis and interpretation.

VI. PROCESSING AND ANALYSIS OF QUALITATIVE DATA

Qualitative data may be collected through open-ended questions in self-administered questionnaires, in individual interviews or focus group discussions or through observations during fieldwork. For a detailed description of the analysis of qualitative data see Module 10C and in particular Module 23, which specify the methods most often used. Here we will concentrate on the analysis of responses obtained from open-ended questions in interviews or self-administered questionnaires.

Commonly solicited data in open-ended questions include:

The data can be analysed in seven steps:

Step 1: Take a sample of (say 20) questionnaires and list all answers for a particular question. Take care to include the source of each answer you list (in the case of questionnaires you can use the questionnaire number), so that you can place each answer in its original context, if required.

Step 2: To establish your categories, you first read carefully through the whole list of answers. Then you start giving codes (A, B, C, for example or key words) for the answers that you think belong together in one category, and write these codes in the left margin. Use a pencil so that it is easy to change the categories if you change your mind.

Step 3: List the answers again, grouping those with the same code together.

Step 4: Then interpret each category of answers and try to give it a label that covers the content of all answers. In the case of data on opinions, for example, there may be only a limited number of possibilities, which may range from (very) positive, neutral, to (very) negative.

Data on reasons may require different categories depending on the topic and the purpose of your question. In the exercise below you will be asked to categorise the reasons why people smoke by grouping them in such a way that it is easy to find entry points for health education aimed at reducing smoking.

After some shuffling you usually end up with 5 to 7 categories.

Step 5: Now try a next batch of 20 questionnaires and check if the labels work. Adjust the categories and labels, if necessary.

Step 6: Make a final list of labels for each category and give each label a code (keyword, letter or number).

Step 7: Code all your data, including what you have already coded, and enter these codes in your master sheet or in the computer.

Note again that you may include a category ‘others’, but that it should be as small as possible, preferably used for less than 5% of the total answers.

If you categorise your responses to open-ended questions in this way you can:

Questions that ask for descriptions of procedures, practices, or beliefs usually do not provide quantifiable answers (though you may quantify certain aspects of them). The answers rather form part of a jigsaw puzzle that you have to put together in order to obtain insight in your problem/topic under study.

IN CONCLUSION, a plan for the processing and analysis of data may include:

GROUP WORK

Prepare your plan for data processing and analysis, considering the following points:

  1. Sorting and quality control of data (10 minutes):
    • How will the sorting be done? When?
    • What quality checks should be made? Who will do them? When?
  2. Processing of data (50 minutes):
    • How will you do it (by hand or by computer)? If by computer: do you have enough experience and is the necessary equipment available?
    • Prepare data master sheets for your proposal (preferably on flip charts).
    • How many open-ended questions do you have that require categorising or coding? Who will do the categorising or coding? How much time will be required for data processing (taking into account the sample size)?
  3. Analysing and interpreting the data (1 hour):
    • Using the specific objectives and the list of variables, prepare dummy tables in which you relate variables to each other to analyse possible (causal) relationships. Select the dummy tables that you plan to fill in before we have our workshop on data analysis and reporting.
    • Make estimates of the time and materials required for the analysis (in our case, only for the period up to the second workshop during which we will continue the analysis of data).
  4. Prepare to present in plenary your master sheet, three dummy tables, a list of other important variables that you would like to cross-tabulate, and rough estimates of manpower, time, and materials required for the analysis of data (15 minutes).

EXERCISE: Analysis of responses to open-ended questions

Please analyse and interpret the following answers from the study of student nurses on smoking which were given to Question 7: ‘Why do you smoke?’

  1. I have tried to give up so many times but I have been unable to.
  2. I like the feel of the cigarette in my hand.
  3. Because it gives me pleasure.
  4. I do not see why I should give up smoking!!
  5. Because I like to blow the smoke through my mouth and nose.
  6. Because I feel confident and in charge when I am smoking.
  7. It helps me to think better.
  8. I like the image that comes with smoking.
  9. I feel that people respect me more as a smoker.
  10. All my friends are smokers.
  11. It helps to make people more friendly and comfortable, especially when offering a cigarette.
  12. Why not?!!
  13. Smoking makes me feel like a man.
  14. I like to blow smoke rings.
  15. I like the taste.
  16. It is too difficult to give up.
  17. It helps me to relax.
  18. It helps me to reduce the pressure and tension at work.
  19. My wife likes a man who smokes.

Analyse and interpret these answers as follows:

In reality you would have tried the coding system out on another sample of answers. Thereafter you would most likely still have slightly adjusted the coding system, after which you would have coded all answers on the questionnaires and entered the codes in the master sheet or computer.

Trainer’s Notes

Module 13: PLAN FOR DATA PROCESSING AND ANALYSIS

Timing and teaching methods

1 hour Introduction and discussion
½ hour Exercise
2½ hours Group work
1 hour Plenary
5 hours TOTAL TIME
Introduction and discussion
Exercise: Analysis of responses to open-ended questions

This can be done in the small working groups, before starting on the group work assignment. Assist the participants in analysing and interpreting the 19 answers given to the open-ended question. One way you might categorise and interpret the answers is given on the next page. Explain that the codes can be inserted on the master sheets.

Annex 13.1: Questionnaire for adults (15+ ) on smoking behaviour

Introduction

May I talk with you for a few minutes? We are students from the nursing school, and we are conducting a small study on smoking. We ask about 10 questions. Would you mind participating? (State you would like to interview non-smokers as well as smokers.)

If the informant agrees:

Everything we discuss will be treated confidentially (no one else will know who said what). But you should feel free to remain silent if you hesitate to answer a particular question. And if you have any questions for us, please feel free to ask them.


Questionnaire

./img/designandco_267_la_0.jpg

Put √ in the applicable box, unless otherwise indicated

./img/designandco_267_la_1.jpg

./img/designandco_268_la_0.jpg

./img/designandco_269_la_0.jpg

./img/designandco_270_la_0.jpg

Annex 13.2: Mastersheet for femal non-smokers

./img/designandco_271_la_0.jpg

Annex 13.2: Mastersheet for femal non-smokers (continued)

./img/designandco_272_la_0.jpg

This page intentionally left blank.

Designing and Conducting Health Systems Research Projects
Part I: Proposal Development and Fieldwork

Module 14
PRE-TESTING THE METHODOLOGY

./img/designandco_275_la_0.jpg

NB: Development of a research process is a cyclical process. The double-headed arrows indicate that the process is never linear.

Module 14: PRE-TESTING THE METHODOLOGY

OBJECTIVES

At the end of this session you should be able to:

  1. Describe the components of a pre-test or pilot study that will allow you to test and, if necessary, revise your proposed research methodology before starting the actual data collection.
  2. Plan and carry out pre-tests of research components for the proposal being developed.

What is a pre-test or pilot study of the methodology?

A PRE-TEST usually refers to a small-scale trial of particular research components.

A PILOT STUDY is the process of carrying out a preliminary study, going through the entire research procedure with a small sample.

WHY do we carry out a pre-test or pilot study?

A pre-test or pilot study serves as a trial run that allows us to identify potential problems in the proposed study. Although this means extra effort at the beginning of a research project, the pre-test and/or pilot study enables us, if necessary, to revise the methods and logistics of data collection before starting the actual fieldwork. As a result, a good deal of time, effort and money can be saved in the long run. Pre-testing is simpler and less time-consuming and costly than conducting an entire pilot study. Therefore we will concentrate on pre-testing as an essential step in the development of research projects.

WHAT aspects of your research methodology can be evaluated during pre-testing?
  1. Reactions of the respondents to the research procedures can be observed in the pre-test to determine:
  2. The data-collection tools can be pre-tested to determine:
  3. Sampling procedures can be checked to determine:
  4. Staffing and activities of the research team can be checked, while all are participating in the pre-test, to determine:

    Note:

    The pre-test can be seen as a period of extra training for the research team in which sensitivity to the needs and wishes of the study population can be developed.

  5. Procedures for data processing and analysis can be evaluated during the pre-test. Items that can be assessed include:
  6. The proposed work plan and budget for research activities can be assessed during the pre-test. Issues that can be evaluated include:
When do we carry out a pre-test?

You might consider:

Which components should be assessed during the pre-test?
  1. Pre-test during the workshop

    Depending on how closely the pre-test situation resembles the area in which the actual field work will be carried out, it may be possible to pre-test:

    Even if you cannot assess all these components fully, the field experience will provide information that will be quite valuable to you in reviewing the methodological aspects of your proposal and in planning your work plan and budget.

  2. Pre-test in the actual research area

    All the issues mentioned above will have to be thoroughly reviewed during a pre-test in the actual field situation. Other issues, such as the functioning of the research team, including newly recruited and trained research assistants, and the feasibility of the work plan, can only be tested in the research area. An important output of the pre-test should be a fully developed work plan.

    If choices have to be made as to what to include in the pre-test, the following considertions may be helpful:

Note:

It is highly recommended that you analyse the data collected during the pre-test right away. Then finalise and adjust the master sheets, if necessary. Make totals for each variable included in the master sheets. Fill in some dummy tables and prepare all the dummy tables you need, considering your research objectives.

Do all this even if you plan to analyse the data by computer. You will detect shortcomings in your research tools that you can still correct!

Who should be involved in the pre-test or pilot study?
How long should the pre-test or pilot study last?

The time required for a pre-test or pilot study will be determined by a number of factors:

Keep in mind that this is the last chance you will have to make adjustments which will help to ensure the quality of your fieldwork. If you have a 20 day field work period you might reserve at least 3-5 days for pre-testing your data collection tools, analysing the results of the pre-test, finalising your tools and elaborating the work plan.

GROUP WORK I – to prepare the pre-test during the workshop (1½ hours+)

Only half a day will be available for conducting a pre-test of your methodology during the course:

  1. Determine what parts of the methodology you would like to test. Include all data collection tools, if possible.
  2. Decide with your facilitator and course manager where in the local area you could best carry out the pre-test.
  3. Decide which members of your team will conduct various aspects of the pre-test. You are advised to work in pairs, so that you can discuss observations during the pre-test.
  4. Prepare a short list of questions you wish to answer during the pre-test. (See Annex 14.1 for suggestions.)

GROUP WORK II – after completion of the pre-test (4 hours)

  1. Answer the questions you developed for the pre-test.
  2. Determine whether your pre-test experience indicates that you need any:
    • changes in your research proposal
    • changes in your data collection tools

    Assign various group members to make these changes.

  3. Determine what aspects of the study you would like to pre-test (again) in your research area and why, with whom, when and where.
  4. After completing items 1-3, summarise the major points on a flip chart and in one or two paragraphs for your research proposal.

GROUP WORK III – (instead of Group Work I and II, if no pre-test is possible during the workshop, 3 hours)

  1. Decide what aspects of the study you would like to pre-test in your research area and why, with whom, when and where. Summarise this information in one or two paragraphs in your research proposal.
  2. Instead of doing a pre-test during the workshop, you might take the time to carefully review your research methodology and your data collection tools, using the checklist provided in Annex 14.2.

./img/designandco_282_la_0.jpg

./img/designandco_283_la_0.jpg

Annex 14.2: Summary of possible fallacies in the design and implementation of
studies

As we have now gone through all steps of the study design, including the planning of data processing and analysis, it may be useful to summarise the critical points at which a researcher can go wrong:

These potential errors should be reviewed while pre-testing your research methodology.

1. Errors in selection of respondents or study elements.

In the selection of respondents we may distinguish several major possibilities for error.

Too limited (or inappropriate) definition of the study population or use of incorrect sampling procedures, for example by:

Errors in the assignment of research subjects to study groups in analytic and experimental studies:

Selective dropouts or non-response

Dropouts or subjects who do not respond to selected questions may represent a special category of respondents. If attrition is high or the rate of non-response excessive, results may be biased.

In cohort studies, follow-up of individuals can be difficult overtime. Bias in follow-up emerges if there is a differential drop out between those exposed to the risk and those not exposed.

2. Errors in data collection

We may obtain:

Invalid data, by applying indicators and measuring techniques or instruments which do not adequately measure what we want to measure.

Unreliable data due to:

Reliability of data collected is always required, but it is of crucial importance if we want to measure changes over time. If we find changes we must be sure that these are not caused by errors in our research methods that could have been prevented.

All the above-mentioned shortcomings may threaten the validity of your findings and conclusions.

The shortcomings can be prevented to some degree by being alert to them when designing and implementing the study; otherwise they have to be mentioned in the study design.

Trainer’s Notes

Module 14: PRE-TESTING THE METHODOLOGY

Timing and teaching methods

½ hour Introduction and discussion
1-½ hours Group work I (to prepare the pre-test)
4 hours Actual pre-test
4 hours Group work II (to discuss pre-test results and revise the data collection tools)
1 hour Plenary
11 hours TOTAL TIME

Note: If no pre-test can be carried out during the workshop, consider using the Group Work III alternative (3 hours) instead of Groups Works I and II.

It is important to note that well before this session (preferably after Module 10) the course manager should ask the groups what data collection tools they want to pre-test and on whom. Each research team, with the assistance of its facilitator, should decide how many interviews or observations will be conducted and should begin making arrangements for obtaining necessary copies of the instruments and other supplies needed. The course manager should make arrangements for all the groups (i.e., look for suitable sites to do the pre-test, inform (health) authorities and/or local leaders of the plans, ask for their consent, and arrange transport).

Introduction and discussion
Group work I

Ask the participants to meet in their working groups to design the pre-test for their project.

Pre-test field exercise

If necessary, an instruction sheet should be prepared for the field exercise including information such as:

Group work II

Arrange for each group to meet after its members have come back from the field to discuss and analyse their experiences and to revise their data-collection tools and possibly other aspects of the research methodology. Ask each group to prepare a short report of its main findings and conclusions.

Plenary

One member of each group should report in plenary on the main findings and conclusions of the pre-test.

Designing and Conducting Health Systems Research Projects
Part I: Proposal Development and Fieldwork

Module 15
WORK PLAN

./img/designandco_289_la_0.jpg

NB: Development of a research process is a cyclical process. The double-headed arrows indicate that the process is never linear.

Module 15: WORK PLAN

OBJECTIVES

At the end of this session you should be able to:

  1. Describe the characteristics and purposes of various project planning and scheduling techniques such as ‘work scheduling’ and ‘GANTT charting’;
  2. Determine the various tasks and the staff you need for your project and justify any additional staff (research assistants, supervisors) apart from the research team that developed the proposal, where you will recruit them, for how long a period you need them, and how you will train and supervise them.
  3. Prepare a work schedule, GANTT chart and staffing plan for the project proposal you are developing.
  1. Introduction
  2. Work scheduling and planning techniques

I. INTRODUCTION

What is a work plan?

A WORK PLAN is a schedule, chart or graph that summarises the different components of a research project and how they will be implemented in a coherent way within a specific time-span.

It may include:

II. VARIOUS WORK SCHEDULING AND PLANNING TECHNIQUES

1. The work schedule

A WORK SCHEDULE is a table that summarises the tasks to be performed in a research project, the duration of each activity and who is responsible for the different tasks.

The version of a work schedule given on the following page includes:

Note:

The period for field research for the course project should not exceed 6 months. Week 1 is the first week after completion of the present workshop.

This work schedule was developed for a study of factors associated with maternal mortality in Mongo district, Zambia. The research team consisted of five persons (mainly district health team members). The study consisted of two main parts: (1) analysis of health facility-based maternal deaths; and (2) analysis of community-based maternal deaths.

EXAMPLE OF A WORK SCHEDULE:

Factors associated with maternal mortality, Mongo district, Zambia

./img/designandco_292_la_0.jpg

You will notice that each team member roughly spent 30 working days on the research, except the Senior Clinical Officer who was the Team Leader. He spent time to recruit research assistants and to visit district authorities to obtain their support for the study. Though he integrated these tasks with his normal clinical duties, he spent about 6 more working days than the other team members did. Five research assistants were recruited to assist with the interviewing. The number of working days required (20) were multiplied by five (for the research team as well as for the research assistants) to arrive at the number of person-days. Another 21 days (6+15) were spent on data analysis and report writing, whereas three days were reserved for feedback of findings and preliminary recommendations to all parties concerned, and two for drafting a plan of action to implement them.

How to develop a work schedule
In reviewing your tentative staffing plan you should ask:

Then fix the dates (in weeks) indicating the period in which each task will have to be carried out and calculate the number of working days per person required to complete each task.

2. The GANTT Chart

A GANTT chart is a planning tool that depicts graphically the order in which various tasks must be completed and the duration of each activity.

The GANTT chart shown on the following page indicates:

The length of each task is shown by a bar that extends over the number of days, weeks or months the task is expected to take.

How can a work plan be used?

A work plan can serve as:

When should the work plan be prepared and when should it be revised?
What factors should be kept in mind when preparing a work plan?

Example of a GANTT Chart: Factors associated with maternal mortality, Mongo district, Zambia

./img/designandco_295_la_0.jpg

Computer-assisted formulation of work plan:

There are a number of computer software packages on the market that one can use to prepare and monitor the implementation of a work plan. Microsoft Project Manager, Excel, and Lotus are among the commonly used software packages.

Budget preparation

The work plan is the starting point for developing your budget. Specify, for each activity in the work plan, what resources are required. Determine for each resource needed the unit cost and the total cost.(See Module 16.)

GROUP WORK (3 hours)

Prepare a work plan for inclusion in your proposal, following the steps below:

  1. Start with the development of a work schedule:
    • List all tasks to be carried out, completing and revising the list of tasks you prepared for your plan for data collection.
    • Consider who will carry out each task, the number of working days required per person to complete each task, the number of staff you will need to finish each task in a given period of time, and the period in which you plan to actually carry out each task.
    • Look at a calendar and note any public holidays or other important activities scheduled for the period (about 6 months) in which you plan to conduct the fieldwork.
    • Include your facilitator in stages of the fieldwork where you feel you would require assistance (e.g., during training of research assistants or during the initial period of data collection in the field). If needed, schedule the use of a local consultant.
    • Do not forget to include support staff required (typists, drivers, for example).
  2. Consider whether the number of days each member of the research team plans to invest in the fieldwork is adequate for the task and will be acceptable. ( It should most likely not exceed 30 working days.)
  3. Prepare a GANTT Chart to include in your proposal.
  4. Include two or three paragraphs on the staff required for your research and their tasks in your work plan, including:
    • Composition of research team itself and the tasks of various members;
    • Reasons for recruiting research assistants/data collectors/supervisors, where you will recruit them, what their tasks will be, for how long you will need them, and how you will train and supervise them;
    • The role of facilitators during the field work and when they will be needed; and
    • Whether any other consultants will be needed and, if so, what skills they should have and what their tasks would be.
  5. Copy your work schedule and GANTT chart on flipcharts or overhead sheets, for use in the exercise below and in the plenary discussion.

EXERCISE (Optional): Project work plan

Review the work plan that another group developed for their research proposal and provide constructive criticism.

Trainer’s Notes

Module 15: WORK PLAN

Timing and teaching methods

½ hour Introduction and discussion
3 hours Group work
¼ hour Exercise (optional)
1 hour Plenary
4½ hours TOTAL TIME
Introduction and discussion
Group work

Ask the participants to prepare work plans for their research proposals taking into account the plans for data collection that they already prepared (Module 12). Ask them to start by listing the tasks to be performed in the correct sequence. Then they should estimate the time involved for each task and assign the tasks to various staff members and consultants (if needed). Encourage each group to think seriously about what staffing pattern would be most cost-effective and efficient for its particular research project.

Exercise: Project work plan (optional)

Ask each group to look at the work plan developed by one of the other groups and provide constructive criticism.

Plenary

Ask each group to present its work schedule, Gantt chart, and staffing plan followed by a short discussion.

Designing and Conducting Health Systems Research Projects
Part I: Proposal Development and Fieldwork

Module 16
BUDGET

./img/designandco_299_la_0.jpg

NB: Development of a research process is a cyclical process. The double-headed arrows indicate that the process is never linear.

Module 16: BUDGET

OBJECTIVES:

At the end of the session you should be able to:

  1. Identify major categories for a budget.
  2. Make reasonable estimates of the expenses in various budget categories.
  3. List various ways a budget can be reduced, if necessary, without substantially damaging a project.
  4. Prepare a realistic and appropriate budget for the project proposal being developed during the course.
Why do we need to design a budget?
When should budget preparation begin?

A complete budget is not prepared until the final stage of project planning. However, cost is usually a major limiting factor and therefore must always be kept in mind during planning so that your proposals will not have an unrealistically high budget. (See Module 4: Analysis and statement of the problem.) Remember that both ministries and donor agencies usually set limits for research project budgets.

The use of locally available resources increases the feasibility of the project from a financial point of view.

How should a budget be prepared?

It is necessary to use the work plan (Module 15) as a starting point. Specify, for each activity in the work plan, what resources are required. Determine for each resource needed. the unit cost and the total cost.

Example:

In the work plan of a study to determine factors affecting maternal mortality in the Mongu District, Western Province, Zambia, it is specified that 5 research team members will each visit 20 households, one per working day, as 100 households of deceased mothers will need to be visited. With two drivers this amounts to 50 working days per driver. Each research team member will be accompanied by one of the research assistants.

The budget for the fieldwork component of the work plan will include funds for personnel, transport and supplies.

Note:

that UNIT COST (e.g., per diem or cost of petrol per km), the MULTIPLYING FACTOR (number of days), and TOTAL COST are required for all budget categories.

EXAMPLE OF A BUDGET: Costs involved in fieldwork for a maternal mortality study

./img/designandco_302_la_0.jpg

The Ministry of Health will make available the necessary manpower to conduct the study, as well
as vehicles. The total budget required from external sources amount to Kwacha (Kw) 27,480. At
an exchange rate of 2,75 to US$ 1, this is equivalent to US$9,990.
If more than one budget source will be used (e.g., the Ministry of Health and a donor), it would be


* NB: If team members do not write the report themselves on computers, one secretary will be required for 10 days during the actual writing of the report.

useful to indicate in the budget which source will pay for each cost. Usually a separate column is used for each funding source. The Ministry of Health may, for example, provide salary costs and operational costs for vehicles, whereas the donor agency is asked to provide per diem (according to local regulations), petrol/public transport expenditures and stationary.

Advice on budget format

The type of budget format to be used may vary depending upon whether the budget will be supported by your own organisation or submitted to the Ministry of Health or a donor organisation for funding. Most donor organisations have their own special project forms, which include a budget format. If you intend to seek donor support it is advisable to write to the potential funding organisation as early as possible during the period of project development.

Advice on budget preparation
Budget justification

It is not sufficient to present a budget without explanation.

The budget justification follows the budget as an explanatory note justifying briefly, in the context of the proposal, why the various items in the budget are required. Make sure you give clear explanations concerning why items that may seem questionable or that are particularly costly are needed and discuss how complicated expenses have been calculated. If a strong budget justification has been prepared, it is less likely that essential items will be cut during proposal review.

How can budgets be reduced?
Obtaining funding for projects

To conduct research, it is usually necessary to obtain funding for the research project. Such funding may be available from local, national or international agencies. In addition to preparing a good research proposal, the following strategies are useful for researchers who need to obtain their own funding:

  1. Familiarise yourself with the policies and priorities of funding agencies. Such policies and priorities may be:

    Obtain the names of such persons and make direct contact with them.

    The funding policies of many agencies may emphasise:

    Annex 16.1 gives a list of some prominent research funding agencies.

  2. Identify the procedures, deadlines and formats that are relevant to each agency.
  3. Obtain written approval and support from relevant local and national health authorities and submit this together with your proposal.
  4. If you are a beginning researcher, associate yourself with an established researcher. Host agencies scrutinise the ‘credibility’ of the researcher to whom funds are allocated. Such credibility is based on previous projects that have been successfully completed.
  5. Build up your own list of successfully completed projects (i.e., your own reports, publications, etc.).

GROUP WORK (2½ hours)

  1. Prepare a budget for your project. Keep in mind the importance of having a realistic budget, for which resources can actually be found.
  2. Examine the work plan in your project proposal and consider See the expenses involved in completing each component. Local rules should be followed for calculating per diems, travel cost and overtime (if required).
  3. Indicate for each item, the UNIT COST as well as the NUMBER OF UNITS. Justify large budget items, travel and allowances in one or two paragraphs attached to the budget.
  4. Consider the ‘cost-effectiveness’ of various budget levels. Will the final results be worth the expense?
  5. Consider the budget level that possible funding authorities would consider appropriate:

    — Examine their guidelines.

    — If appropriate, talk with donor representatives about their policies.

  6. If additional funding is requested from an outside donor, make clear what contribution the Ministry of Health and your own institution is making.

Annex 16.1: INTERNATIONAL SOURCES OF FUNDING FOR RESEARCH

1. International Multi-Lateral Agencies

WHO and Associated Special Programmes:

IMCI (Integrated Management of Childhood Illnesses)
MCH, Reproductive Health, Adolescent Health, etc.
RBM/AIM (Roll Back Malaria/African Initiative on Malaria)
TDR (Tropical Disease Research)
UNAIDS
WHO Headquarters
WHO Regional Offices

African Development Bank
Asian Development Bank
IARC (International Agency for Research on Cancer)
UNICEF (United Nations Children’s Fund)
UNFPA
World Bank etc.

2. Bilateral Agencies

ADAB (The Australian Development Assistance Board)
BOSTID (Board on Science and Technology for International Development)
CIDA (Canadian International Development Agency)
DFID (Department for International Development), United Kingdom
DGIS (Directorate for International Co-operation), The Netherlands
GTZ (Deutsche Gesellschaft für Technische Zusammenarbeit, Germany)
IDRC (International Development Research Centre), Canada
JICA (Japanese International Co-operation Agency)
SAREC (Swedish Agency for Research Co-operation with Developing Countries)
SIDA (Swedish International Development Agency)
USAID (United States Agency for International Development) etc.

3. Private Foundations

Carnegie Corporation of New York
Ford Foundation (Child Health) (USA)
International Health Policy Program (USA)
Kellogg Foundation (Health Services; primary interest in Latin America)
Rockefeller Foundation (USA)
Welcome Trust (UK) etc.

4. National Sources

This will vary from country to country.

NB: You can find addresses through Internet or through the Embassies of the respective countries.

Trainer’s Notes

Module 16: BUDGET

Timing and teaching methods:

¾ hour Introduction and discussion
2½ hours Group work
1 hour Plenary
4¼ hours TOTAL TIME
Introduction and discussion

Before asking participants to work on the budgets for their own projects, it is important to announce STANDARD unit costs both for transport (distance or fuel) and for allowances. These should conform to Ministry of Health regulations. If there are limits to budgets, these should be agreed upon beforehand as well.

Group work

Ask the participants to meet in their groups and prepare the budgets for their own projects, based on the work plan (Module 15). Ask them to specify what the contribution of their own institution/ministry will be to the project and what the external donor is requested to contribute.

Plenary

Ask each group to present its budget proposal in plenary. Allow sufficient time for discussion after each presentation.

Designing and Conducting Health Systems Research Projects
Part I: Proposal Development and Fieldwork

Module 17
PLAN FOR PROJECT ADMINISTRATION
MONITORING, AND UTILISATION OF RESULTS

./img/designandco_309_la_0.jpg

NB: Development of a research process is a cyclical process. The double-headed arrows indicate that the process is never linear.

Module 17: PLAN FOR PROJECT ADMINISTRATION, MONITORING AND UTILISATION OF RESULTS

OBJECTIVES

At the end of this session you should be able to:

  1. List the responsibilities of the team leader and project administrator related to the administration and monitoring of an on-going project.
  2. Prepare a brief plan for administration and monitoring of the research project being developed.
  3. Prepare a plan for actively disseminating and fostering the utilisation of results for the project proposal being developed.
  1. Administrating research projects
  2. Project monitoring
  3. Planning for the dissemination, communication and utilisation of research results

I. ADMINISTRATING RESEARCH PROJECTS

What is project administration?

PROJECT ADMINISTRATION is the term for all the activities involved in managing the human, material, financial and logistical resources of a project.

Why is good administration important in a research project?
What administrative issues should be considered as the project proposal is being finalised?

As a team developing a research project, you should now consider the following issues:

What would the tasks of the team leader, related to project administration, include?
What administrative operations need to be supervised by the team leader at the end of the project?

II. PROJECT MONITORING

What is project monitoring?

MONITORING is the on-going process by which information is gathered concerning the implementation and evolution of the research project. Monitoring involves activities designed to keep track of resources available and used and the quantity and quality of the operations carried out during each phase of the project so that its objectives will be met.

Monitoring should continue throughout the project and be organised so that it is helpful in alerting staff to problems that develop and changes needed. It is a valuable management and learning tool for everyone concerned.

During monitoring sessions you will review:

Note:

Monitoring will usually take place at team meetings during field activities. If there is a gap in the fieldwork, it may be necessary to convene a special meeting.

It is advisable to keep close track of changes in the work plan and problems encountered and solved (or not solved) so that you can inform your facilitator and superiors, and include this information in your preliminary report. (See Module 20.)

III. PLANNING FOR THE DISSEMINATION, COMMUNICATION AND UTILIZATION OF RESEARCH RESULTS

Before you finish drafting your research proposal you should start planning how the results of your study could be used.

Why should researchers be concerned about dissemination, communication and utilisation of research results?

The fundamental reason for undertaking health systems research is to obtain results that can be used to improve health and health care.

Who will be interested in the results?

Depending on the topic you selected, the results may be useful to the community, staff and managers of health and health-related services and to researchers and donor agencies in your own country, as well as others.

However, above all, you as a research team and your program should benefit from the results, as you have developed the proposal to help solve one of your own priority problems.

What strategies can you follow to ensure that the results of your study will be used?

1. Involve relevant authorities, staff and community members in the selection of your topic and in the definition of your problem.

If possible, these groups should be consulted before the proposed development workshop begins. If the final decision for a certain topic is made during the workshop, however, not all parties concerned may have been consulted. If not, they should be consulted immediately after the workshop.

2. List two or three major recommendations you expect to obtain from your study and identify who should be involved in their implementation.

Here we must distinguish between two categories of people who should be involved:

3. Identify which communication channels already exist which can be used to discuss and disseminate results.

Channels for discussing and disseminating results may include, for example:

Keep relevant parties informed of progress during implementation of the study and plan to obtain their input when study findings and recommendations are being drafted.

4. Determine what written materials should be prepared to keep relevant parties informed. They may include:

Make sure that summaries of your findings and preliminary recommendations are adapted to the level of understanding and interests of different audiences. This will increase their motivation to provide thorough feedback and to participate in the implementation of the final recommendations collectively agreed up on.

5. Determine whether additional actions should be taken or mechanisms developed to discuss the study results with all parties concerned and obtain their input, approval and co-operation for the implementation of the recommendations. These may include, for example:

For complex studies of relatively long duration it may be advisable to have a Project Advisory Committee, representing the major parties involved. Since the projects developed during workshops will in general not last longer than six months, you may be able to keep key individuals or representatives informed through ad hoc or even routine meetings.

Note:

Never forget to report the findings to the subjects/community /organisation studied before the report is finalised. This should be done to fulfil an obligation to those studied, to obtain information on possible errors in your draft report, and to discuss your proposed recommendations and obtain useful feedback. (See Module 33)

GROUP WORK (1½ hours)

  1. Develop a plan for administrating and monitoring your project. Consider the following questions as you develop your plan:

    Administration

    • Who will be the team leader for your project?
    • Which organisational unit or which official would be best able to administer the project? (Remember that the team leader cannot also be the project administrator.)
    • Which authorities are likely to fund the project?
    • How can a smooth flow of funds be assured?
    • Who will do the project accounting and file and submit receipts?

    Monitoring

    • What aspects of the project will be monitored and who will be responsible?
    • How will the monitoring activities be organised and when will they take place?
  2. List two or three major recommendations expected from your study in a table and identify who should be involved in their implementation:

    ./img/designandco_316_la_0.jpg

  3. Determine what channels or mechanisms you will use (or develop) to keep parties from whom you require authorisation and/or co-operation for implementing recommendations informed concerning the project (1) before starting the field work; (2) after completing the field work; (3) after preparing the draft report of findings and recommendations.
  4. Identify the one or two authorities who are most crucial for implementation of your recommendations so that they can be invited for the presentation and discussion of your findings and recommendations at the end of the data analysis workshop.
  5. Present the results of your group work on a flipchart and prepare several paragraphs on project administration, monitoring and the utilisation of results for inclusion in your research proposal. Do not forget to include the dissemination and utilisation of the results in your work plan and, if necessary, in the budget.
Trainer’s Notes

Module 17: PLAN FOR PROJECT ADMINISTRATION, MONITORING AND UTILISATION OF RESULTS

Timing and teaching methods:

¾ hour Introduction and discussion
1½ hour Group work
1 hour Plenary
3¼ hours TOTAL TIME
Introduction and discussion
Group work

Ask the participants to meet in their working groups and develop a plan for project administration, monitoring and utilisation of results. A brief summary should be made for presentation in plenary and for inclusion in the research proposal.

Plenary

Have each group present its plan for project administration, monitoring and utilisation of results, followed by discussion.

Designing and Conducting Health Systems Research Projects
Part I: Proposal Development and Fieldwork

Module 18
FINALISING AND REVIEWING
THE RESEARCH PROPOSAL

./img/designandco_319_la_0.jpg

NB: Development of a research process is a cyclical process. The double-headed arrows indicate that the process is never linear.

Module 18: FINALISING AND REVIEWING THE RESEARCH PROPOSAL

OBJECTIVES

At the end of the session you should be able to:

  1. Finalise the research proposal for the presentation to the relevant authorities
  2. Write a brief summary of the completed research proposal
  3. Prepare a letter of intent to send to potential funding agencies
  1. Finalising the research proposal
  2. Writing a summary of the research proposal
  3. Presenting the research proposal to the relevant authorities and potential funding agencies

I. FINALISING THE RESEARCH PROPOSAL

When you have finished the methodological section of your research proposal and have pre-tested the methodology or at least reviewed it thoroughly (Module 14), you can start preparing the final draft of various parts of your research proposal.

If relevant to the problem, you should first review again whether major cross-cutting issues have been addressed in a consistent way. These may include:

You should have considered these issues when analysing and prioritising your research problem (see Module 4); when developing your research objectives (see Module 6), when developing your data collection techniques (see Module 10), when selecting your study populations (see Module 11), and when planning for data analysis and utilisation of results, which should be reflected in your work plan and budget (Modules 13, 15, 16, 17).

You can start working on part I of Module 18 any time after the group work of Module 14 has been completed.

The outline of your research proposal, as presented in Module 1, is as follows:

SUMMARY
TABLE OF CONTENTS
1. INTRODUCTION

1.1 Background information (context of the problem)

1.2 Statement of the problem

1.3 Literature review (partly or fully integrated into 1.1 and 1.2)

2. OBJECTIVES
3. METHODOLOGY

3.1 Study type, data collection techniques used. Variables or research themes can either be annexed or (if succinct) included in the text after study type

3.2 Sample and sampling procedures

3.3 Plan for data collection

3.4 Plan for data processing and analysis

3.5 Ethical considerations

3.6 Pre-test

4. WORK PLAN(including description of project staff)
5. BUDGET (including explanatory note on major budget posts)
6. PLAN FOR ADMINISTRATION, MONITORING, AND UTILIZATION OF RESULTS
REFERENCES
ANNEXES

Annex 1. List of abbreviations, if applicable (can also go to the beginning of the proposal).

Annex 2. Interview guides/ questionnaires (and/or other data collection tools).

How should you proceed?
  1. The first section of your proposal contains background information, the statement of the problem and literature review. This section should convince the reader of the relevance of the study (magnitude, severity of the problem). It should provide enough background data for an outsider to understand the different aspects of the problem, or the different factors influencing the problem and the context in which it occurs. Your review of available literature and reports should further illustrate why the problem is important, not only in your own working area but probably also beyond.

    Remember that, if your problem has a strong global dimension, (e.g., HIV/AIDS, family planning) you might consider a separate section for reviewing international literature as an introduction to your statement of the problem (see Module 5). However, in many cases (especially if you are focussing on smaller, local research problems) it will be possible to include all literature you consulted in the ‘Background information’ and ‘Statement of the problem’ sections.

    Note:

    Remember that review of literature is an ongoing activity that will continue during fieldwork, data analysis and even report writing.

    You can justify your study by pointing to the gaps in available information, which you hope to fill in with the data from your planned research. Finally, you can increase the interest of your readers by summarising what results you hope will emerge from your study and how you plan to use them to help solve or alleviate the problem on which your study concentrates.

    You now have to thoroughly review the various pieces of text that you have produced during earlier sessions of the workshop, and rewrite them to form a coherent study design.

    Note:

    When developing your research methodology you may have somewhat revised the focus on your research problem; you may have become more specific as you proceeded, adding certain factors, or omitting others. Any adjustments should be included throughout the text of your proposal because all parts of the study should be consistent and logically connected to each other.

    When revising your proposal you can best work backward, from the data collection tools to the objectives and then to the statement of the problem, to check for consistency.

  2. Critically review your objectives. Check whether they still cover what you have planned to study, and whether they are specific enough.
  3. The next section presents the methodology. You have already prepared small sections focusing on various aspects of your methodology. You should now check the text for clarity of wording and logical coherence. (An outsider must be able to understand what you mean)
  4. Discussion of various ethical issues affecting your study may be scattered in different parts of your draft. Identify the most important issues and discuss them in a separate section. (Include, for example, issues relative to the selection of your topic, your methodology, and the collection of your data, where you will have to pay attention to by asking consent and avoiding any harm to your informants.)
  5. The last sections of the research proposal, which will focus on project management, the work plan, the budget and the plan for administration, monitoring and utilisation of results, are quite clear-cut. When writing them, consult the directions presented in the GROUP WORK sections of the respective modules.
  6. References should follow in a separate section at the end of the research proposal. Take care that you list the references in a consistent way. (Author(s), year, title, place, publisher, etc., see Module 5.)
  7. Annexes. You should add a list of abbreviations, if there are many, here or after the summary. Your data collection tools should be annexed, each with a number, so that you can easily refer in the text to the various instruments.
  8. Finally, prepare a title page, summary, table of contents and, if appropriate, acknowledgements.

GROUP WORK

  1. Prepare a final draft of your proposal following the guidelines presented above. It is advisable to work in groups of one or two persons, each with the responsibility for one or more sections.

    Take care that you number the sections, for example as in the outline presented in this module.

  2. Two persons should be responsible for final editing. They should review and revise the text so that it flows smoothly from one section to the next.
  3. All members of the group, including the facilitator, should have read all sections of the proposal before the final manuscript is handed in for typing.
  4. The team leader should be responsible for co-ordinating the production of the final draft of the proposal.

It is useful to prepare a list of all sections that have to be written (see Table of Contents) and make a note as group members go through each step in the production process.

For example:

./img/designandco_323_la_0.jpg

II. WRITING A SUMMARY OF THE RESEARCH PROPOSAL

When you have completed writing your research proposal, there is usually a need for the protocol to be reviewed by senior authorities and policy makers or funding agencies. For the purpose of obtaining approval from policy makers or very busy administrators it is advisable to add a summary (of no more than two pages) to the proposal.

A summary usually includes:

One page containing essential information such as:

You should put the summary at the beginning of the proposal, although it is the last thing you prepare.

After the summary, a table of contents should follow. Adding numbers to the pages of your report and including them in your table of contents is one of the last activities involved in preparing your proposal.

Then a title page should be prepared, containing the title of your study, the names of the researchers with their titles, the name of the institution that has organised the course (Ministry of Health, or Health Research Unit of the Ministry of Health, for example) and date of issue. Add also this is a research proposal in order to distinguish it clearly from the research report that will later appear, probably under the same name and with the same authors. Finally you may add a page of acknowledgements, thanking all who enabled you to develop and implement this study.

III. PRESENTING THE RESEARCH PROPOSAL TO THE RELEVANT AUTHORITIES

Before a research project can be implemented, the health systems research proposal usually has to be:

In certain circumstances some of the above steps may be combined.

The procedure for approval may require that the research proposal be submitted with an accompanying letter or prescribed form for the relevant authority. In addition, the researchers may be requested to make a brief verbal presentation or ‘defend’ the proposal in person.

At the end of the proposal development workshop, participants will make 7-10 minutes presentations of their research proposals to a panel so they will gain an appreciation for the concerns of the various approving agencies and acquire the skills to respond briefly and succinctly to questions relating to particular aspects of the proposal.

Presentation to a panel

The panel should consist of experienced researchers who will comment on the research aspects of the proposal and, on the other hand, health managers who are familiar with the problem that is being investigated and will therefore be competent to comment on the focus, scope and usefulness of the proposed study. The panel members should be given a copy of the summary of the research proposal before the presentation begins.

Each participant group should prepare a presentation covering briefly the salient points in each section of its proposal. Participants should be encouraged to use the overhead projector and to practice and time their presentation prior to the actual presentation to the panel. (See Trainer’s Notes on ‘Course Management during the Workshop’ for further details on presentation skills that should be stressed.)

The main points that should be emphasised in the presentation include:

  1. Title of the study
  2. A brief description of the problem, why the study is needed, what information is needed and how such information will be used
  3. Objectives of the study
  4. A brief statement on the type of study design, sample(s) and methods of data collection
  5. A summary of how and when the study will be implemented (where, by whom, when, etc.)
  6. A summary of how data will be analysed to provide the required information
  7. A summary of the main resources required (e.g., manpower, budget, transport)
  8. A brief summary of ethical considerations, and plan for project administration, monitoring and utilisation of results.

Although the presentation itself should be brief, participants should be prepared to respond to detailed questions on any of the aspects of the proposal that have been presented. Keep a list of research themes and/ or variables at hand for questions on content of the study.

Submission of the proposal

Accompanying letters should contain the title, the name of the team leader and project administrator, and the period over which the study will be carried out. If a letter is going to the national research council or a similar group you may briefly refer to your study’s methodology and expected results and mention where further details can be found in your proposal.

Prepare a letter of intent that clearly summarises your research proposal and the estimated resources required, to send to potential funding agencies. In letters to potential donors you should not only state the total amount required but also the account to which the money, if granted, should be credited. Furthermore, it may be advisable to develop a follow-up proposal with a budget line, if implementation of your research recommendations will require additional resources.

Trainer’s Notes

Module 18: FINALISING AND REVIEWING THE RESEARCH PROPOSAL

Timing and training methods

½ hour Introduction and discussion
8 hours+ Group work
Introduction and discussion

To ensure that the research proposals are prepared, word processed, and duplicated in time for the panel presentation, facilitators should monitor each group closely, making sure that the sections of the proposal that have been completed are promptly submitted to the secretariat or group members for word processing. Encourage groups to use the list of proposal sections presented in the group work as a checklist to keep production of the final draft co-ordinated.

Group work

The major task of the facilitator is to assist the team leader in distributing writing tasks among group members, in editing, in organising the word processing and making corrections in the different drafts. All group members should be involved in writing, either working in pairs or individually. They should read all sections written by others. The group as a whole should, in particular, discuss the summary.

If time allows, it is highly recommended that a draft of the proposal (or of the most important sections of the proposal) be given to a facilitator of one of the other groups for comments before the text is finalised.

Presentation of the research proposal to a panel

Trainers should use the presentations to the panel as an opportunity to establish interaction between participants, health managers and experienced researchers. This interaction will be beneficial to all parties. For example:

Briefing of panel members

The course facilitator should brief panel members so that they are familiar with the purpose of the presentation. It is useful to request that they behave as though they were members of a research committee that is responsible for approving the projects.

Time allocated

Allow 10 minutes for each presentation and 10-15 minutes for questions and discussion.

Role of facilitators

During the presentation, course facilitators should refrain, as far as possible, from intervening unless it is obvious that an important point is being misunderstood or overlooked. However, panel members or participants may require assistance on specific issues, and the facilitators should serve as resource persons for this purpose.

Designing and Conducting Health Systems Research Projects
Part I: Proposal Development and Fieldwork

Module 19
FIELDWORK ACTIVITIES

./img/designandco_329_la_0.jpg

MODULE 19: FIELDWORK ACTIVITIES

PROCEDURAL GUIDE
for course participants, so that during the field period they will have:

  1. Briefed managers, health service staff and community members regarding the project.
  2. Obtained the necessary permission to collect data.
  3. Identified and obtained the resources (manpower, materials etc.) needed to collect data.
  4. Reviewed the availability of subjects and information and organised logistics for data collection.
  5. Trained interviewers/research assistants/supervisors.
  6. Refined, pre-tested and revised the research instruments and procedures for data collection.
  7. Collected the required data.
  8. Processed the data.

The activities to be carried out between the workshop on proposal development and the workshop on data analysis and report writing consist of field operations and data processing. The facilitator/ resource person for the project should visit the research team at least once but preferably twice, including:

(S)he should also be available for consultation by telephone.

The activities that should be completed during the inter-workshop period are discussed below:

1. Briefing of managers, health service personnel and community members concerned

The purpose of the briefing is to obtain support for the project. Such support is necessary in order to obtain resources as well as to obtain permission to collect data. Attention to the following points will increase your chances of obtaining permission to conduct the study and being allocated adequate resources:

2. Identifying and obtaining project resources

Identify and obtain the resources (manpower, materials etc.) needed to collect data. Refer back to your project document so as to make sure that all the items needed for the study are included.

3. Reviewing availability of subjects and information*

It is important to make a personal visit to every site where data will be collected in order to understand the physical and manpower limitations, constraints and special circumstances that could influence data collection. During the visit:


* Sections 3 - 9 are partly based on the following: The Population Council (1970) A manual for Surveys of Fertility and Family Planning: Knowledge, Attitude and Practice. New York, and: Institute Kesihatan Umum (1986) National Health and Morbidity Survey, Supervisors’ Manual. Kuala Lumpur. (Monograph document)

4. Organising logistics for data collection

After finishing an inventory of available resources, the logistics for data collection should be organised. This will involve planning in detail how, where and when data collection will be carried out, elaborating the work plan prepared earlier.

5. Preparing fieldwork manuals

Manuals or instruction sheets should be prepared for:

6. Training of research team members, including assistants as well as supervisors

The research team including, in particular, research assistants who join just before the pre-test, must be given explicit training. They should not only be able to collect data properly but also understand other procedures such as the selection of sampling units, map reading and data handling. They may also be involved in the pre-test and in the adjustment of instruction sheets and data collection tools after the pre-test.

The training programme usually consists of:

The research assistants should be trained together with the whole team, including possible additional supervisors.

7. Conducting the pre-test in the research location, with preliminary data analysis and revision of data collection tools
8. Collecting data

Having obtained permission for the study, and having

the data collection can now be carried out.

9. Processing data

After collecting and sorting the data, all data should be checked for errors. The content may be converted into codes or keywords for processing by computer or using master sheets.

The steps during this process include:

  1. editing/cleaning,
  2. categorising and coding,
  3. summarising data on data master sheets, or
  4. if a computer is used, writing instructions to the computer analyst concerning data input and analysis.

Note:

Reread Module 13 for more information on steps in data processing and analysis.

1. Editing

During editing look for:

Editing should be done by the research team or under its direction. If several persons are involved in editing, as in the case of large surveys, an editing manual should be compiled beforehand.

2. Categorising and coding

A coding manual or coding instructions on the data collection tools, if required, should have been completed when the questionnaire was finalised after the pre-test. Look at Module 13 and Module 23 for coding instructions and for instructions on how to process data from open-ended questions, and at Module 10C for processing qualitative data from FGDs with key words.

3. Summarising data on master sheets

After the data have been edited and coded they may be summarised on data master sheets.

4. Computer analysis

If the study is large, or if there are other reasons for the use of a computer, instructions should be written for the computer analyst.

After editing, coding and summarising the data, a preliminary analysis can be made by hand or using the computer. (See Module 20.)

Final note:

Despite all the advise presented in this module, some EMERGENCIES may arise during the field work. WHAT SHOULD YOU DO?

  1. Use common sense
  2. Consult your principal investigator and co-researchers
  3. Consult your research proposal
  4. Consult the modules
  5. Write/phone/fax/email your facilitator
  6. Other (specify)

Trainer’s Notes

Module 19: FIELDWORK ACTIVITIES

Timing and training methods:

10-15 minutes Presentation and discussion
Presentation and discussion (10 minutes)

This module has been prepared to provide course participants with a succinct guide that covers all the tasks they must complete during the fieldwork period. The module need not to be presented in detail, but participants should be made aware of its content so that they will remember to consult it during appropriate stages in their fieldwork.

Role of facilitators during the fieldwork

Research teams should receive at least one, and preferably two visits from a facilitator during the fieldwork period. If only one visit is possible, this visit should focus on:

  1. Checking the progress of the project;
  2. If necessary, assisting in obtaining managerial support;
  3. Observing the real-life situation in which the project will be implemented, identifying problems and anticipating pitfalls;
  4. Evaluating the proposed methodology for data collection with the group (by discussing as well as by pre-testing in the field) and advising any necessary modifications in the research design (sampling, data collection tools);
  5. Assisting in the training of research assistants (if required); and
  6. Finalising and trying out the procedures for data processing and analysis during the pre-test, and, if a second visit is possible, assisting at the onset of data processing and analysis.
Suggested checklist
  1. Determine whether managers and health staff concerned have been adequately briefed.
  2. Determine whether permission for data collection has been obtained. (As resource person you can provide support by making courtesy calls.)
  3. Review each of the proposed methods of data collection, reassessing:
  4. Make sure that the research team has visited data collection sites and identified working procedures and conditions, constraints, possible additional resources.
  5. Determine whether the pre-test of data collection has been well planned. (Assist in any further planning, if necessary.)
  6. Assist with the pre-test and training of research assistants and advise on:

    NB: The sequence of these activities is arbitrary. Revision of instruments may occur twice, before and after pre-testing. Fieldwork manuals may be prepared before the pre-test but adjusted thereafter.

  7. Make arrangements for telephone or written consultation during subsequent stages of the fieldwork, encouraging the participants to contact you when necessary.

Designing and Conducting Health Systems Research Projects
Part I: Proposal Development and Fieldwork

Module 20
FIELDWORK REPORT

This page intentionally left blank.

MODULE 20: FIELDWORK REPORT

OBJECTIVES

After reading this module you should be able to:

  1. Summarise your field experiences and observations, including technical or logistical difficulties encountered in carrying out your research project.
  2. Assess the extent to which you are able to answer the specific objectives with the data you have collected.
  3. Summarise your main findings and preliminary conclusions for each objective.
  4. Identify areas in which you need to do further analyses and specify in what sets of data you will find the data.
  5. Produce a preliminary report which covers all the issues mentioned above.
  1. Introduction
  2. Content of the fieldwork report

I. INTRODUCTION:

Elaborating a fieldwork report is an entry point to HSR training on data analysis and research report writing. The research team, research assistants, if there were any, and facilitators should participate in this activity (see Module 15). Why should you prepare a fieldwork report that summarises your fieldwork experiences, observations, and preliminary conclusions? This will help you to:

II. CONTENT OF THE FIELDWORK REPORT

Your fieldwork report should include:

1. Fieldwork experience

Review your fieldwork experience and evaluate how well you were prepared technically (in terms of the methodology developed in your research proposal), and organisationally (work plan, budget and administrative procedures). Summarise your experience and your evaluation of it in at most two pages. Address questions such as those posed below:

2. Preliminary research findings

When presenting research findings:

Note:

If you have gone further with preliminary data processing and analysis, state what you have done and what remains to be done.

GROUP WORK

  1. Select a team member to be responsible for taking notes on your discussions and divide responsibilities for writing specific parts of the preliminary report.
  2. Complete the review of your fieldwork experiences as proposed in section 1 of this module, stressing the problems you experienced and whether and how they were overcome. This review should be very brief, at most two pages. Include a summary description of your sample populations (persons and/or records).
  3. Discuss your specific objectives one by one, brainstorming on whether you have data to answer the research questions implied in your objectives. Remember to consider not only data obtained from record reviews or interviews but also from informal interviews with key informants, FGDs and your own observations during fieldwork.
  4. State tentative conclusions you can draw from your research at this stage, using all data available.
  5. Give a brief overview of how far you have proceeded with data analysis and what remains to be done:
    • Processing of material:

      —Have master sheets been filled (or if a computer is used, has data entering been completed)?

      —Has all qualitative data been listed and categorised/coded?

    • Preliminary analysis:

      —Have all straight frequency counts been done?

      —To what extent have cross-tabulations been made?

      —Has the interpretation of qualitative data been completed?

    The preliminary report should not exceed four to five pages. Try to have it computerised or clearly written. It can be distributed at the beginning of the data analysis workshop. The main points from the report can be put on overhead sheets on the first workshop day for presentation in plenary.

Trainer’s Notes

Module 20: FIELDWORK REPORT

Timing and teaching methods

15 minutes Introduction and discussion

It is necessary to present this module during the workshop. Facilitators should stress that participants should consult the module when they have filled in their master sheets or have received the first printouts from the computer. It will guide them in beginning data analysis and preparing the preliminary report that the team leader will present on the first day of the data analysis workshop.

If a facilitator can pay two visits to the field, the second visit should preferably take place at the onset of data processing. He or she can then work with the group on this task, using Modules 13 and 20 as reference material.

Note:

The discussion of Modules 19 and 20 can easily be combined. The best opportunity to discuss them may be just before the course evaluation, when groups have handed in the last draft of their research proposals for finalising by a course secretary.

This page intentionally left blank.

ANNEX to the modules: GUIDELINES FOR ORGANISING SHORT HSR COURSES; Part I: PROPOSAL DEVELOPMENT AND FIELDWORK*

I. Planning for the workshop

II. Management during the workshop

III. Training methodology

IV. Implementation of projects (the fieldwork period)

Annex 1: Example of a workshop schedule

Annex 2: Guidelines for budgeting an HSR training course

Annex 3: Example of information circular for course participants (used in Malaysia)

Annex 4: Reply format for participants (used in Malaysia)

Annex 5: Example of a course evaluation form


* See also: The Population Council (1970) A manual for Surveys of Fertility and Family Planning: Knowledge, Attitude and Practice. New York and Institute Kesihatan Umum (1986) National Health and Morbidity Survey, Supervisor Manual Kuala Lumpur, Malaysia. (Monograph document)

This page intentionally left blank.

I. PLANNING FOR THE WORKSHOP

1. Selection of course facilitators

The course co-ordinator will be the chief organiser of the course. A course co-ordinator will usually be supported by at least four facilitators, depending on the number of research teams. Together, they will be responsible for planning the course content, preparing instructional objectives and guiding the learning process throughout the course. They will give lectures, facilitate group sessions and guide research projects. The course co-ordinator is usually attached to the HSR Unit of the Ministry of Health.

Course facilitators may be selected according to the following criteria:

2. Course administrator

Although the course co-ordinator is responsible for the overall functioning of the course, it is highly recommended that he delegate administrative tasks to a course administrator. The course administrator will, for example, make administrative arrangements, supervise support staff, (typists, drivers), ensure that participants and facilitators receive the necessary support to travel to and from the course site, and make sure that necessary payments are made and various other support tasks during and after the workshop are carried out promptly. The course administrator should attend all meetings of facilitators so that logistic support for the participants can be arranged for appropriate times.

3. Requesting consent for conducting the workshop

To obtain approval and funding to conduct the workshop, a proposal may have to be submitted to the relevant authorities for approval and funding approximately 12–18 months prior to the workshop. The proposal should include:

As workshops in general take place at the initiative of the HSR Unit of the MOH, a training institution or NGOs active in health, approval is usually a matter of rubber-stamping, if needed at all. However, sometimes (additional) funding has to be identified for the workshops or for implementation of the research proposals developed, which requires time and energy.

4. Request for consultants (if needed)

Requests for consultants should be based on specific terms of reference and made through the Ministry of Health or other initiating organisation to the relevant donor agency. The workshop proposal should be included with the request.

5. Pre-workshop preparatory activities

A course management team consisting of the course manager, a core group of facilitators, and a course administrator should be set up at least 5 months prior to the workshop, to:

(1) Selection of participants and research topics (about 4 months before the workshop)

The selection of participants is usually delegated by the HSR Unit or organising NGOs to the provincial levels, hence the need to give clear guidelines.

Number of participants: 20–25 (5–6 per team)

Criteria for selection:

The criteria for selection of the participants should be clearly defined, taking into consideration the types of participants who are available, their educational background, the feasibility of their incorporating research into their functions and their interest in research. The following factors have been found to be useful in selecting participants for the basic course described in this volume:

Communications with national and regional or provincial health managers or institutes that are invited to provide participants should be made 3-5 months before the workshop.

First, informal contact should be made with these authorities to inform them of the possibility of authorising staff to participate in an HSR training course, to enlist their interest and support in selecting problems for the research projects. The criteria for selection of participants and topics should be discussed, and the managers should be requested to explore potential topics with staff members whom they plan to select as participants. These should in turn consult other parties concerned (field staff, community members).

(2) Communication with selected participants

An information circular should be sent to all the participants selected, providing them with preliminary information on the workshop (similar to the one sent to their superiors). It should emphasise that they will be expected to do a research project themselves. The relevant sections of Module 3 can also be sent, providing there will be guidance for the selection of topics. (See a sample of an information circular in Annex 3).

On receiving confirmation of their participation, the prospective participants can be sent background-reading materials on HSR.

(3) Discussions on training methodology and training procedures

It is extremely important that the course management team as a whole takes time to discuss the course content and methodology. All facilitators should be very familiar with the training materials. Consensus will have to be reached concerning who will introduce the different modules and the role of facilitators during group work and plenaries. The capabilities of each facilitator will have to be discussed in relation to the training requirements.

(4) Ordering or duplicating sufficient copies of the training modules

This can be done through WHO Geneva, AFRO or other regional offices, or IDRC Ottawa.

(5) Selection of additional local resource persons

Additional local expertise in disciplines such as epidemiology, statistics, qualitative data collection and analysis, may be required if these specialisations are not sufficiently represented in the course management team. A local librarian or a researcher who is presently involved in an interesting HSR project may also provide ad hoc assistance.

Outside resource persons should generally not be asked to present modules, unless they are very familiar with the course and its training methodology. However, it is useful to invite them to one or more of the course sessions to familiarise them with the course; to introduce them to the participants as valuable resource persons (both during the course and afterwards); to make their expertise available during group work; and, finally, to enlist their support for the implementation of the proposals being developed.

(6) Invitation of authorities to open or close the course

Usually a high official from the Ministry of Health, or, if appropriate, a representative from another agency supporting the course, should be invited to open the course. This is a useful strategy for making high-level officials aware of HSR and motivating them to support it.

Usually, the official opening of courses takes place on the first morning. It might be worthwhile, however, to officially open the course the evening of the day previous to the first day of the workshop. This will save time. Alternatively, the opening may take place in the later afternoon of the first day or, better, the morning of the second day, when participants can present the final selection of their research topics.

(7) Invitation of donors

If you are considering inviting donors to explain what types of research projects they presently support and to provide details on research priorities and funding procedures, it is advisable to invite them for a panel session all together one evening. Time could be scheduled after the panel for teams to talk with the donors individually.

(8) Invitation of panel members for the presentations of proposals and research results

The plenary sessions at the end of the first and of the second workshop are extremely important. At the end of Workshop I each group presents its research proposal while at the end of Workshop II each group presents its research report, including a first draft of major findings, conclusions and recommendations. Each of these gives the participants a chance to gain experience in presenting research proposals or research papers. It also provides the opportunity to invite senior managers, researchers, academicians, etc., as panel members so that they gain a better understanding of health systems research and can have an input in the formulation of conclusions and recommendations. This will likely enhance their support for implementing them. Selection of appropriate panel members is important. Health managers of different levels and interested researches should be invited. If possible (distant-wise), it is highly recommended to include the direct superior(s) of the research team will utilise the research findings in the panel.

(9) Selection of support staff

Support staff for the workshop should include two secretaries and one driver/messenger. For the last 3 days of the workshop, four full time secretaries would be desirable, unless participants themselves do the word processing of part of the protocol. Secretaries may have to work overtime to finish typing the research tools before the pre-test and to finalise research proposals. For the data analysis workshop, one secretary may be sufficient during the first week, but when the groups start writing their reports four secretaries should preferably by available, unless participants are (partly) self-reliant. Ensure that there is a computer available for each research team, and two diskettes per group.

(10) Site preparation

Space required:

Materials required:

II. COURSE MANAGEMENT DURING THE WORKSHOP

1. Course co-ordinator

The course co-ordinator will have overall responsibility for the workshop. Some of the essential functions include:

2. Chairpersonship of plenary sessions

It may be useful to rotate the chairpersonship, depending on the subject being discussed. For instance, the facilitator presenting a specific module and guiding the following discussion could be the chairperson of that session.

3. Allocating facilitators to working groups

When the participants have selected their research topics, a final decision will have to be made as to which facilitators, considering their interest and expertise, would best be in charge of particular groups. Facilitators will, in principle, stay with the same groups throughout the course, in order to ensure continuity and the quality of the end product.

In addition, each facilitator may have overall responsibility for certain technical aspects of the research process in which he or she is specialised, and assist other groups as well. Also local resource persons may assist on an ad hoc basis (e.g., on sampling).

4. Facilitators’ meetings

It is desirable to have a daily meeting of facilitators to monitor course progress and give an opportunity to the facilitators to discuss possible problems This meeting is best held in the evening and will usually last between half an hour and an hour. The course co-ordinator should be responsible for convening this meeting. It is probably helpful to have a secretary record at least the action points for each meeting.

5. Approval of projects

The national agencies that will need to endorse the research proposals (the National Research Council, for example) will also have to be mobilised before as well as after the workshop, to speed up procedures.

6. Workshop report

The official report of the workshop should be as brief as possible. After a one page introduction (when, where, why, organisers, sponsors, management team (functions), type of participants in the course), a summary report of 2-4 pages could follow, describing the training process, starting with the topics chosen and ending with evaluation results. A list of participants and course facilitators and their addresses (by research group) could be annexed to the report, as well as (summaries of) opening speeches.

The report should contain the final drafts of the groups’ research proposals. It is highly recommended that the course facilitators as a group screen the proposals immediately after the workshop, because some items may have been dropped out or added to the proposal that would need some clarification. The final polishing up of the proposals can be completed in the month following the workshop. Usually procedures to obtain consent for implementing the proposals take up the first month, so there is some time to do any final editing needed.

7. Meeting the participants’ needs for technical support during research implementation

The facilitators that have assisted in the development of the proposals will also assist the groups in the implementation of the proposals. However, sometimes additional support may be required, (e.g., the assistance of an experienced sociologist or statistician for data collection and data processing).

The participants will need to state this in their proposals and include it in the projects’ budgets.

All groups will need assistance when they start sorting and processing their research data.

III. TRAINING METHODOLOGY

Sessions in this training course on health systems research contain the following components:

1. Introduction and discussion

The introduction and discussion period is used to briefly explain new concepts and their application. Inviting responses and suggestions from participants and listing these on a flipchart or using them as a starting point for discussion is an essential element of the training method. This increases the interest of participants and may bring up valuable points of view that would be missed in classical (pure lecture-type) classroom teaching. Encourage all to participate in the discussion.

Depending on the level of the participants, the facilitator can delete or add details in the introduction, preferably using the research proposals that are being developed as examples, in interaction with the respective teams.

The text of the sessions as given in the training modules is not meant to be followed word by word. Each introduction and discussion period should not last longer than at most one hour to 75 minutes.

2. Group work

The purpose of the group work is to develop four to five research proposals (one per working group) that should be ready for implementation by the end of Part I of the training course. Thus the facilitators need to always keep in mind that the proposals being developed need to be feasible and of good quality.

To increase the efficiency of the group work, a chairperson and rapporteur should be appointed for each group. The chairperson is not only responsible for leading the discussion, but also for dividing the work among group members. It is recommended that, after discussion within the work group, the group should split up into groups of two or three persons to work on separate parts of the task to be completed. The work of each sub-group can then be discussed and amended before presentation in plenary.

Each facilitator should be responsible for one group throughout the course, in order to ensure continuity. Facilitators should only change groups if they have major problems in assisting their own groups. Other facilitators and resource persons, of course, can be consulted at any time on technical issues. The amount of time the facilitator spends with his/her group will depend on the needs and demands of that group. In the beginning of the course the needs may be greater than towards the end. In principle facilitation is a full-time activity. Even if a facilitator is not permanently participating in the group work, (s)he should be available at all times for consultation.

The facilitator’s role in discussion is primarily to stimulate the group to find its own solutions. However, if the group is clearly going in the wrong direction the facilitator should provide more direct guidance. In the beginning a facilitator may have to keep the group from wasting time on less relevant issues, or prevent relevant issues brought up by group members from being dropped because not everyone sees their importance.

3. Exercises

There are two types of exercises. In some exercises groups practice the use of new concepts by working with case studies prepared in advance. For these, it is probably a good idea to organise groups of a different composition than for the group work, so that all participants get to know each other well. In the second type of exercise, each group will examine a component of the proposal that is being developed by another group and provide constructive criticism. Groups should be encouraged to put the summaries of their comments on flipcharts or transparencies for presentation in plenary and for reference by the group developing the proposal. Not all modules contain exercises. Exercises can be omitted or added, depending on the needs of the participants and the time available.

4. Plenary

Presentations of the results of group work or exercises in plenary require special skills. Before the first plenary (in which the research topics considered for the development of research proposals are presented) the importance of presenting the group reports clearly and audibly, and of using readable visual aids, should be discussed with the participants. The working groups can use either flipcharts or transparencies for the presentations. Flipcharts have the advantage that they can be easily referred to or elaborated on later in the working groups. However, if the plenary exceeds 25 persons, it may become difficult for all to read the flipcharts. The use of transparencies and an overhead projector, in that case, may be indicated.

It should be stressed that there are limitations to what one can put on a transparency or a flipchart. Prepare two examples, one of a readable and one of an unreadable transparency, and let the participants give suggestions concerning how much information a transparency should contain (12-14 lines is the limit).

Stress also that one should never turn one’s back to the audience when presenting. (A pointer can be used to indicate various points on the transparency, rather than on the screen.)

In general, the presentation of one working group should not exceed 15 minutes, discussion included. Sometimes even less time is required. If necessary, the facilitator chairing the session should let presenters know when they have just a minute or so left.

IV. SUPERVISION OF RESEARCH PROJECTS: INTER-WORKSHOP ACTIVITIES

The activities during the inter-workshop period consist of preparing and implementing the studies and (preliminary) processing of data. The facilitator or resource person for the project should visit at least once, preferably during the pre-test, and if possible also during the preparations for data processing.

1. Activities that should be carried out by participants

During this period, the participants should:

  1. Brief managers, health service staff and concerned community members regarding the project;
  2. Obtain the necessary permission to collect data;
  3. Identify and obtain the resources (manpower, materials, etc.) needed to collect data;
  4. Review the availability of subjects or respondents, information, ethical aspects of the study and adjust the methodology if necessary;
  5. Train research assistants, if required. Refine, pre-test and revise the research instruments and procedures for data collection and data analysis with the entire research team, preferably together with the facilitator;
  6. Collect data;
  7. Prepare for processing of data and do some of the processing by hand and by computer (if feasible and useful); and
  8. Prepare a preliminary report.
2. Guidelines for participants

Modules 19 and 2 0 should be presented to participants at the end of Workshop I for their use during the inter-workshop period. The content of the modules should be reviewed before they leave for the field, so participants will know what they contain and why they are important.

Module 19 will serve as a checklist as well as a guideline for field activities.

3. Visit by the facilitator/resource person

The purpose of the visit by the facilitator near the beginning of the fieldwork is to:

  1. Check the progress of the project;
  2. Observe the real life situation in which the project will be implemented and identify problems and anticipate pitfalls;
  3. Review the proposed methodology of data-collection;
  4. Advise on modifications/adaptation of the research design, sampling, data collection procedures, etc.;
  5. Assist in training team and possible research assistants in research- and interview techniques; and
  6. If necessary, assist in obtaining managerial support.

The trainer’s notes at the end of Module 19 provide a useful checklist for facilitators to use during their site visits.

Annex 1: Example of a course schedule (used in southern Africa)

WORKSHOP I: PROPOSAL DEVELOPMENT AND FIELDWORK

./img/designandco_357_la_0.jpg

./img/designandco_358_la_0.jpg

./img/designandco_359_la_0.jpg

./img/designandco_360_la_0.jpg

Annex 2: Guidelines for budgeting an HSR training course

The following items will probably have to be budgeted. Indicate for each item who will cover the cost (Ministry of Health or donor, for example). Salaries of local participants and transport are usually provided by the Ministry of Health, whereas accommodations and meals are usually covered by the donor.

(1) Accommodation and meals

Board and lodging or lodging and an allowance for meals for:

24 participants
5 facilitators
2 secretaries (perhaps 4 at the end of the workshop)

Make sure the workshop site has available:

Also include:

Consider the inclusion of a:

(2) Salaries and allowances

Facilitatorssalary at US$ . . . . . . ./day x 14-16 days
allowance at US$ . . . . . . ./day x 14-16 days
 
Participants salary at US$ . . . . . . ./day x 14-16 days (MOH)
allowance, if applicable at US$ . . . . . . ./day x 14-16 days (according to MOH rules)

(3) Transport

(4) Supplies

If photocopying is used during the workshop for duplication of the research protocol, questionnaires/tools, final report; and other documents:

If all duplicating is done using stencils:

40 note pads, 40 pens, 40 pencils, 40 rubbers, 40 file holders

35 name tags

paper clips, staplers, staples, paper hole punchers, scissors, chalk

200 overhead sheets, markers

5 flip charts, markers

35 copies of the modules, to be ordered through KIT, Amsterdam, AFRO or IDRC Ottawa, at least three months before the beginning of the workshop

Annex 3: Example of information circular for course participants (used in Malaysia)

Background

Health systems research (HSR) has been identified as an important tool to provide managers with information they can use in decision-making processes aimed at improving health care. In this context ‘managers’ could be those responsible for planning or implementing health programmes at district, state or national level or those responsible for managing hospitals, clinical units within hospitals or clinical outpatient services in hospitals, clinics etc.

Objective

The aim of the course is to enable you to develop and implement health systems research projects assisting managers, including yourselves, to improve the effectiveness and efficiency of health services.

Expected outcome and future functions

Once you have successfully completed the course, you will be expected to incorporate the conduct of HSR into your regular duties. As staff members who have had training in research, you will design and supervise projects and train your staff to collect data, analyse it, etc. You will also serve as ‘resource persons’ for your programmes, hospitals, etc., and provide assistance with the analysis of problems, design of studies, preparation of study reports.

Research skills can be acquired only through real life practice. Therefore this training programme is designed to give both theory as well as practical experience in conducting research. The practical experience will be in the form of a project that you will carry out in your place of work as a supervised training exercise. The course will be conducted in two sequential ‘parts’ with an ‘inter-workshop period’ during which you will actually carry out your study.

The structure of workshop will be:

1 . Pre-workshop assignment:

2. Part 1: Protocol development workshop (12-14 days)

3. Inter-workshop period (5-6 months)

4. Part 2: Data analysis/report writing workshop (14 days)

(The respective state/programme directors will attend these presentations and participate in the discussions.)

5. Time and research will have to be reserved for implementation the recommendations

Background reading

This workshop will be very intensive and you will need to do a considerable amount of background reading both before and during the workshop. Pre-workshop reading consists of: (List a number of relevant short papers on the concept and purpose HSR).

Selection of projects for the training exercise

Development and implementation of a research project will be the most important part of this training programme. The first step in doing research is to select a problem that is an appropriate topic for research. You will have to do this before you come to the workshop.

One of the basic principles of HSR is that research should focus on priority problems. Although the project that you will do during the workshop will be designated as a training exercise, the only difference from any other research project will be that the scope might have to be limited to enable you to complete the project before the Data Analysis/Report Writing Workshop. Therefore, the problems that are selected should conform to all the criteria that would be used in selecting projects for research, and the process of selection should be the same as in actual practice.

We suggest that you should meet with your state or regional/provincial and district medical officer, in order to identify one or two priority problems that need additional information that can be provided through research. (N.B. If sufficient information is already available either through routine data or from other studies, it is not suitable for a research project, even if the problem is a priority.)

Criteria for selecting a problem for research include:

If two or three problems emerge as suitable, a final decision concerning which to choose as the focus of the research can be made during the Protocol Development Workshop.

Preparation for the project

Before coming for the workshop, you should be able to answer the following questions regarding the problem that you will focus on in your research:

Available data on the problem

Manpower for data collection

Identify members of your staff such as nurses, medical assistants, health inspectors and others who can assist you in collecting data for your research project during the implementation period. You will need this information to help you determine how much data you can collect within the given time period.

./img/designandco_366_la_0.jpg

Evaluation questions: HSR Proposal Development Workshop in . . . . . . . . . . . . . . . . . .

  1. Have the objectives of the workshop been achieved?

    1.1 Have you gained the expertise to develop a research proposal? (explain)



    1.2 Have you developed a proposal that you think can be carried out by your group within the coming 5 months? (explain)



    1.3 Do you feel (1) motivated and (2) confident enough to start other small research projects in the future in your own working situation? (explain) □ Yes □ No



  2. Do you have any comments on the course content?

    Would certain parts need extension?

    Should certain parts be slimmed down?

    How clear were the presentations?

  3. What is your opinion of the training methods used in the workshop (compared, for example, to the ‘lecture’ type of teaching)?
  4. Did you find the division of time between lectures, group work and plenary satisfactory or would you propose more or less time for any of these three components of the course?


  5. Were you satisfied with the type of assistance provided by the facilitators?

    □ Yes □ Partly □ No

    Would you have any suggestions for improving the work of facilitators for similar courses in the future?


  6. How did you function as a group?

    Do you feel that every group member had an equal chance to gain from and contribute to the course?

    □ Yes □ Partly □ No

    (Explain)


    Would you have any suggestions for improving the opportunities for group members to gain from or contribute to similar courses in the future?


  7. Acknowledging that you are all busy people, but that the course was quite compact: Would you have liked the duration of the course longer, shorter or was it the right length as it was?


  8. What is your opinion on the organisation/accommodation/working conditions of the course?


    Do you have any suggestions for the next workshop in this respect?


  9. ANY OTHER COMMENTS

This page intentionally left blank.

ABOUT THE AUTHORS

Corlien M. Varkevisser, MA, PhD, MPH, is a medical sociologist-anthropologist by profession who specialised in public health. As a staff member of the Royal Tropical Institute, Amsterdam, and former head of the Primary Health Care (PHC) Unit, she has gained extensive experience in health systems research and PHC management in sub-Saharan Africa. She was one of the co-initiators of the Joint HSR Project (WHO/Netherlands Ministry for Development Cooperation/ Royal Tropical Institute) for Southern Africa and was based at the WHO Sub-regional Office in Harare as manager of the Joint HSR Project from its onset in April 1987 till 1992. Thereafter she became manager of the MPH course at the RTI and professor in HSR at the Faculty of Political and Social Sciences, University of Amsterdam.

Indra Pathmanathan, MMBS, MPH, is a physician specialised in public health who, as Head of the HSR program of the Ministry of Health in Malaysia since its inception, has been responsible for developing and implementing several strategies for HSR that have been replicated in other countries. These included training programs in HSR and Quality Assurance for decision-makers in ministries, for physicians, and for staff in district health teams, hospitals, and universities. She was a member of the Advisory Group on HSR, WHO-Geneva and served on the editorial board of BRIDGE. Over the past ten years she has been a consultant to the World Bank in the field of health.

Ann Brownlee, MA, PhD, is a medical sociologist who specialised in HSR, planning and evaluation, and cross-cultural aspects of health care. She served as Research and Evaluation Coordinator for the Project for Strengthening Health Delivery Systems in West and Central Africa for a number of years, where she worked closely with WHO’s Regional Office for Africa and with colleagues from Africa and elsewhere to develop an HSR training program and to publish the HSR Training Course that was a forerunner of this volume. She currently works as a consultant in international health for groups such as WHO, IDRC, and Wellstart and teaches at the University of California at San Diego.