Research and development 2018

research and development key performance indicators and research & development responsibilities
Dr.ThorasRyder Profile Pic
Dr.ThorasRyder,Hong Kong,Researcher
Published Date:07-07-2017
Your Website URL(Optional)
Comment
Research for Development 1 von Kurt Bangert Introduction: Every now and then, developmental NGOs are confronted with requests and suggestions to fund research projects. Since NGOs are not academic institutions, the question arises: why should they support or engage in research, and if so: which kind of research should be supported and which should not. Furthermore: if research is considered useful and in tune with the NGOs principles and mandate, then which standards should guide the research? Why Research may be Needed: There are three main reasons for developmental NGOs to engage in research: 1. Public Expectation: There is an increased public expectation for improved nonprofit governance of charitable agencies, including the need for better impact control. In the past, NGOs were often satis- fied with measuring input and output but not so much outcome and impact. Much of NGOs impact documentation has been anecdotal and incidental, rather than scientifically validated. 2. Donor Expectation: In addition to the above, more and more donor agencies (such as USAID, AUSAID, CIDA etc.), when granting funds for development programs, are requiring NGOs to demon- strate their professionalism and effectiveness through sound empirical and evaluative (i.e. evidence based) research. Needless to say, our sponsors, too, want to know if their donations have an effect and which. 3. Self-Expectation: NGOs have an interest of their own to know whether or not they are creating the impact they expect to create. That in itself may not always require scientific documentation, but sci- entific research may help them to verify (or falsify) the claims they make. Purpose for Research: Developmental NGO’s primary concern is to alleviate human suffering, overcome poverty and injus- tice, and assist people in developing countries to reach their full potential. NGOs do not engage in 1 Kurt Bangert is Research Director of the World Vision Institute for Research and Innovation, and Director of the German Institute for Poverty Reduction (Deutsches Institut für Armutsbekämpfung). research for purely academic reasons. It is not l’art pour l’art or research for research’s sake. Rather, it must help to increase the effectiveness and efficiency of development programs. Research should, for the most part, attempt to answer questions such as: Do development programs work? What are their impacts? Do existing methodologies work better than alternative choices? What is the rele- vance or viability of the current methods and how can they be improved? What new approaches can be pursued in order to achieve better results? If developmental NGO are to engage in research, it may be mainly for the following reasons: 1. to document that development programs are indeed creating the impact they are set out to cre- ate; 2. to improve programs for the sake of better cost-effectiveness (efficiency and effectiveness); 3. to ascertain new methodologies and solutions for addressing ever more complex problems in combating poverty, deprivation and injustice; 4. to back up demands for political policy change through scientifically sound information. In pursuing research for the sake of measuring, and improving, impact, we must not only address the question of what has been achieved (or not), but also of why it has been achieved. Types of Research: It will be good to know the different types of research, in order to understand which types are useful for development and which are not. In the scientific literature, there are no clear-cut and unanimous definitions of the different types of research. For this purpose, we are distinguishing between the following types (for a more detailed explanation, refer to the annex). We can group them into basic research and applied research. 1. BASIC RESEARCH Basic Research (also known as Pure Research) is designed to add to our fundamental understanding and knowledge of the world regardless of their immediate practical implications. Basic Research may involve developing and testing theories and hypotheses that are intellectually challenging to the re- searcher but may or may not have practical application at the present time or in the future. Basic research usually is the task of university departments, government-funded or otherwise funded re- search institutions established to advance general knowledge. There are different types of basic re- search: - Exploratory Research - Descriptive Research - Explanatory Research For a definition of these, please consult Annex I. Basic research is, under normal circumstances, not a priority for development NGOs. 2. APPLIED RESEARCH Applied Research is intended to be useful in the immediate future; it tests understanding and knowledge in terms of its application to a practical situation, such as policy or program implementa- tion. It suggests action or is designed to increase effectiveness in a particular area. Applied research is often done to solve specific, practical questions for policy formulation, administration and under- standing of a phenomenon. Applied research is meant to provide information that can and should be utilized for implementation. It is often commissioned by companies and/or organizations interested to improve products or services. There are different types of applied research such as (1) Implemen- tation and/or Action Research, (2) Evaluation Research, (3) Experimental Research, and (4) Opera- tions Research. It is mostly these types of research that a development NGO will and must engage in Implementation Research and Action Research can be distinguished from one another, but for this purpose they may be seen together. The objective of either one is to apply theoretical knowledge to programs and to make them more effective. Implementation research seeks to translate knowledge about effective interventions into practical delivery systems. Action research is to bring about social changes for the subjects’ own benefits. Improvement and Involvement are key to the success of this type of research. The target group is involved from the start. Evaluation Research is well known to development NGOs. Evaluation Research is to assess the im- pacts of programs, policies or legal changes and to determine whether they have achieved their in- tended outcomes. Evaluations are designed to improve these or similar programs, and their results and implications are usually communicated to clients and decision-makers for the future develop- ment and adaptation. Evaluation research is a complex form of research that may have different purposes; and different types of evaluation research have been identified such as: needs assessment, formative evaluation, cost-benefit evaluation, and summative evaluation. The differences between these four types of evaluation research are according to when they are carried out and also for what purpose they are carried out. - Needs assessment should be done mostly prior to program implementation but can also be done after the program has already started. - Formative evaluation is looking at whether the program is creating the impact it was de- signed to create; it is usually done after the program has run for some time but can some- times be done at an early stage; it is formative in that it will allow for adaptive changes of the program to make it more effective. - Cost-benefit evaluation may not so much look at the impact of a program (which is assumed or demonstrated to have taken place) but will investigate the cost-benefit ratio in order to make the program more efficient. Sometimes, too small an impact is affected at too high a cost. - Summative evaluation is usually done at the end of a program in order to assess whether it has reached its envisioned goals; its results may affect other, similar programs. Experimental Research is an applied research method designed to test the effect of certain variables (for instance: the number of visits by health workers to pregnant women). It assumes these variables have a direct causal effect on certain properties (such as child diseases) and that by changing the variables one will influence these properties in a desired direction. Experimental Research requires a Design of Experiment (DoE) which defines which variables to adjust and which indicators (outcome) to measure. Experimental Research often makes use of control groups which must be similar to the experimental groups, but for which the variables will not be adjusted in the same way. The use of control groups will help to demonstrate that the variables being tested are actually the causal factors producing the predicted results. The DoE will also serve to limit time, effort and budget while maxim- izing results. Operations Research, although often used for any kind of evidence-based experimental research, is little understood by most people. It is a sophisticated research method designed to test different methods in order to change programs for the purpose of improving their efficiency or effectiveness. It is usually carried out only in conjunction with a university department specializing in operations research. Despite its sophistication, it is increasingly required by donor agencies such as USAID in order to scientifically document program impact and make changes as needed. Operations research will usually involve control groups in order to compare them with those groups which will benefit from interventions. For further details of research types, see Annex I. Research Techniques Decision makers should be aware not only of different research types but also of different research techniques so they can ask valid questions regarding methodologies and identify weaknesses in the research design. The term techniques is used here for want of a better term. One may also think of them as instruments or components of research. Here are some of the most common research tech- niques that may be relevant for different research types: - Purpose Statement - Mathematical Modeling (MM) - Problem Statement - Control Group - Research Question - Management Involvement - Research Design - Partner Involvement - Literature review - Stakeholder Involvement - Data review - University Involvement - Critical appraisal - Conclusions - Hypothesis statement - Recommendations - Theory formation - Implementation Plan - Data gathering - Budget Acquisition Plan - Case study - Budget Control - Qualitative Survey - Quality Control - Quantitative Survey - Ethics Control - Representative Survey - Time Management Control - Experiment - Report - Design of Experiment (DoE) - Publication For brief definitions of these research techniques, see Annex 2. Research Applicable to Development NGOs Basic Research will not be a first priority for development NGOs and should be funded only when there is a special need of informing the public about general issues of need, poverty, and discrimina- tion, or when we need to back up policy demands to governments and other stakeholders. In such cases exploratory, explanatory and descriptive or other such research may be considered from time to time, if well justified and funds available. When, for advocacy purposes, so-called policy research is requested for funding, for which only a simple desk analysis or literature review is required, this should be called policy analysis rather than research. Policy Analyses may be pursued, and funded, if it helps to support advocacy issues with more substantial back-up information. In cases, when such literature reviews or policy analyses are extended for academic purposes (for the purpose of a Bachelor or Master thesis or a doctoral disser- tation), funding is normally not recommended unless the NGO has a high stake in the issue to be researched. Applied Research is the type of research most relevant to development NGOs, especially when it helps to answer questions such as these:  Do existing programs work?  What are their impacts?  Do existing methodologies work better than alternative choices?  What is the relevance or viability of practiced methods and how can they be improved?  What new approaches can be pursued in order to achieve better results? Among the different types of Applied Research, Evaluation Research will be a main focus. Among the different evaluation types, the Formative (or: Process) Evaluation is to be preferred as it is designed to adapt ongoing programs for the sake of improving their efficiency and effectiveness and their cost- benefit ratio. Summative Evaluation (ex-post or outcome) will be of less interest except perhaps to donors who wish to know if the funded programs have actually achieved the results they were de- signed to achieve. While one should have an interest in knowing whether a completed program ac- tually had the impact one had hoped, priority should be given to ensuring already at an early stage that the program is effective. Implementation or Action Research may also be considered, if and when - the community for which a program is to be designed is not yet fully sure of its needs and the direction of the future development is not yet fully clear; or when - it appears that interventions known to be effective have not yet been implemented in a community due to lack of knowledge, capacities, infrastructure or other obstacles; In these cases, such research ought to be carried out in close conjunction with the community con- cerned. Experimental Research will be needed to test certain interventions for their benefits to recipients. Such tests (or experiments) will be required if such intervention are thought to improve the effec- tiveness and/or efficiency of a service program, when there is as yet too little evidence that they actually will have that positive effect. In most cases, such research would involve control groups to ensure that the interventions tested are actually responsible for the positive effect. Such research can be carried out as part of a development program or prior to initiating such a program. It will normally involve the management of the program, and such research should be carried out only if and when provision is made to actually implement the program changes recommended by the re- search report. Sometimes, this type of research is called Operations Research (OR) although OR en- tails some further requirements. Operations Research will be needed from time to time, either to satisfy donor expectations or for inherent reasons, i.e. in order to design – and improve – programs in such a way as to aim for opti- mal impact. The more we professionalize our programs, the more we will be required to not only create any kind of impact, but an optimal impact with an optimal cost-benefit ratio. Operations Re- search will only be useful if management staff will be involved from the start and are committed to implementing the best possible solutions within a large catchment area. Research Standards The research development NGOs will engage in should meet scientifically accepted standards. For the sake of simplification, research standards are defined here by correlating research types and their respective research techniques (instruments or components). For the sake of a quick overview, the information about which types of research will require which techniques is given in the following table . In some cases, one may deviate from the table, depending on the exact nature of the re- search. In any case, it is assumed here that research standards are observed as long as all research techniques required for certain types of research are carried out. The table distinguishes between mandatory and optional techniques. Research Standards: Table “Types and Techniques” The following table lists the different research types at the top and the different research techniques at the left. An x indicates which technique is required for which type. An (x) is considered optional. There may be some variation from time to time, depending on the exact nature of the research. It is generally assumed that if the techniques indicated for a respective type are included in the research, then good standards will be achieved. ANNEX 1 Research Types: Definitions First, we can make a distinction between Basic Research and Applied Research. Basic research is non- utilitarian, whereas applied research is supposed to be useful. Basic Research (also known as Pure Research) is designed to add to our fundamental understanding and knowledge of the world regardless of their immediate practical implications. Basic Research may involve developing and testing theories and hypotheses that are intellectually challenging to the re- searcher but may or may not have practical application at the present time or in the future. Basic research usually is the task of university departments, government-funded or otherwise funded re- search institutions established to advance general knowledge. Applied Research is intended to be useful in the immediate future; it tests understanding and knowledge in terms of its application to a practical situation, such as policy or program implementa- tion. It suggests action or is designed to increase effectiveness in a particular area. Applied research is often done to solve specific, practical questions for policy formulation, administration and under- standing of a phenomenon. Applied research is meant to provide information that can and should be utilized for implementation. It is often commissioned by companies and/or organizations interested to improve products or services. Some experts distinguish further between basic, applied, and practical research. Practical research is thought to go one step beyond applied research as it is supposed to deal with concrete and practical real-life situations. BASIC RESEARCH Research, rather than being categorized according to its utilitarian or non-utilitarian nature, can also be categorized according to its main objective. In that sense, research has been classified as being exploratory, descriptive and explanatory. Exploratory Research can be described as research on a relatively unstudied topic or in a new area. Exploratory research is fundamental in nature and does not have any immediate applicability and does not provide sufficient evidence to base far-reaching decisions on. Exploratory studies are for the purpose of gaining new insights, discovering new ideas, and increasing knowledge of particular phe- nomena not yet fully understood. The researcher starts with observations about the subject of study and tries to develop tentative generalizations about it. This research uses an inductive approach which moves from observable phenomena to some general interpretations. The conclusions it may formulate, are drawn with extreme care and caution. Often, qualitative data is collected, analyzed and interpreted. In the social sciences, the question may be asked: What is going on? Or: What is the problem? An example could be to investigate how female prisoners manage motherhood from pris- on. While conclusions may still be preliminary and somewhat hypothetical in nature and not yet based on hard and solid evidence, they should make use of empirical (qualitative) data, either al- ready available or collected during the research, however difficult that may be. Definition: Exploratory Research is inductive research on an unstudied topic or in a new area through gathering and analyzing qualitative data that is gained from a few cases, with the overall goal to become familiar with this area, to develop some general ideas about it, to generate theoretical per- spectives and statements on the topic and to spot themes for further research. Descriptive Research intends to describe groups, activities, situations, phenomena or events with a focus on structure, attitudes or behavior. Typically, the researcher already has a general under- standing of the topic before he/she starts to collect data. Descriptive Research answers the questions who, what, where, when and how... The overall goal of this kind of research is to gain an accurate picture of the topic under study. It describes how things are or how things have changed over time. Descriptive studies are a means of discovering new meaning, describing what exists, determining the frequency with which something occurs, and categorizing information. One way of conducting de- scriptive research is a census. An example of descriptive research would be a study which finds that Christian couples are twice as likely to divorce as Jewish couples. Typically, the data is collected through surveys or content analysis and involves a large number of cases, which requires the use of quantitative data analysis (statistics). Although the data description is factual, accurate and systemat- ic, the research does not normally describe what caused a situation. Thus, Descriptive Research can- not be used to make statements about a causal relationship where one variable affects another. In 2 other words, descriptive research can be said to have a low requirement for internal validity. Definition: Descriptive Research is designed to answer questions such as who, what, where, when and how. It describes a topic or state by describing structures, attitudes or behavior, about which the re- searcher typically already knows something before data is collected. Data collection is usually done by gathering and analyzing data of a large number of cases using quantitative surveys with the overall goal to gain an accurate and precise picture of the topic or phenomenon under investigation. Explanatory Research is to answer the question of why. It seeks to identify causes for a given phe- nomenon. Explanatory research goes beyond descriptive research as it looks for reasons why things are the way they are. For instance, if descriptive research found out that Christian couples are twice as likely to divorce as Jewish couples, explanatory research would be interested in the reasons be- hind that phenomenon. Or it could ask: Why are some people more likely than others to do work for pay after they retire from their regular lifetime work? This kind of research uses a deductive ap- proach by moving from general statements to statements less general. Pre-existing theories are of- ten used to develop preliminary ideas of a possible explanation, and then collect further data to veri- fy or modify that initial explanation. This kind of research typically requires a large number of cases with additional questions to be asked to as to allow for further in-depth analyses. Definition: Explanatory research is intended to explain observed variations by identifying causes and reasons for these variations using preliminary explanatory theories that are tested by collecting fur- ther data from a relatively large number of cases and doing quantitative data analysis by moving deductively from a more general statement to more specific statements. 2 We speak of Internal Validity when a causal relationship can be demonstrated between two variables. APPLIED RESEARCH In Applied Research, we can distinguish between Implementation Research, Action Research, Evalua- tion Research, and Operations Research. Implementation Research has the purpose of turning knowledge into delivery systems. For instance, research in maternal, newborn and child health, and research regarding neglected infectious diseases have advanced significantly in recent years; most research, however, has focused on the develop- ment of new interventions rather than optimizing the delivery of existing interventions. Implementa- tion research addresses these delivery issues, exploring the challenges of applying research findings to the real world. We may have the academic and even experimental knowledge of how to address certain health and social issues, but we may not yet have implemented the avenues or infrastruc- tures to actually provide the services to those who need them. Implementation research looks into how these knowledge and service gaps can be overcome. It also looks at why service providers have not been as successful as they might have been, and what can be done to make them more effective. Implementation Research has also been taken to denote research that is simply looking at “what is happening” in a program. As such, it may be considered to be part of evaluation research, with the focus, however, being not so much on impact alone but on other aspects such as goals, concepts, designs, suitability, target population etc. It will ask such questions as: Is the program based on sound theory and practice? Is it suitable for the particular environment? Does the agency carrying it out have sufficient know-how, capacities and resources? Is data being collected and documented to evaluate impact? Definition: Implementation Research seeks to translate knowledge about effective interventions into practical delivery systems. It looks at obstacles to interventions and methods to overcome them. It will address social issues not for the sake of simply developing new intervention, but for the sake of optimizing delivery of existing interventions. Action Research generally aims at initiating and sustaining processes designed to bring about social changes, particularly those having to do with the subjects’ own attitude or behavior. The social changes targeted are thought to come about mostly through people’s own efforts. Improvement as well as involvement is essential for action research. Central to this kind of research is the collabora- tion between researchers and those who are the focus of the research (e.g. community). Hence, this type of research is both process-oriented as well as participation-oriented by definition. The action that comes from the research is intended to help the community or organization rather than the research community. Definition: Action Research targets changes in social attitude and behavior in the target group to bring about social changes for the subjects’ own benefits. Improvement and Involvement are key to the success of this research. The target group must be involved in the whole process from the start. Evaluation Research has the purpose of assessing the impacts of programs, policies, or legal changes. It is a form of applied research as it is carried out to provide organizations with information that can be utilized in the immediate future. Often, this type of research focuses on the question of whether a program or policy has reached the intended or anticipated outcomes. The results and implications of an evaluation research normally have to be communicated both to the institution having ordered the evaluation and to the decision makers of the program under investigation for the purpose of further planning, adaptation and monitoring of that program. Within evaluation research, we can further distinguish between certain types of evaluation research which differ according to their specific pur- pose or focus: Summative (or: Outcome) Evaluation, Formative (or: Process) Evaluation, Needs As- sessment and Cost-Benefit Evaluation. Summative Evaluation is carried out in programs already completed in order to arrive at a fi- nal opinion regarding its impact; i.e. the fulfillment of its anticipated results. This evaluation is result-oriented and compares the anticipated outcome with the final outcome. The results of this evaluation may have implications for other, similar or future programs which may be adapted accordingly. In research, this type of evaluation may also be called ex-post evalua- tion. Formative Evaluation focuses on the design or early implementation stages of a program or policy before it is fully implemented or while it is still being carried out, or “in process”, as it were (hence, this is also called process evaluation). Formative evaluation is done at a time when it still expedient to change the program under investigation in order to correct weak- nesses discovered by the evaluation. It is evaluation for the sake of intervention. Results of the evaluation may lead to changing the approach and intervention methods of the program or even the objectives and outcomes to be achieved. Needs Assessment can be described as an analysis that ascertains social or health needs in a specific geographic area or a population group. It is carried out prior to program implementa- tion and will help to determine the objectives and/or methods of that program. A Cost-benefit Evaluation is research comparing a program’s costs to its benefits. In order to do so, such an analysis usually compares one specific program with other, similar types of programs in order to arrive at an opinion regarding the social returns on the money invested. It asks: What is the cost for a certain desired outcome? And: Is the outcome worth the in- vestment? The costs are usually calculated in financial terms, but may also involve non- monetary costs. Definition: Evaluation Research is a complex form of research that may have different purposes, but its overall goal is to assess the impacts of programs, policies or legal changes and to determine whether they have achieved their intended outcomes. Evaluations are designed to improve these or similar programs, and their results and implications are usually communicated to clients and decision- makers for the future development and adaptation of the program or policy. Experimental Research is an applied research method designed to test the effect of certain variables (for instance: the number of visits by health workers to pregnant women). It assumes these variables have a direct causal effect on certain properties (such as child diseases) and that by changing the variables one will influence these properties in a desired direction. Experimental Research requires a Design of Experiment (DoE) which defines which variables to adjust and which indicators (outcome) to measure. Experimental Research often makes use of control groups which must be similar to the experimental groups, but for which the variables will not be adjusted in the same way. The use of control groups will help to demonstrate that the variables being tested are actually the causal factors producing the predicted results. The DoE will also serve to limit time, effort and budget while maxim- izing results. One can distinguish between true experimental designs and quasi-experimental designs. True experimental design is regarded as the most accurate form of experimental research in that it tries to prove or disprove a hypothesis mathematically, with statistical analysis. For an experiment to be true-type, the sample group must be assigned randomly; the control group must be truly comparable, and only one variable will be tested at one time. A true type ex- periment would not only test for “if A, then B”, but also for “if not A, then not B”. However, this is sometimes difficult to achieve in the social sciences. Quasi-experimental designs involve selecting groups, upon which a variable is tested, without any random pre-selection processes. Experiments of this kind lend themselves for social sci- ence experiments where pre-selection and randomization of groups is often difficult. “E.g. if we study the effect of maternal alcohol use when the mother is pregnant, we know that al- cohol does harm embryos. A strict experimental design would include that mothers were randomly assigned to drink alcohol. This would be highly illegal because of the possible harm 3 the study might do to the embryos.” Definition: Experimental Research will test the effect of certain measures on the behavior or well- being of people. It is an applied research method involving an experiment in which one or more varia- bles are manipulated in order to influence specific desirable outcomes which can be measured on the basis of clearly defined indicators. Experimental Research usually involves a control group. Operations Research (OR) is understood to be a highly-scientific research on “operations”, i.e. on processes within an organization or programs of an organization. The objective of this research is to find practical solutions to operational or strategic problems in order to improve a program’s (or an organization’s) effectiveness and/or efficiency. OR usually involves a mathematical calculation de- signed to find the best solution possible within a given set of parameters. Hence, this research is the search for optimality. Usually, there will be multiple solutions that will need to be tested in order to find the optimum. The test involves trying out different variables or, in the social field, working with control groups so as to find the best possible course of action to achieve the best possible outcome. Operations Research originally constituted a sub-discipline of mathematics; and mathematical mod- eling still constitutes the “core” of OR. However, in the social field, most of the research involves gathering data following the implementation of certain types of measures designed to address social problems. Operations Research must involve decision makers from the very start in order to apply the leverage to actually implement the solution that the research will identify as being the best. Op- erations Research usually involves the following steps or phases: (1) defining a problem; (2) formulat- ing (several) solutions; (3) designing an experiment to test these solution and carrying out the exper- iment in the field; (4) gathering data from the experiment; (5) calculating the best possible solution from the data (the solution may actually turn out to be different from the solutions tested); (6) im- plementing the best possible solution (under a given set of parameters and/or constraints). The fol- lowing definition of OR includes all the necessary steps to be included in this type of research: Definition: Operations Research (OR) is the process of identifying and analyzing a problem within the operations (activities) of an organization or a program to be set up or already in place, and of finding 3 Example taken from: http://www.experiment-resources.com/quasi-experimental-design.html an optimal solution for this problem by testing alternative ways of action (models) which will serve as a basis for decision making regarding successive modifications of key variables to achieve the overall goal of enhancing or even optimizing the efficiency and effectiveness of the program or organization. ANNEX 2 Research Techniques: Definitions Different Research Types require different techniques (instruments or components); some tech- niques are applicable to all types. Which technique is required for each type of research is given in table of Annex 3. It is assumed that high quality research is achieved if all techniques applicable to a respective research type is implemented. Below, please find definitions of the various techniques: - Purpose Statement: a statement as to why this research is proposed or carried out; - Problem Statement: a statement describing precisely the problem to be researched; - Research Question: a statement in the form of a question that should be answered by the re- search under consideration; - Research Design: a description of the structure of the research holding the different parts of the research together; it helps to turn the research question into a research strategy; - Literature review: an overview of what others have written on a certain topic. It is mostly for information gathering and to sound out if similar research has already been done by some- one else; - Data review: an overview of relevant data that has been gathered/published prior to this re- search; - Critical appraisal: an unbiased analysis of available information or data; - Hypothesis statement: an attempt at describing a problem by theorizing about the relation- ship between cause and effect and about a possible solution; it can be phrased as a question; - Theory formation: a statement of hypothetical nature that makes predictions about the out- come of an empirical test or experiment; it can also be phrased as a question; - Data gathering: an attempt at retrieving relevant data either from available sources, or – if they should be lacking – by way of a practical, hands-on approach of collecting data through other means such as observation, interviews, surveys etc. - Case study: an empirical in-depth enquiry (rather than a large-scale representative data col- lection) that investigates a phenomenon by looking at a single instance or “case” in its real- life context; case studies may not allow for sweeping generalizations, but may give some in- dication for formulating a hypothesis; - Qualitative Survey: an in-depth survey of a small, but typical group representing the target population; usually carried out through face-to-face interviews; sometimes focus-groups are collectively questioned; - Quantitative Survey: an information-gathering method surveying a sample group more or less representative of the target group under investigation; this method usually involves a 4 standardized questionnaire; - Representative Survey: an information or opinion-gathering method by way of surveying a sample (population) that is representative of the target group in question; to be truly repre- 5 sentative, the sample must not only be high in number but also fair in composition; - Experiment: a method of gathering empirical data in order to test a theory for its validity; it usually involves the manipulation of one or more variables to better understand the relation- ship of cause and effect; - Design of Experiment (DoE): a detailed description of how an experiment is to be carried out to generate valid statistical data; it assumes a causal relationship between set parameters and dependent variables; the DoE will plan how inputs will be varied and which outputs (in- dicators) will be observed/measured. The DoE must have strong controls to ensure variables tested are sufficiently isolated and not compromised by incidental influences; a major aim of the DoE is to limit time, effort and budget while maximizing results; - Mathematical Modeling (MM): can be described as “thought experiments” about real-world phenomena, which – with the help of mathematical equations – can simulate, and hence predict, social changes based upon the modification of key variables; MM may be crucial in designing experiments and predicting their outcome; - Control Group: a group (of people) similar to the group selected for testing certain variables. In the control group the variables are not applied and can therefore not influence the out- come to be measured. The control group must be clearly separated from the experimental group(s). Using control groups as a base line allows researchers to demonstrate that the vari- ables being tested on the experiment group are actually the causal factors producing the re- sults that were predicted; 4 The size of the sample should be selected according to the total number of the group in question. As a rule of thumb, taking the root of a target population makes a good sample. 5 If the target population has 30% below the age of a5, then the sample must also contain 30% below 15. A sample of 400 can be representative, but will have an error margin of some 5%; with a sample of 4000-5000 the margin of error will be reduced to below 1%. - Management Involvement: Inclusion of key decision makers responsible for the (develop- ment) program, for the twin-purpose of (a) supporting the research/experiment and (b) for implementing the recommendations growing out of the research/experiment; - Partner Involvement: Inclusion of target group representation for the purpose of offering in- sight and support to the research to be carried out on that group; - Stakeholder Involvement: Inclusion of other players having a vested interest in the topic un- der investigation, such as donor agencies, governments, universities, publishers etc. - University Involvement: Partnership with a university department that is well-positioned to assist in, or lead out in, the research to be undertaken. As for Operations Research it will be required in most cases; for other types of research it is optional. - Conclusions: a summary of how the research (including a possible experiment) has contribut- ed to answering the research question, solved the problem that had been defined, attained the purpose of the research, and yielded knowledge about the phenomenon being studied; - Recommendations: a set of statements growing out of the research and the conclusions reached, that are meant to be acted upon; - Implementation Plan: a strategy or schedule for implementing the recommendations grow- ing out of the research in order to maximize on the lessons learnt through the research; - Budget Acquisition Plan: a strategy for soliciting financial support for the research to be car- ried out; - Budget Control: a system of checks and balances to be set up for ensuring that the budget al- located for the research is closely adhered to according to the budget designation; - Quality Control: a system for ensuring high research standards are maintained at all times so that the research will actually produde the results it was anticipated to yield; - Ethics Control: a supervision designed to watch over design, methods, and implementation of the research to ensure ethical standards are adhered to with regard to protecting the integri- ty of target groups involved; - Time Management Control: supervision to ensure time schedules are adhered to according to plan; - Report: written documentation about the whole research process including problem defini- tion, theory formulation, design of experiment, data gathering and analysis, conclusions, rec- ommendations etc. - Publication: information regarding when and where the report will be published to achieve wide dissemination for the purpose of maximizing on lessons learnt from the research. BIBLIOGRAPHY Austin, Sara L.: “WV’s Draft Guidelines for R&D with and/or about Children”, 2006 Borrmann, Axel and Reinhard Stockmann: Evaluation in der deutschen Entwicklungszusammenarbeit, Bd. 1: Systemanalyse, Waxmann: Münster 2009. Borrmann, Axel and Reinhard Stockmann: Evaluation in der deutschen Entwicklungszusammenarbeit, Bd. 2: Fallstudien, Waxmann: Münster 2009. Carboni, Isabelle: “Guiding Principles for National Offices. Measuring and Reporting the Child Well- being Targets”, May 2011. Hillier, Frederick S. and Gerald J. Lieberman: Introduction to Operations Research, McGraw-Hill: New York 2010. Laws, Sophie with Caroline Harper and Rachel Marcus: Research for Development: A Practical Guide, Sage/Save the Children 2003. Siegel, Lena: “Operations Research. Definition, Purpose and Applicability”, Background Paper, World Vision Institute, 2011. Stebbins, Robert A.: Exploratory Research in the Social Sciences, Sage: Thousand Oaks/London/N.Delhi 2001. Stockmann, Reinhard (Ed.): Handbuch zur Evaluation. Eine praktische Handlungsanleitung, Waxmann: Münster 2007. Werner, Alan: A Guide to Implementation Research, Urban Institute Press: Washington 2003. Online Sources General http://www.socialresearchmethods.net/kb/index.php http://en.wikipedia.org/wiki/Research_design Research Design http://www.nyu.edu/classes/bkg/methods/005847ch1.pdf http://en.wikipedia.org/wiki/Research_design Research Methods http://www.webster.edu/woolflm/statmethods.html http://www.ischool.utexas.edu/palmquis/courses/RsrchHome.html http://www.lindaswebs.org.uk/Page3_Orglearning/Resmeth/ResMeth.htm Basic Research http://www.lbl.gov/Education/ELSI/research-main.html Applied Research http://en.wikipedia.org/wiki/Applied_research http://www.wisegeek.com/what-is-applied-research.htm Exploratory Research http://en.wikipedia.org/wiki/Exploratory_research http://www.marketresearchworld.net/index.php?option=com_content&task=view&id=798 Descriptive Research http://en.wikipedia.org/wiki/Descriptive_research http://www.aect.org/edtech/ed1/41/41-01.html http://spirit.lib.uconn.edu/punj/m3507.pdf http://facultyweb.berry.edu/nmiller/classinfo/323/1.2/conducting_descriptive.htm http://www.marketresearchworld.net/index.php?option=com_content&task=view&id=800&Itemid= 64 Explanatory Research http://www.blurtit.com/q415229.html http://en.wikipedia.org/wiki/Explanatory_style Implementation Research http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0030186 http://www.who.int/alliance-hpsr/projects/implementationresearch/en/index.html http://www.urban.org/toolkit/data-methods/implementation.cfm http://www.globalhealthmagazine.com/cover_stories/implementation_research Action Research http://www.web.net/robrien/papers/arfinal.html http://www.lab.brown.edu/pubs/themes_ed/act_research.pdf http://en.wikipedia.org/wiki/Action_research http://de.wikipedia.org/wiki/Aktionsforschung Evaluation Research http://www.socialresearchmethods.net/kb/evaluation.php http://en.wikipedia.org/wiki/Evaluation Experimental Research http://www.aect.org/edtech/ed1/38.pdf Operations Research http://en.wikipedia.org/wiki/Operations_research http://de.wikipedia.org/wiki/Operations_Research Research Techniques http://www.wisegeek.com/what-are-the-different-types-of-research-techniques.htm http://www.experiment-resources.com/research-methodology.html http://tqu-group.com/downloads/doedownload.pdf (on Design of Experiment)