How Research methods is important

what research methods can i use for my dissertation and what is quantitative research methods with examples | download free pdf
Dr.KeiraCollins Profile Pic
Dr.KeiraCollins,United States,Professional
Published Date:07-07-2017
Your Website URL(Optional)
CLES Research Methods Handbook Introductory guide to research methods for social research Stuart MacDonald & Nicola Headlam, CLESAbout CLES Preface e C Th entre for Local Economic Strategies (CLES) is a not-for-profit Ee ff ctive research methods are the tools by which information is gath- think-tank, consultancy and network of subscribing organisations ered. Without the appropriate design and use of research methods, we specialising in regeneration, local economic development and local are unlikely to gather quality information and as such create a shaky governance, which was founded in 1986. foundation to any review, evaluation or future strategy. For CLES, the group of research methods contained within this handbook are the tool box, and like any tools need to be used in the right way - for the right job. Research methods, if understood and used appropriately can make your job a lot easier. At CLES we use a range of research methods in our policy and consultancy work and are skilled in using them. How- ever, we do not believe that the use of research methods is the preserve of so called ‘experts’ and in all instances requires sophisticated knowl- edge and practice. Clearly, to be experts in their use, requires practice, but like any tool, the basic principle behind its use can be understood and applied, by all. Furthermore, for those who have no intention of actually using a research method, it is important, perhaps in your work in overseeing or commissioning research activity, to know what the purpose of particular research methods are. This enables you to assess the appropriateness of their use. This handbook, therefore attempts on the one hand to explain and demystify the world of research methods, whilst on the other it seeks to provide a starting point for their use. In this, we are not suggesting that using research methods is easy, but merely that it can be appreciat- ed and undertaken by practitioners and non-research experts. We hope you enjoy this handbook which is linked to our annual training pro- gramme. This handbook, reflects our wider charitable aim to develop and improve the performance of the regeneration and local economic development sector. In this, we hope this handbook goes some way in helping to address some of the persistent issues faced by local places Centre for Local Economic Strategies and communities and contributes to ensuring positive local change. Express Networks • 1 George Leigh Street Neil McInroy Manchester M4 5DL Chief Executive, Centre for Local Economic Strategies. tel 0161 236 7036 fax 0161 236 1891 CLES is a registered charity no.1089503 Company limited by guarantee no. 4242937 • vat no. 451 4033 86 CLES Consulting is the trading name of CLES European Research Network Company limited by guarantee no. 2467769 • vat no. 519 493 812 ISBN: 1870053656 printed on100% post-consumer recycled paper 2 3Contents Introduction 7 Types of method 8 Quantitative Research Methods 11 Quantitative survey 11 Secondary data collation and analysis 20 Statistical analysis 24 Qualitative Research Methods 35 Social surveys/questionnaires 35 Interviews 39 Discussion Groups 43 Workshops 47 Observation 50 Visual Techniques 53 Research Issues 59 Glossary 61Introduction We are all “users” of social research. How we apprehend and absorb information and use our critical faculties feeds what we believe about the social world. From surveys reported in the media to market research, opinion poll- ing and large scale datasets such as the British Crime Survey. Social Research Methods are the tools used to explain social phenomena and often it is more possible to challenge conclusions if you are at least conversant with the variety of methodologies and tools applied. What is this book about? This book provides an introduction to the reader to a whole range of research methods. It aims to introduce a toolkit of methods, explain- ing how to use them, their appropriateness and some of the pitfalls of using them in practice. e s Th ections explain in turn: • What is the method? • When should it be used? • What do I need to consider? • What is the output? • How should it be analysed? • Examples from practice • Pros/Cons • Further Reading er Th e is also a glossary of technical terms at the back of the book. Who should use this book? This handbook will be useful for those working in local authorities, re- generation partnerships or other public agencies where research is con- ducted and commissioned. It can help in undertaking research in the selection of appropriate methods, making decisions about the mix of methods, and the planning of a research strategy where it is necessary to make explicit judgements about a hierarchy of evidence, the weight afforded to the various elements, and how they might combine to give a rounded perspective. It can also help in understanding research propos- als, and the methodologies presented to address a particular research question. 6 7Types of method In any form of research, you will be required to either count things and/ Quantitative Qualitative or talk to people. We can broadly classify research methods using this The aim is to count things in The aim is a complete, detailed Aim distinction. These two types of research method and their output data an attempt to explain what is description of what is observed. are classified as: observed. Purpose Generalisability, prediction, causal Contextualisation, interpretation, Quantitative - as the name suggests, is concerned with trying to explanations understanding perspectives quantify things; it asks questions such as ‘how long’, ‘how many’ or ‘the degree to which’. Quantitative methods look to quantify Tools Researcher uses tools, such as Researcher is the data gathering surveys, to collect numerical data. instrument. data and generalise results from a sample of the population of interest. They may look to measure the incidence of various Data collection Structured Unstructured views and opinions in a chosen sample for example or aggregate Output Data is in the form of numbers and Data is in the form of words, pictures or results. statistics. objects. Qualitative – concerned with a quality of information, Sample Usually a large number of cases Usually a small number of non- representing the population representative cases. Respondents qualitative methods attempt to gain an understanding of the of interest. Randomly selected selected on their experience. underlying reasons and motivations for actions and establish respondents how people interpret their experiences and the world around Objective/ Objective – seeks precise Subjective - individuals’ interpretation of them. Qualitative methods provide insights into the setting of a Subjective measurement & analysis events is important problem, generating ideas and/or hypotheses. Researcher role Researcher tends to remain Researcher tends to become subjectively e f Th ollowing table provides a breakdown of the key features of each of objectively separated from the immersed in the subject matter. these categorisation of research method and data. subject matter. Analysis Statistical Interpretive 8 9Quantitative Research Methods Quantitative methods are research techniques that are used to gather quantitative data, data that can be sorted, classified, measured. This following section outlines the core quantitative research methods used in social research. Quantitative survey What is the method? Surveys are a popular method of collecting primary data. The broad area of survey research encompasses any measurement procedures that involve asking questions of respondents. They are a flexible tool, which can produce both qualitative and quantitative information depending on how they are structured and analysed. In this section we focus on the quantitative use of surveys, and in later sections we explore the more qualitative use of survey methods. When should it be used? When you need to generate primary data from a large number of sources to answer your research question. Surveys are a useful a means of gathering data from businesses, community organisations and resi- dents, and survey research is one of the most important areas of meas- urement in applied social research. However, health warnings need to be attached to the use of quantitative surveys and careful consideration needs to be taken before embarking on any large-scale survey. What do I need to consider? In undertaking a survey it is important to understand who you want to survey, how you are going to select them, how you are going to survey them, what you want to ask them and how you are going to organise the task. The following section outlines some key considerations that need to be made before embarking on a large-scale survey. Population – A number of questions about the proposed popula- tion for a survey need to be considered. Such as: 10 11 1. How many roads must a man walk down? a) less than 10 b) 10 to 20 c) more than 20 d) don’t knowCan the population be counted? Some populations will business members of a Chambers of Commerce in a particular location. be easy to count, in a given geographical area there will be Detailed consideration of sampling needs to be made to ensure the secondary data sources that will give you a population count validity of your results, and the following issues need consideration: (Census), in a membership organisation there may be a list of all Who is the respondent? The first thing you need to under - members, however in a newly arrived ethnic community such as stand is who your respondent is going to be. This is the person the recent arrivals of Polish and Eastern European communities that will provide the data you are asking for. If the survey is there is less chance that you can obtain a reliable count of the distributed amongst households, who in particular will be filling population. A bias in your survey results can occur if the survey in the survey? Do you want to specify who the survey is to be sample does not accurately represent the population. Having a completed by? And do you understand why you are specifying count of the population is also important in order to establish this person? The same is true when surveying organisations or the significance of your results to allow a generalisation to the groups. A survey will have much greater success if it is directed population as a whole. to the right respondent. Identifying the person best suited to Are there language issues? Respondents may have vary- completing a survey will help to increase the response rate and ing capacities for being able to complete written surveys or generate more accurate data. questionnaires. While telephone and street surveys do not What is your sampling frame? A sampling frame is a list of require the respondent to be able to read or write in English, members of a population from which members of a sample are postal surveys involve respondents completing the survey or then selected. A sampling frame needs to be accurate, complete, questionnaire themselves. You should consider the oer o ff f help up-to-date and relevant to the purposes of the survey for in self-administered surveys for respondents to complete a form which it is to be used. Once you have an established sampling either in person or over the telephone, this will help address frame, depending on its size you may need to adopt a sampling potential language or basic skills issues. If surveying an ethnic technique to extract your final sample. For example random minority population you may wish to translate questionnaires sampling, simple random sampling or stratified sampling (see into community languages, or have people who speak the com- further reading for more details on sampling techniques). munities’ language to assist where necessary. Are response rates likely to be a problem? With any survey, What are the geographic restrictions? The geographic spread you need to look at the profile of the people who did responded of the population to be surveyed will determine the method and satisfy yourself that they are about the same as the people used for collecting your data. If you are surveying people from a who didn’t respond – and also, that they’re about the same as particular location or organisation it may be possible to conduct the overall population that you’re sampling. If you send out a a survey using an interviewer, however if you have a population survey to a population, which is 50% male, and 50% female, but sample that is geographically dispersed then you would look to your responses are 80% from females then your findings will use a different method, such as a telephone or postal survey. not represent your target population. Response rates can be Sampling low for surveys, under 20% for a postal survey is not uncom- e s Th ample is the section of the wider population that will be engaged mon. However, all the considerations in this section can help to in the survey and sampling is the process of identifying who you will improve your response rate. aim to contact from that population. The word ‘population’ is used to Statistical significance: Understanding your population, describe the target group, and while this may be the national popula- sample size, and response rates are important for calculating tion as a whole, it may also be a smaller group such as lone parents, or 12 13interval and confidence levels, which are vital in determining ask respondents to choose from a list of categories, such as New how many people you need to interview in order to get results Deal for Communities, Neighbourhood Renewal Funding and so that reflect the target population as precisely as needed. You on). Usually, closed questions include an ‘other’ option to enable can use online calculators to establish this type of information, respondents to add any categories that have been omitted; but it is important to understand the terms and the reasons for Ranking scales – these are most commonly used when trying to doing this (see section on statistical analysis for more detail). ascertain the level of importance of a number of items. A list of Format choices are provided and respondents are asked to put them in It is important to understand what format of survey you are looking to order (e.g. when undertaking a feasibility study for a new town undertake. There are broadly two survey formats that you may use and centre, a question using a ranking scale may show a list of items it is important to understand which you are using: that are commonly found in town centres and ask respondents to rank which ones are most important to them); Cross-sectional surveys are used to gather information on a population at a single point in time. An example of a cross- Sliding scales – these are used to discover respondents’ strength sectional survey would be a questionnaire that collects data on of feeling towards an issue. Respondents are given a series of peoples’ experiences of a particular initiative or event. A cross- statements and asked how much they agree or disagree with sectional survey questionnaire might try to determine the the statement by using a sliding scale where numbers represent relationship between two factors, like the impact of a pro- different strengths of feelings. For example, 1 = strongly agree gramme of activity on the level of benefits claims for example. and 5 = strongly disagree. Longitudinal surveys gather data over a period of time. This Write questions that are clear, precise, and relatively short would allow analysis of changes in the population over time and Because every question is measuring something, it is important for attempt to describe and/or explain them. The three main types each to be clear and precise. Your goal is for each respondent to inter- of longitudinal surveys are trend studies, cohort studies, and pret the meaning of each survey question in exactly the same way. If panel studies (for more details see further reading). A longitu- your respondents are not clear on what is being asked in a question, dinal study will also seek to determine the relationship between their responses may result in data that cannot or should not be applied factors, but the difference is that the examination will be of a in your survey findings. change in factors over time, so for example the relationship Do not use “loaded” or “leading” questions between health and employment. A loaded or leading question biases the response given by the respond- Questions ent. A loaded question is one that contains loaded words. Loaded or er Th e are a whole range of questions to be asked in survey design, leading questions may hint to the respondent how you expect the such as: What types of questions can be asked? How complex will/can question answered, for example ‘Do you think your neighbourhood is the questions be? Will screening questions be needed? Can question still run down?’, by including the word ‘still’ a bias is introduced as it sequence be controlled? Will lengthy questions be asked? Will long presupposes that the respondent thought the area was previously run response scales be used? Here we outline the main types of questions down. used in quantitative surveys: Ambiguous or compound questions can be confusing, leaving respond- Closed questions – these have a number of possible answers ents unsure as to how to answer. Compound questions are ones that in a list for respondents to choose from (e.g. a closed question ask several things which might require different answers, for example about the sources of funding for a community project would ‘Would you like to see more community support oc ffi ers on the streets, 14 15allowing a reduction in investment in CCTV?’. The respondent may It is vitally important to conduct a trial run or pilot of any survey, as wish to provide multiple answers to this question, answering yes to those that have designed a survey and are close to its subject, may take having more community support oc ffi ers, but disagreeing with the for granted that the questions and layout will work as a survey with the reduction in investment for CCTV. See the section on further reading wider intended population. A survey may be piloted with colleagues or for more information on question types and constructing survey ques- friends that have the same level of involvement in the subject you are tions. surveying as the wider intended population. Feedback should be sought on the ease upon which the survey can be followed and completed. A pi- Administration lot survey may also be conducted with a subset of the selected sample. e c Th osts, required facilities, time, and personnel needed to conduct an This would give opportunities to detect and resolve problems before ee ff ctive survey are often underestimated. The most common resource they obscure or distort the result of the wider survey. underestimated is time. You need to factor in time to pilot or test your survey, time to deliver your survey, time to give respondents to complete surveys and then have them returned (this may be via mail Pros Cons and therefore take time to return), and you also need to factor in the Postal Can reach a large geographical area No clarification available during time required to analyse surveys. When conducting a large scale survey, completion. inputting data to generate your analysis can be very time consuming. People are used to completing Need a motivated population to e b Th est approach is to often work up your timeline backwards from paper-and-pencil surveys return the survey when you need your results, calculating the time required for each step, Can take the survey with you and Respondents must be able to read, this way you can establish when things need to start by. complete it anywhere and anytime see, and write How should it be used? Great for sensitive issues Selecting the type of survey you are going to use is one of the most Telephone/ Information is obtained immediately Possible bias from the critical decisions in many social research contexts. In a similar way to administered administrator interviews, surveys can be delivered in a variety of ways: Can explore answers with Higher level of resources respondents • postal surveys; • telephone surveys; Negligible distribution costs Respondent must be “online” email/internet • email/internet surveys; Only “acceptable” answers can be Respondents must be able to • street surveys/administered surveys. allowed (validation) use a computer, a mouse, and/or keyboard e d Th elivery method for any survey should be carefully considered, and Require the question to be Respondent must be able to use a in many ways will be decided by consideration of factors listed above, answered web browser such as population, sample size and respondent. Having a good under- standing of these will inform the best method of delivery. For example, Can give respondent links that give Reliant on technology that can fail additional explanation if the survey is to be distributed to a particular local authority oc ffi er role across the country, then a postal or email survey would work best, as it is likely there will be over 350 in the population, geographically dispersed and literate. 16 17SurveyMonkey Excel What is the output? Microsoft Excel is useful for data summary, presentation, and for other SurveyMonkey is an Survey data is the question answers, such as ‘yes’ or ‘no’ or perhaps a basic statistical analysis. The program provides a set of data analysis online survey tool number, where a person has ranked a question on a scale. The survey that enables people tools called the Analysis ToolPak which you can use to save steps when data output will depend on the way in which the survey was construct- of all experience you develop complex statistical analyses. You provide the data and pa- ed, it will be shaped by the survey questions asked, the format of the levels to create rameters for each analysis and the tool uses the appropriate statistical survey itself and the method in which data was collected. For example, their own surveys macro functions and then displays the results in an output table. Some quickly and easily. if the survey was completed by the respondent, in a written form, then tools generate charts in addition to output tables. The Analysis ToolPak It has an online you will have a collection of written documents which require analysis survey designer, is not loaded by default, instructions for installing it, along with guides of the question answers. If the survey has been completed by a re- which contains on how to use it can be found on the Microsoft website. searcher, then a more sophisticated method of data collection may have many questions and occurred e.g. tallies and counts of responses. If using an internet or SPSS (Statistical Package for Social Scientists) formats. It collects responses and email survey, a computer programme may have collected the data in a SPSS is among the most widely used program for statistical analysis in analyses them in format which can easily be analysed. Consideration of the output needs social science. This is a data analysis package for quantitative research. real time, producing to be made at the outset of the process, and time considerations need It is particularly useful for the analysis of survey data as it covers a charts and graphs to be given as to how this data will be collected and analysed. broad range of statistical procedures. There are other packages available with available such as SAS, Stata or Minitab however all are expensive to purchase, information. All responses can be How should it be analysed? especially if only to be used for a one off survey. It may be possible to downloaded in a work with an academic institution to utilise their statistical packages, Before you can input your data in a computer program or application variety of formats and organisations such as the Cathie Marsh Centre for Census and Sur- you will need to undertake a process of coding. This involves assign- to allow further vey Research (CCSR) provide training on the use of these packages. statistical analysis in ing a code (often numeric) to each possible answer in your survey. So computer packages if question 1 in your survey asked the gender of the respondent, you For more detail on data analysis see section on Statistical Analysis. such as SPSS. may seek to code the answer ‘male’ with the number 0, while you may seek to code the answer ‘female’ with the number 1. Establishing these www.surveymonkey. Further reading com ‘codes’ on the distributed questionnaire can help at data entry time, but A practical guide to sampling, National Audit Oc ffi e obviously has the downside of putting numbers on the questionnaire that are of no relevance to the respondent and therefore could make the questionnaire look more confusing than it needs to. Question Bank is an information resource, in the field of social re- search, with a particular emphasis on quantitative survey methods. Web based programmes Internet based survey tools can distribute your survey via email and also collect your results, often allowing you to view your results as they A general introduction to the design of questionnaires for sur- are collected in real-time. You can download live graphs and charts vey research, University of Leeds of the responses, and often filter the responses and dig down to get individual responses. While this oer ff s significant benefits there needs The Centre for Applied Social Surveys, University of Southampton to be careful consideration of the pros and cons of email or internet – runs a programme of short courses in survey methods across the UK. surveys and whether this method of collection suits the population you are targeting. 18 19Key principles Secondary data collation and analysis er Th e are a number of key principles it is useful to follow when collect - ing and analysing secondary information. What is the method? 1. Think about the key issues and topics that need to be ad- This method refers to the review of existing information, and in the dressed. Having a clear idea of what information is required will quantitative context may involve the manipulation of statistical data. make the collection of secondary information a lot easier; It differs from primary research techniques in that the researcher does not collect the data directly and cannot control the actual data col- 2. Search for the information and data sources; lected, but can bring to bear new insights through interpretation or 3. Having collected the information, the next step is to read it presentation. Managing large data sets and large amounts of quantita- and analyse it; tive material does require some specialist skill. The Policy Action Team Reports in the early Blair Administration described the lack of avail- 4. Collate information from secondary data into key headings. ability of relevant datasets in order to support neighbourhood working, Referencing and over the last decade more statistics have been made more readily A key issue when using secondary data is ensuring that all information accessible to a wider range of people. is properly referenced and that it is clear where the information has come from. You must be very careful that comparisons are genuine and When should it be used? meaningful. e c Th ollection of secondary data can be an important first stage. The main use for this sort of information is that it can provide a starting What is the output? point for an evaluation or analysis to gain some background knowledge e inf Th ormation gathered from secondary data analysis can produce and understanding. Secondary data collection is also useful for contrib- various outputs depending on the type of information collated and re- uting to the analysis and commentary throughout a research report. viewed. Some of the most common types include statistics, data tables and charts and maps. What do I need to consider? e inf Th ormation may show how changes have occurred over time in a What types of data sources are there? particular area. It could also be comparative, which allows the research- er Th e are a number of different types of secondary information. Some er to make comparisons between a number of different areas. of the most common types are identified as follows: Official statistics - This refers to national data sets relating to How should it be analysed? issues such as population, employment and unemployment and Secondary data can be analysed using the same techniques as for businesses. Much of this information can be acquired from the primary data. See the following section on statistical analysis for more Oc ffi e for National Statistics and www.neighbourhood.statistics. details.; Other statistics - A wide range of other types of numerical data can be drawn on for evaluation purposes. e.g. project monitor- ing information of benec fi iaries, funding information, service data. 20 21Case study: Using secondary data to create a baseline Pros Cons Baseline assessments refer to a number of headline indicators or statistics for a specific Robust, accurate data enabling comparison across Health warnings around use of statistics – need to time/area make sure of appropriateness in context area at a particular moment in time e.g. the percentage of unemployed economically ac- tive males in Newcastle-upon-Tyne in 2002. Baselines are particularly useful when meas- Very credible and clear picture can emerge when Using only national statistics can suggest overly uring the impact of a regeneration project or programme, as by knowing what the area presented well clear picture – reality is often messier. was like before it commenced, it enables an evaluator to gauge the extent the project/ As part of mixed method strategy can provide You have no direct control over what is collected, programme has changed an area. “bedrock” for field of enquiry and suggest other only how it is presented and manipulated. appropriate techniques/interventions Working with baseline data relating to local social, economic, cultural and environmental conditions is a core feature in many policy interventions. In developing and updating a baseline, you may need to: Further reading • access secondary data (as detailed above) for establishing and up- How oc ffi ial statistics are collected has improved a lot in recent years, dating baselines; the neighbourhood statistics website is very user-friendly and uses maps, graphs and tables for a wide range of oc ffi ial statistics, present - • develop baselines retrospectively. This entails deciding key indica- ing data at many scales, from the smallest unit of data collection; the tors and collecting data for a period of time in the past (e.g. the Super Output Area (population 1000-1500) to Parliamentary Constitu- start of a regeneration programme); ency level, Local Authority level or regional Government Oc ffi e Area. • revise existing baseline indicators to ensure they reflect local priori- ties and are SMART; The Office for National Statistics website is also excellent and con- • recommend new baseline indicators where gaps exist and devising tains all census and oc ffi ial data as well as population projections and a entirely new baseline assessments where one exist; wealth of data on the economy. • review and aligning baseline indicators with those used nationally (e.g. Quality of Life indicators, Best Value Performance Indicators, Oxford Consultants for Social Inclusion are experts in manipulat- Public Service Agreement (PSA) indicators and National Floor Tar- ing datasets in order that they can help to inform decision making. get indicators) whilst maintaining a local focus; e Th y specialise in map data. • identify the best sources of data, frequency of updates and lead responsibilities for collection to aid future baseline updates; • collect and collate primary (e.g. survey) data to inform the baseline assessment. 22 23What do I need to consider? Statistical analysis Variables A variable is any measured characteristic or attribute that differs for What is the method? different subjects. Quantitative variables are measured on an ordinal, Statistical analysis is a mathematical method of interrogating data. interval, or ratio scale, whereas qualitative variables are measured on This is done by looking for relationships between different sets of data. a nominal scale (note in SPSS the Interval and Ratio levels are grouped Statistical analysis can be complex, and this following section aims to together and called scale). There are a range of variables that need to be explain some of the basic considerations, to an audience without an as- understood, dependent/independent, controlled/continuous/discrete sumed mathematical background. At the end of this section there are a in the application of statistical tests. The independent variable answers wide variety of links to further reading, which can help you through the the question “What do I change?”, the dependent variable answers the process of statistical analysis. question “What do I observe?” and the controlled variable answers the er Th e are two types of statistics: question “What do I keep the same?”. A variable which can have any numerical value is called a continuous variable (e.g. time). A variable • Descriptive statistics: numerical summaries of samples (what which can only have whole numbers (integers) is called a discrete vari- was observed); able (e.g. the number of people in a group). It is important to under- • Inferential statistics: from samples of populations (what could stand the variable you have for analysis of data in statistical packages have been or will be observed). such as SPSS. It is important to understand which type of statistics you are working Inference with before embarking on analysis. If working with inferential statistics you need a sound understanding of your population (the set of individuals, items, or data, also called When should it be used? universe) and your sample (a subset of elements taken from a popula- tion). See the section on quantitative surveys for further discussion on e g Th eneral idea of statistical analysis is to summarise and analyse data populations and samples. We make inferences (conclusions) about a so that it is useful and can inform decision-making. You would analyse population from a sample taken from it, therefore it is important that descriptive statistics if you wanted to summarise some data into a population and sampling is well understood, as any error will influence shorter form, where as, you would use inferential statistical analysis your inferences (conclusions). In some situations we can examine the when you were trying to understand a relationship and either general- entire population, then there is no inference from a sample. ise or predict based on this understanding. Statistical analysis, through a range of statistical tests, can give us a way to quantify the confidence Confidence & Significance we can have in our inferences or conclusions. • The confidence interval is an interval estimate of a popula- tion parameter, this is the plus-or-minus figure reported in, for Statistical analysis should only be used where there is a clear under- example, newspaper or television opinion poll results. If you standing of the reasons for doing so. The use of statistical tests (as use a confidence interval of 4 for example, and 54% percent of detailed above) will provide you with valuable findings if you know how your sample picks one answer, you can be “sure” that if you had to interpret the results and use them to inform your research. asked the question of the entire relevant population, between 50% and 58% would have picked that answer (plus or minus 4). er Th e are three factors that determine the size of the confidence 24 25interval for a given confidence level. These are: sample size, increases the closer the percentage is to 50%. In survey 1 (above) the percentage and population size (see below). confidence interval for a value of 50% is 3.02. This confidence interval would fall to 0.6 if the survey returned a value of 99% or 1%. • The confidence level tells you how sure you can be that this inference is correct. Most social science researchers use the 95% It is important that the survey sample size is considered for statistics confidence level, which means you can be 95% certain; while where 50% of the population answer both ‘yes’ and ‘no’ as this is when the 99% confidence level means you can be 99% certain. When the confidence level is broadest and so provides the general level of ac - you apply the confidence level and the confidence interval curacy for a sample. together, you could say that you are 95% sure that between 50% Population Size and 58% would have picked that answer. e p Th opulation size refers to the number of people within a group that In statistics, a result is called statistically significant if it is unlikely to have a similar characteristic. This could be the total number of people have occurred by chance. In statistics, “significant” means probably living in a town or the number of people with a more specific attribute true, and not ‘important’. The findings of your research may be proved such as suffering from a disability or residents from a specific ethnic to be ‘true’ but this does not necessarily mean that the findings are group. Population size is of greatest importance when the population is ‘important’. In social science, results with a 95% confidence level are relatively small and is known. accepted as significant. Examples Factors that affect the confidence interval Confidence A survey of 1,000 households has been completed, in a town of 20,000 e c Th onfidence interval is affected by three factors. These are the sam- households. 54% of households felt that crime had the largest impact on their quality of ple size, percentage and population size. life. Using a 95% confidence level a confidence interval of 3.01 can be assumed. So you Sample Size can say that between 51% and 57% of the town’ss population feel the crime has the larg- e l Th arger your sample, the more confident you can be that their est impact on quality of life. answers truly reflect the population. The relationship between the Significance A survey is distributed to all 20,000 households in a town, there are 1,000 confidence interval and sample size is not linear. An example can be responses to the survey, equal to a 5% response. In accepting an interval level of 3, the found below: sample size needed for significant results at the 95% confidence level is 1013, therefore Survey 1 Survey 2 the response rate is just short of significance at the 95% level. Sample 1,000 2,000 Population 20,000 20,000 e s Th ignificance of change over time in survey findings In measuring the confidence interval of survey data when survey % of respondents answering 50% 50% results are compared over time, it is important to understand if, for ‘yes’ to a specific question example, economic activity has changed over time or if the change in Confidence Interval +/-3.02 +/-2.08 results is caused by survey error. To understand whether actual change has taken place, this requires the confidence interval of the difference Percentage between the two means to be tested (see further reading for a link to a e c Th onfidence interval is also determined by the percentage of web tool for measuring the confidence interval between two means). the sample that provides the same answer. The confidence interval 26 27Example • Parametric tests – include Mean, Standard Deviation, t test, analysis of variance (ANOVA), Pearson correlation, regression Survey 1 finds that economic activity stands at 49% using a sample of 1,000 residents. (linear and non linear); Another sample is selected one year later. Survey 2 finds that 51% of residents are eco- nomically active. In this case the 95% confidence interval is from -0.05 to 0.03 meaning • Non-parametric tests – include Median, interquartile range, that we cannot be sure whether the economic activity rates have actually increased or Spearman correlation, Wilcoxon test, Mann-Whitney test, whether this is a result of survey error. This is because the 95% confidence interval has Kruskal-Wallis test, Friedman test. values which are either side of zero. Choosing the right test Choosing between these two families of tests can be difficult. The If economic activity increases to 55%, the 95% confidence interval is from -0.09 to -0.01 following section outlines some of the basic rules for deciding which meaning we can be 95% confident that economic activity has actually increased. family of tests suits your data. Considerations: Both surveys must be based on a sample that is representative of the • You should choose a parametric test if your data is sampled from population. The sample used in survey 2 also needs to be independent from the sample a population that follows a normal distribution (or Gaussian used in survey 1. distribution). The normal distribution is a pattern for the distri- bution of a set of data, which follows a bell shaped curve. This Cross-tabulation means that the data has less of a tendency to produce unusually Cross-tabluation is about taking two variables and tabulating the extreme values, compared to some other distributions. results of one variable against the other variable. This can be done quite • You should choose a non-parametric test if the population simply in data analysis tools such as Microsoft Excel or SPSS. A cross- clearly does not follow a normal distribution. Where values tabulation gives you a basic picture of how two variables inter-relate, may be “off the scale,” that is, too high or too low to measure, so for example you may have a question in your survey about employ- a non-parametric test can assign values too low or too high to ment, by running a cross tabulation of the survey data obtained for this measure. question against that of age or gender for example (or both), would give you a table showing the employment status of both males and females, What do these tests tell you? broken down by the age ranges you coded in your survey. This can pro- Parametric tests vide quite powerful levels of information and is a useful way of testing Mean - The mean is more commonly called the average, however the relationships between variables. this is incorrect if “mean” is taken in the specific sense of Statistical tests “arithmetic mean” as there are different types of averages: the For more complex statistical analysis there are a range of statistical mean, median, and mode. tests that can be applied to your data. To select the right test, you need Standard Deviation - The standard deviation measures the to ask yourself two questions: spread of the data about the mean value. It is useful in compar- 1. What kind of data have you collected? ing sets of data, which may have the same mean but a different 2. What variables are you looking to establish a relationship range. between? t test - The t-test assesses whether the means of two groups are Choosing the right test to compare measurements can be a tricky one, statistically different from each other. This analysis is appropri- as you must choose between two families of tests: parametric and non- ate whenever you want to compare the means of two groups. parametric: 28 29Analysis of variance (ANOVA) – This is used to test hypotheses Mann-Whitney test - The Mann-Whitney test is a non-paramet - about differences between two or more means as in the t-test, ric test for assessing whether two samples of observations come however when there are more than two means, analysis of from the same distribution, testing the null hypothesis that the variance can be used to test differences for significance without probability of an observation from one population exceeds the increasing the error rate (Type I). probability of an observation in a second population. Pearson correlation – This is a common measure of the cor - Kruskal-Wallis test - A non-parametric method for testing relation between two variables. A correlation of +1 means that equality of population medians among groups, using a one-way there is a perfect positive linear relationship between variables. analysis of variance by ranks. A correlation of -1 means that there is a perfect negative linear Friedman test - The Friedman test is a nonparametric test that relationship between variables. compares three or more paired groups. Regression (linear and non linear) - A technique used for the modelling and analysis of numerical data. Regression can be What is the output? used for prediction (including forecasting of time-series data), e o Th utput of statistical analysis will depend on the statistical test you inference, hypothesis testing, and modelling of causal relation- apply to your data, a detailed understanding of the test is required to be ships. able to interpret the results. The output will most probably be further tables of data, with a number of things being reported. It is important Non-parametric tests to understand the information you need from a table of results, as you Median - The median is the middle of a distribution: half the may only require a single figure, but be presented with a range of infor - scores are above the median and half are below the median. The mation which may be confusing if you are new to statistical analysis. median is less sensitive to extreme scores than the mean and this makes it a better measure than the mean for highly skewed distributions. The median income is usually more informative How should it be analysed? than the mean income for example. Microsoft Excel Microsoft Excel includes a collection of statistical functions, within the Interquartile range - The interquartile range (IQR) is the add-on Data Analysis ToolPak. Excel can analyse descriptive statistics distance between the 75th percentile and the 25th percentile. at a simple level and when used ee ff ctively, can be very useful in the e IQR is ess Th entially the range of the middle 50% of the data. exploratory analysis of data, cross tabulations (pivot charts), viewing Because it uses the middle 50%, the IQR is not affected by data in graphs to detect errors, unusual values, trends and patterns and outliers or extreme values. summarising data with means and standard deviations. However, Excel Spearman correlation - Spearman’s Rank Correlation is a tech- is of very limited use in the formal statistical analysis of data unless nique used to test the direction and strength of the relationship your experimental design is very simple. The Analysis ToolPak is also no between two variables. In other words, it’s a device to show easier to use than more formal statistical packages, however there are whether any one set of numbers has an ee ff ct on another set of plenty of guides and tutorials to be found on the internet. numbers. Formal Statistical Packages (SPSS, SAS, Stata) Wilcoxon test - The Wilcoxon test compares two paired groups Inferential statistics are more often analysed in specialist statistical of data. It calculates the differences between each set of pairs, packages such as SPSS which provide greater functionality compared and analyses the list of differences. to Excel. The package used by the researcher often depends on which 30 31package the researcher is familiar with and has access to. These formal Further reading statistical packages can summarise data (e.g. frequencies), determine Guide to Good Statistical Practice - This resource is based at the Sta- whether there are significant differences between groups (e.g. t-tests, tistical Services Centre, University of Reading and consists of a series analysis of variance) and examine relationships among variables (e.g. of guides on good statistical practice, intended primarily for research correlation, multiple regression). Further, these packages can produce and support staff in development projects. Guides can be downloaded charts, graphs and tables from the results of the analysis. in HTML and PDF on subjects such as Data Management and Analysis. Links include: training courses and workshops; consultancy; resources (such as publications, software, external links) Pros Cons Microsoft Excel Commonly used and widely available It is not possible to see a record of Introduction to Central Tendency, David Lane the analysis you have previously - A useful guide explaining conducted some basics to Statistical Analysis. Easy to use for basic data analysis Statistical analysis is only possible if data is sorted or in blocks Statsoft Electronic Textbook, - This Electronic Easy to import information from other Limited by space – MS Excel has a Statistics Textbook oer ff s training in the understanding and application packages. size limitation of 256 columns and over 65,500 rows meaning it has of statistics. Creating and amending charts is simple limited capacity for analysing larger Simple Interactive Statistical Analysis, datasets - SISA allows you to do statistical analysis Formal Widely used. Expensive to purchase. directly on the Internet. User friendly guides are available for statistical Statistical More recent versions are more user Need to buy add-ons to get full Packages procedures. friendly than earlier versions (menus to functionality (SPSS, SAS, select rather than having to use syntax) Excel For Statistical Data Analysis, Stata) Allows a wider range of statistics test to Output isn’t user friendly for be conducted compared to Excel beginners Raynald’s SPSS Tools - - A website oer ff ing Easy to analyse survey / questionnaire Charts are poor quality and difficult tools and tips for users of SPSS software, the site oer ff s an archive of responses to amend - need to copy information 400+ sample SPSS syntax, scripts and macros classified by purpose, as into Excel File size is only dependent on your well as an FAQ, tips, tutorials and a Newbie’s Corner. It invites contri- computers capacity butions from other SPSS users to create a shared, open-source resource. Survey data can be given assigned labels Choosing the correct Statistical Test - It is easy to analyse sub groups of a large dataset Confidence interval between two means - The following link pro- vides a tool for measuring the confidence interval between two means 32 33Qualitative Research Methods Qualitative methods are generally associated with the evaluation of social dimensions. Qualitative methods provide results that are usually rich and detailed, oer ff ing ideas and concepts to inform your research. Qualitative methods can tell you how people feel and what they think, but cannot tell you how many of the target population feel or think that way as quantitative methods can. Social survey/questionnaire What is the method? Social surveys are a questionnaire-based method of research that can produce both qualitative and quantitative information depending on how they are structured and analysed. This section focuses on the use of surveys to collect and analyse qualitative data. Many of the issues and considerations are the same as for the quantitative use of surveys, and more detail can be found in the earlier section of this handbook. When should it be used? Questionnaire surveys can be used in a wide range of settings and to gather a variety of different types of information. You may be evaluat - ing a programme in which a wide range of projects have been com- missioned, and want to gather the views of a wide range of project managers, or you may be measuring the impact of an initiative on the business community in a specific geographical area. A small-scale quali- tative survey may be conducted to explore in more detail the findings of qualitative research. What do I need to consider? Many of the considerations for a social survey are the same as for a quantitative survey, however we define a social survey as one where less statistical rigour is required, where sample sizes are not as large, and with results not expected to be significant of the wider population. A social survey may have a greater focus on collecting rich and detailed qualitative data. Population A number of questions about the proposed population for a social survey need to be considered. Such as are there language issues? And 34 35 1. Is this the real life? yes/no If no, please answer question 2 2. Is it just fantasy? yes/no Please provide reasons for your answer .................................................. .................................................. .................................................. .................................................. ..................................................what are the geographic restrictions? These are the same issues as for Administration quantitative surveys. e c Th osts, required facilities, time, and personnel needed to conduct an ee ff ctive survey are often underestimated, even when it is not on a large Sampling scale. There should be an administrative system in place to deal with the e s Th ample is the section of the wider population that will be engaged in questionnaires for when they are returned/completed. This may include the survey. Detailed consideration of sampling still needs to be made even numbering the questionnaires, recording what action has been taken when not striving for statistical significance. It is still important to under - with them, entering the results into a spreadsheet/database etc. stand who the respondent is and what your sampling frame is going to be. Format How should it be used? A social survey will usually be a cross-sectional survey used to gather Surveys can be carried out by phone, post, email, website or face-to- information on a small sample population at a single point in time. face, for detailed pros and cons of these delivery methods see the ear- An example of a cross-sectional survey would be a questionnaire that lier section on qualitative surveys. In collecting rich qualitative survey collects data on peoples’ experiences of a particular initiative. How- data, the most ee ff ctive method would be via face to face, administered ever, a qualitative survey could equally be used in a longitudinal study, surveys, as the researcher would be able to use prompts to encourage perhaps returning to particular individuals over time to measure the people to give more detailed answers. This does however introduce a impact of an intervention on the direction of someone’s life. bias, which needs to be understood and controlled as much as possible, i.e. by using standard prompts. In qualitative surveys, it is necessary Questions that the interviewer conduct the interview with total objectivity, so er Th e are a whole range of questions to be asked in relation to survey that respondents are not influenced by any outside source in their design, such as: What types of questions can be asked? How complex responses. For this reason, interviews should be conducted by well- will the questions be? Will screening questions be needed? Can ques- trained and qualified interviewers. tion sequence be controlled? Will lengthy questions be asked? Will long response scales be used? A social survey will be more interested in qual- itative findings, in recording peoples’ opinions and perceptions, and What is the output? therefore will make more use of open questions where respondents can e d Th ata that a social survey can produce is very much dependent on give their own responses to a set question. Open questions will begin how the questionnaire is constructed. However, the data can be very with what, why, how, or describe, to elicit rich qualitative information. useful for providing an overall picture of the way in which a project or programme is being implemented and how ee ff ctively it is impacting Open questions can be used in a variety of ways: upon its target audience. Qualitative data output will be in a text, audio Usage Example or picture format, and each answer may be very different from another. This can make collection of data more difficult, and a way of collating As a follow-on from closed questions, to develop a ‘If answering yes to question 7, please provide the more detailed response. reasons for this’ data needs to be considered early in the process. To find out more about a person, their thoughts, Why is that so important to you? needs, problems, etc. How should it be analysed? To get people to realise the extent of their What effect does this have on your family life? e Q Th uantification of Qualitative Survey Data problems. Surveys can be analysed by collating the frequency of responses to each of the questions on the survey form. This can be done manually using a To get people to reflect on the impact of How has this made a difference to you? something or some change. 36 37“frequency table”, which can be easily set up on an Excel spreadsheet to Further Reading analyse descriptive statistics. See Question Bank for details of question design - QSR NUDIST and NVIVO are qualitative data analysis packages, which enable non-statistical information from interviews, group work, obser- Computer Assisted Qualitative Data Analysis (CAQDA) – vations, audio, video, pictures or documents to be analysed according - provides practical support, training to chosen criteria. For example, it is possible to use the package to ‘pull and information in the use of a range of software programs designed out’ all material relating to key words or phrases (e.g. neighbourhood to assist qualitative data analysis. Also provides various platforms for renewal) and then sub-divide the data into more specific areas of analy - debate concerning the methodological and epistemological issues aris- sis (e.g. statement of use, problems, projects). This is a powerful piece ing from the use of such software packages. of software that can provide clarity to wide range of often complicated Research Observatory, University of the West of England - written or media materials. - the site is divided into topic areas with each topic area containing a number of Case study: Using surveys to evaluate a project learning units and a collection of resources about a particular subject related to research. A programme targeted on helping young people back into work through training wants to evaluate how well it is achieving its objectives. It uses a survey to canvas the views of young people who have been on the programme to date. The survey asks them closed Interviews questions about what training they have attended and how useful they have found the training (on a scale of 1:4). The survey also uses open questions to ask young people What is the method? about what their plans are for the future as a result of the training (i.e. has it helped One of the most popular and frequently used methods of gathering them to consider applying for full time work? Or further education opportunities?). The information from people about anything is by interviewing them. It is qualitative data is analysed and this shows that the young people have gained in confi- also the most popular method used within the social sciences. There is dence, are looking to go into further education or training or have already secured job a continuum of formality around interviewing and it covers a multitude interviews in a range of occupational fields, however there is a distinct focus on work in of techniques, from informal “chats” maybe arranged as “vox-pops” the field of construction. right through to highly structured, formal interviews, taped and tran- scribed. e r Th esults of the survey are analysed and this provides conclusions about overall success of the programme, which allows the programme manager to draw conclusions and con- e diff Th erent types and styles of interview elicit very different types of sider design issues for making the programme more ee ff ctive in the future. information. Conducting interviews is an interpersonal process and as an investigator you must be very aware of your own behaviours and assumptions in the context. Interviews are not “neutral” social spaces and you must be respectful and maintain appropriate boundaries at all times. What do I need to consider? Interviews are a qualitative method of research often used to obtain the interviewees’ perceptions and attitudes to the issues. The key issue 38 39

Advise: Why You Wasting Money in Costly SEO Tools, Use World's Best Free SEO Tool Ubersuggest.