Quantitative Research methodology sample

how to critique quantitative research methodology and quantitative research methodology dissertation example and How to use Quantitative Research methods
Dr.JohnParker Profile Pic
Published Date:01-07-2017
Your Website URL(Optional)
 Quantitative  Topic Topic Research  8 8 Methods  LEARNING OUTCOMES By the end of this topic, you should be able to: 1. Explain what is quantitative research method; 2. Describe the three types of descriptive research designs; 3. Differentiate the two types of survey; and 4. Apply the common statistical approaches in quantitative data analysis. 8.1 DEFINITION OF QUANTITATIVE METHODS Quantitative research methods are research methods dealing with numbers and anything that is measurable in a systematic way of investigation of phenomena and their relationships (Figure 8.1). It is used to answer questions on relationships within measurable variables with an intention to explain, predict and control a phenomena (Leedy 1993).  TOPIC 8 QUANTITATIVE RESEARCH METHODS 88 Figure 8.1: Description of Quantitative Method An entire quantitative study usually ends with confirmation or disconfirmation of the hypothesis tested. Researchers using the quantitative method identify one or a few variables that they intend to use in their research work and proceed with data collection related to those variables. In the field of ICT, quantitative methods often deal with results computation and system analysis using a scientific approach. The objective of the quantitative method is to develop and employ models based on mathematical approach, hypotheses and theories pertaining to the nature of an ICT phenomenon. The process of measurement (which we have come across in the previous topic) is the focus of quantitative method due to its connectivity between empirical observation and mathematical expression of quantitative relationships. This method is also known as iterative process where evidence is evaluated, and hypotheses and theories are refined with some technical advances, leveraging on statistical approach. Quantitative method typically begins with data collection based on a hypothesis or theory and it is followed with application of descriptive or inferential statistics. Surveys and observations are some examples that are widely used with statistical association. We will see different types of quantitative research methods in the next section. For example, when a researcher is interested to investigate the „effectiveness of expert system for managing e-commerce application in open source environment‰, the researcher will formulate the TOPIC 8 QUANTITATIVE RESEARCH METHODS  89 research question such as, „How effective is the expert system in comparison to case-based reasoning for e-commerce module development?‰ The researcher finds 10 software developers using e-commerce module with expert system in open source environment and 10 software developers using case-based reasoning e-commerce module in propietary programming language environment. The researcher will administer the results and compute them using statistical approach and then summarise it. Here, we can say the researcher used the quantitative method for the work mentioned. SELF-CHECK 8.1 (a) Define the quantitative research method in your own words. (b) Scenario: A researcher formulates a research question, „The effectiveness of a network topology set up in a campus‰. He finds several network administrators that use the network topology, and concluded the reasons with one common conclusion without any numerical figures/ statistics. Does the researcher in the above scenario perform the quantitative research method? If yes, explain why. If no, provide the reasons why you think so. 8.2 DESCRIPTIVE RESEARCH Quantitative research methods fall under the broad heading of descriptive research. This type of research corresponds to identifying the characteristics of an observed phenomenon, or exploring correlations between two or more entities. The three types of descriptive research are illustrated by Figure 8.2 below. Figure 8.2: Types of descriptive research  TOPIC 8 QUANTITATIVE RESEARCH METHODS 90 8.2.1 Observation Studies Observation studies are involved in both quantitative and qualitative research methods. However, in quantitative methods, the focus of observation studies is on a particular factor of behaviour and it is quantified. In this type of design, a researcher will try to maintain objectivity in assessing the behaviour being studied. Table 8.1 summarises some strategies used in this design. Table 8.1: Strategies used in observation studies No. Strategies Description 1 Using rating scale Using rating scale (e.g. Likert Scale) to evaluate the behaviour in terms of specific factor or reasons. 2 Defining the behaviour Defining the behaviour being studied in a precise and solid manner so that the behaviour is easily recognised during its occurence. 3 Rated by two or more Having two or more individual ratings the individuals same behaviour independently, without the knowledge of one anotherÊs ratings. 4 Clustering the observation Divide observation period into small clusters periods and then record whether the behaviour does or does not occur during each cluster or segment. Time period may be assigned with some intervals depending on the studies requirement. 5 Train the rater(s) Train the rater(s) of the behaviour to follow some specific requirement until consistent ratings are obtained during any of the behaviour occurences. Next, let us look at the next type of descriptive research, the correlational research. TOPIC 8 QUANTITATIVE RESEARCH METHODS  91 8.2.2 Correlational Research Correlational research or studies examine differences of characteristics or variables of two or more entities. A correlation exists when one variable increases or decreases correspondingly with the other variable. A researcher will gather data about two or more variables in a particular group. These data are numbers that reflect measurement of the characteristics of research questions such as test scores, baud rate, device jitter or network performance using 2 different transmission mediums. For example, as storage becomes larger, it is easier to manage databases. In other words, there is a correlation between storage size and database capacity. Correlational results can be represented using various means of visualisation. One such example would be using the scatterplot which allows a visual inspection of the relationship between two variables. In ICT research, scatterplot is commonly used in representing results obtained for journal or proceedings publications. Figure 8.3 below shows an example of a scatterplot. Figure 8.3 A scatterplot By examining the scatterplot, we can identify or describe the homogeneity or heterogeneity of the two variables. Secondly, we can describe the degree of which the two variables are intercorrelated or using statistical approach known as correlation coefficient. Finally, we can interpret these data and give them meaning.  TOPIC 8 QUANTITATIVE RESEARCH METHODS 92 8.2.3 Survey Research In ICT research, one of the common methodologies used is survey research. It is widely used as a method due to the nature of the ICT field that involves multiple interests of public, infrastructure and technologies associated with it. Kerlinger (1973) defined survey research as a study on large and small populations by selecting samples chosen from the desired population and to discover relative incidence, distribution and interrelations. The ultimate goal of survey research is to learn about a large population by surveying a sample of the population; thus we may also call it descriptive survey or normative survey. In this method, a researcher poses a series of questions to the respondents, summarises their responses in percentages, frequency distribution and some other statistical approaches (Figure 8.4). Survey research typically employs face-to-face interviews, telephone interviews or the common approach using questionnaires (we will look into questionnaires in the next section). Basically, information is acquired by asking respondents questions by using interviews or questionnaires concerning the following: What are the beliefs? Should Service Oriented Architecture (SOA) be adopted in seeking human capital for Human Resource Management? What do they know? What was the first IT enterprise architecture that dominated Malaysian scenario? What have they done? How often do you perform virus scan What do they expect? and monitoring in your Do you think Open Source policy organisation? should be implemented in all educational agencies for wider acceptance and cost savings? Figure 8.4: Example of questions asked in survey research TOPIC 8 QUANTITATIVE RESEARCH METHODS  93 There are two types of survey (see Figure 8.5). Normally, the type of survey method used depends on the scope of the research work. If the research needs a pool of opinions and practices, a cross-sectional survey would be appropriate. On the other hand, if a researcher specifies the objective as to compare differences in opinion and practices over time, a longitudinal survey would be the ideal method. Figure 8.5 : Types of survey In cross-sectional survey, a researcher collects information from a sample drawn from a population. It involves collecting data at one point of time. The period of data collection can vary and it depends on the study weightage. For example: You administer a questionnaire on broadband usage among 500 university students for information dissemination using research network. The university students may comprise 20-23 year old students. The students could be males and females from different course backgrounds in a particular university. In this case, the data you obtain is derived from a cross-section of the population at one point of time. In longitudinal surveys, data collection is done at different points of time to observe the changes. Two common types of longitudinal surveys are Cohort Studies and Panel Studies. (a) Cohort Studies In Cohort Studies, a researcher specifies population (e.g. IT subordinates in an organisation dealing with security audit and penetration tests) and lists the names of all members of this population. At each data collection point, a researcher will select a sample of respondents from the population of IT subordinates doing security audit and penetration testing and administer a questionnaire. This is then repeated at another point of time. Although the population remains the same, different respondents are sampled each time. The researcherÊs aim here is to see if there are changes in perceptions or trends that occur in the study.  TOPIC 8 QUANTITATIVE RESEARCH METHODS 94 (b) Panel Studies In Panel Studies, a researcher can identify a sample from the beginning and follow the respondents over a specified period of time to observe changes in specific respondents and highlight the reasons why these respondents have changed. ACTIVITY 8.1 Summarise the characteristics of quantitative research methods. 8.3 QUANTITATIVE DATA ANALYSIS Data collected from questionnaires or other instruments in quantitative research methods have to be analysed and interpreted. Generally, statisical procedures are quantitative data approaches. In this section, we will look at these common statistical approaches and emphasis on a conceptual understanding for quantitative data analysis. Figure 8.6 summarises the statistical components that we will be looking at in this section. TOPIC 8 QUANTITATIVE RESEARCH METHODS  95 Figure 8.6: Summary of statistical components in quantitative data analysis (a) Mean Mean is also known as average. A mean is the sum of all scores divided by the number of scores. The mean is used to measure central tendency or centre of a score distribution generally. For example, the mean for the following set of integers: 3, 4, 5, 7 and 6 = 5.  TOPIC 8 QUANTITATIVE RESEARCH METHODS 96 Figure 8.7 : Mean in distributed scores Adapted from http://www.tarorigin.com/art/Omasory/Uncertainty/ (b) Standard Deviation A standard deviation tells us how close the scores are centred around the mean. By referring to the above Figure 8.7, when the scores are bunched together around the mean, the standard deviation is small and the bell curve is steep. When the scores are spread away from the mean, the standard deviation is large and the bell curve is relatively flat. Figure 8.8: Standard deviation To explore better what standard deviation means, we shall refer to Figure 8.8. The mean is 20 and the standard deviation(SD) is 5. Figure 8.4 represents the score obtained on grid test for two organisation terminals using cluster computing with the same mean of 20. TOPIC 8 QUANTITATIVE RESEARCH METHODS  97  One standard deviation (SD= 5) from the mean in either direction on the horizontal axis accounts for around 68% of the organisation in this group. In other terms, 68% terminals obtained 15 and 25 optimal time.  Two standard deviation (5+5=10) away from the mean accounts roughly 95% of terminals. In other words, 95% terminals obtained between 10 and 30 optimal time.  Three standard deviations ( 5+5+5=15) away from the mean accounts for roughly 99% terminals. In other words, 99% terminals obtained 5 and 35 optimal time. 8.3.1 Testing for Significant Differences between Two Means Using the t-Test (Independent Groups) LetÊs say you are conducting a study to compare the effectiveness of the use of Âservice discovery protocolÊ (independent variable) in enhancing network appliances detection in home networks. The mean score and standard deviation for the application test are shown in Table 8.2 and you want to test the null hypothesis. H : There is no significant difference between the experimental group and the 0 control group in terms of enhancing network appliances detection. To solve this, you may use the statistical approach called t-test to obtain the t- value for independent means. In this case, independent means that two groups consist of different subjects. The t-test gives the probability that the difference between the two means is caused by chance. For testing the siginificance, you will need to set a risk level called the alpha level. Similar to social science research, in ICT research, we set the alpha level at .05. This means that the obtained result which is significant at .05 level could occur by chance only 5 times in trial of 100.  TOPIC 8 QUANTITATIVE RESEARCH METHODS 98 Table 8.2 Means and Standard Deviations Obtained for the Experimental and Control Groups N Mean Standard Deviation Experimental group 10 13.8 2.10 Control group 10 11.4 1.96 t value = 2.65; degrees of freedom = 18; p0.02 (a) Table 8.2 displays t-value of 2.65 obtained. If you are using statistical software like SPSS or SAS, the probablity value is given (i.e.p0.02). We could also refer to the table of critical values to find out whether the t-value is large enough to say that the difference between the groups is not likely to have been a chance finding. (b) We can also determine the degrees of freedom (df) for the test which is the sum of the terminals in both groups minus 2 (i.e.n-2). By the given alpha level, the df and the t-value, we can refer to the t-value in the table of critical values. (c) Refer Table 8.3. The obtained t-value(2.65) is bigger than the critical value (2.1009) for 18 degreee of freedom (20-2=18). From this, we can conclude that the differences between the means for two organisations is significantly different at the 0.05 level of significance. Table 8.3 Extract from the Table of Critical Values df p = 0.05 p = 0.01 17 2.1098 2.8982 18 2.1009 2.8784 19 2.1009 2.8609 (d) Please note the difference is NOT SIGNIFICANT at the 0.01 level of significance because t-value (2.65) is smaller than the critical value (2.8784) for 18 degress of freedom. TOPIC 8 QUANTITATIVE RESEARCH METHODS  99 8.3.2 Testing for Significant Differences between Two Means Using the t-test (Dependent groups) LetÊs say you would like to conduct a study to compare the effectiveness of the use of Âservice discovery protocolÊ (independent variable) in enhancing network appliances detection (dependent variable) in ONE home network (IEEE 802.11). You gave a pretest and after testing the protocol with the IEEE 802.11 network, you give a posttest. Here, the same group of subjects are tested 2 times. The mean score and standard deviation obtained for network detection are in Table 8.4. You want to test the null hypothesis. H : There is no significant difference between the pretest mean and the posttest 0 mean in terms of network appliances detection enhancement. Table 8.4: Means and Standard Deviation Obtained for the Pretest and Posttest Scores Mean Standard Deviation Pretest 9.90 1.66 Posttest 10.90 0.99 N=10; t value=1.94; degrees of freedom=9; p0.09 (a) By using the t-test for dependent groups, we can obtain value of 1.94. In this case, dependent means that the two means are obtained from the same groups. (b) From Table 8.5, we can highlight that for 9 degree of freedom, the critical value is 2.2622, which is larger than the t-value 1.94. We can conclude that the means are NOT significantly different at the 0.05 level of significance. Table 8.5 Extract from the Table of Critical Values of t df p = 0.05 p = 0.01 8 2.3060 3.3554 9 2.2622 3.2498 10 2.2281 3.1693  TOPIC 8 QUANTITATIVE RESEARCH METHODS 100 8.3.3 Testing for Differences between Means Using One-Way Analysis of Variance (ANOVA) In ANOVA, the mechanism is similar to t-test, although the method differs. Whenever a researcher wants to compare more than two means, he/ she will always opt to use One-Way Analysis of Variance or widely known as ANOVA. For example, say you want to conduct an experiment on three types of buffering methods for video streaming over mobile device. The means and standard deviation obtained are shown in Table 8.6. Table 8.6 Means and Standard Deviation for Three Types of Buffering Methods N Mean Standard Deviation Buffer Method 1 10 14.6 1.83 Buffer Method 2 10 15.6 2.22 Buffer Method 3 10 18.0 2.10 Table 8.7: Summary of the Buffering Method Analysis Summary Sum of Square df Mean Squares F p Treatment 61.066 2 30.533 7.1811 0.003 Within 114.800 27 - TOTAL 175.866 29 In this example, a researcher used One-Way ANOVA and obtained an F-value of 7.1811 which is significant at 0.003 (Table 8.7). Therefore, the null hypothesis of no differences between means is rejected. However, it is unsure which of the differences contributes to the significance. To overcome this problem, another statistical approach needs to be considered. A researcher can perform Pos Hoc Comparisons such as Scheffe Test or Tukey Test. These tests are usually applied after an analysis of variance. Table 8.8: Tukey Test for the Analysis Buffer Method 1 vs Buffer Method 1 Not significant Buffer Method 1 vs Buffer Method 3 Significant at p.01 Buffer Method 2 vs Buffer Method 3 Significant at p.05 TOPIC 8 QUANTITATIVE RESEARCH METHODS  101 From Table 8.8, we can conclude that there is no significant difference between the performance using Buffer Method 1 and Buffer Method 2. Buffer Method 3 performed significantly better at significance level 0.05. Buffer Method 3 also outperformed Buffer Method 1 at the 0.01 level of significance. 8.3.4 Correlation Coefficient To find the relationship or correlation between two variables, the approach used is called correlation coefficient. For example, in a research work, you collected data on Âbandwidth rateÊ and also ÂjitterÊ. You may want to find out if there is a correlation between bandwidth rate and jitter in network performance. It is important to note that correlation has direction and can be positive or negative. The Pearson product-moment correlation coefficient ( represented by ÂrÊ) is used to show the strength of relationship between two variables. A coefficient can range from r = +1.00 to -1.00. Figure 8.9 shows what the coefficient means. Figure 8.9: Graphs showing Different Correlation Coeffiecients. Source: http://arts.uwaterloo.ca/jfsulliv/Lectures%2015%20&%2016.htm  TOPIC 8 QUANTITATIVE RESEARCH METHODS 102 Figure 8.9 shows perfect postive correlation ( r=+1) which means an increase in variable y is also followed by an increase in variable x. The second diagram shows a perfect negative correlation( r = -1) which means an increase in variable y is followed by a decrease in variable x or vice-versa. The third diagram shows a zero correlation(r=0.00) which means there is no relationship between variable y and variable x. SELF-CHECK 8.2 1. Describe the types of survey for quantitative research methods. 2. Identify the common statistical components for quantitative methods.  Quantitative methods deal with numbers and anything that is measurable in a systematic way of investigation of phenomena.  Three types of descriptive designs are observation, correlation and survey research.  Likert Scale is the common rating scale widely used in questionnaires.  There are two types of survey: cross sectional survey and longitudinal survey.  Questionnaires are used widely due to their cost effectiveness and easier management nature.  The t-test is used for significant differences between means for independent and dependent groups.  ANOVA is used when comparing the means of more than two groups.  Correlation coefficient is used to test the strength of relationship between two variables. TOPIC 8 QUANTITATIVE RESEARCH METHODS  103 ANOVA Observation Cohort studies Panel studies Correlation Quantitative method Cross-sectional Standard deviation Longitudinal Survey research Mean T-test 1. When would you use a longitudinal survey rather than a cross- sectional survey? Discuss. 2. Discuss a situation of using Analysis of Variance (ANOVA) in a research work (use an example with a research topic of your own). 3. Discuss some ethical issues you think could arise during survey research. Books Bailey, K. D. (1978). Methods of social research. (3rd ed.). New York: The Free Press. Grinnell (1988). Social work research and evaluation. Itasca, Illinois: F. E. Peacock Publishers. Leedy, P. D. (1993). Practical research: planning and design. New Jersey: Prentice-Hall. Kerlinger, F. (1973). The structure of scientific revolution. Chicago: University of Chicago Press. Kumar, R. (2005). Research methodology. SAGE Publications.  TOPIC 8 QUANTITATIVE RESEARCH METHODS 104 Internet Resources http://www.chssc.salford.ac.uk/healthSci/rem99/resmeth/chap2.htm http://www.socialresearchmethods.net/kb/survsel.htm ve Methods Electronic version Salford University (2007). Quantitati Trochim,W. K.(2007). Survey methods Electronic version