Beginning Research  Action Research  Case Study  Interviews  Observation Techniques  Education Research in the Postmodern
Evaluation Research in Education  Narrative Presentations  Qualitative Research  Quantitative Methods  Questionnaires  Writing up Research
Quantitative Methods in Education Research
Dr Ulrike Hohmann
Originally prepared by Professor John Berry
© J Berry, Centre for Teaching Mathematics,
University of Plymouth, 2005
(links updated August 2006)
This component is unable to do more than help you to begin thinking about quantitative methods in educational research. Its aim is to give you an insight into the issues should you choose quantitative methods as part of your research methodology.
We will briefly address the following questions:
Education research has moved away largely from the numbers approach in recent years, and the emphasis has been on qualitative methods. However, the use of numbers can be a very useful tool, either as part of a larger project that employs many different methods or as a basis for a complete piece of work. With the use of sophisticated software packages such as SPSS it is relatively easy to deal with the computation side of things and it is possible to come up with numerous tables and charts almost instantly once your data is installed. However, it is very important that the underlying principles of statistical analysis are understood if sense is to be made of the results spewed out by such a package in terms of your research.
This component consists of two sections; we begin with an overview of quantitative methods and finish with a brief introduction to some of the basic statistical concepts to be looking for when you read research papers that use quantitative methods of research.
B. QUANTITATIVE AND QUALITATIVE RESEARCH
In simple terms we can think of two approaches to investigations in educational research: qualitative and quantitative. In the former we use words to describe the outcomes and in the latter we use numbers.
Quantitative research methods were originally developed in the natural sciences to study natural phenomena. However examples of quantitative methods now well accepted in the social sciences and education include:
Qualitative research methods were developed in the social sciences to enable researchers to study social and cultural phenomenon. Examples of qualitative methods include:
Other components of this module cover various qualitative research methods.
Structure of Research Papers
When setting out on educational research you will be (have been) encouraged by your supervisor to read appropriate publications and this is a good way of identifying the methods of research that seem most used in your research area. A typical structure for a research paper is summarised in the table below:








Table 1 Structure of typical research paper
Activity
1 Scan read the following three papers: Identify the research method being used in each paper. Answer the question 'Why use numbers in education research?' with reference to these examples. 
C. INGREDIENTS OF QUANTITATIVE RESEARCH
As part of your research you will be looking at certain characteristics (variables) and endeavouring to show something interesting about how they are distributed within a certain population. The nature of your research will determine the variables in which you are interested. A variable needs to be measured for the purpose of quantitative analysis.
We may collect data concerning many variables, perhaps through a questionnaire, or choose to measure just two or several variables by observation or testing. The variables we are interested in may be dependent or independent. There will be other features present in the problem that may be constant or confounding.
Using the data that you have collected then you can:
See the section on Variablesin The Research Methods Knowledge Base.
Often it is not possible to undertake a true experiment as part of your research and a common research approach in educational research is called quasiexperimental design represented in the following diagram:
Experimental Group  O1  X  O2 
Control Group  O3  O4 
In this figure O1 and O3 represent initial testing of the two groups; X represents some intervention or experimentation strategy with one of the groups and O2 and O4 represent final testing of the two groups. We would use the test results to investigate whether the experimental teaching approach has led to an improvement in the feature being tested.
Perhaps the best way to begin to appreciate the kind of statistics that you might employ in your own research is to have a look at what others have done.
Read the paper by John Berry and Pasi Sahlberg: Investigating Pupils' Ideas of Learning. Learning and Instruction. 6 No 1, pp 1936, 1996 and attempt to identify the statistics that are used.
At the heart of quantitative research methods is some very sound statistical theory. If you are planning to carry out a research investigation using quantitative research methods you do not need a thorough grounding in this theory but you will need an understanding of the statistical methods. We use statistical software packages to do the arithmetical calculations so the important skill is not doing the mathematics but is interpreting the results.
In what follows we have gathered together some of the essential statistical ideas needed for quantitative research. It is a summary with some examples to provide a flavour of the ideas.
E. STATISTICAL CONCEPTS & QUANTITATIVE PROCEDURES
NB What follows is merely an introductory overview of some of the relevant concepts and procedures. To find out more go first to a general textbook such as Denscombe (1998), Chapter 10, and then, for a much fuller account, try Peers (1996). 
1. Variables
See also the section on Variables in The Research Methods Knowledge Base.
2. Basic Measures
3. Frequency Distribution
Example AA new hybrid apple is developed with the aim of producing larger apples than a particular previous hybrid. In a sample of 1000 apples, the distribution of weights was as follows:
Suppose that we graph the data using columns to show the amount in each group. We get a frequency distribution.
From the data there are 168 apples whose weight is less than 150 g and 832 apples whose weight is greater than 150 g. There are 1000 apples altogether. We can deduce that the proportion = 16.8 % cannot be sold to the supermarket. The mean weight of the apples is 223.35 g and the standard deviation is 78.9 g. The difference in weight between 150 g and the mean is 73.35 g and this is of a standard deviation. 
4. Measures of Location and Dispersion
A distribution is symmetrical if the difference between the mean and the median is zero. An appropriate pictorial representation of the data, (histogram, stem and leaf diagram etc.) would produce a mirror image about the centre:
A distribution is positively skewed (or skewed to the right) if the mean  median is greater than zero. Such data when represented by a histogram would have a right tail that is longer than the left tail:
A distribution is negatively skewed (or skewed to the left) if the mean  median is less than zero. Such data when represented by a histogram would have a left tail that is longer than the right tail:
If data are skewed then the best measure of location is the median and the best measure of dispersion is the interquartile range.
If data are symmetrical then the best measure of location is the mean and the best measure of dispersion is the standard deviation or variance.
5. Probability
This is an important concept in statistics and is an important part of our story.
It is defined in the following way: if an experiment has n equally likely outcomes and q of them are the event E, then the probability of the event E, P(E), occurring is
P(E) =
Some simple examples:
the probability of getting a head from the toss of a fair coin is
the probability of getting a six from the throw of a fair die is
the probability of getting the ace of spades from cutting of a pack of cards is
The smaller the probability the more unlikely the event is to happen. This is an important concept in quantitative methods in education research as we shall see.
There is an important link between probability and the frequency distribution. Consider again the hybrid apple example above. We saw that the proportion of apples weighing less than 150 grams was 0.168. If we pick up one of the apples ‘at random’ then it could weigh less than 150 g and be rejected or it could weigh more than 150 g and be accepted by the supermarket.
The probability of picking such an apple is 0.168.
ExerciseTo illustrate this idea further complete the following:
The important idea here is that the probability is associated the amount of data under the distribution graph. 
6. Testing an hypothesis
There are two basic concepts to grasp before starting out on testing an hypothesis.
This is referred to as the null hypothesis and is labelled H_{0}. The conclusions of a hypothesis test lead either to acceptance of the null hypothesis or its rejection in favour of the alternative hypothesis H_{1}.
Hypothesis testing: a hypothesis test or significance test is a rule that decides on the acceptance or rejection of the null hypothesis based on the results of a random sample of the population under consideration.
step
1: Formulate the practical problem in terms of hypotheses. The null hypothesis needs to be
very simple and represents the status quo, i.e. there is no difference between the
processes being tested. step 2. Calculate a statistic that is a function purely of the data. All good statistics should have two properties: (i) they should tend to behave differently when H_{0} is true from when H_{1} is true; and (ii) its probability distribution should be calculable under the assumption that H_{0} is true. step 3: Choose a critical region. We must be able to decide on the kind of values of the test statistic, which will most strongly point to H_{1} being true rather than H_{0}. The value of the test statistic in this critical region will lead us to reject H_{0} in favour of H_{1}; otherwise we are not able to reject H_{0} in favour of H_{1}. We should never conclude by accepting H_{0}. step 4: Decide the size of the critical region. i.e. a 1% probability of H_{0} being rejected etc. 
For more on this see the section on Hypotheses in The Research Methods Knowledge Base.
Example BIn an educational research programme two groups of students are taught a topic in different ways. An experimental group uses a spreadsheet to explore the topic and a control group uses a more traditional pen and paper activity. Each group contains 20 students. At the end of the topic the teacher tests the two groups on their understanding of the topic and obtains the following data:
Extra data:
Experimental Group Control Group How would you interpret these findings?
Some analysisThe researcher might be tempted to conclude that the experiment has had little or no effect on the performance of the experimental group as judged by the means. However the large difference in standard deviations might suggest that the experimental group is much more variable in performance than the control group. The researcher might also be tempted to deduce that the experiment has turned some of the pupils off the task. Look at the three low scores! Suppose that we investigate the difference in the means: Let H_{0}: there is no difference between the means of the two groups: m _{1} = m _{2} H_{1}: the score of the experimental group is greater than the score of the control group: m _{1} > m _{2} We use a twosample ttest to get
The p value is 0.39 (39%) so we deduce that there is not enough evidence to reject the null hypothesis. The researcher could not deduce that there was an improvement in student performance. 
7. Statistical tests
t tests
In hypothesis testing, the t test is used to test for differences between means when small samples are involved. (n £ 30 say). For larger samples use the z test.
The t test can test
i) if a sample has been drawn from a Normal population with known mean and variance. (Single sample)
ii) if two unknown population means are identical given two independent random samples. (Two unpaired samples)
iii) if two paired random samples come from the same Normal population. (Two paired samples (paired differences))
Any hypothesis test can be one tailed or two tailed depending on the alternative hypothesis, H_{1}.
Consider the null hypothesis, H_{0}: m =3
A one tailed test is one where H_{1} would be of the form m > 3.
A twotailed test is one where H_{1} would be of the form m ¹ 3.
Click here for more information on ttests.
Single sample test
Let X_{1}, X_{2}, ¼ , X_{n} be a random sample with mean and variance s^{2}. To test if this sample comes from a Normal population with known mean m and unknown variance s^{2}, the test statistic
is used to test the null hypothesis H_{0}: the population mean equals m. If the test statistic lies in the critical region whose critical values are found from the distribution of T_{n, a}, H_{0} is rejected in favour of the alternative hypothesis H_{1}. n are the degrees of freedom and for a single sample test n = n1, and a is the significance level of the test.
Two unpaired samples
Let X_{1}, X_{2}, ¼ , X_{m }be a random sample with mean and variance s_{x}^{2} drawn from a Normal population with unknown mean m_{x} and unknown variance s_{x}^{2}.
Let Y_{1}, Y_{2}, ¼ , Y_{n} be a random sample with mean and variance s_{y}^{2} drawn from a Normal population with unknown mean m_{y} and unknown variance s_{y}^{2}.
To test the null hypothesis that the two unknown population means are the same we use the test statistic
where , the estimate of the common population standard deviation.
The test statistic T is distributed T_{n}, where n =(m1)+(n1) for two unpaired samples. If the test statistic lies in the critical region whose critical values are found from the distribution of T_{n, a}, H_{0} is rejected in favour of the alternative hypothesis H_{1}.
Example CIt is claimed that the concentration period of students doing a particular task is normally distributed with mean 44mins. A sample of 21 students were taken, and their concentration period measured. The mean time of the sample was found to be 42mins and the sample variance was calculated to be 36min. Is there any evidence at the 5% level of significance against the claim that the population mean is 44min? Solution Here m = 44, = 42, s = Ö 36 = 6 and n = 21. This is a twotailed test since we are looking for any difference. H_{0}: m = 44 H_{1}: m ¹ 44
Since p = 0.1423 = 14.23% there is insufficient evidence to reject the null hypothesis. We therefore conclude that the population mean concentration period is 44 minutes. 
Example DA researcher investigating the effects of pollution on two rivers takes an independent random sample of fish of a certain species from each river, measures their mass in ounces and obtains the following results.
Test at the 5% level of significance if there is any evidence of a difference in the mean weight between the two rivers. Solution Assume that each sample is taken from a normal population. Here m = 6, = 13.667, s_{x}^{2} = 23.556 n = 9, = 8.333, s_{y}^{2} = 5.556. Let m _{1} be the population mean of river 1, and m _{2} be the population mean of river 2. H_{0}: m _{1} = m _{2} H_{1}: m _{1} ¹ m _{2}
Since p = 0.043 = 4.3% < 5% we deduce that there is sufficient evidence to reject the null hypothesis at the 5% level of significance. We therefore conclude that the mean weight of fish in River 1 is not equal to the mean weight of fish in River 2. 
(NB: only for those University of Plymouth students undertaking the ‘Research in Education’ module as part of the preparation for the submission of a MA dissertation proposal)
Tasks, once completed, should be sent to resined@plymouth.ac.uk, making clear:
It will then be passed on to the component leader (and copied to your supervisor). The component leader will get back to you with comments and advice which we hope will be educative and which will help you in preparing your dissertation proposal once you are ready. (Remember that these tasks are formative and that it is the proposal which forms the summative assessment for the MERS501 (resined) module.) This email address is checked daily so please use it for all correspondence about RESINED other than that directed to particular individuals for specific reasons.
Before tackling either of the two tasks (B or C) you might want to consider ...
TASK B (DATA COLLECTION)
TASK C (DATA ANALYSIS)
Student number  1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
11 
12 
Examination score  30 
31 
20 
17 
25 
32 
35 
29 
30 
27 
32 
30 
Resit score  42 
38 
30 
21 
40 
45 
31 
32 
38 
50 
34 
40 
Blaxter, I., Hughes C. and Tight M. (1996) How to Research. Buckingham, Open University Press.
Bryman, A. and Cramer D. (1999) Quantitative Data Analysis with SPSS 8 Release for Windows: a guide for social scientists. London, Routledge.
Cohen, L ; Manion, L & Morrison, K (2000) Research Methods in Education (5th edition). London, RoutledgeFalmer.
Denscombe, M. (1998) The Good Research Guide. Buckingham, Open University Press.
Greenfield, Tony (ed) (1996) Research Methods – Guidance for Postgraduates. London, Arnold.
Kanji, Gopal (1993) 100 Statistical Tests. London, SAGE Publications.
Peers, Ian (1996) Statistical Analysis for Education & Psychology Researchers. London, Falmer.
Plewis, Ian (1997) Statistics in Education. London, Arnold.
Robson, C. (1990) Experiment, Design and Statistics in Psychology. Middlesex, Penguin Books.
Rose, D. and Sullivan, O. (1993) Introducing Data Analysis for Social Scientists. Buckingham, Open University Press.
http://www.cf.ac.uk/socsi/capacity/References.html
Trochim, William M. The Research Methods Knowledge Base, 2nd
Edition. Internet WWW page, at URL: http://www.socialresearchmethods.net/kb/
(version current as of
December 20, 2006
).
(This is the excellent site referred [and linked] to several times in the sections
presented above.)
SURFSTAT australia
http://www.anu.edu.au/nceph/surfstat/surfstathome/surfstat.html
Electronic Statistics Textbook
http://www.statsoft.com/textbook/stathome.html
Beginning Research  Action Research  Case Study  Interviews  Observation Techniques  Education Research in the Postmodern
Evaluation Research in Education  Narrative Presentations  Qualitative Research  Quantitative Methods  Questionnaires  Writing up Research
© J Berry, Centre for Teaching Mathematics, University of Plymouth, 2005