STATISTICS III: ESTIMATION
 

Characteristics of Estimators

When a parameter is being estimated, the estimate can be either a single number or it can be a range of scores. When the estimate is a single number, the estimate is called a "point estimate"; when the estimate is a range of scores, the estimate is called an interval estimate. Confidence intervals are used for interval estimates.

As an example of a point estimate, assume you wanted to estimate the mean time it takes 12- year-olds to run 100 yards. The mean running time of a random sample of 12-year-olds would be an estimate of the mean running time for all 12-year-olds. Thus, the sample mean, M, would be a point estimate of the population mean, m.

Often point estimates are used as parts of other statistical calculations. For example, a point estimate of the standard deviation is used in the calculation of a confidence interval for m. Point estimates of parameters are often used in the formulas for significance testing.

Point estimates are not usually as informative as confidence intervals. Their importance lies in the fact that many statistical formulas are based on them. 

Characteristics of Estimators

Statistics are used to estimate parameters. There are three important attributes of statistics as estimators: unbiasedness, consistency, and relative efficiency.

When we say that a measurement is unbiased we mean that the average of a large set of unbiased measurements will be close to the true value. For example, the sample mean, M, is an unbiased estimate of the population mean, m.

When more than one statistic can be used to estimate a parameter, one will naturally be more efficient than the other(s). In general the relative efficiency of two statistics differs depending on the shape of the distribution of the numbers in the population. Statistics that minimize the sum of squared deviations such as the mean are generally the most efficient estimators for normal distributions but not for skewed distributions.

Estimating variance

The formula for the variance computed in the population, s2, is different from the formula for an unbiased estimate of variance, S2, computed in a sample. The two formulas are shown below:

The unexpected difference between the two formulas is that the denominator is N for s2 and is N-1 for S2. That there should be a difference in formulas is very counterintuitive. To understand the reason that N-1 rather than N is needed in the denominator of the formula for S2, consider the problem of estimating s2 when the population mean, m, is already known.

Assume that you knew that the mean amount of practice it takes student pilots to master a particular maneuver is 12 hours. If you sampled one pilot and found he or she took 14 hours to master the maneuver, what would be your estimate of s2 ? The answer lies in considering the definition of variance: It is the average squared deviation of individual scores from m.
With only one score, you have one squared deviation of a score from m. In this example, the one squared deviation is:
(X-m)2 = (14-12)2= 4.
This single squared deviation from the mean is the best estimate of the average squared deviation and is an unbiased estimate of s2. Since it is based on only one score, the estimate is not a very good estimate although it is still unbiased. It follows that if m is known and N scores are sampled from the population, then an unbiased estimate of s2 could be computed with the following formula:

Now it is time to consider what happens when m is not known and M is used as an estimate of m. Which value is going to be larger for a sample of N values of X:  
or
?

Since M is the mean of the N values of X and since the sum of squared deviations of a set of numbers from their own mean is smaller than the sum of squared deviations from any other number, the quantity

will always be smaller than

The argument goes that since

is an unbiased estimate of s^2 and since

will always be smaller than

then  must be biased and will have a tendency to underestimate s2. It turns out that dividing by N-1 rather than by N increases the estimate just enough to eliminate the bias exactly. 

Confidence intervals

Before a simple research question such as " What is the mean number of digits that can be remembered?" can be answered, it is necessary to specify the population of people to which it is addressed. The researcher could be interested in, for example, adults over the age of 18, all people regardless of age, or students attending high school. For the present example, assume the researcher is interested in students attending high school.

Once the population is specified, the next step is to take a random sample from it. In this example, let's say that a sample of 10 students were drawn and each student's memory tested. The way to estimate the mean of all high school students would be to compute the mean of the 10 students in the sample. Indeed, the sample mean is an unbiased estimate of m, the population mean. But it will certainly not be a perfect estimate. By chance it is bound to be at least either a little bit too high or a little bit too low.

For the estimate of m to be of value, one must have some idea of how precise it is. That is, how close to m is the estimate likely to be?

An excellent way to specify the precision is to construct a confidence interval. If the number of digits remembered for the 10 students were: 4, 4, 5, 5, 5, 6, 6, 7, 8, 9 then the estimated value of m would be 5.9 and the 95% confidence interval would range from 4.71 to 7.09.

The wider the interval, the more confident you are that it contains the parameter. The 99% confidence interval is therefore wider than the 95% confidence interval and extends from 4.19 to 7.61.

Below are shown some examples of possible confidence intervals. Although the parameter m2 - m2 represents the difference between two means, it is still valid to think of it as one parameter; π2 - π2 can also be thought of as one parameter.


 
 

A confidence interval is a range of values that has a specified probability of containing the parameter being estimated. The 95% and 99% confidence intervals which have .95 and .99 probabilities of containing the parameter respectively are most commonly used. If the parameter being estimated were m, the 95% confidence interval might look like the following:

12.5 <= m <= 30.2

What this means is that the interval between 12.5 and 30.2 has a .95 probability of containing m.

A confidence interval only has the specified probability of containing the parameter if the sample data on which it is based is the only information available about the value of the parameter.
As an extreme example, consider the case in which 1000 studies estimating the value of m in a certain population all resulted in estimates between 25 and 30. If one more study were conducted and if the 95% confidence interval on m were computed (based on that one study) to be:

35 <= m <= 45

then it would be absurd to say that the probability that m is between 35 and 45 is .95. It almost certainly is not. However, if the only data you had to go on were that one study, then, from your point of view, the probability is .95.

It is important to be very precise about the sense in which a confidence interval has a specified probability of containing a parameter: If the procedure for computing a 95% confidence interval is used over and over, 95% of the time the interval will contain the parameter.

Next you'll see how to compute a confidence interval for the mean of a normally-distributed variable for which the population standard deviation is known. In practice, the population standard deviation is rarely known. However, learning how to compute a confidence interval when the standard deviation is known is an excellent introduction to how to compute a confidence interval when the standard deviation has to be estimated.

Three values are used to construct a confidence interval for m: the sample mean (M), the value of z (which depends on the level of confidence), and the standard error of the mean (σM).
The confidence interval has M for its center and extends a distance equal to the product of z and σM in both directions. Therefore, the formula for a confidence interval is

M - z*σM <= m <= M + z*σM .

Assume that the standard deviation of SAT verbal scores in a school system is known to be 100. A researcher wishes to estimate the mean SAT score and compute a 95% confidence interval from a random sample of 10 scores.
The 10 scores are: 320, 380, 400, 420, 500, 520, 600, 660, 720, and 780. Therefore, M = 530, N = 10, and = = 31.62. The value of z for the 95% confidence interval is the number of standard deviations one must go from the mean (in both directions) to contain .95 of the scores. It turns out that one must go 1.96 standard deviations from the mean in both directions to contain .95 of the scores. The value of 1.96 was found using a z table. Since each tail is to contain .025 of the scores, you find the value of z for which 1-.025 = .975 of the scores are below. This value is 1.96.



All the components of the confidence interval are now known: M = 530, σM = 31.62, z = 1.96.
Lower limit = 530 - (1.96)(31.62) = 468.02
Upper limit = 530 + (1.96)(31.62) = 591.98

Confidence intervals can be constructed for any estimated parameter, not just m. For example, one might estimate the proportion of people who could pass a training program or the difference between the mean for subjects taking a drug and those taking a placebo.

More about confidence intervals

HOMEWORK

1. Which is wider, a 93% or a 98% confidence interval? Explain.

2. When you construct a 95% confidence interval, what are you 95% confident about?

3. What is the effect of sample size on the width of a confidence interval? Explain.

4. Why is the construction of confidence intervals considered part of inferential statistics?

5. A population is known to be normally-distributed with a standard deviation of 2.8. Compute the 95% confidence interval on the mean based on the following sample of 6 numbers: 8, 9, 12, 13, 14, 16.