Statistical estimates of distribution parameters. Point estimation of distribution parameters Statistical estimation and its properties
The distribution of a random variable (general population distribution) is usually characterized by a number of numerical characteristics:
- for a normal distribution, N(a, σ) is the mathematical expectation a and the standard deviation σ ;
- for a uniform distribution R(a,b) are the boundaries of the interval in which the values of this random variable are observed.
When an estimate is defined by a single number, it is called point estimate. Point Estimation, as a function of the sample, is a random variable and varies from sample to sample during repeated experiments.
Point estimates are subject to requirements that they must satisfy in order to be "good" in any sense. This unbiasedness, efficiency And solvency.
Interval Estimates are determined by two numbers - the ends of the interval that covers the estimated parameter. Unlike point estimates, which do not give an idea of how far the estimated parameter can be from them, interval estimates allow you to establish the accuracy and reliability of estimates.
As point estimates of the mathematical expectation, variance and standard deviation, sample characteristics are used, respectively, the sample mean, sample variance and sample standard deviation.
Estimation unbiased property.
A desirable requirement for estimation is the absence of systematic error, i.e. with repeated use, instead of the parameter θ of its estimate, the average value of the approximation error is zero - this is valuation unbiased property.
Definition. An estimate is called unbiased if its mathematical expectation is equal to the true value of the estimated parameter:
The sample arithmetic mean is an unbiased estimate of the mathematical expectation, and the sample variance 
- biased estimate of the general variance D. The unbiased estimate of the general variance is the estimate
Evaluation consistency property.
The second requirement for an estimate - its consistency - means an improvement in the estimate with an increase in the sample size.
Definition. Grade 
is called consistent if it converges in probability to the estimated parameter θ as n→∞.
Convergence in probability means that with a large sample size, the probability of large deviations of the estimate from the true value is small.
Efficient Estimation Property.
The third requirement allows you to choose the best estimate from several estimates of the same parameter.
Definition. An unbiased estimator is efficient if it has the smallest variance among all unbiased estimators.
This means that the effective estimate has minimal scatter about the true value of the parameter. Note that an efficient estimator does not always exist, but one can usually choose a more efficient estimator from two estimators, i.e., with less dispersion. For example, for an unknown parameter a of a normal general population N(a,σ), both the sample arithmetic mean and the sample median can be taken as an unbiased estimate. But the variance of the sample median is approximately 1.6 times greater than the variance of the arithmetic mean. Therefore, a more efficient estimate is the sample arithmetic mean.
Example #1. Find an unbiased estimate of the variance of measurements of some random variable by one device (without systematic errors), the measurement results of which (in mm): 13,15,17.
Solution. Table for calculating indicators.
| x | |x - x cf | | (x - x sr) 2 |
| 13 | 2 | 4 |
| 15 | 0 | 0 |
| 17 | 2 | 4 |
| 45 | 4 | 8 |
simple arithmetic mean(unbiased expectation estimate)
Dispersion- characterizes the measure of spread around its mean value (measure of dispersion, i.e. deviation from the mean - a biased estimate).
Unbiased estimator of variance- consistent estimate of the variance (corrected variance).
Example #2. Find an unbiased estimate of the mathematical expectation of measurements of some random variable by one device (without systematic errors), the measurement results of which (in mm): 4,5,8,9,11.
Solution. m = (4+5+8+9+11)/5 = 7.4
Example #3. Find the corrected variance S 2 for a sample size of n=10 if the sample variance is D = 180.
Solution. S 2 \u003d n * D / (n-1) \u003d 10 * 180 / (10-1) \u003d 200
statistical estimate distribution sample
An estimate is an approximation of the values of the desired value, obtained on the basis of the results of a selective observation. Estimates are random variables. They provide the possibility of forming a reasonable judgment about the unknown parameters of the general population. An example of estimating the general mean is the sample mean of the general variance - sample variance, etc.
In order to evaluate how “well” the assessment meets the corresponding general characteristic, 4 criteria have been developed: consistency, unbiasedness, efficiency and sufficiency. This approach is based on the fact that the quality of an estimate is determined not by its individual values, but by the characteristics of its distribution as a random variable.
Based on the provisions of probability theory, it can be proved that of such sample characteristics as the arithmetic mean, mode and median, only the arithmetic mean is a consistent, unbiased, efficient and sufficient estimate of the general mean. This determines the preference given to the arithmetic mean in a number of other sample characteristics.
unbiased evaluation is manifested in the fact that its mathematical expectation for any sample size is equal to the value of the estimated parameter in the general population. If this requirement is not met, then the estimate is displaced.
The condition of unbiased estimation is aimed at eliminating systematic estimation errors.
When solving evaluation problems, they also use asymptotically unbiased estimates, for which, with an increase in the sample size, the mathematical expectation tends to the estimated parameter of the general population.
solvency statistical estimates is manifested in the fact that with an increase in the sample size, the estimate approaches the true value of the estimated parameter more and more, or, as they say, the estimate converges in probability to the desired parameter, or tends to its mathematical expectation. Only consistent estimates are of practical importance.
This is the estimate of the unbiased parameter that has the smallest variance for a given sample size. In practice, the variance of the estimate is usually identified with the error of the estimate.
As evaluation effectiveness measures take the ratio of the minimum possible variance to the variance of another estimate.
An estimate that ensures the completeness of the use of all the information contained in the sample about an unknown characteristic of the general population is called sufficient(exhaustive).
Compliance with the properties of statistical estimates discussed above makes it possible to consider the sample characteristics for estimating the parameters of the general population as the best possible.
The most important task of mathematical statistics is to obtain the most rational, "truthful" statistical estimates of the desired parameters of the general population from sample data. There are two types of statistical inference: statistical evaluation; testing of statistical hypotheses.
The main task of obtaining statistical estimates is to select and justify the best estimates that provide the possibility of a meaningful assessment of the unknown parameters of the general population.
The problem of estimating unknown parameters can be solved in two ways:
- 1. An unknown parameter is characterized by one number (point) - the method of point estimation is used;
- 2. interval estimation, that is, an interval is determined in which the desired parameter can be found with some probability.
Point Estimation of the unknown parameter lies in the fact that a specific numerical value of the sample estimate is taken as the best approximation to the true parameter of the general population, that is, the unknown parameter of the general population is estimated by one number (point) determined from the sample. With this approach, there is always a risk of making a mistake, so the point estimate must be supplemented by an indicator of the possible error at a certain level of probability.
Its standard deviation is taken as the average estimation error.
Then the point estimate of the general mean can be represented as an interval
where is the sample arithmetic mean.
In point estimation, several methods are used to obtain estimates from sample data:
- 1. the method of moments, in which the moments of the general population are replaced by the moments of the sample;
- 2. least squares method;
- 3. maximum likelihood method.
In many problems, it is required to find not only a numerical estimate of the parameter of the general population, but also to evaluate its accuracy and reliability. This is especially important for relatively small samples. A generalization of a point estimate of a statistical parameter is its interval estimation- finding a numerical interval containing the estimated parameter with a certain probability.
Due to the fact that there is always some error in determining general characteristics from sample data, it is more practical to determine the interval centered in the found point estimate, within which the true desired value of the estimated parameter of the general characteristic is located with a certain given probability. Such an interval is called a confidence interval.
Confidence interval is a numerical interval that, with a given probability r, covers the estimated parameter of the general population. This probability is called confidence. Confidence probability r is the probability that can be recognized as sufficient within the framework of the problem being solved to judge the reliability of the characteristics obtained on the basis of sample observations. the value
the probability of making a mistake is called significance level.
For a selective (point) estimate AND * (theta) of the parameter AND of the general population with an accuracy of ( marginal error) D and confidence probability r the confidence interval is determined by the equality:
Confidence probability r makes it possible to establish confidence limits random fluctuation of the studied parameter And for a given sample.
The following values and their corresponding values are often taken as a confidence level significance levels
Table 1. Most commonly used confidence levels and significance levels
For example, a 5 percent significance level means the following: in 5 cases out of 100, there is a risk of making an error in identifying the characteristics of the population from sample data. Or, in other words, in 95 cases out of 100, the general characteristic identified on the basis of the sample will lie within the confidence interval.
Let it be required to study, for example, a quantitative sign of the general population. Assume that, from theoretical considerations, it was possible to establish which distribution has a feature. Naturally, the problem of estimating the parameters that determine this distribution arises. For example, if it is known in advance that the trait under study is normally distributed in the general population, then it is necessary to estimate (approximately find) the mathematical expectation a and the standard deviation s, since these two parameters completely determine the normal distribution.
Usually, the researcher has only sample data at his disposal, for example, the values of a quantitative trait x 1, x 2, ..., x n, obtained as a result of n observations. Through these data and express the estimated parameter.
Let q * be a statistical estimate of the unknown parameter q of the theoretical distribution. Distinguish unbiased And displaced estimates.
unbiased is called a statistical estimate q * , the mathematical expectation of which is equal to the estimated parameter q for any sample size, that is
Otherwise, that is, if M(q *) ¹ q, the estimate is called displaced.
The requirement of unbiasedness means that there should not be a systematic deviation in the same direction of the observed values from q.
There is also a requirement for statistical evaluation. efficiency, which implies (for a given sample size) the smallest possible variance, and in the case of a large sample size, the requirement solvency, that is, the practical coincidence of the observed values of the random variable with the estimated parameter.
If the statistical material is presented in the form of a variational series, then its subsequent analysis is carried out, as a rule, with the help of some constant values that quite fully reflect the patterns inherent in the studied general population.
These constants include average values, among which the most significant is arithmetic mean- it is simpler than others both in meaning, and in properties, and in the method of obtaining.
Since sampling is carried out in the study of the general population, the constant characterizing the sample is called sample mean and is denoted.
It can be shown that there is unbiased estimator the arithmetic mean value of the sign of the general population, that is
Let some set be divided into parts - groups, not necessarily the same size. Then the arithmetic mean distributions of group members are called group averages, and the arithmetic mean of the distribution on the same basis of the entire population - general average. The groups are called disjoint if each member of the population belongs to only one group.
The overall mean is equal to the arithmetic mean of the group means of all non-overlapping groups.
Example. Calculate the average wages of the workers of the enterprise according to the table
Solution. By definition, the overall average is
. (*)
n 1 \u003d 40, n 2 \u003d 50, n 3 \u003d 60
The average wage of the workers of shop No. 1. To find it, we compiled the arithmetic average salary for the entire shop: 75, 85, 95 and 105 (c.u.). For convenience, these values \u200b\u200bcan be reduced five times (this is their largest common divisor): 15, 17, 19, 21. The rest is clear from the formula.
Having done similar operations, we find , .
Substituting the obtained values into (*), we get
Averages are constant values that characterize distributions in a certain way. Some distributions are judged only by means. For example, to compare levels wages in various branches of industry it is enough to compare the average wages in them. However, the averages cannot be used to judge either the differences between the wage levels of the highest and lowest paid workers, nor what deviations from the average wages take place.
In statistics, of greatest interest is the spread of the values of a feature around their arithmetic mean. In practice and in theoretical studies, the dispersion of a feature is more often characterized by dispersion and standard deviation.
Sample variance D B is called the arithmetic mean of the squares of the deviation of the observed values of the trait from their mean value.
If all values х 1 , х 2 , … х n of the feature of the sample size n are different, then
. (3)
If the values of the attribute x 1, x 2, ... x k have frequencies n 1, n 2, ... n k, respectively, and n 1 + n 2 + ... + n k \u003d n, then
. (4)
If there is a need for the scattering index to be expressed in the same units as the characteristic values, then you can use the summary characteristic - standard deviation
To calculate the variance, the formula is usually used
If the population is divided into non-overlapping groups, then to characterize them, we can introduce the concepts of group, intra-group, inter-group and total variance.
Group variance is the variance of the distribution of members of the j-th group relative to their average - group average, that is
where n i is the frequency of the value x i , is the volume of the group j.
Intragroup variance is the arithmetic mean of group variances
where N j (j = 1, 2, …, m) are the volumes of disjoint groups.
Intergroup variance is the arithmetic mean of the squared deviations of the group means of all non-overlapping groups from the common mean, that is
.
General variance is the variance of the values of the attribute of the entire population relative to the total average
,
where n i - frequency value x i ; - general average; n is the volume of the entire population.
It can be shown that the total variance D is equal to the sum, i.e.
Example. Find the total variance of the population consisting of the following two groups
| First group | Second group | |||
| x i | n i | x i | n i | |
Solution. Let's find the group averages
Let's find the group variances
Let's find the common average
Required total variance
The estimates considered above are usually called pinpoint, since these estimates are determined one number. When small volume sample, an interval estimate is used, determined by two numbers, called the ends of the interval.
Interval estimates make it possible to establish accuracy and reliability ratings. Let us explain the meaning of these concepts. Let the statistical characteristic q * found from the sample data serve as an estimate of the unknown parameter q. It is clear that q * the more accurately will determine the parameter q, the smaller the absolute value . In other words, if d > 0 and , then the smaller d, the more accurate the estimate.
Thus, the number d > 0 characterizes accuracy estimates. But on the other hand, statistical methods do not allow us to state categorically that the estimate q * satisfies the inequality . Here we can only talk about probabilities g, with which this inequality is realized. This probability g is called reliability (confidence probability) estimating q by q * .
Thus, it follows from what has been said that
The relation (*) should be understood as follows: the probability that the interval (q * - d, q * + d) contains (covers) the unknown parameter q is equal to g. The interval (q * - d, q * + d) covering the unknown parameter with a given reliability g is called the confidence interval.
Example. The random variable X has a normal distribution with a known standard deviation s = 3. Find the confidence intervals for estimating the unknown mathematical expectation a from the sample means if the sample size is n = 36 and the reliability of the estimate is given g = 0.95.
Solution. Note that if random value X is normally distributed, then the sample mean , found from independent observations, is also normally distributed, and the distribution parameters are: , (see page 54).
We require that the relation
.
Using formula (**) (see p. 43), replacing X by and s by in it, we obtain
Statistical estimates of the parameters of the general population. Statistical hypotheses
LECTURE 16
Let it be required to study the quantitative sign of the general population. Assume that, from theoretical considerations, it was possible to establish which distribution has a feature. This gives rise to the problem of estimating the parameters that determine this distribution. For example, if it is known that the trait under study is distributed in the general population according to the normal law, then it is necessary to estimate (approximately find) the mathematical expectation and standard deviation, since these two parameters completely determine the normal distribution. If there are reasons to believe that the feature has a Poisson distribution, then it is necessary to estimate the parameter , which determines this distribution.
Usually, in the distribution, the researcher has only sample data, for example, the values of a quantitative trait obtained as a result of observations (hereinafter, the observations are assumed to be independent). Through these data and express the estimated parameter.
Considering as values of independent random variables 
, we can say that to find a statistical estimate of an unknown parameter of a theoretical distribution means to find a function of the observed random variables, which gives an approximate value of the estimated parameter. For example, as will be shown below, to estimate the mathematical expectation of a normal distribution, the function (arithmetic mean of the observed values of a feature) is used:
.
So, statistical evaluation unknown parameter of the theoretical distribution is called a function of the observed random variables. The statistical estimate of an unknown parameter of the general population, written as a single number, is called point. Consider the following point estimates: biased and unbiased, effective and consistent.
In order for statistical estimates to give “good” approximations of the estimated parameters, they must satisfy certain requirements. Let's specify these requirements.
Let there be a statistical estimate of the unknown parameter of the theoretical distribution. Assume that when sampling the volume, an estimate is found. Let's repeat the experiment, that is, we will extract another sample of the same size from the general population and, using its data, we will find an estimate, etc. Repeating the experiment many times, we get the numbers 
, which, generally speaking, will differ from each other. Thus, the estimate can be considered as a random variable, and the numbers as possible values.
It is clear that if the estimate gives an approximate value with an excess, then each number found from the data of the samples will be greater than the true value of . Therefore, in this case, the mathematical (mean value) of the random variable will be greater than , that is, . Obviously, if it gives an approximate value with a disadvantage, then .
Therefore, the use of a statistical estimate, the mathematical expectation of which is not equal to the estimated parameter, leads to systematic (one sign) errors. For this reason, it is natural to require that the mathematical expectation of the estimate be equal to the estimated parameter. Although compliance with this requirement will not, in general, eliminate errors (some values are greater than and others less than ), errors of different signs will occur equally often. However, compliance with the requirement guarantees the impossibility of obtaining systematic errors, that is, eliminates systematic errors.
unbiased called a statistical estimate (error), the mathematical expectation of which is equal to the estimated parameter for any sample size, that is, .
Displaced called a statistical estimate, the mathematical expectation of which is not equal to the estimated parameter for any sample size, that is.
However, it would be erroneous to assume that an unbiased estimate always gives a good approximation of the estimated parameter. Indeed, the possible values may be highly scattered around their mean, i.e. the variance may be significant. In this case, the estimate found from the data of one sample, for example, may turn out to be very remote from the average value , and hence from the estimated parameter itself. Thus, taking as an approximate value, we will make a big mistake. If, however, the variance is required to be small, then the possibility of making a large error will be excluded. For this reason, the requirement of efficiency is imposed on the statistical evaluation.
efficient called a statistical estimate, which (for a given sample size ) has the smallest possible variance.
Wealthy is called a statistical estimate, which tends in probability to the estimated parameter, that is, the equality is true:
.
For example, if the variance of the unbiased estimator at tends to zero, then such an estimator also turns out to be consistent.
Consider the question of which sample characteristics best estimate the general mean and variance in terms of unbiasedness, efficiency, and consistency.
Let a discrete general population be studied with respect to some quantitative attribute .
General secondary is called the arithmetic mean of the values of the feature of the general population. It is calculated by the formula:
§ - if all values of the sign of the general population of volume are different;
§ 
– if the values of the sign of the general population have frequencies, respectively, and . That is, the general average is the weighted average of the trait values with weights equal to the corresponding frequencies.
Comment: let the population of the volume contain objects with different values of the attribute . Imagine that one object is randomly selected from this collection. The probability that an object with a feature value, for example , will be retrieved is obviously equal to . Any other object can be extracted with the same probability. Thus, the value of a feature can be considered as a random variable, the possible values of which have the same probabilities equal to . It is not difficult, in this case, to find the mathematical expectation:
So, if we consider the examined sign of the general population as a random variable, then the mathematical expectation of the sign is equal to the general average of this sign: . We obtained this conclusion, assuming that all objects of the general population have different values of the feature. The same result will be obtained if we assume that the general population contains several objects with the same attribute value.
Generalizing the result obtained to the general population with a continuous distribution of the attribute , we define the general average as the mathematical expectation of the attribute: 
.
Let a sample of volume be extracted to study the general population with respect to a quantitative attribute.
Sample mean called the arithmetic mean of the values of the feature of the sample population. It is calculated by the formula:
§ - if all values of the sign of the sample population of volume are different;
§ 
– if the values of the feature of the sampling set have, respectively, frequencies , and . That is, the sample mean is the weighted average of the trait values with weights equal to the corresponding frequencies.
Comment: the sample mean found from the data of one sample is obviously a certain number. If we extract other samples of the same size from the same general population, then the sample mean will change from sample to sample. Thus, the sample mean can be considered as a random variable, and therefore, we can talk about the distributions (theoretical and empirical) of the sample mean and the numerical characteristics of this distribution, in particular, the mean and variance of the sample distribution.
Further, if the general mean is unknown and it is required to estimate it from the sample data, then the sample mean is taken as an estimate of the general mean, which is an unbiased and consistent estimate (we propose to prove this statement on our own). It follows from the foregoing that if several samples of a sufficiently large volume from the same general population are used to find sample means, then they will be approximately equal to each other. This is the property stability of sample means.
Note that if the variances of two populations are the same, then the proximity of the sample means to the general ones does not depend on the ratio of the sample size to the size of the general population. It depends on the sample size: the larger the sample size, the less the sample mean differs from the general one. For example, if 1% of objects are selected from one set, and 4% of objects are selected from another set, and the volume of the first sample turned out to be larger than the second, then the first sample mean will differ less from the corresponding general mean than the second.
) problems of mathematical statistics.
Let us assume that there is a parametric family of probability distributions (for simplicity, we will consider the distribution of random variables and the case of one parameter). Here, is a numeric parameter whose value is unknown. It is required to estimate it by the available sample of values generated by this distribution.
There are two main types of assessments: point estimates And confidence intervals.
Point Estimation
Point estimation is a type of statistical estimation in which the value of an unknown parameter is approximated by a single number. That is, you must specify the function of the sample (statistics)
,whose value will be considered as an approximation to the unknown true value .
Common methods for constructing point estimates of parameters include: maximum likelihood method, method of moments, quantile method.
Below are some properties that point estimates may or may not have.
solvency
One of the most obvious requirements for a point estimate is that one can expect a reasonably good approximation to the true value of the parameter given enough large values sample size . This means that the estimate must converge to the true value at . This evaluation property is called solvency. Since we are talking about random variables for which there are different types convergence, then this property can be precisely formulated in different ways:
When just using the term solvency, then we usually mean weak consistency, i.e., convergence in probability.
The consistency condition is practically obligatory for all estimates used in practice. Inconsistent estimates are rarely used.
Unbiasedness and asymptotic unbiasedness
The parameter estimate is called unbiased, if its mathematical expectation is equal to the true value of the estimated parameter:
.The weaker condition is asymptotic unbiasedness, which means that the mathematical expectation of the estimate converges to the true value of the parameter with an increase in the sample size:
.Unbiasedness is a recommended property of estimators. However, its importance should not be overestimated. Most often, unbiased parameter estimates exist, and then one tries to consider only them. However, there may be some statistical problems in which unbiased estimates do not exist. The most famous example is the following: consider a Poisson distribution with a parameter and set the problem of estimating the parameter . It can be proved that there is no unbiased estimator for this problem.
Grade Comparison and Efficiency
To compare different estimates of the same parameter with each other, the following method is used: choose some risk function, which measures the deviation of the estimate from the true value of the parameter, and the best one is considered to be the one for which this function takes a smaller value.
Most often, the mathematical expectation of the squared deviation of the estimate from the true value is considered as a risk function
For unbiased estimators, this is simply the variance.
There is a lower bound on this risk function called Cramer-Rao inequality.
(Unbiased) estimators for which this lower bound is met (i.e. having the smallest possible variance) are called effective. However, the existence of an effective estimate is a rather strong requirement for the problem, which is by no means always the case.
The weaker condition is asymptotic efficiency , which means that the ratio of the variance of the unbiased estimate to the lower Cramer-Rao bound tends to unity at .
Note that under sufficiently broad assumptions about the distribution under study, the maximum likelihood method gives an asymptotically efficient estimate of the parameter, and if there is an effective estimate, then it gives an efficient estimate.
Sufficient statistics
The statistic is called sufficient for the parameter if the conditional distribution of the sample provided that , does not depend on the parameter for all .
The importance of the concept of sufficient statistics is due to the following approval. If is a sufficient statistic and is an unbiased estimate of the parameter , then the conditional expectation is also an unbiased estimate of the parameter , and its variance is less than or equal to the variance of the original estimate .
Recall that the conditional expectation is a random variable that is a function of . Thus, in the class of unbiased estimators, it suffices to consider only those that are functions of sufficient statistics (provided that such a statistic exists for the given problem).
The (unbiased) effective parameter estimate is always a sufficient statistic.
We can say that a sufficient statistic contains all the information about the estimated parameter that is contained in the sample.
Loading...
