The Effect of Instrumental Errors on the Accuracy of Regression Estimates in Multiparameter Monitoring
Abstract Generated abstract
This paper analyzes how instrumental measurement errors affect linear regression estimates used in multiparameter product quality monitoring. Under assumptions of large sample size, stationary error moments, and independence of errors from variables and from one another, it derives expressions for the bias and residual variance changes caused by systematic and random errors in dependent and informative variables. The authors show that random error in the dependent variable does not reduce the accuracy of estimating the true parameter, while random errors in informative variables alter regression weights and increase residual variance, and they provide formulas for recovering the true model when error moments are known. The paper also proposes criteria for specifying channel accuracy in instrument design and discusses correction channels as a means to reduce the influence of measurement errors.
Full Text
EFFECT OF INSTRUMENTAL ERRORS ON THE ACCURACY OF REGRESSION ESTIMATES IN MULTIPARAMETER MONITORING
N. M. OSKORBIN, V. S. PLOTNIKOV
(Presented by the scientific-technical seminar of the Department of Information-Measuring Technology)
In an instrument for multiparameter monitoring of product quality, the measured parameter \(h\) is determined from a set of informative variables \(x_1, x_2, \ldots, x_n\).
In developing such an instrument, a predesign stage and a design stage may be distinguished. At the predesign stage, a mathematical model is found for the interrelation between the monitored parameter \(h\) and the informative variables \(X_i (i = 1, 2, \ldots, n)\), and, by analyzing the model, the question of the fundamental possibility of constructing the instrument is resolved. Then an instrument is designed that implements the model found and satisfies the specifications for accuracy, reliability, etc.
In most cases the mathematical model may be taken to be linear,
\[ \hat{h}_1 = \sum_{i=1}^{n} c_i X_i + c_0, \tag{1} \]
where \(\hat{h}_1\) is an estimate of the measured parameter (dependent variable);
\(c_i, c_0\) are, respectively, the weighting coefficients and the constant term of the equation.
The construction of the mathematical model (1) can be carried out using the method of least squares [1]. In determining the coefficients of expression (1), the method of least squares leads to the solution of the system of equations
\[ c_i D[X_i] + \sum_{\substack{j=1 \\ j \ne i}}^{n} c_j R_{ij} = R_{ih}, \quad i = 1, 2, \ldots, n, \tag{2} \]
where \(D[x_i]\) are the variances of the independent variables;
\(R_{ij}\) are the correlation moments between the quantities \(X_i\) and \(X_j\);
\(R_{ih}\) are the correlation moments between the independent and dependent variables.
The constant term of equation (1) can be found from the equation
\[ c_0 = \bar{h} - \sum_{i=1}^{n} c_i \bar{X}_i, \tag{3} \]
where \(\bar{h}, \bar{X}_i\) are the mathematical expectations of the variables \(h\) and \(X_i\), respectively.
The variance of the error of the constructed model is found from the expression
\[ D[e_1]=M[h-\hat{h}_1]^2=D[h]-\sum_{i=1}^{n} c_i R_{ih}, \tag{4} \]
where \(M\) is the mathematical expectation sign;
\(D[h]\) is the variance of the dependent variable.
The variances, mathematical expectations, and correlation moments in formulas (2), (3), (4) are found from experimental data measured by laboratory equipment at the predesign stage; moreover, both the values of the parameter of interest \(h\) are measured with error \(\varepsilon_h\), and each of the informative variables is recorded with error \(\varepsilon_i\) \((i=1,2,\ldots,n)\). Obviously, errors in observing \(h\) and \(X_i\) reduce the accuracy of the mathematical model. The instrument estimates \(h\) from the set of informative variables \(X_i\), and each of them is likewise recorded with some error \(\varepsilon_i^*\).
The main purpose of the present work is to determine the degree to which measurement errors of the quantities \(X_i\) and \(h\) affect monitoring accuracy. First, the influence of instrumental errors on the accuracy of the regression equation is investigated, and the true model is found, i.e., the model in which the influence of measurement errors is excluded. Then the accuracy conditions for the channels of the instrument are determined, and ways of reducing the influence of channel errors on monitoring accuracy are also investigated. In analyzing the influence of instrumental errors, we shall find the increment in the variance of the error of the regression equation, and all conclusions will be drawn under the following assumptions:
1) the sample size is infinitely large;
2) the mathematical expectations and variances of measurement errors do not change with time;
3) the random errors are linearly independent both of the quantities \(h\) and \(X_i\), and of one another.
Influence of measurement errors during sample formation on the accuracy of regression estimates
The measured values of the quantities \(X_i\) and \(h\) can be represented in the following form:
\[ Y_i=X_i+\varepsilon_i,\quad i=1,2,\ldots,n; \]
\[ z=h+\varepsilon_h. \tag{5} \]
The measurement errors of the corresponding variables have both systematic and random components, with
\[ M[\varepsilon_i]=\bar{\varepsilon}_i,\quad M[\varepsilon_i-\bar{\varepsilon}_i]^2=D[\varepsilon_i],\quad i=1,2,\ldots,n; \]
\[ M[\varepsilon_h]=\bar{\varepsilon}_h,\quad M[\varepsilon_h-\bar{\varepsilon}_h]^2=D[\varepsilon_h]. \]
For the measured values of the variables one can construct the regression equation:
\[ \hat{h}_2=\sum_{i=1}^{n} b_i Y_i+b_0. \tag{6} \]
In view of the fact that \(D[Y_i]=D[X_i]+D[\varepsilon_i]\), \(i=1,2,\ldots,n\), and the correlation moments of the corresponding quantities do not depend on measurement errors [2], for example,
\[ M[(z-\bar{Z})(Y_i-\bar{Y}_i)] = M[(h-\bar{h})(X_i-\bar{X}_i)] = R_{ih}, \]
then the system of equations for determining the coefficients \(b_i\) \((i=1,2,\ldots,n)\) has the form
\[ b_i\{D[X_i]+D[\varepsilon_i]\}+\sum_{\substack{j=1\\ j\ne i}}^{n} b_j R_{ij}=R_{ih},\qquad i=1,2,\ldots,n, \tag{7} \]
and
\[ b_0=\bar z-\sum_{i=1}^{n} b_i\bar Y_i . \]
Analysis of the expressions given shows that systematic measurement errors can cause only a bias of the estimate \(h_2\) and do not affect its scatter. The bias of the estimate \(h_2\) is due only to the systematic component of the measurement error of the dependent variable and can be found from the expression
\[ M[\hat h_2-\bar h] = M\left[ \sum_{i=1}^{n} b_iY_i+\bar z-\sum_{i=1}^{n} b_i\bar Y_i-\bar h \right] = \bar\varepsilon_h . \]
The random component of the measurement error of the dependent variable \(\varepsilon_h\), as follows from the system of equations (7), does not cause a change in the weighting coefficients and therefore has no effect on the accuracy of the estimate \(h\). Usually the residual variance of the regression equation (6) is determined from the expression
\[ D[\varepsilon_z]=D[z]-\sum_{i=1}^{n} b_iR_{ih} = D[h]+D[\varepsilon_h]-\sum_{i=1}^{n} b_iR_{ih}, \tag{8} \]
and is used in regression analysis for statistical estimates of the equation, for example, to determine confidence intervals of the coefficients found. However, when this equation is used to estimate the parameter \(h\), the residual variance will be overestimated by the value \(D[\varepsilon_h]\), since the true value of the variable \(h\), and not its measured value \(z\), will be estimated.
The random components of the recording errors of the independent variables \(X_i\) \((i=1,2,\ldots,n)\) affect both the magnitude of the weighting coefficients and the residual variance. Approximate expressions for estimating the increments of the weighting coefficients and of the residual variance caused by these errors are given in [2]. An exact formula for calculating the increment of the residual variance caused by the measurement errors of all variables can be found as follows:
\[ D[\varepsilon_2/\varepsilon_i,\varepsilon_h] = M[\hat h_2-\hat h_1-\bar\varepsilon_h]^2 = M\left[ \sum_{i=1}^{n} b_i(Y_i-\bar Y_i) - \sum_{i=1}^{n} c_i(X_i-\bar X_i) \right]^2 . \]
Simple transformations of this expression, carried out with allowance for (2), (5), and (7), lead to the final formula
\[ D[\varepsilon_2/\varepsilon_i,\varepsilon_h] = \sum_{i=1}^{n} b_ic_iD[\varepsilon_i]. \tag{9} \]
It follows directly from expression (9) that the conclusion made earlier about the absence of influence of the measurement errors of the dependent variable on the accuracy of regression estimates is valid.
The true model (1) can be obtained from model (6) if the mathematical expectations and variances of the measurement errors of the quantities are known-
quantities \(h\) and \(X_i (i = 1, 2, \ldots, n)\). In this case the true weight coefficients are found from the system of equations:
\[ c_i\{D[Y_i]-D[\varepsilon_i]\}+\sum_{\substack{j=1\\ j\ne i}}^{n} c_j R_{ij}=R_{ih},\quad i=1,2,\ldots,n, \tag{10} \]
\[ c_0=\bar z-\bar\varepsilon_h-\sum_{i=1}^{n} c_i(\bar Y_i-\bar\varepsilon_i). \]
The residual variance of the true equation is determined by the expression
\[ D[\varepsilon_1]=D[z]-D[\varepsilon_h]-\sum_{i=1}^{n} c_i R_{ih}. \tag{11} \]
If the mathematical expectations and variances of the instrumental errors remain constant both when determining the coefficients of the regression equation and when estimating the measured quantity \(h\), then the following conclusions may be drawn:
1) the random component of the measurement error of the dependent variable and the systematic components of the measurement errors of the informative variables have no effect on the accuracy of the estimate;
2) the systematic component of the measurement error of the dependent variable can be taken into account by introducing the corresponding correction;
3) the influence of the random components of the measurement errors of the informative variables should be taken into account when designing a multiparameter instrument.
Specifying the Accuracy of the Channels When Designing a Multiparameter Monitoring Instrument
The design of a multiparameter monitoring instrument can be begun if there is a margin in the accuracy of the true regression equation, i.e., if the condition
\[ \Delta D[\varepsilon]=D[\varepsilon_n]-D[\varepsilon_1]>0, \]
is satisfied, where \(D[\varepsilon_n]\) is the accuracy specification for the instrument being designed.
The possible errors of the channels of the instrument being designed can be characterized by the quantities \(D[\varepsilon_i^*]\) \((i=1, 2,\ldots,n)\). Then the value of the residual variance \(D[\varepsilon^*]\) can be found from expression (11) by replacing the weight coefficients \(c_i\) \((i=1, 2,\ldots,n)\) by the weight coefficients \(b_i^*\), obtained from the system of equations
\[ b_i^*\{D[X_i]+D[\varepsilon_i^*]\}+\sum_{\substack{j=1\\ j\ne i}}^{n} b_j^* R_{ij}=R_{ih},\quad i=1,2,\ldots,n, \]
and the remaining accuracy margin must satisfy the condition
\[ \Delta D[\varepsilon]=D[\varepsilon_n]-D[\varepsilon^*]>D_0, \tag{12} \]
where \(D_0\) is the total variance of the errors of the decision device, output indicator, and other units whose errors are not taken into account in the present method.
One of the possible ways of specifying the permissible error of the channels may be a method of successive refinements, in which the channel variances \(D[\varepsilon_i^*]\) are initially specified on the basis of the possibilities of the selected circuit variants and the weight of the errors of individual channels in the instrument errors, and then the satisfiability of condition (12) is checked. If
if condition (12) is not satisfied, then the following ways of increasing the accuracy of the instrument are possible:
1) finding a new composition of informative variables;
2) reducing the value of \(D_0\);
3) increasing the accuracy of the channels;
4) introducing new channels for correcting errors in the measurement of informative variables.
The choice of a way to increase the accuracy of the instrument is determined by the specific design conditions.
Of special interest are channels for correcting errors in the measurement of informative variables. Their introduction differs from finding a new composition of informative variables in that, in the true regression equation, the value of the residual variance is not reduced by them. The variables used for correction may be signals proportional both to the factors causing the errors of some channel and to certain parameters of the monitored item; and a distinctive property of these variables is that their weight coefficients in the true regression equation are equal to zero. The ability of correcting variables to reduce the influence of channel errors is explained by the properties of the regression equation. The reduction of the residual variance of the instrument due to the introduction of a channel for correcting errors in the measurement of informative variables can be found as the difference between the variances caused by channel errors without correction and with correction:
\[ \Delta D[\varepsilon] = \sum_{i=1}^{n} c_i b_i^{*} D[\tilde{\varepsilon}_i] - \sum_{i=1}^{n+1} c_i (b_i^{*}+\Delta b_i^{*}) D[\tilde{\varepsilon}_i] = \sum_{i=1}^{n} c_i \Delta b_i^{*} D[\tilde{\varepsilon}_i], \]
where \(\Delta b_i^{*}\) \((i=1,2,\ldots,n,n+1)\) are the increments of the weight coefficients when the correction channel is introduced.
In practical cases it is possible to find certain independent variables that both increase the accuracy margin of the true regression equation and possess correcting properties.
LITERATURE
-
Yu. V. Linnik. The Method of Least Squares and the Foundations of the Theory of Processing Observations. Moscow, Fizmatgiz, 1962.
-
V. P. Boroduk, E. K. Letskii. Statistical Description of Industrial Objects. Moscow, “Energiya,” 1971.