round-off error

(redirected from Roundoff error)

round-off error

[′rau̇n‚dȯf ‚er·ər]
(mathematics)
McGraw-Hill Dictionary of Scientific & Technical Terms, 6E, Copyright © 2003 by The McGraw-Hill Companies, Inc.
References in periodicals archive ?
Using Nigam method to calculate ground motion response spectrum, Nigam method is that in all of the computation process without introducing any approximate calculation method, will not produce any truncation and roundoff error and has a high precision, so in the current commonly used this exact solution of Nigam method for general processing seismic engineering earthquake record.
We performed a roundoff error analysis of the CholeskyQR2 algorithm for computing the QR decomposition of an m x n real matrix X, where m [greater than or equal to] n.
We note that if A is normal, then [parallel]A[parallel] = [rho](A) but when the matrix A is not normal the spectral radius gives no indication of the magnitude of the roundoff error for finite M.
So we can easily obtain the following roundoff error equation:
For the non-diagonal element of the matrix, the roundoff error is given as follows,
Everything I've done in my career, in dollars and cents, is a roundoff error for a good-sized Phase III trial.
Reducing the spatial or temporal steps caused smaller discretization error, but larger roundoff error due to the increased number of calculation.
Note, however, that there is a significant amount of roundoff error, particularly in evaluating partial derivatives, and IEEE standard single precision is not sufficient.
Buckettop and width must be double precision to avoid fatal accumulation of roundoff error in the often repeated update operation: buckettop += width;
Roundoff error greatly affected the value of [V.sub .6].
Other roundoff error considerations are studied in detail in Ward [1977].