precision
The number of digits used to express the fractional part of a number. The more digits, the more precision. See single precision and double precision. See also accuracy.precision [prə′sizh·ən]
(mathematics)
The number of digits in a decimal fraction to the right of the decimal point.
(navigation)
In air operations, pertaining to a navigational facility which provides a combined azimuth and glide slope guidance to a runway.
(science and technology)
The measure of the range of values of a set of measurements; indicates reproducibility of the observations.
| (mathematics) | precision - The number of decimal places to which a number
is computed. Compare accuracy. |
Precision
a measure of the dispersion of the values of a random variable. The precision h is related to the standard deviation σ by the formula
This method of measuring dispersion is justified since, in the case of a normal distribution, the probability density of a random variable with precision h and mathematical expectation a is described by the formula
The precision is used as a measure of the dispersion primarily in gunnery theory and the theory of errors.
Want to thank TFD for its existence? Tell a friend about us, add a link to this page, add the site to iGoogle, or visit the webmaster's page for free fun content.
| ?Page tools | ||
|---|---|---|
|