From Wikipedia, the free encyclopedia In computer science, the precision of a numerical quantity is a measure of the detail in which the quantity is expressed. This is usually measured in bits, but sometimes in decimal digits. It is related to precision in mathematics, which describes the number of digits that are used to express a value. Precision is often the source of rounding errors in computation. The number of bits used to store a number will often cause some loss of accuracy. An example would be to store "sin(0.1)" in IEEE single precision floating point standard. The error is then often magnified as subsequent computations are made using the data (although it can also be reduced). Arbitrary-precision arithmetic Extended precision IEEE754 (IEEE floating point standard) Integer (computer science) Significant figures Truncation Retrieved from "https://en.wikipedia.org/w/index.php?title=Precision_(computer_science)& oldid=708962480" Categories: Computer data This page was last modified on 8 March 2016, at 12:30. Text is available under the Creative Commons Attribution-ShareAlike License; additional terms may apply. By using this site, you agree to the Terms of Use and Privacy Policy. Wikipedia® is a registered trademark of the Wikimedia Foundation, Inc., a non-profit organization. Precision (computer science) - Wikipedia, the free encyclopedia https://en.wikipedia.org/wiki/Precision_(computer_science) 1 of 1 2016/4/26 上午 12:18

Precision (Computer Science)

Download PDF Report

Upload
alex-chen
View
213
Download
0

Embed Size (px)

DESCRIPTION

Precision (Computer Science) wiki

Citation preview

From Wikipedia, the free encyclopedia

In computer science, the precision of a numerical quantity is a measure of the detail in which thequantity is expressed. This is usually measured in bits, but sometimes in decimal digits. It is related toprecision in mathematics, which describes the number of digits that are used to express a value.

Precision is often the source of rounding errors in computation. The number of bits used to store anumber will often cause some loss of accuracy. An example would be to store "sin(0.1)" in IEEEsingle precision floating point standard. The error is then often magnified as subsequentcomputations are made using the data (although it can also be reduced).

Arbitrary-precision arithmeticExtended precisionIEEE754 (IEEE floating point standard)Integer (computer science)Significant figuresTruncation

Retrieved from "https://en.wikipedia.org/w/index.php?title=Precision_(computer_science)&oldid=708962480"

Categories: Computer data

This page was last modified on 8 March 2016, at 12:30.Text is available under the Creative Commons Attribution-ShareAlike License; additional termsmay apply. By using this site, you agree to the Terms of Use and Privacy Policy. Wikipedia® is aregistered trademark of the Wikimedia Foundation, Inc., a non-profit organization.

Precision (computer science) - Wikipedia, the free encyclopedia https://en.wikipedia.org/wiki/Precision_(computer_science)

1 of 1 2016/4/26 上午 12:18