noun[mass noun] Computing
The use of twice the usual number of bits to represent a number, giving greater arithmetic accuracy: [as modifier]: run on a computer in double-precision arithmetic
More example sentences
- My binary math is a little rusty, but that limit seems to correspond to 32-bit double precision real arithmetic.
- When writing software, this can be achieved by using statistical software or a programming language offering double precision rather than a spreadsheet, and by obtaining error estimates using the methods reviewed in this paper.
- The type is notorious because of its single precision overflow behavior, so most people end up using even when double precision is not required.
Definition of double precision in:
- The US English dictionary