Calculate calculator buffer size?
Today when I was in math class I came up with the thought "How many bytes can my calculator hold". And after a while I came up with this: (2^8)^byte. For example: (2^8)^2 = 65536, that's the biggest number you can put in 2 bytes. My calculator gave me the overflow error on (2^8)^42.
Do you think this is right? Or is there any other way?
How many digits can your calculator hold?
For example: My calculator holds 10 decimal digits. That means the largest integer it can hold is 9999999999. Now, the rest depends on how your calculator handles decimal numbers. It may be that it just gives each digit a one byte, meaning it hold 10 bytes.
Or, if it needs to convert the decimal number to binary, 34 bits are necessary to represent the number 9999999999 in binary. However, 34 bits allow you to represent the number (2^34)-1, which is more than 10 decimal digits. So if it's doing things that way, it cuts off precision based on the decimal number, not the binary.
EDIT:
(2^8)^2 = 65536, that's the biggest number you can put in 2 bytes. The biggest number you can put in two bytes is 65535 (which is 2^16 - 1).
ynori7 wrote: [quote]Orillian wrote: what level maths is this? Binary? It's not generally taught in math classes. It's computer science.[/quote]
Just to be a dick…
Binary is not a level of math, just a number system. The same math from algebra to calculus can be applied to it. In this case its algebra so level 7th grade.