Math primitives on larger .net num representations. INT64...
2 posts
• Page 1 of 1
Math primitives on larger .net num representations. INT64...
What is the recommended way of dealing with simple math on these larger .NET values?
I.e. UInt64 / 12
I.e. Int64 * 12
etc..
The interpreter does not extend the primitives to these larger values like Int32 and Floats.
As you see I get a DOMAIN ERROR.
Converting the number to text and executing it is one dirty workaround.
Using a class that contains Divide is one option
but is that the best way?
Norbert
I.e. UInt64 / 12
I.e. Int64 * 12
etc..
- Code: Select all
⎕using←'System' 'Dyalog'
val←Convert.ToUInt64⊂ '23424234'
val
23424234
The interpreter does not extend the primitives to these larger values like Int32 and Floats.
As you see I get a DOMAIN ERROR.
- Code: Select all
val÷12
DOMAIN ERROR
val÷12
∧
Converting the number to text and executing it is one dirty workaround.
- Code: Select all
(1⊃⎕vfi val.ToString ⍬)÷12
1952019.5
(⍎ val.ToString ⍬)÷12
1952019.5
Using a class that contains Divide is one option
- Code: Select all
Decimal.Divide val 12
1952019.5
but is that the best way?
Norbert
-
norbertjurkiewicz84 - Posts: 62
- Joined: Mon Nov 01, 2010 7:26 pm
Re: Math primitives on larger .net num representations. INT6
Int64s and UInt64s are returned as objects (aka namespaces) to the interpreter. To convert these to DECFs on which (if ⎕FR is set to 1287), Dyalog can perform arithmetic, I'd recommend something like:
Convert.ToDecimal val
This will work on all sort of "types" of val:
Convert.ToDecimal⊂'23424234'
Note, that if you don't need the extra precision that DECFs give you over doubles (which only offer 53 bits of integer precision), you can Convert.ToDouble, and leave DR at 645.
Note also, that technically we could auto-convert all incoming Int64 and UInt64s to DECFs but that would be a breaking change from pre-DECF versions of the interpreter so we decided not to do that.
Convert.ToDecimal val
This will work on all sort of "types" of val:
Convert.ToDecimal⊂'23424234'
Note, that if you don't need the extra precision that DECFs give you over doubles (which only offer 53 bits of integer precision), you can Convert.ToDouble, and leave DR at 645.
Note also, that technically we could auto-convert all incoming Int64 and UInt64s to DECFs but that would be a breaking change from pre-DECF versions of the interpreter so we decided not to do that.
-
JohnD|Dyalog - Posts: 74
- Joined: Wed Oct 01, 2008 9:35 am
2 posts
• Page 1 of 1
Who is online
Users browsing this forum: No registered users and 1 guest
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group