Proposal: APL Language extensions

APL-related discussions - a stream of APL consciousness.
Not sure where to start a discussion ? Here's the place to be
Forum rules
This forum is for discussing APL-related issues. If you think that the subject is off-topic, then the Chat forum is probably a better place for your thoughts !

Proposal: APL Language extensions

Postby neeraj on Thu Dec 03, 2009 5:27 am

As we all know we live in the array world. So everything that we need or use we arrayize. For e.g. []XML returns an array from essentially a tree structured text file. I wonder if the lanuage should / could be extended so that we can deal with other collections as arrays which are not necessarily square or rectangle for e.g. a tree. How about associative arrays or dictionaries.

Just throwing it out into the open to see what the users of this forum think.
neeraj
 
Posts: 81
Joined: Wed Dec 02, 2009 12:10 am
Location: Ithaca, NY, USA

Re: Proposal: APL Language extensions

Postby StefanoLanzavecchia on Thu Dec 03, 2009 10:50 am

Here are a couple of classes that implement an associative array (a general one and one specialised for case-insensitive strings as keys). There are probably references to utility functions that anybody should be able to easily substitute with their private versions.

Code: Select all
:Class Dict
    :field public Instance _Keys←⍬
    :field public Instance _Values←⍬

    :field public Instance _ErrorIfNotFound←0
    :field public Instance _Default←''

    :property Keyed Default Item
    :Access Public Instance
        ∇ v←get arg;k
          k←⊃arg.Indexers
          v←GetString k
          v←{(⊂⍣((0≠⊃⍴⍴v)∧0=⊃⍴⍴k))⍵}v
        ∇
        ∇ set arg;v;k
          v k←arg.(NewValue Indexers)
      ⍝    k←{(⊂⍣(2≠≡⍵))⍵}k
          (⊃k){Put ⍺ ⍵}¨v
        ∇
    :endProperty

    ∇ Put(k v);i
      i←_Keys⍳⊂k←NormalizeKey k
      :If (⍴_Keys)<i
          _Keys,←⊂k ⋄ _Values,←⊂''
      :EndIf
      (i⊃_Values)←v
    ∇

    ∇ v←GetString k
      :Access Public Instance
      k←{(⊂⍣(2≠≡⍵))⍵}k
      v←(_Values,_ErrorIfNotFound↓⊂_Default)[_Keys⍳NormalizeKey¨k]
      v←{(⊃⍣(0=⊃⍴⍴k))⍵}v
    ∇

    ∇ v←GetStringWithDefault(k def)
      :Access Public Instance
      k←{(⊂⍣(2≠≡⍵))⍵}k
      v←(_Values,⊂def)[_Keys⍳NormalizeKey¨k]
      v←{(⊃⍣(0=⊃⍴⍴k))⍵}v
    ∇

    ∇ v←GetNumber k
      :Access public instance
      :If ''≢v←GetString k
          v←⊃(//)⎕VFI v
      :EndIf
    ∇

    ∇ key←NormalizeKey key
      :Access Public Overridable
    ∇

    ∇ tbl←FromMatrix mx
      :Access Public Shared
      tbl←⎕NEW ⎕THIS
      tbl.(_Keys _Values)←↓[1]mx
    ∇


    ∇ RemoveValue v
      :Access Public Instance
      _Keys _Values⌿⍨←⊂v∘≢¨_Values
    ∇

    ∇ r←ToMatrix
      :Access Public Instance
      r←_Keys,[1.5]_Values
    ∇

:endclass

:Class DictCI : Dict
 ⍝ Dictionary with Case Insensitive keys
   
    ∇ key←NormalizeKey key
      :Access Public Override
      key←#.UPPERCASE key
    ∇
:EndClass


No documentation.
But a quick sample could be as follows. Don't expect to be able to run it... it's a snippet from production code so there's stuff missing around, but it should give a taste for the syntax.

Code: Select all
 vars←'@PDFPages'(⍕PDF.CountPages path,p)
 vars⍪←'@NomeLotto'fname

 vars⍪←'@XML'(fname,'_I.xml')
 vars⍪←'@INF'(fname,'.inf')
 vars⍪←'@CPX'(fname,'.cpx')

 vars⍪←'@Path'path

 tbl←Dict.FromMatrix vars
 tbl._Default←'*UNKNOWN*'

 dt←0 0 2000+3↑2⊃'/'⎕VFI⊃tbl[⊂'data fine reg']     ⍝ 30/11/06
 tbl[⊂'MeseRiferimento']←dt[2]
 tbl[⊂'AnnoRiferimento']←dt[3]

 attr←⎕NEW Dict
 attr[⊂'Producer']←⊂'Utente'
 attr['Id_lotto' 'Id_file' 'Info_filename' 'Prn_filename']←tbl['@NomeLotto' '@XML' '@INF' '@CPX']
 attr[⊂'Language']←⊂'PDF'
User avatar
StefanoLanzavecchia
 
Posts: 84
Joined: Fri Oct 03, 2008 9:37 am

Re: Proposal: APL Language extensions

Postby alexbalako on Thu Dec 03, 2009 4:26 pm

Do not forget that base language already has "nested" arrays, which may represent tree structure.
alexbalako
 
Posts: 16
Joined: Mon Nov 30, 2009 8:58 pm

Re: Proposal: APL Language extensions

Postby Phil Last on Thu Dec 03, 2009 10:19 pm

alexbalako wrote:Do not forget that base language already has "nested" arrays, which may represent tree structure.


And "ordinary" namespaces allow one to implement dictionaries but without having to quote all the targets.

dict←⎕NS''
dict.(this that and the other)←Tom Dick and a Harry
User avatar
Phil Last
 
Posts: 557
Joined: Thu Jun 18, 2009 6:29 pm

Re: Proposal: APL Language extensions

Postby neeraj on Fri Dec 04, 2009 3:42 am

Thanks to everbody who replied. I would like to draw attention to the fact that []XML has been implemented as returning an array. I am not suggesting even for a moment that this is wrong in any way, just to the fact that we did not have a data structure which would be more appropriate hence it was implemented to return a rectangular array.

While the language is rich enough to provide workarounds to achieve similar effects, the purpose of my email was to point to where the APL language is deficient. It could be argued that everything that Dfns can do can be done by explicit functions so there is no need for Dfns, but look at the elegance imparted to code by DFns.
neeraj
 
Posts: 81
Joined: Wed Dec 02, 2009 12:10 am
Location: Ithaca, NY, USA

Re: Proposal: APL Language extensions

Postby JohnS|Dyalog on Fri Dec 04, 2009 7:08 am

I believe that Ken Iverson's original notation in his book "A Programming Language" did indeed have trees.

It seems to me that there is some merit in our current position that "everything is an array". Otherwise, we would have to define the effect of applying each of our primitive functions to the new data types. This would be possible but it would increase the size of the language significantly.

I wonder also, if some algebraic reasoning and expression transformation is possible only if we know that the arguments are arrays?

It's amusing to compare our predicament with that of some of the FP languages, where "everything is a list": we implement lists using arrays and they implement arrays using lists :-).
JohnS|Dyalog
 

Re: Proposal: APL Language extensions

Postby Morten|Dyalog on Mon Dec 07, 2009 1:36 pm

neeraj wrote:... So everything that we need or use we arrayize. For e.g. ⎕XML returns an array from essentially a tree structured text file. I wonder if the lanuage should / could be extended so that we can deal with other collections as arrays which are not necessarily square or rectangle for e.g. a tree. How about associative arrays or dictionaries.


I don't know whether I like the idea of adding data structures to APL which are not "arrays", but arrays already support at least two kinds of trees: nested arrays and nested objects.

I think it *is* worth thinking about whether we could add an operator or two which would allow the application of APL functions to nodes in a the trees that we can already represent.

We'll take a close look at this in the months to come and see whether any elegant ideas come to the surface.

Morten

P,S, With respect to ⎕XML, can you explain about some operations which you would find easier to express if the data was represented as a tree?
User avatar
Morten|Dyalog
 
Posts: 394
Joined: Tue Sep 09, 2008 3:52 pm

Re: Proposal: APL Language extensions

Postby neeraj on Wed Dec 09, 2009 4:05 am

I was exposed to []XML for the first time at the APL conference and the first thought that crossed my mind was that xml is inherently a tree like structure and is it the case that

1. Are we using an array because that is what we know

or

2. An array is the best solution even if we had a tree structure.

I have not worked sufficiently with XML that I can give you a cogent example right now where a tree structure would definitely benefit (other forum members who have more experience, please contribute). But my feeling is that we have stayed with arrays for almost 50 years now. It may be time to explore new frontiers.

While we are at it I think we should also look at text processing facilities in Dyalog natively. I think when dealing with more than a few megabytes the search facility used to become painfully slow. A solution exists in the form of using .NET external code but I believe that there are a sufficiently large no. of apps out there that would be written or benefit from superior text processing natively in APL. Once again these are my opinions and stuff may have changed since I last explored. To be fair the datasets that I have dealt with range from 100s of megabytes to terabyte size.
neeraj
 
Posts: 81
Joined: Wed Dec 02, 2009 12:10 am
Location: Ithaca, NY, USA

Re: Proposal: APL Language extensions

Postby AndyS|Dyalog on Wed Dec 09, 2009 9:31 am

That thought had occurred to us :-)

Some people wish to be able to do string matching on APL code, and others are concerned more with parsing on text files - so the trick is to come up with a solution that works with all versions of Dyalog APL, irrespective of platform, and meets as many requirements as possible. The discussions so far have been based on using PCRE-compliant functionality, allowing both pattern matching and string replacement ..
User avatar
AndyS|Dyalog
 
Posts: 235
Joined: Tue May 12, 2009 6:06 pm

Re: Proposal: APL Language extensions

Postby DanB|Dyalog on Wed Dec 09, 2009 11:00 am

There is already a user command (named WSLOC) in 12.1 - which is V11 compatible via Spice - that allows to search/replace the ws using regular expressions. It uses .Net but is similar to PCRE.
DanB|Dyalog
 

Next

Return to APL Chat

Who is online

Users browsing this forum: No registered users and 1 guest