Intuition, aesthetics and performance

General APL language issues

Intuition, aesthetics and performance

Postby Phil Last on Sat Feb 20, 2010 9:36 am

Two equivalent algorithms:
      ⊃{⍺,'%',⍵}/⍵
      1↓⊃,/'%',¨⍵

Both chain a list of strings with intervening separators (represented here by '%').

I don't get to examine other people's code much but several of us are working on a system that's currently about 10MB in about 5000 individual functions, 100 operators, 300 namespaces and 75 classes.

There are 76 occurrences of the former and 8 of the latter.

You can see why. I didn't really need to explain what they did because the first one says so on the tin. It's nice to use self documenting algorithms and we all know code clarity is important for maintenance as well as pleasing to the eye.

Unfortunately:
At 10 strings the nice one takes twice as long.
At 100 strings it's 5 times.
At 1000, 100.
At 10000, my comparison function gets a domain error equivalent to divide by zero.

There is hope; the former is also equivalent to:
      ⊃'%'{⍺,⍺⍺,⍵}/⍵

which is marginally slower (10%) than the first at less than 10 strings and equivalent above 100. But its advantage is that it's invariant from the left brace rightwards.

In other words      {⍺,⍺⍺,⍵}/      might be a candidate for a recognised idiom.
User avatar
Phil Last
 
Posts: 572
Joined: Thu Jun 18, 2009 6:29 pm

Re: Intuition, aesthetics and performance

Postby harsman on Sun Feb 21, 2010 10:10 pm

Why don't you refactor all of the repeated code into a function? Call it "join". That way it can be ugly if it needs to for speed (and it's less characters to type).

We have exactly such a function and it used to look just like your first example but we re-wrote it into the second form since it was so much quicker (quadratic performance is really bad for something simple like concatenation) The re-write was easy because the definition was in one place, the join function.

While idioms are nice I would much rather have general optimisations that apply everywhere, like say reducing interpreter overhead or adding some sort of loop fusion to reduce memory traffic.
harsman
 
Posts: 27
Joined: Thu Nov 26, 2009 12:21 pm

Re: Intuition, aesthetics and performance

Postby Phil Last on Sun Feb 21, 2010 11:17 pm

Exactly the approach I should have taken in the past. But in present circumstances it would have to be at the least "##.Core.join" if I wanted to ensure its availability wherever used unless it was cloned into the twenty different namespaces that use it.
User avatar
Phil Last
 
Posts: 572
Joined: Thu Jun 18, 2009 6:29 pm

Re: Intuition, aesthetics and performance

Postby harsman on Mon Feb 22, 2010 8:46 am

Ah, too bad. This is why I want a []PATH with namespace scope, less unnecessary typing without naming conflicts. You could make []PATH search "outwards" for all namespaces, but that might cause other issues depending on how they're structured, and it doesn't help you with sibling in the namespace tree.
harsman
 
Posts: 27
Joined: Thu Nov 26, 2009 12:21 pm

Re: Intuition, aesthetics and performance

Postby kai on Mon Feb 22, 2010 11:28 am

I consider the introduction of []PATH one of the biggest mistakes ever made in APL. For a start, it doesn't avoid naming conflicts, it makes them more effective. You can't double-click a name, or press <Shift+Enter> on it. With []PATH clarity and readability is sacrificed in order to avoid a bit of typing. We have Autocompletion to save typing time.
User avatar
kai
 
Posts: 122
Joined: Thu Jun 18, 2009 5:10 pm
Location: Hillesheim / Germany

Re: Intuition, aesthetics and performance

Postby Morten|Dyalog on Mon Feb 22, 2010 12:51 pm

harsman wrote:While idioms are nice I would much rather have general optimisations that apply everywhere, like say reducing interpreter overhead or adding some sort of loop fusion to reduce memory traffic.

This year, we will start a research project into new, more general optimizations and "just-in-time-compilation" techniques. With luck, this will eventually lead to significant reductions in overhead in various parts of the interpreter, loop fusion and significantly more efficient use of memory - and allow us to distribute work efficiently across multiple cores. It will be some time before the results of this work make it into the "main stream", but we expect the impact to be "significant".

---

Regarding ⎕PATH, do you think it would solve your problem if ⎕USING was extended to allow it to refer to APL namespaces?
User avatar
Morten|Dyalog
 
Posts: 406
Joined: Tue Sep 09, 2008 3:52 pm

Re: Intuition, aesthetics and performance

Postby Phil Last on Mon Feb 22, 2010 3:05 pm

Re. ⎕PATH I agree with Kai. I'd go further. Although I've learned to live with it I consider :Include very much in the same light as ⎕PATH and other obfuscations. I didn't mention it above but my preference so far is to define a shortcut ref at the top of a function or even globally in the ns that points to the real source.

f00←{
      ut←##.Core.Utils
      ...
      z←ut.whatever ⍵
⍝ even renaming remote functions locally if I need to use them frequently
      join←ut.join
      x←', 'join list
      ...
}
User avatar
Phil Last
 
Posts: 572
Joined: Thu Jun 18, 2009 6:29 pm

Re: Intuition, aesthetics and performance

Postby Dick Bowman on Tue Feb 23, 2010 8:26 am

Well, with my duty to present an alternative point of view...

I'm happy to use ⎕PATH when it suits me, taking the view that I have a computer to handle menial jobs like remembering where I put things. The only real beef I have is that it's a simple character vector, I'd have thought it more "APLey" if it had been a nested vector. There are plenty more evil things in this world than ⎕PATH, even in APL.

I've used Phil's "abbreviation" approach occasionally (mostly because fully qualified names are just so tedious, not just to type but also to read) - works (for me) in simple cases but I think it breaks down if there are too many namespaces involved (just looked at a little "messing around" workspace and seen it contain a dozen "application" namespaces and thirty "utility" - can't abbreviate all of them).

Maybe I'm just lucky (surely not well-disciplined), but I don't ever seem to run into name clashes.
Visit http://apl.dickbowman.com to read more from Dick Bowman
User avatar
Dick Bowman
 
Posts: 235
Joined: Thu Jun 18, 2009 4:55 pm

Re: Intuition, aesthetics and performance

Postby harsman on Tue Feb 23, 2010 9:10 am

I don't really care about []PATH one way or the other, I just want a somewhat sane way to handle modularization and naming conflicts. If []PATH has namespace scope I could build a reasonable facsimile of a module system without re-writing most of our code base, but I'd be open to alternatives.

What I would like:

    A standard way to say "this module/namespace uses and depends on these other modules".

    A choice on whether to refer to these other modules I depend on with their full prefix (core.vec.join), something shorter I invented myself (v.join) or without any prefix at all (join).

    A standard way to map namespaces/modules to the file system to enable loading them dynamically from text files or work spaces.

    A pony.

[]USING working for APL-namespaces or []PATH having namespace scope would solve most of these and I could build a module system on top of it, by having some magic function called import that did all the dirty work.

This is essentially how our current module system works, except it pre-dates namespaces so it is based on prefixing everything with the module name, and there is a huge pile of "utility" functions that are available in the global namespace.
harsman
 
Posts: 27
Joined: Thu Nov 26, 2009 12:21 pm

Re: Intuition, aesthetics and performance

Postby harsman on Tue Feb 23, 2010 9:18 am

Morten|Dyalog wrote:
harsman wrote:While idioms are nice I would much rather have general optimisations that apply everywhere, like say reducing interpreter overhead or adding some sort of loop fusion to reduce memory traffic.

This year, we will start a research project into new, more general optimizations and "just-in-time-compilation" techniques. With luck, this will eventually lead to significant reductions in overhead in various parts of the interpreter, loop fusion and significantly more efficient use of memory - and allow us to distribute work efficiently across multiple cores. It will be some time before the results of this work make it into the "main stream", but we expect the impact to be "significant".


That sounds awesome. Have you looked at LuaJIT? It has very impressive numerical performance, especially for a dynamically typed language like Lua. I think the semantics for numbers in Lua match APL pretty well, everything can be promoted to doubles.
harsman
 
Posts: 27
Joined: Thu Nov 26, 2009 12:21 pm

Next

Return to Language

Who is online

Users browsing this forum: No registered users and 1 guest