Re: Possible activity heading towards XyWrite replacement

[wbass@xxxxxxxx]

Harry Binswanger wrote:
>Wally Bass wrote:
>>   replacement string for portions match by "wildcards". For
>>   example, if X denotes some kind of "wildcard" expression,
>>   and a search argument like "abXcdXef" finds matchs of
>>   abJOEcdFREDef and abMARYcdJANEef, then the change command
>>   can be made to replace

>>    abJOEcdFREDef with abMIKEcdJOHNef, and
>>    abMARYcdJANEef with abSUEcdMISTYef,

>>   by appropriate specification of from and to arguments.

>I don't think I understand your example.

And well you shouldn't, because I messed it up completely. The last four
lines of my quoted text above should have read:

>>   can be made to replace

>>    abJOEcdFREDef with 12JOE34FRED56, and
>>    abMARYcdJANEef with 12MARY34JANE56,

>>   by appropriate specification of from and to arguments.

which is basically very close to the "wildcard search-and-replace"
function that you describe in XY IV. I didn't know that had been added to
XY IV. However, the epsilon function in this area is still quite a bit
richer, and, for example can even reorder the data. To (approximately)
quote the epsilon manual:

 === start of quote from epsilon manual ===

In the replacement text of a regex replace, the # character followed by a
digit n has a special meaning. Epsilon finds the nth parenthesized
expression in the pattern, counting left parentheses from 1. It then
substitutes the match of this subpattern for the #n in the replacement
text. For example, replacing

     ([a-z]+) = ([a-z]+)
with
     #2 := #1
would change
     variable = value;
to
     value := variable;

 === end of quote from epsilon manual ===

>>  f. Coloring. Each character in a "buffer" for a file being
>>    edited has a settable *and queryable* "color" attribute,
>>    which controls the background and foregroud colors used
>>    in displaying that character. This facility is used for
>>    the "code coloring" that epsilon does.

>You can do that in TAMEdos Xy. Maybe not exactly what you mean,
>but any MoDe can be given any Windows color (of the millions of
>colors Windows has available).

I think you missed the point. The issue is not how the character is
displayed -- it's that the color attribute is information that can be
attached individually to each character in a buffer, without changing the
text itself in the buffer, and the attribute can later be examined by
macros, for whatever purpose. For example, in a file of 100000 characters,
I could mark (as the result of a search, for example) 5 or 50 or 500 or
5000 different words (or whatever) that I wanted to "come back to" for
some reason, by setting a particular color attribute for the individual
characters in the 5/50/500/5000 words, and later find these "marked" words
with a macro, but without having modified the actual text of the file one
iota. As I "come back to" the 5/50/500/5000 "marked words," I could then
inspect and process them -- in any order -- one at time, "unmarking" each
such word (by reseting it's color attribute) as I inspected it -- and
still instantly be able to locate those "originally marked" words that I
have not yet processed. Lots of different useful functionality scenarios
are possible, I think, both for direct exposure to a user, or possibly
just for use inside the logic of some larger functionality of some macro
that one is writing.

It's the facility for use in programming and data marking/manipulation
that I'm interested in, not the color display functionality itself.

>>    All of the user commands are written in EEL, and all EEL
>>    source code is provided. This is not like XPL code --
>>    it's semi-compiled a thousand times faster. So it's
>>    really a different ballgame than XyWrite.

>A thousand times faster than already virtually instantaneous? I
>have some *very* long XPL programs that do heavy processing, and
>the longest one takes about four seconds to run. User commands
>in XPL have no noticeable delay.

I think you're looking at the wrong level, or something. Assume, for
example, that you didn't like XPL, and wanted to change it slightly, into
a new language called XPL2, which is very similar. Starting with XY IV, it
would be possible (I think) to write such an interpreter for XPL2 in XPL,
and then execute an XPL2 program using the XPL2 interpreter that was
written in XPL. The interpreter would, of course, have to process an XPL2
program by examining it byte by byte, decoding it, doing the requested
function for whatever was decoded, and so on.

However, if you did so, my guess is that XPL2 would be interpreted (by the
interpreter, written in XPL) at least 5000 times slower than an XPL
program would be, and you would more than just notice the difference. But
I could write an XPL2 interpreter in EEL, and I would expect it to run no
slower than about 10 times slower than XyWrite's XPL interpreter.

Having a "*very* long" XPL program, as you do, really isn't a measure of
much -- it's a much more a question of how much looping is done. For
example, I have one XPL program which examines every character in a file,
by defining one character, placing that char in a S/G, canceling the
define, doing a quick test on the char in the S/G, and then defining the
next character, and so on, to the end of the file. On an 800MHz Pentium
III, that XPL program takes 50 seconds to scan a *single* file that is
about 180KB in size. On the same system, XyWrite III can search 430
different files which total 5.36MB in size, in about 1 second (on the
second try, where all of the files are already in the Windows cache and no
disk accesses are required - the first try takes longer because of
physical disk accesses). (This also involves *at least* a character by
character examination of the data in the in all of the files, *plus* one
heck of a lot of other processing, including opening and reading the 430
files.) So the XPL program in this case is *at least* 1500 times slower
(probably more like 10000 or 50000 times slower, considering all the other
processing that is done in the multifile search) than XyWrite's search
code, and would hence be totally unacceptable for writing the byte-by-byte
examination routine involved in such a search.

There is just a lot of code that couldn't, and can't, be written in XPL,
because XPL is too slow. EEL does not have the same problem of being
unsuitable for these purposes. To do my XyWrite "simulation" in Epsilon,
for example, I probably *will* be writing an XPL interpreter in EEL, and I
think that it will be plenty fast enough. It wouldn't work to do the
reverse, however -- an EEL interpreter written in XPL wouldn't be viable,
for example.

I think we all have an implicit understanding of what kind of code
shouldn't be written in XPL, due to speed, and as a result, we don't
generally write that kind of XPL code. So we can think of XPL as being
"fast enough." But the fact that we don't, and in fact really can't, write
some kinds of code really is in fact a serious limitation, when one is
considering some really serious extensions, such as emulating a different
editor or word processor, as I am talking about.

>The disadvantages you list are deal-breakers for me . . . unless you
develop a more Xy-like front-end.

Yeah, they are kind of deal breakers for me too, which is why I'm
considering writing such a "front-end." It may be harder than I think, but
after thinking about it some, I've reached the tentative conclusion that
writing such a "front-end" on top of the environment that epsilon provides
might be a surprisingly small amount of work.

It does seem to me that Epsilon really does provide a nice package for
what I'm considering, and I don't know of anything else that provides a
similar package. Lugaru provides a complete 'C' (EEL)
edit/compile/link/debug environment to me, and also to anyone else who
choses to work with and modify what I produce (using their copy of
Epsilon). The code I write runs probably only about 10 times slower than
code compiled to machine language (I wrote such a similar
compiler/interpreter in the mid 70's, and the speed ratio was about 1/10
compared to machine language). So, performance wise, I can apply a coding
solution to most any problem that I encounter (particularly considering
the speed of todays machines, which are a lot more than 10 time faster
than the older machines, which ran XPL code plenty fast enough).
Functionally, Lugaru's done the hard work. They provide built-in "buffers"
functionality, for holding files while editing or other processing, where
allocation, text insertion and deletion, file reading and writing, and
such are handled by 'primitives' known to the compiler, and which are easy
to use. (The functionality of being able to quickly insert or delete bytes
in the *middle* of what is essentially a million byte string (for a 1MB
file), and provide the *appearance* of continued continuity of the
resulting "string" for search and other purposes, but doing so without
having to move 500000 bytes each insert/delete to "make room" (for insert)
or "close ranks" (for delete), is a pretty significant piece of
functionality.) Lugaru provides all kinds of search and replace primitives
for data in the buffers, including regular expressions (making it easy for
me to find XyWrite type  and  markups in a file being edited,
for example). They provide undo/redo. They provide for display of data in
the buffers. (When I want the display to "look different" than the data
that is actually in the buffer, as is usually the case with XY files, I
use another buffer, put something in that buffer that "looks like" what
the display should look like, and I display that buffer instead.) They
provide keyboard and mouse input primitives, with flexible keyboard
binding. They allow whatever I write to run on a good variety of host
types. There just ain't that much left to write, and it sure seems a lot
better than starting from scratch.

But, I think I'm getting my answer. For one reason or another, I'm not
seeing a lot of XY folks that are much interested in going where I'm
going, so I think that I should mostly consider myself to be the only
target user for I do. I really use a pretty small subset of XY markup
functionality (I never have used a footnote, for example), so I suspect
that I will, at least in the beginning, only implement what I use.

Wally Bass