>>>>Wally Bass said: ... in XyWrite 3, every time a file is loaded from disk into an
editing buffer, all NUL bytes are stripped out of the data stream and are discarded (they are
stripped out only when a file is loaded, but they are not stripped out, if present, when a file is
saved).
>>Carl Distefano says: I don't think this is universally true, in either Xy3 or Xy4. At least
it's not true with an artificially constructed file, Nulls.txt, which you can download here ...
>>Carl continues: If you open it [Nulls.txt] in Xy3 or Xy4, you'll see 26 chars, A-Z. But a
DIRectory listing will show that the file has 52 bytes, because, in reality, each letter is followed
by an Ascii-0. When I save that file in either Xy3 or Xy4, the NULs are stripped out, and DIR shows
the file now as having only 26 bytes.
So, I wrote a response that said: "Okay, it should be only 26 bytes (or 27, with an EOF added).
But what is it that makes you think that the bytes weren't stripped out as the file was being loaded
into the edit buffer for editing, rather than as it was being saved back to disk? The fact the DIR
shows 52 is is just showing you information about file on disk, not what is in the editing text
buffer."
That question is still valid, but ...
I went off and did some experimenting. I started with Carl's Nulls.txt file, but I had been working
on a technique which would allow me to load a file containing NULs in for editing, but would do so
WITHOUT deleting any NULs in the process of getting the file loaded into an editing buffer. That
resulted in some results that were rather mind-blowing for me, so I thought I would write them down.
I was already aware of a fact that I don't think is widely known, namely that when one does LDPM to
load a "macro" (any file, actually) into a S/G, that read-in process is much different
that the read-in process for normal editing. The read-in process is strictly controlled by the DIR
indicated length of the file, it doesn't look at the data coming in, and it will continue reading
through both NULs and EOF (1Ah, "End-Of-File) until the file length count is exhausted. (One
can tell how many bytes have been read in, just by using XPL's @SIZ(<ISnn>) XPL function.)
So what I was working on was a process where I first read-in the NUL containing file (I used Carl's
Nulls.txt, which is a reasonable test case) normally. Then I "defined" the entire contents
being displayed, and delete the define, so that I was now editing an empty file buffer, but one that
still has the original (Nulls.txt) file name associated with it.
The the next step is to use LDPM, and do a second read-in (of Nulls.txt) into a S/G, where I know
will still have the NULs intact.
So the first try is to just do a <PVnn> to dump the S/G into the now empty edit buffer. But
that doesn't work, the S/G is treated a subroutine, and it deletes the NULs during the transfer.
<GTnn> is another way to do the same thing, and it fails for the same reason, BUT, the
<GT..> command is supposed to pass the data unchanged unless the S/G is marked as a
subroutine. So the obvious problem here is that LDPM marks any S/G that it loads as a
"subroutine" (that is well documented, and is nothing new).
So I added another step, which uses the <SXnn,<ISmm>> statement to make a copy to a
different S/G. I though would get rid of the "subroutine" flag in the copy.
Still doesn't work, with either PV or GT
So I then tried using just plain old <SVnn,...> to load a S/G. To do this, I used a <SV>
that defined a string AABBCCDDEE, and then edited the binary for the macro containing the <SV>
statement with a hex editor to change every other letter in the above string to be a NUL.
So, SUCCESS, finally. When using the <GTnn> command to do the transfer. NULs were retained. So
this told me that my <SXnn,<ISmm>> step apparently copied the subroutine flag when doing
the transfer. That is an interesting observation to be aware of, IMO.
So, now comes the weird part. If the <SXnn,<ISmm>> tactic didn't work in getting rid of
the subroutine flag, what other copy (or other) options are left?
So what I did was
<SV04,{escape}X><XSmm,04,03,01,02><SX01,<IS01>+<ISO2>>, which uses the
XS parse function and a "wildcard" as a way to do the copy -- we use the "any
byte" wildcard to break up the string into "the first byte" and "the rest",
and then put the parts back together.
And that worked, when followed by a <GTnn> operation to transfer the data to the edit buffer.
The NULs don't show on the screen, but they were present after saving the file to disk. And one can
also use a JMP nn command to check what is happening -- a JMP 26 command will get you past the
ending 'Z' char if the NULs have been deleted, but it will only get you half way through the
alphabet if the NULs are really still in the edit buffer (when using Carl's Nulls.txt file).
To me, that was an utterly bazaar reason to have to use the XS command. But it did work. If anyone
has another idea, I'd sure like to hear it.
While I was messing around, I thought I'd try one more experiment. Before doing the experiment, I
changed the 5th byte in Carl's Nulls.txt file from "C" to an EOF (1Ah) byte. I knew that
that the EOF wouldn't truncate the initial read-in by LDPM, but I didn't know what would happen
after that. So, after doing that, repeating the process as described above, and looking at the
resulting file in the edit buffer, it seemed like everything after the 'B' was gone. But upon saving
the file and examining that file with a hex editor, the length of the file was the expected 53 byte
(not 52, because XyWrite always adds an EOF), and the EOF at byte 5 was also still there, as were
letters D thorough Z, and all of the NULs.
So, that is the first time I have ever seen XyWrite write a file that had an EOF anywhere other than
as the last byte of the file. The fact that my added EOF deleted everything after the 1st EOF in the
display suggested one thing, but then the fact that everything following the 1st EOF was still there
when the file was saved was is in some way a major contradiction that said something else. That is
the first time I have ever seen XyWrite save a file that had an EOF anywhere, other than as the last
byte.
In any case, it seems that I finally have found a way to load a file for editing which preserves
existing NULs, and that can be useful for a file that actually contains an existing, intended NUL. I
have some macros that contain a <SVnn,{NUL}> statement (and hence contain a DESIRED, embedded
NUL, which is necessary for easily checking for a NUL in a string), so that is useful for me, and I
hope it is useful for some other XPL'ers also.
Wally Bass