VanguardLH,

Piping uses a buffer to pipe data from stdout to stdin.
Redirection uses a file pointer (not necessarily a file in
the file system) to transfer data.

In this day-and-age ? True. But only when the consumer can run at the
same time as the sourcer.

So I have to wonder if the problem isn't with the program
generating the stdout stream.

Possible. But in that case, how would you explain that just the first few
lines go wrong -- and in different ways -- with the rest going fine ?

Not all programs work with pipes.

I'm not quite certain how that applies to a program which expects a
write-only handle to the console. As far as I'm aware a handle to a file,
a device or a pipe respond the same in that regard.

That is, for "programA | programB", programA might fill up
the buffer and still be in write mode so programB cannot
read the buffer.

:-) That would cause any program using a blocking read or write to
permanently freeze within seconds (which does not happen). As far as I know
a pipe is just a FIFO with a write, and a read end. If it gets full the
writing end cannot place more data into (the write blocks) it until the
reading end removes some data from its end.

As I recall, cmd.exe's pipe is only 512 bytes. Very small.
And probably why piping is slow.

Such a small buffer means that the writing program might need to do a *lot*
of small writes, and what you than see is probably the overhead of it all.

I don't bother using MODE to define the buffer size for the
console

I don't either. What I do use it for is not having a windowed console.
Ofcourse, the fact that 80x25 is what most legacy console apps expect to see
is a factor too. :-)

After all, you are using more.com to paginate, anyway, so scrolling
or not shouldn't be a concern to you.

Huh ? I like my daily shower just after getting outof bed. I would not
like it if that shower is cold, of a natural origine or when I'm clothed.
:-)

In other words: I want to have scrolling when *I'm* ready for it, not
allways. Besides, the scrolling buffer of a windowed console is limited
too, and stuff could scroll outof that buffer without me having a chance to
see it (yes, I sometimes have that much output to look at).

I've created (somewhat of) a solution though: I've taken a 16-bit
environment MORE.COM, removed the version check and use it in the console
window. The only drawback is that I have to wait for the sourcing program
to finish before the 16-bit MORE program gets its first data and thus can
display it. It will do for now.

Regards,
Rudy Wieser


-- Origional message:
VanguardLH schreef in berichtnieuws
...
Piping uses a buffer to pipe data from stdout to stdin. Redirection
uses a file pointer (not necessarily a file in the file system) to
transfer data. With redirection using file pointers, the file has to
get closed to reuse it in another redirection (except as noted when
using &1 to merge data streams by having the output from one handle
write into to the input of another handle). So I have to wonder if the
problem isn't with the program generating the stdout stream. stdin for
more.com looks to be working because you tested with "... file & more
file" and that works (all lines are captured). Not all programs work
with pipes.

Note: Although I've mentioned stdin, stdout, and stderr, cmd.exe
actually permits up to 10 handles. 0 = stdin, 1 = stdout, 2 = stderr,
and 3-9 are defined by the program and are specific to it. To specify
which handle, put a number before the redirection character. No number
defaults to 1 (stdout) for and defaults to 0 (stdin) for . You can
specify a file or handle for the stream source. For example, "1&2"
redirects from handle 2 (stderr) into handle 1 (stdout) while "dir path
file 2&1" sends the dir's stdout and stderr to the same file. See:

http://www.microsoft.com/resources/d...l/proddocs/en-
us/redirection.mspx

Piping uses a buffer while redirection uses file handles. I have read
that using | is that a command can produce enough output to fill up the
pipe's buffer which causes a block on the next program to read it. That
is, for "programA | programB", programA might fill up the buffer and
still be in write mode so programB cannot read the buffer. The size of
the pipe's buffer differs with different operating systems. There are
bunch of rules as to size and it can change within an OS. I read where
Mac OS/X has a pipe buffer capacity of 16,384 bytes but will switch to
65,336 bytes (although since Linux 2.6.11 it defaults to 65.336) if a
large write is made to the pipe, or switch to a single system page if
too much memory is already used by the pipe. fcntl is called by a
program to change the pipe's buffer size. win32api calls are used in
Windows. So a program could adjust the size of the pipe. Alas, I don't
know if a console-mode program can adjust the size of cmd.exe's piping
buffer. As I recall, cmd.exe's pipe is only 512 bytes. Very small.
And probably why piping is slow.

So perhaps you are hitting against the pipe's max buffer size versus
using redirection with file handles since files are rather indefinite in
size (with restrictions within the OS and hardware).

With the mode command, it looks like the args specify the buffer size.
"mode con:cols=80 lines=25" only gives you a 2000 byte buffer. If
instead you used "mode con:cols=132 lines=2500" then you would get a
322kB buffer (and use scrolling in the console window to get at scrolled
off output - but then you are using more.com to paginate that output).

I don't bother using MODE to define the buffer size for the console
(cmd.exe). I might if I needed in in a script (.bat file). Instead I
load the command shell and click on its control menu to select
Properties where I set the buffer size (and window size) under the
Layout tab. Because I don't want a huge window size (I set it at 132 x
50) which would end up with much of it being offscreen, I set a larger
buffer size (132 x 5000) and use the vertical scroll bar to move up and
down through the output. I could set width to 80 and use horizontal
scrollbar to move left and right for output greater than 80 characters
but the screen is usually wide enough that I can set width to 132 (few
DOS-mode programs ever exceed that line length).

So move away from using MODE to define some ancient 80x25 screen size
(whether by using mode.com or setting properties of cmd.exe's window)
and up the buffer size and use scroll bars. After all, you are using
more.com to paginate, anyway, so scrolling or not shouldn't be a concern
to you.