Application test puzzler - FFMPEG with 10074 x64

OK, so I decide to transcode a movie.

The encoder is Cinepak in FFMPEG. It's a single
threaded video encoder, compute bound (doesn't
seem to make large demands on memory, and definitely
not on disk).

First I try to encode the video with a single instance.
It encodes at 0.2FPS or 150x slower than real time. It
would take 450 hours to complete the encoding, if left
to its own devices.

So I use avidemux and chop the movie into enough
pieces, so each core of the processor gets a piece.
That reduces the estimated encoding time by the number
of virtual cores on my processor.

I open cmd.exe windows (one per core), and issue the command
for each movie chunk. (I've already tested this procedure on
1 minute sample chunks, then used the concat filter
to glue the pieces sequentially. The filelist.txt
has the names of the completed encodings, to be
glued together without re-encoding. The video playback
was smooth, without any "glitch" between segments glued
together.)

ffmpeg -f concat -i G:\WORK\filelist.txt -c copy concattest.avi

So when I leave the machine, I have SpeedFan running,
temperatures are normal (maybe 45C max), certainly
not near any thermal limits for CPU.

The benchmark is CPU bound, because as you add program
runs, the speed of the first encoder to run, is unaffected.
As well, if you open Task Manager, the graphs fill to
100% on all cores.

I go take a nap, and come back hours later.

All but two of the FFMPEG runs have stopped.

There is *no* error message.

Each run looks like it did a normal run and
stopped when it ran out of input file to process.

The runs are supposed to complete 26268 frames each.
Some completed around ~6000, some fewer. (The encoding
rate is not constant, the content, the level of action
makes a difference to the encoding time.)

Two runs were still going, but I manually stopped
them with control-C, and as usual, FFMPEG writes
a trailer to the end of each encoding, so the
(partial) encodings are complete. Right now, I don't
see a difference between the two stopped manually,
and the ones that failed to finish encoding their chunk.

In Power Options, the disk drive is set to "Never"
for spindown. I've seen one warning in Windows 10, that
this does not work properly. The person making the
comment, said he could set the disk spindown to a
finite time (like five hours), and the ACPI subsystem
will pay attention to that setting. But if you set
it to Never, it has a mind of its own, and your
disk will spin down.

Other than stare at the stupid test computer for
hours on end, looking for an "event", what would
people recommend for debugging ? I checked Event
Viewer, and there is nothing of note in there.
Windows Update whines a bit, because the power
to the "networking tree" was switched off, and
the Ethernet interface was unavailable for
regular visits to Windows Update. (That's to
prevent a Windows Update from scheduling a
reboot that would destroy my session.)

I've done four hours of Prime95 Torture Test
on the machine, which is my minimum acceptance
test. All cores running, only a little memory
untested (leaving headroom for normal OS nonsense).
So it has passed a torture test. The machine
seems to run a bit cooler when FFMPEG runs are
in progress, so that might not be quite as
intensive as Prime.

What's most puzzling, is the lack of feedback.
Everything appears "normal", except the computing
jobs didn't do all their work. It's possible,
if some system thing sent a command to "stop"
to FFMPEG, it would intercept the call, quietly
write the trailer for the video, and give the
impression of normal termination. But why
would the OS do this on its own ? And two
of the runs were still in progress when
I got there, so not all got the "stick a fork
in it" treatment.

For now, I'm going to boot into Windows 8.1 and
see if I can make any kind of forward progress.
I don't seen any reason at the moment, to repeat
the Windows 10 TP 10074 test case, because
I have no tools to "catch it doing something".
Not even a hint of trouble in Event Viewer.
No Delayed Write failure. Nothing. The disk
I/O rate is minimal, because when encoding
one frame of 1280x720 video every five seconds,
you don't need to read or write very much stuff.
You can see brief random flashes from the disk light
on the front of the case, as each run finished a
frame. The disk isn't under any sort of I/O
pressure - no queue is building. And in Task
Manager, memory usage was minimal.

I've had FFMPEG runs, where the "queue" that FFMPEG
uses on high I/O rate stuff, grows in memory.
On Windows 10, a run of that nature, stops
when it has used around 10GB of memory, and
garbage collection seems to "clean" whatever is
going on there. This test was not even remotely
like one of those runs. FFMPEG chose not to queue
a damn thing. Memory usage for each of these
FFMPEG runs might be in the 100-150MB region,
with a bit of "slosh" as frame buffers are
malloced and released. So the memory usage isn't
a constant, but you might only see the memory
usage vary by about 5MB up and down. Not a
high amplitude of memory usage variance.

It's about as tame a run as you can imagine,
except it just "stopped" for no reason.

And during the run, all cores show exactly 100% usage
in Task Manager. Not like a 7ZIP run, where 7ZIP
isn't clever enough to prefetch data, and the
CPU usage goes up and down, like the "motion of the
ocean". These runs really had the CPU pegged. But
with no signs of discomfort. I could start a copy
of Firefox, and surf the web, while all the runs
were in flight. No abnormal behavior there, or
signs of stress. I didn't try to oversubscribe the
machine or anything. I just filled it up, chopping
the movie into pieces, and making the conversion
process "manually multi-threaded" :-)

ffmpeg-20141016-git-0e406ab-win64-static.7z
13,075,558 bytes
SHA1 = 0ab00d6a099eabc72e3e9b1c2cc9beb4d55e25c4 (for the whole file)

"This is a FFmpeg Win64 static build by Kyle Schwarz.
Zeranoe's FFmpeg Builds Home Page: http://ffmpeg.zeranoe.com/builds/
This build was compiled on: Oct 16 2014, at: 01:40:02"

Run on Win10 TP 10074

ffmpeg -i G:\WORK\00.avi -vcodec cinepak -acodec pcm_s16le G:\WORK\a00.avi

Paul

Paul wrote:
OK, so I decide to transcode a movie.

The encoder is Cinepak in FFMPEG. It's a single
threaded video encoder, compute bound (doesn't
seem to make large demands on memory, and definitely
not on disk).

First I try to encode the video with a single instance.
It encodes at 0.2FPS or 150x slower than real time. It
would take 450 hours to complete the encoding, if left
to its own devices.

So I use avidemux and chop the movie into enough
pieces, so each core of the processor gets a piece.
That reduces the estimated encoding time by the number
of virtual cores on my processor.

I open cmd.exe windows (one per core), and issue the command
for each movie chunk. (I've already tested this procedure on
1 minute sample chunks, then used the concat filter
to glue the pieces sequentially. The filelist.txt
has the names of the completed encodings, to be
glued together without re-encoding. The video playback
was smooth, without any "glitch" between segments glued
together.)

ffmpeg -f concat -i G:\WORK\filelist.txt -c copy concattest.avi

So when I leave the machine, I have SpeedFan running,
temperatures are normal (maybe 45C max), certainly
not near any thermal limits for CPU.

The benchmark is CPU bound, because as you add program
runs, the speed of the first encoder to run, is unaffected.
As well, if you open Task Manager, the graphs fill to
100% on all cores.

I go take a nap, and come back hours later.

All but two of the FFMPEG runs have stopped.

There is *no* error message.

Each run looks like it did a normal run and
stopped when it ran out of input file to process.

The runs are supposed to complete 26268 frames each.
Some completed around ~6000, some fewer. (The encoding
rate is not constant, the content, the level of action
makes a difference to the encoding time.)

Two runs were still going, but I manually stopped
them with control-C, and as usual, FFMPEG writes
a trailer to the end of each encoding, so the
(partial) encodings are complete. Right now, I don't
see a difference between the two stopped manually,
and the ones that failed to finish encoding their chunk.

In Power Options, the disk drive is set to "Never"
for spindown. I've seen one warning in Windows 10, that
this does not work properly. The person making the
comment, said he could set the disk spindown to a
finite time (like five hours), and the ACPI subsystem
will pay attention to that setting. But if you set
it to Never, it has a mind of its own, and your
disk will spin down.

Other than stare at the stupid test computer for
hours on end, looking for an "event", what would
people recommend for debugging ? I checked Event
Viewer, and there is nothing of note in there.
Windows Update whines a bit, because the power
to the "networking tree" was switched off, and
the Ethernet interface was unavailable for
regular visits to Windows Update. (That's to
prevent a Windows Update from scheduling a
reboot that would destroy my session.)

I've done four hours of Prime95 Torture Test
on the machine, which is my minimum acceptance
test. All cores running, only a little memory
untested (leaving headroom for normal OS nonsense).
So it has passed a torture test. The machine
seems to run a bit cooler when FFMPEG runs are
in progress, so that might not be quite as
intensive as Prime.

What's most puzzling, is the lack of feedback.
Everything appears "normal", except the computing
jobs didn't do all their work. It's possible,
if some system thing sent a command to "stop"
to FFMPEG, it would intercept the call, quietly
write the trailer for the video, and give the
impression of normal termination. But why
would the OS do this on its own ? And two
of the runs were still in progress when
I got there, so not all got the "stick a fork
in it" treatment.

For now, I'm going to boot into Windows 8.1 and
see if I can make any kind of forward progress.
I don't seen any reason at the moment, to repeat
the Windows 10 TP 10074 test case, because
I have no tools to "catch it doing something".
Not even a hint of trouble in Event Viewer.
No Delayed Write failure. Nothing. The disk
I/O rate is minimal, because when encoding
one frame of 1280x720 video every five seconds,
you don't need to read or write very much stuff.
You can see brief random flashes from the disk light
on the front of the case, as each run finished a
frame. The disk isn't under any sort of I/O
pressure - no queue is building. And in Task
Manager, memory usage was minimal.

I've had FFMPEG runs, where the "queue" that FFMPEG
uses on high I/O rate stuff, grows in memory.
On Windows 10, a run of that nature, stops
when it has used around 10GB of memory, and
garbage collection seems to "clean" whatever is
going on there. This test was not even remotely
like one of those runs. FFMPEG chose not to queue
a damn thing. Memory usage for each of these
FFMPEG runs might be in the 100-150MB region,
with a bit of "slosh" as frame buffers are
malloced and released. So the memory usage isn't
a constant, but you might only see the memory
usage vary by about 5MB up and down. Not a
high amplitude of memory usage variance.

It's about as tame a run as you can imagine,
except it just "stopped" for no reason.

And during the run, all cores show exactly 100% usage
in Task Manager. Not like a 7ZIP run, where 7ZIP
isn't clever enough to prefetch data, and the
CPU usage goes up and down, like the "motion of the
ocean". These runs really had the CPU pegged. But
with no signs of discomfort. I could start a copy
of Firefox, and surf the web, while all the runs
were in flight. No abnormal behavior there, or
signs of stress. I didn't try to oversubscribe the
machine or anything. I just filled it up, chopping
the movie into pieces, and making the conversion
process "manually multi-threaded" :-)

ffmpeg-20141016-git-0e406ab-win64-static.7z
13,075,558 bytes
SHA1 = 0ab00d6a099eabc72e3e9b1c2cc9beb4d55e25c4 (for the whole file)

"This is a FFmpeg Win64 static build by Kyle Schwarz.
Zeranoe's FFmpeg Builds Home Page: http://ffmpeg.zeranoe.com/builds/
This build was compiled on: Oct 16 2014, at: 01:40:02"

Run on Win10 TP 10074

ffmpeg -i G:\WORK\00.avi -vcodec cinepak -acodec pcm_s16le
G:\WORK\a00.avi

Paul

How ready is Ffmpeg for Windows 10?
I'd go to the horse's mouth before spending hours trying to solve this.
There's a forum here;
https://ffmpeg.zeranoe.com/forum/
and they have a group for developers using Windows software.

Ed

Ed Cryer wrote:


How ready is Ffmpeg for Windows 10?
I'd go to the horse's mouth before spending hours trying to solve this.
There's a forum here;
https://ffmpeg.zeranoe.com/forum/
and they have a group for developers using Windows software.

Ed


I just tested in Windows 8.1, and the same thing
happened. The *exact* same thing. Which is pointing
to an FFMPEG problem and not Windows.

And the goofy part, is again, two instances are
still running. That means, there is a problem
with the content in the other segments. And the
remaining two segments don't have the same problem
(even though they were all created with the same
tool, one after another). So there's a pattern
sensitivity of some sort.

One difference between Windows 10 and Windows 8.1,
is in Windows 10, the FFMPEG status line remains
at the bottom of the command window, so you cannot
see each line of output later. Windows 8.1 Command
Prompt window on the other hand, doesn't support the
gotoxy curses method properly, so the lines of output
are all recorded. This leaves more table-scraps for
Paul to analyze.

http://i60.tinypic.com/63zziq.png

What you can see here (that I couldn't see on
Windows 10), is the last three lines of progress,
the frame number does not increment. That means
the tool has spent 15 seconds working on that frame,
and still hasn't managed to finish the lossless
compression. The letter "L" appears out of nowhere.

My guess is, regular status report output goes to
"stdout". Error output goes to "stderr", and in this
case, the program errors out and terminates, before
stderr is flushed. And the rest of the message,
whatever it was, is lost. For my next run, I'm
going to try redirecting the two output streams
to text files, and maybe then I can get the entire
message.

So this is most likely to be a bug in that
particular encoder. And not a Windows problem.
If it was a Windows level problem, I would have
expected different instances of the runs to
terminate. Instead, the same two are still running.
And Windows wouldn't be clever enough to do that.

The reason I got off on the Windows kick, is there
were no error messages. I couldn't see any evidence
FFMPEG was upset about something.

Paul

Ed Cryer wrote:


How ready is Ffmpeg for Windows 10?
I'd go to the horse's mouth before spending hours trying to solve this.
There's a forum here;
https://ffmpeg.zeranoe.com/forum/
and they have a group for developers using Windows software.

Ed

Well, I know exactly what's wrong now.

I ran ffprobe on the input file, a ~16GB AVI file.
FFProbe also stops in an inappropriate manner.
But when you examine the tail of the output text file
from FFProbe, the file position in bytes, is almost
exactly at the 4GB mark. This basically grinds up
your whole movie, and stores it in a text file,
as if dissected with a hex editor. Very nice, when
you need some "info".

C:\FFMPEG\bin\ffprobe -show_packets -show_data F:\test.avi F:\test.txt

I opened the 18GB text file in VIM on Linux. That
took just a few tries and a LiveCD change to get
it right (needed a true x64 DVD - my Mint DVD barfed
at the 4GB mark of that file).

vim -n -R test.txt

It means my AVI 2.0 OpenDML file is not compliant.
And when FFMPEG hits that point in the file (where
a regular AVI format file would die), so too does
FFMPEG die at that exact instant. And it dies quietly.

So now I have to backtrack, and do it a much harder
way. A way that will require multiple experiments
until I get the incantation right. Three-cheers
for a command line universe with hundreds of options o.O

It's beginning to look like leaving the thing running,
single threaded, for 450 hours, will get me a result
faster :-)

Paul

Paul wrote:
Ed Cryer wrote:


How ready is Ffmpeg for Windows 10?
I'd go to the horse's mouth before spending hours trying to solve
this. There's a forum here;
https://ffmpeg.zeranoe.com/forum/
and they have a group for developers using Windows software.

Ed

Well, I know exactly what's wrong now.

I ran ffprobe on the input file, a ~16GB AVI file.
FFProbe also stops in an inappropriate manner.
But when you examine the tail of the output text file
from FFProbe, the file position in bytes, is almost
exactly at the 4GB mark. This basically grinds up
your whole movie, and stores it in a text file,
as if dissected with a hex editor. Very nice, when
you need some "info".

C:\FFMPEG\bin\ffprobe -show_packets -show_data F:\test.avi
F:\test.txt

I opened the 18GB text file in VIM on Linux. That
took just a few tries and a LiveCD change to get
it right (needed a true x64 DVD - my Mint DVD barfed
at the 4GB mark of that file).

vim -n -R test.txt

It means my AVI 2.0 OpenDML file is not compliant.
And when FFMPEG hits that point in the file (where
a regular AVI format file would die), so too does
FFMPEG die at that exact instant. And it dies quietly.

So now I have to backtrack, and do it a much harder
way. A way that will require multiple experiments
until I get the incantation right. Three-cheers
for a command line universe with hundreds of options o.O

It's beginning to look like leaving the thing running,
single threaded, for 450 hours, will get me a result
faster :-)

Paul

I transcode lots of movies; and I've never used FFMPEG.
I use things like Handbrake, ConvertXtoDVD, DVDShrink.

I hate to trust my ego in your problem because you've proved over years
and years to be a pioneer out further than the rest of us. But I'd
welcome a simple explanation of why you're acting as systems tester for
that package. Are they paying you?

Ed

Ed Cryer wrote:
Paul wrote:
Ed Cryer wrote:


How ready is Ffmpeg for Windows 10?
I'd go to the horse's mouth before spending hours trying to solve
this. There's a forum here;
https://ffmpeg.zeranoe.com/forum/
and they have a group for developers using Windows software.

Ed

Well, I know exactly what's wrong now.

I ran ffprobe on the input file, a ~16GB AVI file.
FFProbe also stops in an inappropriate manner.
But when you examine the tail of the output text file
from FFProbe, the file position in bytes, is almost
exactly at the 4GB mark. This basically grinds up
your whole movie, and stores it in a text file,
as if dissected with a hex editor. Very nice, when
you need some "info".

C:\FFMPEG\bin\ffprobe -show_packets -show_data F:\test.avi
F:\test.txt

I opened the 18GB text file in VIM on Linux. That
took just a few tries and a LiveCD change to get
it right (needed a true x64 DVD - my Mint DVD barfed
at the 4GB mark of that file).

vim -n -R test.txt

It means my AVI 2.0 OpenDML file is not compliant.
And when FFMPEG hits that point in the file (where
a regular AVI format file would die), so too does
FFMPEG die at that exact instant. And it dies quietly.

So now I have to backtrack, and do it a much harder
way. A way that will require multiple experiments
until I get the incantation right. Three-cheers
for a command line universe with hundreds of options o.O

It's beginning to look like leaving the thing running,
single threaded, for 450 hours, will get me a result
faster :-)

Paul

I transcode lots of movies; and I've never used FFMPEG.
I use things like Handbrake, ConvertXtoDVD, DVDShrink.

I hate to trust my ego in your problem because you've proved over years
and years to be a pioneer out further than the rest of us. But I'd
welcome a simple explanation of why you're acting as systems tester for
that package. Are they paying you?

Ed


I'm trying to create a movie that will work in the one
player here, that has nice controls on it. I need a movie
that is "seekable" - meaning when you move the time point
in the player, the movie can be immediately played. The
sound does not disappear. The sound does not "stutter".
Both have happened to me so far. I'm trying to fix this!

Of course the procedure isn't practical, but every once
in a while, I dabble in stupid stuff.

The format I've been working on, happens to meet both
criterion. Low overhead, and seekable. I tried mencoder,
and it created the same kind of file, but the file could
not be seeked (move the cursor on the timeline, and
the video would no longer even try to play). That's
why at the moment, I'm using FFMPEG.

Paul

Paul wrote:
Ed Cryer wrote:
Paul wrote:
Ed Cryer wrote:


How ready is Ffmpeg for Windows 10?
I'd go to the horse's mouth before spending hours trying to solve
this. There's a forum here;
https://ffmpeg.zeranoe.com/forum/
and they have a group for developers using Windows software.

Ed

Well, I know exactly what's wrong now.

I ran ffprobe on the input file, a ~16GB AVI file.
FFProbe also stops in an inappropriate manner.
But when you examine the tail of the output text file
from FFProbe, the file position in bytes, is almost
exactly at the 4GB mark. This basically grinds up
your whole movie, and stores it in a text file,
as if dissected with a hex editor. Very nice, when
you need some "info".

C:\FFMPEG\bin\ffprobe -show_packets -show_data F:\test.avi
F:\test.txt

I opened the 18GB text file in VIM on Linux. That
took just a few tries and a LiveCD change to get
it right (needed a true x64 DVD - my Mint DVD barfed
at the 4GB mark of that file).

vim -n -R test.txt

It means my AVI 2.0 OpenDML file is not compliant.
And when FFMPEG hits that point in the file (where
a regular AVI format file would die), so too does
FFMPEG die at that exact instant. And it dies quietly.

So now I have to backtrack, and do it a much harder
way. A way that will require multiple experiments
until I get the incantation right. Three-cheers
for a command line universe with hundreds of options o.O

It's beginning to look like leaving the thing running,
single threaded, for 450 hours, will get me a result
faster :-)

Paul

I transcode lots of movies; and I've never used FFMPEG.
I use things like Handbrake, ConvertXtoDVD, DVDShrink.

I hate to trust my ego in your problem because you've proved over
years and years to be a pioneer out further than the rest of us. But
I'd welcome a simple explanation of why you're acting as systems
tester for that package. Are they paying you?

Ed


I'm trying to create a movie that will work in the one
player here, that has nice controls on it. I need a movie
that is "seekable" - meaning when you move the time point
in the player, the movie can be immediately played. The
sound does not disappear. The sound does not "stutter".
Both have happened to me so far. I'm trying to fix this!

Of course the procedure isn't practical, but every once
in a while, I dabble in stupid stuff.

The format I've been working on, happens to meet both
criterion. Low overhead, and seekable. I tried mencoder,
and it created the same kind of file, but the file could
not be seeked (move the cursor on the timeline, and
the video would no longer even try to play). That's
why at the moment, I'm using FFMPEG.

Paul

I've just tested out two DVDs on my Acer desktop. One a store-bought
movie; one burned locally after transcoding from Xvid. I tried them with
both WMP and VLC.
All four tests gave seekable access; and all four tests passed with
flying colours. No matter how high I sought, the stutter never lasted
for more than a couple of nanoseconds, and I watched for half a minute
thereafter to see if lips stayed in sync with audio.

This machine is a store-bought Acer Aspire X3990, with a Windows
Experience value of 5.1 (it's the graphics that put it there; all other
factors have a high 7.+ rating).

Isn't this problem you're having with "seekability" something to do with
hardware? DVD speed, CPU speed, graphics card?

Ed

Ed Cryer wrote:
Paul wrote:
Ed Cryer wrote:
Paul wrote:
Ed Cryer wrote:


How ready is Ffmpeg for Windows 10?
I'd go to the horse's mouth before spending hours trying to solve
this. There's a forum here;
https://ffmpeg.zeranoe.com/forum/
and they have a group for developers using Windows software.

Ed

Well, I know exactly what's wrong now.

I ran ffprobe on the input file, a ~16GB AVI file.
FFProbe also stops in an inappropriate manner.
But when you examine the tail of the output text file
from FFProbe, the file position in bytes, is almost
exactly at the 4GB mark. This basically grinds up
your whole movie, and stores it in a text file,
as if dissected with a hex editor. Very nice, when
you need some "info".

C:\FFMPEG\bin\ffprobe -show_packets -show_data F:\test.avi
F:\test.txt

I opened the 18GB text file in VIM on Linux. That
took just a few tries and a LiveCD change to get
it right (needed a true x64 DVD - my Mint DVD barfed
at the 4GB mark of that file).

vim -n -R test.txt

It means my AVI 2.0 OpenDML file is not compliant.
And when FFMPEG hits that point in the file (where
a regular AVI format file would die), so too does
FFMPEG die at that exact instant. And it dies quietly.

So now I have to backtrack, and do it a much harder
way. A way that will require multiple experiments
until I get the incantation right. Three-cheers
for a command line universe with hundreds of options o.O

It's beginning to look like leaving the thing running,
single threaded, for 450 hours, will get me a result
faster :-)

Paul

I transcode lots of movies; and I've never used FFMPEG.
I use things like Handbrake, ConvertXtoDVD, DVDShrink.

I hate to trust my ego in your problem because you've proved over
years and years to be a pioneer out further than the rest of us. But
I'd welcome a simple explanation of why you're acting as systems
tester for that package. Are they paying you?

Ed


I'm trying to create a movie that will work in the one
player here, that has nice controls on it. I need a movie
that is "seekable" - meaning when you move the time point
in the player, the movie can be immediately played. The
sound does not disappear. The sound does not "stutter".
Both have happened to me so far. I'm trying to fix this!

Of course the procedure isn't practical, but every once
in a while, I dabble in stupid stuff.

The format I've been working on, happens to meet both
criterion. Low overhead, and seekable. I tried mencoder,
and it created the same kind of file, but the file could
not be seeked (move the cursor on the timeline, and
the video would no longer even try to play). That's
why at the moment, I'm using FFMPEG.

Paul

I've just tested out two DVDs on my Acer desktop. One a store-bought
movie; one burned locally after transcoding from Xvid. I tried them with
both WMP and VLC.
All four tests gave seekable access; and all four tests passed with
flying colours. No matter how high I sought, the stutter never lasted
for more than a couple of nanoseconds, and I watched for half a minute
thereafter to see if lips stayed in sync with audio.

This machine is a store-bought Acer Aspire X3990, with a Windows
Experience value of 5.1 (it's the graphics that put it there; all other
factors have a high 7.+ rating).

Isn't this problem you're having with "seekability" something to do with
hardware? DVD speed, CPU speed, graphics card?

Ed


Normally, you expect a movie to be seekable to the nearest keyframe.
However, when you do that, and are using pcm_s16le (an uncompressed audio
format), what excuse can one make, for audio disappearing (until you
reload the movie) ? On a second attempt at it, the sound was there,
it played every time a new cursor position was arranged, but there
was a sound effect suggesting the CPU was out of horsepower. And yet
Task Manager was reading maybe 35% on one core.

So I'm playing with formats, trying to find something that
actually works.

I have various movie samples that play just fine. Just
not this one.

I always start out with the best of intentions, but these
kinds of projects have a life of their own :-) It's
what happens when you "won't take No for an answer",
over and over again :-)

Paul

Paul wrote:
Ed Cryer wrote:
Paul wrote:
Ed Cryer wrote:
Paul wrote:
Ed Cryer wrote:


How ready is Ffmpeg for Windows 10?
I'd go to the horse's mouth before spending hours trying to solve
this. There's a forum here;
https://ffmpeg.zeranoe.com/forum/
and they have a group for developers using Windows software.

Ed

Well, I know exactly what's wrong now.

I ran ffprobe on the input file, a ~16GB AVI file.
FFProbe also stops in an inappropriate manner.
But when you examine the tail of the output text file
from FFProbe, the file position in bytes, is almost
exactly at the 4GB mark. This basically grinds up
your whole movie, and stores it in a text file,
as if dissected with a hex editor. Very nice, when
you need some "info".

C:\FFMPEG\bin\ffprobe -show_packets -show_data F:\test.avi
F:\test.txt

I opened the 18GB text file in VIM on Linux. That
took just a few tries and a LiveCD change to get
it right (needed a true x64 DVD - my Mint DVD barfed
at the 4GB mark of that file).

vim -n -R test.txt

It means my AVI 2.0 OpenDML file is not compliant.
And when FFMPEG hits that point in the file (where
a regular AVI format file would die), so too does
FFMPEG die at that exact instant. And it dies quietly.

So now I have to backtrack, and do it a much harder
way. A way that will require multiple experiments
until I get the incantation right. Three-cheers
for a command line universe with hundreds of options o.O

It's beginning to look like leaving the thing running,
single threaded, for 450 hours, will get me a result
faster :-)

Paul

I transcode lots of movies; and I've never used FFMPEG.
I use things like Handbrake, ConvertXtoDVD, DVDShrink.

I hate to trust my ego in your problem because you've proved over
years and years to be a pioneer out further than the rest of us. But
I'd welcome a simple explanation of why you're acting as systems
tester for that package. Are they paying you?

Ed


I'm trying to create a movie that will work in the one
player here, that has nice controls on it. I need a movie
that is "seekable" - meaning when you move the time point
in the player, the movie can be immediately played. The
sound does not disappear. The sound does not "stutter".
Both have happened to me so far. I'm trying to fix this!

Of course the procedure isn't practical, but every once
in a while, I dabble in stupid stuff.

The format I've been working on, happens to meet both
criterion. Low overhead, and seekable. I tried mencoder,
and it created the same kind of file, but the file could
not be seeked (move the cursor on the timeline, and
the video would no longer even try to play). That's
why at the moment, I'm using FFMPEG.

Paul

I've just tested out two DVDs on my Acer desktop. One a store-bought
movie; one burned locally after transcoding from Xvid. I tried them
with both WMP and VLC.
All four tests gave seekable access; and all four tests passed with
flying colours. No matter how high I sought, the stutter never lasted
for more than a couple of nanoseconds, and I watched for half a minute
thereafter to see if lips stayed in sync with audio.

This machine is a store-bought Acer Aspire X3990, with a Windows
Experience value of 5.1 (it's the graphics that put it there; all
other factors have a high 7.+ rating).

Isn't this problem you're having with "seekability" something to do
with hardware? DVD speed, CPU speed, graphics card?

Ed


Normally, you expect a movie to be seekable to the nearest keyframe.
However, when you do that, and are using pcm_s16le (an uncompressed audio
format), what excuse can one make, for audio disappearing (until you
reload the movie) ? On a second attempt at it, the sound was there,
it played every time a new cursor position was arranged, but there
was a sound effect suggesting the CPU was out of horsepower. And yet
Task Manager was reading maybe 35% on one core.

So I'm playing with formats, trying to find something that
actually works.

I have various movie samples that play just fine. Just
not this one.

I always start out with the best of intentions, but these
kinds of projects have a life of their own :-) It's
what happens when you "won't take No for an answer",
over and over again :-)

Paul

I think I understand; and I approve very strongly. It's the scientist's
driving instinct, and it takes you into a learning curve that sometimes
produces that proverbial diamond in the mine.

I'm slowing down in my mid 60s. There was a time when I would have
quickly got a 400watt PSU and 2GB graphic card for this box of tricks,
but now, ......

I've never been a gamer. I did look at Grand Theft Auto V for a few
minutes, and got bored. These on-board Intel graphics and 220watt PSU
are good enough for Freecell. (And that's not a joke!)

Ed

Ed Cryer wrote:
Paul wrote:
Ed Cryer wrote:
Paul wrote:
Ed Cryer wrote:
Paul wrote:
Ed Cryer wrote:


How ready is Ffmpeg for Windows 10?
I'd go to the horse's mouth before spending hours trying to solve
this. There's a forum here;
https://ffmpeg.zeranoe.com/forum/
and they have a group for developers using Windows software.

Ed

Well, I know exactly what's wrong now.

I ran ffprobe on the input file, a ~16GB AVI file.
FFProbe also stops in an inappropriate manner.
But when you examine the tail of the output text file
from FFProbe, the file position in bytes, is almost
exactly at the 4GB mark. This basically grinds up
your whole movie, and stores it in a text file,
as if dissected with a hex editor. Very nice, when
you need some "info".

C:\FFMPEG\bin\ffprobe -show_packets -show_data F:\test.avi
F:\test.txt

I opened the 18GB text file in VIM on Linux. That
took just a few tries and a LiveCD change to get
it right (needed a true x64 DVD - my Mint DVD barfed
at the 4GB mark of that file).

vim -n -R test.txt

It means my AVI 2.0 OpenDML file is not compliant.
And when FFMPEG hits that point in the file (where
a regular AVI format file would die), so too does
FFMPEG die at that exact instant. And it dies quietly.

So now I have to backtrack, and do it a much harder
way. A way that will require multiple experiments
until I get the incantation right. Three-cheers
for a command line universe with hundreds of options o.O

It's beginning to look like leaving the thing running,
single threaded, for 450 hours, will get me a result
faster :-)

Paul

I transcode lots of movies; and I've never used FFMPEG.
I use things like Handbrake, ConvertXtoDVD, DVDShrink.

I hate to trust my ego in your problem because you've proved over
years and years to be a pioneer out further than the rest of us. But
I'd welcome a simple explanation of why you're acting as systems
tester for that package. Are they paying you?

Ed


I'm trying to create a movie that will work in the one
player here, that has nice controls on it. I need a movie
that is "seekable" - meaning when you move the time point
in the player, the movie can be immediately played. The
sound does not disappear. The sound does not "stutter".
Both have happened to me so far. I'm trying to fix this!

Of course the procedure isn't practical, but every once
in a while, I dabble in stupid stuff.

The format I've been working on, happens to meet both
criterion. Low overhead, and seekable. I tried mencoder,
and it created the same kind of file, but the file could
not be seeked (move the cursor on the timeline, and
the video would no longer even try to play). That's
why at the moment, I'm using FFMPEG.

Paul

I've just tested out two DVDs on my Acer desktop. One a store-bought
movie; one burned locally after transcoding from Xvid. I tried them
with both WMP and VLC.
All four tests gave seekable access; and all four tests passed with
flying colours. No matter how high I sought, the stutter never lasted
for more than a couple of nanoseconds, and I watched for half a minute
thereafter to see if lips stayed in sync with audio.

This machine is a store-bought Acer Aspire X3990, with a Windows
Experience value of 5.1 (it's the graphics that put it there; all
other factors have a high 7.+ rating).

Isn't this problem you're having with "seekability" something to do
with hardware? DVD speed, CPU speed, graphics card?

Ed


Normally, you expect a movie to be seekable to the nearest keyframe.
However, when you do that, and are using pcm_s16le (an uncompressed audio
format), what excuse can one make, for audio disappearing (until you
reload the movie) ? On a second attempt at it, the sound was there,
it played every time a new cursor position was arranged, but there
was a sound effect suggesting the CPU was out of horsepower. And yet
Task Manager was reading maybe 35% on one core.

So I'm playing with formats, trying to find something that
actually works.

I have various movie samples that play just fine. Just
not this one.

I always start out with the best of intentions, but these
kinds of projects have a life of their own :-) It's
what happens when you "won't take No for an answer",
over and over again :-)

Paul

I think I understand; and I approve very strongly. It's the scientist's
driving instinct, and it takes you into a learning curve that sometimes
produces that proverbial diamond in the mine.

I'm slowing down in my mid 60s. There was a time when I would have
quickly got a 400watt PSU and 2GB graphic card for this box of tricks,
but now, ......

I've never been a gamer. I did look at Grand Theft Auto V for a few
minutes, and got bored. These on-board Intel graphics and 220watt PSU
are good enough for Freecell. (And that's not a joke!)

Ed

Well, what keeps me going, is the one decent test clip
I got out of the exercise. it's an AVI file, with the video
compressed in CinePak, which is a format you use on gutless
computers. It's also supposed to be a "compatible" format,
in the sense it's been around for a while, so older devices can
play it.

Right now, the attractive part, is I can load it in Avidemux,
and move the video cursor, and it updates the movie frame
on screen *at around 30 FPS*. In other words, while you
do a fast forward by moving the cursor, the screen is
updating in close to real time. While doing that, my
3Ghz E8400 dual core runs at around 35%. If I do this with
the original movie, and load the original movie in WMP, it
updates the frame on the screen at 2 FPS. Using the CinePak
gives a relatively smooth way to preview a movie, by just
sliding it along at high speed. When I stop moving the cursor,
and click play, the sound works properly each time.

But it's an expensive format to make with a price of around
150 hours compute per hour of video. And it is single threaded.
And when I tested a competing compressor that is supposed
to do the same thing, the "seek" feature is broken. Probably
some problem with time stamps inside the file.

*******

I think these little larks of mine, used to drive my boss
nuts at work. But occasionally something useful would come of it.
Maybe I have a short attention span or something :-)

Paul

Paul wrote:
Ed Cryer wrote:
Paul wrote:
Ed Cryer wrote:
Paul wrote:
Ed Cryer wrote:
Paul wrote:
Ed Cryer wrote:


How ready is Ffmpeg for Windows 10?
I'd go to the horse's mouth before spending hours trying to solve
this. There's a forum here;
https://ffmpeg.zeranoe.com/forum/
and they have a group for developers using Windows software.

Ed

Well, I know exactly what's wrong now.

I ran ffprobe on the input file, a ~16GB AVI file.
FFProbe also stops in an inappropriate manner.
But when you examine the tail of the output text file
from FFProbe, the file position in bytes, is almost
exactly at the 4GB mark. This basically grinds up
your whole movie, and stores it in a text file,
as if dissected with a hex editor. Very nice, when
you need some "info".

C:\FFMPEG\bin\ffprobe -show_packets -show_data F:\test.avi
F:\test.txt

I opened the 18GB text file in VIM on Linux. That
took just a few tries and a LiveCD change to get
it right (needed a true x64 DVD - my Mint DVD barfed
at the 4GB mark of that file).

vim -n -R test.txt

It means my AVI 2.0 OpenDML file is not compliant.
And when FFMPEG hits that point in the file (where
a regular AVI format file would die), so too does
FFMPEG die at that exact instant. And it dies quietly.

So now I have to backtrack, and do it a much harder
way. A way that will require multiple experiments
until I get the incantation right. Three-cheers
for a command line universe with hundreds of options o.O

It's beginning to look like leaving the thing running,
single threaded, for 450 hours, will get me a result
faster :-)

Paul

I transcode lots of movies; and I've never used FFMPEG.
I use things like Handbrake, ConvertXtoDVD, DVDShrink.

I hate to trust my ego in your problem because you've proved over
years and years to be a pioneer out further than the rest of us. But
I'd welcome a simple explanation of why you're acting as systems
tester for that package. Are they paying you?

Ed


I'm trying to create a movie that will work in the one
player here, that has nice controls on it. I need a movie
that is "seekable" - meaning when you move the time point
in the player, the movie can be immediately played. The
sound does not disappear. The sound does not "stutter".
Both have happened to me so far. I'm trying to fix this!

Of course the procedure isn't practical, but every once
in a while, I dabble in stupid stuff.

The format I've been working on, happens to meet both
criterion. Low overhead, and seekable. I tried mencoder,
and it created the same kind of file, but the file could
not be seeked (move the cursor on the timeline, and
the video would no longer even try to play). That's
why at the moment, I'm using FFMPEG.

Paul

I've just tested out two DVDs on my Acer desktop. One a store-bought
movie; one burned locally after transcoding from Xvid. I tried them
with both WMP and VLC.
All four tests gave seekable access; and all four tests passed with
flying colours. No matter how high I sought, the stutter never lasted
for more than a couple of nanoseconds, and I watched for half a minute
thereafter to see if lips stayed in sync with audio.

This machine is a store-bought Acer Aspire X3990, with a Windows
Experience value of 5.1 (it's the graphics that put it there; all
other factors have a high 7.+ rating).

Isn't this problem you're having with "seekability" something to do
with hardware? DVD speed, CPU speed, graphics card?

Ed


Normally, you expect a movie to be seekable to the nearest keyframe.
However, when you do that, and are using pcm_s16le (an uncompressed
audio
format), what excuse can one make, for audio disappearing (until you
reload the movie) ? On a second attempt at it, the sound was there,
it played every time a new cursor position was arranged, but there
was a sound effect suggesting the CPU was out of horsepower. And yet
Task Manager was reading maybe 35% on one core.

So I'm playing with formats, trying to find something that
actually works.

I have various movie samples that play just fine. Just
not this one.

I always start out with the best of intentions, but these
kinds of projects have a life of their own :-) It's
what happens when you "won't take No for an answer",
over and over again :-)

Paul

I think I understand; and I approve very strongly. It's the
scientist's driving instinct, and it takes you into a learning curve
that sometimes produces that proverbial diamond in the mine.

I'm slowing down in my mid 60s. There was a time when I would have
quickly got a 400watt PSU and 2GB graphic card for this box of tricks,
but now, ......

I've never been a gamer. I did look at Grand Theft Auto V for a few
minutes, and got bored. These on-board Intel graphics and 220watt PSU
are good enough for Freecell. (And that's not a joke!)

Ed

Well, what keeps me going, is the one decent test clip
I got out of the exercise. it's an AVI file, with the video
compressed in CinePak, which is a format you use on gutless
computers. It's also supposed to be a "compatible" format,
in the sense it's been around for a while, so older devices can
play it.

Right now, the attractive part, is I can load it in Avidemux,
and move the video cursor, and it updates the movie frame
on screen *at around 30 FPS*. In other words, while you
do a fast forward by moving the cursor, the screen is
updating in close to real time. While doing that, my
3Ghz E8400 dual core runs at around 35%. If I do this with
the original movie, and load the original movie in WMP, it
updates the frame on the screen at 2 FPS. Using the CinePak
gives a relatively smooth way to preview a movie, by just
sliding it along at high speed. When I stop moving the cursor,
and click play, the sound works properly each time.

But it's an expensive format to make with a price of around
150 hours compute per hour of video. And it is single threaded.
And when I tested a competing compressor that is supposed
to do the same thing, the "seek" feature is broken. Probably
some problem with time stamps inside the file.

*******

I think these little larks of mine, used to drive my boss
nuts at work. But occasionally something useful would come of it.
Maybe I have a short attention span or something :-)

Paul


Albert Einstein was working in the Patents' Office in Bern while delving
into E=Mc². He used to ride into work on a tram and speculate about what
it would be like to ride on a light-beam.
I wonder what his boss thought of him? And his fellow passengers too?
He must have appeared right out of it to the guy in the street. A bit
like professor Emmett Brown alongside Michael J Fox in the Back To The
Future films.

But I'll bet Albert felt really alive with all that. He published his
papers, waited for the scientific world to respond, met silence and the
dumb stare and rejection of total incomprehension.
"But that contradicts Newton".

Ah, ah.

"The origin of all technical achievements is the divine curiosity and
the play instinct of the working and thinking researcher as well as the
constructive fantasy of the technical inventor".
(Einstein's: Der Urquell aller technischen Errungenschaften ist die
göttliche Neugier und der Spieltrieb des bastelnden und grübelnden
Forschers und nicht minder die konstruktive Phantasie des technischen
Erfinders)

Ed

On Wed, 13 May 2015 05:16:07 -0400, Paul wrote:

Ed Cryer wrote:


How ready is Ffmpeg for Windows 10?
I'd go to the horse's mouth before spending hours trying to solve this.
There's a forum here;
https://ffmpeg.zeranoe.com/forum/
and they have a group for developers using Windows software.

Ed

Well, I know exactly what's wrong now.

I ran ffprobe on the input file, a ~16GB AVI file.
[snip]


I was going to gently remind you that AVI, as a video container format, had
pretty well become obsolete by the mid to late 90's, but after reading the
rest of the thread I see that you're having fun with the experiments, so
play on. :-)

Char Jackson wrote:
On Wed, 13 May 2015 05:16:07 -0400, Paul wrote:

Ed Cryer wrote:


How ready is Ffmpeg for Windows 10?
I'd go to the horse's mouth before spending hours trying to solve this.
There's a forum here;
https://ffmpeg.zeranoe.com/forum/
and they have a group for developers using Windows software.

Ed

Well, I know exactly what's wrong now.

I ran ffprobe on the input file, a ~16GB AVI file.
[snip]


I was going to gently remind you that AVI, as a video container format, had
pretty well become obsolete by the mid to late 90's, but after reading the
rest of the thread I see that you're having fun with the experiments, so
play on. :-)



I went up to Seascale last week for a break. (Yes I know it's right next
door to Sellafield, but I didn't break my heart (Ah, ah! Only a fellow
Brit will understand that)) - anyway, I took my Samsung camera with me.
It's an ES67 and I bought it brand new a couple of years ago.
So then, I took some stills and movies.
Back home I download them onto the PC; and, guess what format the videos
are in! Yes, you guessed, AVI.

Ed

On Wed, 13 May 2015 22:35:11 +0100, Ed Cryer wrote:

Char Jackson wrote:
On Wed, 13 May 2015 05:16:07 -0400, Paul wrote:

Ed Cryer wrote:


How ready is Ffmpeg for Windows 10?
I'd go to the horse's mouth before spending hours trying to solve this.
There's a forum here;
https://ffmpeg.zeranoe.com/forum/
and they have a group for developers using Windows software.

Ed

Well, I know exactly what's wrong now.

I ran ffprobe on the input file, a ~16GB AVI file.
[snip]


I was going to gently remind you that AVI, as a video container format, had
pretty well become obsolete by the mid to late 90's, but after reading the
rest of the thread I see that you're having fun with the experiments, so
play on. :-)



I went up to Seascale last week for a break. (Yes I know it's right next
door to Sellafield, but I didn't break my heart (Ah, ah! Only a fellow
Brit will understand that)) - anyway, I took my Samsung camera with me.
It's an ES67 and I bought it brand new a couple of years ago.
So then, I took some stills and movies.
Back home I download them onto the PC; and, guess what format the videos
are in! Yes, you guessed, AVI.

Yeah, there are still some oddballs like that.

And for the record, I didn't get the Brit humour, but you knew that!

Ed Cryer wrote:
Char Jackson wrote:
On Wed, 13 May 2015 05:16:07 -0400, Paul wrote:

Ed Cryer wrote:


How ready is Ffmpeg for Windows 10?
I'd go to the horse's mouth before spending hours trying to solve this.
There's a forum here;
https://ffmpeg.zeranoe.com/forum/
and they have a group for developers using Windows software.

Ed

Well, I know exactly what's wrong now.

I ran ffprobe on the input file, a ~16GB AVI file.
[snip]


I was going to gently remind you that AVI, as a video container
format, had
pretty well become obsolete by the mid to late 90's, but after reading
the
rest of the thread I see that you're having fun with the experiments, so
play on. :-)



I went up to Seascale last week for a break. (Yes I know it's right next
door to Sellafield, but I didn't break my heart (Ah, ah! Only a fellow
Brit will understand that)) - anyway, I took my Samsung camera with me.
It's an ES67 and I bought it brand new a couple of years ago.
So then, I took some stills and movies.
Back home I download them onto the PC; and, guess what format the videos
are in! Yes, you guessed, AVI.

Ed


AVI is fine, as long as you keep your movies... short :-)

That's what I'm learning.

I've got another test run underway. This time, the container
is .mov :-) I don't have a working player, but that's never
stopped me before :-) [If I had to, I could always boot
up the Mac.]

For this run, the movie goes from 6GB .mp4 to 800GB RGB .mov,
and eventually (after the baking session), will be delivered
as piping hot CinePak in AVI. Ready to pop into AviDemux for
a test. Only 30 hours to go... The purpose of the 800GB
intermediate step, is a format without keyframes. Then, if
any snipping or joining needs to be done, there are fewer
issues with time granularity. For example, there are
a few comments about joining videos, that the first two
seconds of sound get blanked out. And that could be related
to the sound compression method. I'm using PCM as an
attempt to avoid that issue.

Paul