Have hardware prices gone crazy during Covid?

On Tue, 30 Jun 2020 12:02:41 +0100, "J. P. Gilliver (John)"
wrote:

On Tue, 30 Jun 2020 at 09:12:20, Chris wrote:
[]
The only downside to SSDs is the per GB cost.

I think the failure mode is another downside. (Sure, it isn't if you
have a robust and frequent backup strategy.) [And I'm talking
statistically/probabilistically; yes, I know, HDDs _can_ fail in those
ways too (other than going read-only).]

Whether you're talking about failure _mode_ or failure _likelihood_, I
think SSDs win out over HDDs. As mentioned above, HDDs still win on cost
per GB, but that gap is closing, especially for smaller amounts of storage.

On 30/06/2020 16:44, Stan Brown wrote:
On Mon, 29 Jun 2020 00:18:33 +0200, Carlos E.R. wrote:

On 28/06/2020 21.52, tesla sTinker wrote:
bill gates is the corona virus.Â* He made it on purpose.\

Kill this subthread.

Please don't respond to trolls -- it just encourages them to keep
posting.

Hard as it may be, the most effective strategy, over time, is to make
no comment of any kind.

Well, yes and no, it depends on the nature of the trolling. There is a
particular problem with fake news, and while most people may be
unaffected by it and just ignore it, and that's fine for them, I think
there's now so much of it that it becomes almost accepted as 'fact',
'pseudo-fact' if you like, just by its ubiquity and that no-one can be
arsed to contradict it. Hence my debunking of it in this case. I don't
suppose for one moment I will persuade an empty-headed repeater of fake
news like the one at fault here, but at least anyone else who unthinking
comes along and steps in his **** has some facts to wipe their boots on.

On 30/06/2020 15.42, nospam wrote:


you're also ignoring hard drive seek time, which is a significant
overhead for smaller files as well as fragmented larger files.

Absolutely.


seek time on an ssd is effectively zero. that alone is a benefit, even
without the faster i/o.

Indeed it is. This is what makes the change to SSD brilliant in use, not
only boot. This is the issue with my database example.


The point being that in the vast majority of things
that people do, an SSD offers no advantage. You're not
going to is a difference if MS Word is saving your DOC
to disk once every 5 minutes, spending 1 ms to do it
instead of 2 ms.

that is simply false.

replacing a hard drive with an ssd will result in a substantial overall
improvement in performance.

the difference might be a little less noticeable with ms word since the
limiting factor is the user typing, but that's not the only app people
use.

Nor is typing the only thing done with Word. Try to shape a brochure or
magazine article, any complex document.

--
Cheers, Carlos.

J. P. Gilliver (John) wrote:
On Tue, 30 Jun 2020 at 09:12:20, Chris wrote:
[]
The only downside to SSDs is the per GB cost.

I think the failure mode is another downside.

You should always work in the basis that hardware will fail
catastrophically. Don't depend on being able to rescue stuff at first signs
of failure. At those first signs you should be in a position to just junk
it.

Mayayana wrote:
"Chris" wrote

| It's important for *any* file I/O. The massive gain is the lack of latency
| with all disk operations. Given that all applications touch a lot of files
| in their general operation every micro/milli-second gain adds up. It is
| genuinely transformative.
|

Yes, that's true. That's what Carlos was talking about
with his on-disk database work. But most people are
not doing massive file access.

Most desktop applications are 100s of megabytes in size and read a great
many files upon launching.

I open a file in a text editor
or graphic editor. Most of what's happening is in RAM until
I save to disk.

Overly simplistic.

I load a webpage. It' save cache, etc, to
disk. Even on older disks, the speed of loading a file is
in the range of maybe 1 ms per MB. (I've tested it before
and don't remember exactly, but it's extremely fast.)

Browsers read a lot of library file, check caches, cookies and the
downloads folder for new or existing files. They also load plugins and
other addons.

The point being that in the vast majority of things
that people do, an SSD offers no advantage. You're not
going to is a difference if MS Word is saving your DOC
to disk once every 5 minutes, spending 1 ms to do it
instead of 2 ms.

Word has always been awful when saving files. It seems to check every drive
responds before it even considers saving your existing open file in the
place it has always been. I remember Word 6 would make the floppy drive
seek every bloody time you hit save. Despite there being no disk it and you
wanting to save to C:

Anything that makes saving files better with word is *always* noticeable.


| On my desktop I've both an SSD on C: and an HDD on D:. The difference in
| application performance if it's installed on C or D is like night and day.
|

Then you must be using I/O-intensive software, or
there's something wrong with your system. It depends
on what you mean by "performance".

I'm used to my responsive MacBook Pro with its very fast SSD so any kind of
lag is very noticeable.

Mayayana wrote:
"Chris" wrote

| It's important for *any* file I/O. The massive gain is the lack of latency
| with all disk operations. Given that all applications touch a lot of files
| in their general operation every micro/milli-second gain adds up. It is
| genuinely transformative.
|

Yes, that's true. That's what Carlos was talking about
with his on-disk database work .

Poorly performing databases are always due to poor design of the database
or poorly constructed queries. It should not be I/O bound. That's the point
of them, otherwise you're better off with text files.

Chris wrote:
Mayayana wrote:
"Chris" wrote

| It's important for *any* file I/O. The massive gain is the lack of latency
| with all disk operations. Given that all applications touch a lot of files
| in their general operation every micro/milli-second gain adds up. It is
| genuinely transformative.
|

Yes, that's true. That's what Carlos was talking about
with his on-disk database work .

Poorly performing databases are always due to poor design of the database
or poorly constructed queries. It should not be I/O bound. That's the point
of them, otherwise you're better off with text files.


I'll have a bench soon, and we can discuss it.

*******

OK, this benchmark involves a million files on RAMDisks,
using two different file systems.

The directory "0" contains a tree of folders, with
16 files per folder at the bottom. If all million files
were placed in one folder, NTFS would "sink like a rock".
I used the tree approach to give it a fighting chance.

This filesystem is TMPFS which is a RAMDisk-based thing in Linux.
This filesystem has a bit uneven performance, and can slow down
a bit as it "ages". It's not perfect, but it does turn in
some nice benchmarks. Since it completes the benchmark
in ten seconds, that's roughly 100,000 files a second.

time hashdeep -c md5 -j 12 -r /tmp/0 output.txt
real 0m9.936s === 10 seconds
user 0m19.495s
sys 0m25.902s

I also have NTFS on RAMDisk of the same size, in Windows. DataRAM RAMDisk.
The TMPFS can do around 7.5GB/sec, while the Windows RAMdisk
does around 4GB/sec.

F:\timeit hashdeep64 -c MD5 -j 12 -r 0 output.txt

Version Number: Windows NT 6.2 (Build 9200)
Exit Time: 6:15 pm, Tuesday, June 30 2020
Elapsed Time: 0:01:49.568 === 110 seconds
Process Time: 0:04:56.078
System Calls: 40544444
Context Switches: 12817643
Page Faults: 2772716
Bytes Read: 3098178548
Bytes Written: 70984398
Bytes Other: 665675206

NTFS is eleven times slower. It's holding back my
Windows result.

And this is a relatively good result for NTFS. I've
seen worse. It makes some of my experiments... take hours.

Paul

"Carlos E.R." wrote

| The point being that in the vast majority of things
| that people do, an SSD offers no advantage. You're not
| going to is a difference if MS Word is saving your DOC
| to disk once every 5 minutes, spending 1 ms to do it
| instead of 2 ms.
|
| You do if you handle a 20 page document with photos or figures on every
| page. Or with active links to calc sheet portions. I can hear the disk
| working as I go around.
|

But you're not waiting for disk writes, right?
I think SSDs are great, personally. I just think
it's important to distinguish between theoretical
and practical advantages.

On 6/29/2020 4:19 PM, Carlos E.R. wrote:
On 29/06/2020 21.30, Yousuf Khan wrote:
On 6/29/2020 7:08 AM, Carlos E.R. wrote:
Well, the attribute/permission set is different, which affects both
directions. Maybe more issues? Legal?

I'm sure one of the older public-domain filesystems, such as Ext3fs
can be modified to include Microsoft attributes instead? I mean the
source code is completely available for free.

Yes, but then your own code based on it would also have to be available
for free to competitors, and is not in the M$ DNA ;-p

Yeah, but if it's just to add the information like the permissions and
ownership information, then what kind of competitive information is left
in that now? At this point in time, NTFS has been thoroughly
reverse-engineered already, and most external OS's can handle NTFS as a
side-gig these days, so they already know most of the attributes inside
it. Besides that was the old Microsoft, this is the new Linux-friendly
Microsoft. :-)

Maybe they would need a team to extract the full detailed specs, and
another team that never ever had a look at the code, create another code
set from scratch.

It seems like a total waste of time, when the source code is already
available for them for free. All they'd have to do is reveal the
hierarchy for their permission and ownership attributes. I don't see
where there is any competitive advantage left for hiding that
information anymore.

Yousuf Khan

On 6/30/2020 5:27 AM, Chris wrote:
Some years back I was hearing rumours of Microsoft developing its MS-SQL
database software as its new filesystem. I guess that didn't pan out?
It was called WinFS

https://en.m.wikipedia.org/wiki/WinFS

Yeah, that's the one.

Yousuf Khan

On 6/30/2020 12:54 PM, Java Jive wrote:
Well, yes and no, it depends on the nature of the trolling.Â* There is a
particular problem with fake news, and while most people may be
unaffected by it and just ignore it, and that's fine for them, I think
there's now so much of it that it becomes almost accepted as 'fact',
'pseudo-fact' if you like, just by its ubiquity and that no-one can be
arsed to contradict it.Â* Hence my debunking of it in this case.Â* I don't
suppose for one moment I will persuade an empty-headed repeater of fake
news like the one at fault here, but at least anyone else who unthinking
comes along and steps in his **** has some facts to wipe their boots on.

Yep, you got to do a little to combat it, but don't go overboard in
trying to convince anyone, once they start showing signs that they can't
be convinced, then start ignoring them.

Yousuf Khan

On 01/07/2020 11.41, Yousuf Khan wrote:
On 6/29/2020 4:19 PM, Carlos E.R. wrote:
On 29/06/2020 21.30, Yousuf Khan wrote:
On 6/29/2020 7:08 AM, Carlos E.R. wrote:
Well, the attribute/permission set is different, which affects both
directions. Maybe more issues? Legal?

I'm sure one of the older public-domain filesystems, such as Ext3fs
can be modified to include Microsoft attributes instead? I mean the
source code is completely available for free.

Yes, but then your own code based on it would also have to be
available for free to competitors, and is not in the M$ DNA ;-p

Yeah, but if it's just to add the information like the permissions and
ownership information, then what kind of competitive information is left
in that now? At this point in time, NTFS has been thoroughly
reverse-engineered already, and most external OS's can handle NTFS as a
side-gig these days, so they already know most of the attributes inside
it. Besides that was the old Microsoft, this is the new Linux-friendly
Microsoft. :-)

Maybe they would need a team to extract the full detailed specs, and
another team that never ever had a look at the code, create another
code set from scratch.

It seems like a total waste of time, when the source code is already
available for them for free. All they'd have to do is reveal the
hierarchy for their permission and ownership attributes. I don't see
where there is any competitive advantage left for hiding that
information anymore.

It is available for free, if you comply with the requirements of the
license of the code you use. It is not that simple. Possibly, all
licenses of M$ libraries that use the library containing the imported
code would have to be opened as well. It has been called a viral license.

--
Cheers, Carlos.

On 01/07/2020 01.10, Mayayana wrote:
"Carlos E.R." wrote

| The point being that in the vast majority of things
| that people do, an SSD offers no advantage. You're not
| going to is a difference if MS Word is saving your DOC
| to disk once every 5 minutes, spending 1 ms to do it
| instead of 2 ms.
|
| You do if you handle a 20 page document with photos or figures on every
| page. Or with active links to calc sheet portions. I can hear the disk
| working as I go around.
|

But you're not waiting for disk writes, right?

Of course we are.

I think SSDs are great, personally. I just think
it's important to distinguish between theoretical
and practical advantages.

Yes, and I can measure the practical advantages in overall speed of the
machines where I change to SSD.

--
Cheers, Carlos.

On 30/06/2020 23.47, Chris wrote:
Mayayana wrote:
"Chris" wrote

| It's important for *any* file I/O. The massive gain is the lack of latency
| with all disk operations. Given that all applications touch a lot of files
| in their general operation every micro/milli-second gain adds up. It is
| genuinely transformative.
|

Yes, that's true. That's what Carlos was talking about
with his on-disk database work .

Poorly performing databases are always due to poor design of the database
or poorly constructed queries. It should not be I/O bound. That's the point
of them, otherwise you're better off with text files.

Exactly.


--
Cheers, Carlos.

On 01/07/2020 00.38, Paul wrote:

I'll have a bench soon, and we can discuss it.

*******

OK, this benchmark involves a million files on RAMDisks,
using two different file systems.

The directory "0" contains a tree of folders, with
16 files per folder at the bottom. If all million files
were placed in one folder, NTFS would "sink like a rock".
I used the tree approach to give it a fighting chance.

This filesystem is TMPFS which is a RAMDisk-based thing in Linux.
This filesystem has a bit uneven performance, and can slow down
a bit as it "ages". It's not perfect, but it does turn in
some nice benchmarks. Since it completes the benchmark
in ten seconds, that's roughly 100,000 files a second.

time hashdeep -c md5 -j 12 -r /tmp/0 output.txt
realÂ*Â*Â* 0m9.936sÂ* === 10 seconds
userÂ*Â*Â* 0m19.495s
sysÂ*Â*Â* 0m25.902s

I also have NTFS on RAMDisk of the same size, in Windows. DataRAM RAMDisk.
The TMPFS can do around 7.5GB/sec, while the Windows RAMdisk
does around 4GB/sec.

F:\timeit hashdeep64 -c MD5 -j 12 -r 0 output.txt

Version Number:Â*Â* Windows NT 6.2 (Build 9200)
Exit Time:Â*Â*Â*Â*Â*Â*Â* 6:15 pm, Tuesday, June 30 2020
Elapsed Time:Â*Â*Â*Â* 0:01:49.568Â*Â* === 110 seconds
Process Time:Â*Â*Â*Â* 0:04:56.078
System Calls:Â*Â*Â*Â* 40544444
Context Switches: 12817643
Page Faults:Â*Â*Â*Â*Â* 2772716
Bytes Read:Â*Â*Â*Â*Â*Â* 3098178548
Bytes Written:Â*Â*Â* 70984398
Bytes Other:Â*Â*Â*Â*Â* 665675206

NTFS is eleven times slower. It's holding back my
Windows result.

And this is a relatively good result for NTFS. I've
seen worse. It makes some of my experiments... take hours.

That's very interesting.

I noticed, years ago, that accessing in Linux an NTFS disk (mounted via
fuse, thus not kernel) was slow and CPU intensive. At that time my CPU
was not very powerful: people with more powerful CPUs did not see the
problem. Of course, a reverse engineered code running with fuse has to
be slower, but your testing confirms that there are other issues.

It heard that NTFS will now be implemented in the Linux kernel :-? That
would speed things a bit.

--
Cheers, Carlos.