If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below. |
|
|
Thread Tools | Rate Thread | Display Modes |
#1
|
|||
|
|||
SSD and surveillance camera?
SSD and surveillance camera?
On the system I'm migrating from win7 to win10 I have a sidebar gadget called "coconutview" which monitors an IP surveillance camera and displays the image. It creates a 22KByte file every second. If my math is correct, that's about 63GB/year times write amplification. Seems like a lot of unnecessary small writes to the SSD. Multiply that by all the other old programs that had no concept of SSD limitations and it seems worth looking into. I started looking into hard links and junctions in an effort to move the writes to the HDD. The more I read, the more confused I got. I have no use for these files. Does windows have a /dev/null? There's also the issue that these writes go to: C:\Users\mike\AppData\Local\Microsoft\Windows\Temp orary Internet Files\Content.IE5 Moving that directory probably has far reaching ramifications. And how does windows 10 manage links to drives that have been removed or swapped? Seems like it might be a maintenance nightmare. I'm just trying to understand SSD limitations and reduce writes wherever I can. |
Ads |
#2
|
|||
|
|||
SSD and surveillance camera?
mike wrote:
SSD and surveillance camera? On the system I'm migrating from win7 to win10 I have a sidebar gadget called "coconutview" which monitors an IP surveillance camera and displays the image. It creates a 22KByte file every second. If my math is correct, that's about 63GB/year times write amplification. Seems like a lot of unnecessary small writes to the SSD. Multiply that by all the other old programs that had no concept of SSD limitations and it seems worth looking into. I started looking into hard links and junctions in an effort to move the writes to the HDD. The more I read, the more confused I got. I have no use for these files. Does windows have a /dev/null? There's also the issue that these writes go to: C:\Users\mike\AppData\Local\Microsoft\Windows\Temp orary Internet Files\Content.IE5 Moving that directory probably has far reaching ramifications. And how does windows 10 manage links to drives that have been removed or swapped? Seems like it might be a maintenance nightmare. I'm just trying to understand SSD limitations and reduce writes wherever I can. Unnecessary. SSDs are rated for *much* more writes per year than this. |
#3
|
|||
|
|||
SSD and surveillance camera?
In article , mike
wrote: SSD and surveillance camera? On the system I'm migrating from win7 to win10 I have a sidebar gadget called "coconutview" which monitors an IP surveillance camera and displays the image. It creates a 22KByte file every second. If my math is correct, that's about 63GB/year times write amplification. Seems like a lot of unnecessary small writes to the SSD. Multiply that by all the other old programs that had no concept of SSD limitations and it seems worth looking into. 63 gig is nothing these days. start to worry when it's in the petabyte range. https://techreport.com/review/27436/...eriment-two-fr eaking-petabytes/2 Flash failures started piling up after 600TB of writes. Apart from a few undulations, the retirement rate has been fairly consistent since. The 840 Pro is now up to 5591 reallocated sectors, which translates to over 8GB of flash. That may sound like a lot, but it's only 3% of the drive's 256GB total. The SMART data indicates that we're only 61% into the used block reserve. https://techreport.com/r.x/endurance-2pb/vitals-840pro-reallocated.gif https://techreport.com/r.x/endurance-2pb/vitals-hyperxcomp-life.gif |
#4
|
|||
|
|||
SSD and surveillance camera?
One option would be to use a RAM disk. Then just periodically wipe
the files if not needed. I use a RAM disk for camera images and periodically write them to the SSD. There must be a better way for you, though. If you don't want or need the files and your software can't be told to throw the images away, find new software. There are plenty of camera handling packages available for free. Good luck. On Fri, 14 Dec 2018 10:04:59 -0800, mike wrote: SSD and surveillance camera? On the system I'm migrating from win7 to win10 I have a sidebar gadget called "coconutview" which monitors an IP surveillance camera and displays the image. It creates a 22KByte file every second. If my math is correct, that's about 63GB/year times write amplification. Seems like a lot of unnecessary small writes to the SSD. Multiply that by all the other old programs that had no concept of SSD limitations and it seems worth looking into. I started looking into hard links and junctions in an effort to move the writes to the HDD. The more I read, the more confused I got. I have no use for these files. Does windows have a /dev/null? There's also the issue that these writes go to: C:\Users\mike\AppData\Local\Microsoft\Windows\Tem porary Internet Files\Content.IE5 Moving that directory probably has far reaching ramifications. And how does windows 10 manage links to drives that have been removed or swapped? Seems like it might be a maintenance nightmare. I'm just trying to understand SSD limitations and reduce writes wherever I can. |
#5
|
|||
|
|||
SSD and surveillance camera?
mike wrote:
SSD and surveillance camera? On the system I'm migrating from win7 to win10 I have a sidebar gadget called "coconutview" which monitors an IP surveillance camera and displays the image. It creates a 22KByte file every second. If my math is correct, that's about 63GB/year times write amplification. Seems like a lot of unnecessary small writes to the SSD. Multiply that by all the other old programs that had no concept of SSD limitations and it seems worth looking into. I started looking into hard links and junctions in an effort to move the writes to the HDD. The more I read, the more confused I got. I have no use for these files. Does windows have a /dev/null? There's also the issue that these writes go to: C:\Users\mike\AppData\Local\Microsoft\Windows\Temp orary Internet Files\Content.IE5 Moving that directory probably has far reaching ramifications. And how does windows 10 manage links to drives that have been removed or swapped? Seems like it might be a maintenance nightmare. I'm just trying to understand SSD limitations and reduce writes wherever I can. I was working on something the other day (the "fragmentation" project), and noticed file I/O is being buffered in 64KB chunks on the current Win10. If the output was a "video" instead of a "picture", then the 64KB buffered chunk might be better for an SSD to deal with. In previous fragmentation experiments, I could fragment at the cluster level. If I had two writers in a C program, I could do a 4KB write with one, then a 4KB write with the other, and get a really fragmented file. (One 4KB cluster in each fragment.) Some of my fragmentation programs, use 16 writers, to prevent the OS from optimizing away my test case. My last attempt to do that, the fragment size is larger, and this implies buffering at the handle level, and less overall fragmentation as a result. There's a limit to how much buffering they can do regarding this issue, so don't expect this idea to "solve all fragmentation for all time". That's not what it's for. The idea is to increase the write size for selected operations a tiny bit, and reduce SSD write amplification problems perhaps. I didn't spot this from ProcMon, I analyzed after the run was over and noticed the expected fragment size was not evident in nfi.exe output. I was a bit annoyed at the time, because the purpose of the small fragments, is to "break" the file system. And this change makes it 16 times harder to break it. ******* For IP cameras that have an SDK, you could write your own recorder or handler, to make "better output". Don't expect a browser plugin to cure cancer or solve world hunger. Paul |
#6
|
|||
|
|||
SSD and surveillance camera?
On 12/14/2018 3:42 PM, Paul wrote:
mike wrote: SSD and surveillance camera? On the system I'm migrating from win7 to win10 I have a sidebar gadget called "coconutview" which monitors an IP surveillance camera and displays the image. It creates a 22KByte file every second. If my math is correct, that's about 63GB/year times write amplification. Seems like a lot of unnecessary small writes to the SSD. Multiply that by all the other old programs that had no concept of SSD limitations and it seems worth looking into. I started looking into hard links and junctions in an effort to move the writes to the HDD.Â* The more I read, the more confused I got. I have no use for these files.Â* Does windows have a /dev/null? There's also the issue that these writes go to: C:\Users\mike\AppData\Local\Microsoft\Windows\Temp orary Internet Files\Content.IE5 Moving that directory probably has far reaching ramifications. And how does windows 10 manage links to drives that have been removed or swapped?Â* Seems like it might be a maintenance nightmare. I'm just trying to understand SSD limitations and reduce writes wherever I can. I was working on something the other day (the "fragmentation" project), and noticed file I/O is being buffered in 64KB chunks on the current Win10. If the output was a "video" instead of a "picture", then the 64KB buffered chunk might be better for an SSD to deal with. In previous fragmentation experiments, I could fragment at the cluster level. If I had two writers in a C program, I could do a 4KB write with one, then a 4KB write with the other, and get a really fragmented file. (One 4KB cluster in each fragment.) Some of my fragmentation programs, use 16 writers, to prevent the OS from optimizing away my test case. My last attempt to do that, the fragment size is larger, and this implies buffering at the handle level, and less overall fragmentation as a result. There's a limit to how much buffering they can do regarding this issue, so don't expect this idea to "solve all fragmentation for all time". That's not what it's for. The idea is to increase the write size for selected operations a tiny bit, and reduce SSD write amplification problems perhaps. I didn't spot this from ProcMon, I analyzed after the run was over and noticed the expected fragment size was not evident in nfi.exe output. I was a bit annoyed at the time, because the purpose of the small fragments, is to "break" the file system. And this change makes it 16 times harder to break it. ******* For IP cameras that have an SDK, you could write your own recorder or handler, to make "better output". Don't expect a browser plugin to cure cancer or solve world hunger. Â*Â* Paul I'm not trying to solve world hunger...or write more code. The thought was to use the existing application/sidebar gadget and just intervene to send the files somewhere other than the SSD. /dev/null would be appropriate in this case, but I don't have any control over the gadget. This should also come in handy with other hardware and sw that likes to write useless garbage to the SSD. This is particularly troublesome because it uses C:\Users\mike\AppData\Local\Microsoft\Windows\Temp orary Internet Files\Content.IE5 I fear that trying to hard link that elsewhere would be troublesome to other inbuilt applications. And I have zero experience with hard links in windows. The camera is a Panasonic BB-HCM511A. I've tried ipcam viewers without much success. They want to put up a window with decorations that take up a lot of space. It's a PTZ camera, so any viewer that seeks to control that has to add a bunch of control buttons and UI. I don't want any of that clutter. I like the coconut gadget because it doesn't have a lot of extra decoration. It's very light on cpu and network usage. Why it needs to save the files is known only to the developer. The camera SD card has everything I'd ever want to view later. I'd be interested to hear about an IPCAM viewer that can display a compact window without a lot of overhead. I can't use anything that captures images from a live stream. That uses way too much local network bandwidth. |
#7
|
|||
|
|||
SSD and surveillance camera?
On 12/14/2018 12:19 PM, Pat wrote:
One option would be to use a RAM disk. Then just periodically wipe the files if not needed. I use a RAM disk for camera images and periodically write them to the SSD. There must be a better way for you, though. If you don't want or need the files and your software can't be told to throw the images away, find new software. There are plenty of camera handling packages available for free. Good luck. I'd be interested in particular recommendations for free viewers that don't write to the C: drive, don't put up big windows with lots of frames and buttons and..., and don't require capturing a stream. That wastes way too much local network bandwidth. The coconut viewer was designed specifically for this camera and is a minimalist approach regarding resource usage and screen area. Only real concern I have is wear on the SSD. On Fri, 14 Dec 2018 10:04:59 -0800, mike wrote: SSD and surveillance camera? On the system I'm migrating from win7 to win10 I have a sidebar gadget called "coconutview" which monitors an IP surveillance camera and displays the image. It creates a 22KByte file every second. If my math is correct, that's about 63GB/year times write amplification. Seems like a lot of unnecessary small writes to the SSD. Multiply that by all the other old programs that had no concept of SSD limitations and it seems worth looking into. I started looking into hard links and junctions in an effort to move the writes to the HDD. The more I read, the more confused I got. I have no use for these files. Does windows have a /dev/null? There's also the issue that these writes go to: C:\Users\mike\AppData\Local\Microsoft\Windows\Temp orary Internet Files\Content.IE5 Moving that directory probably has far reaching ramifications. And how does windows 10 manage links to drives that have been removed or swapped? Seems like it might be a maintenance nightmare. I'm just trying to understand SSD limitations and reduce writes wherever I can. |
#8
|
|||
|
|||
SSD and surveillance camera?
In article , mike
wrote: I'd be interested to hear about an IPCAM viewer that can display a compact window without a lot of overhead. I can't use anything that captures images from a live stream. That uses way too much local network bandwidth. most output an mpeg stream, which uses very little bandwidth, however, there are cameras that will upload snapshots at a user-defined interval to a user-defined server (or public) if that's what you want. |
#9
|
|||
|
|||
SSD and surveillance camera?
mike wrote:
I like the coconut gadget because it doesn't have a lot of extra decoration. It's very light on cpu and network usage. Why it needs to save the files is known only to the developer. The camera SD card has everything I'd ever want to view later. I have a theory. No developer really wants to write their own HTTP engine. If they called a system function with respect to the web, the temporary file would end up in the location you found it in. That's the temporary location an engine would use. The developer didn't make a conscious decision that the folder in question "was the perfect place for it". But any developer who uses existing services to do stuff, generally tolerates whatever the side effects are. Maybe that makes the program into a five line script, instead of hundreds of line of C code or something. ******* Your job is to reverse engineer the design enough, to see whether the "temp" location is an environment variable or a registry setting. Paul |
#10
|
|||
|
|||
SSD and surveillance camera?
On 12/14/2018 10:38 AM, nospam wrote:
In article , mike wrote: SSD and surveillance camera? On the system I'm migrating from win7 to win10 I have a sidebar gadget called "coconutview" which monitors an IP surveillance camera and displays the image. It creates a 22KByte file every second. If my math is correct, that's about 63GB/year times write amplification. Seems like a lot of unnecessary small writes to the SSD. Multiply that by all the other old programs that had no concept of SSD limitations and it seems worth looking into. 63 gig is nothing these days. Can't argue with that. BUT...if you have an older drive that's near full, how does that 63GB relate to actual TBW? One of the drives I'd like to use is a 256GB Kingston SSDNOW 200V. I don't know the history of this drive, but the SMART data seems OK. It was made near the transition when SSDs went from crap to marginally usable. Virtually all the Kingston data relates to the SSDNOW 300V series. Best info I can get out of Kingston is, "some of our older drives don't support TRIM". I put win10 on it and ran TRIM from the optimization dialog. It claimed it was trimming. But, I also read that TRIM is merely a suggestion to the drive and there's no feedback whether it actually TRIMmed. Is there a test to confirm this? Without TRIM support, doesn't that put a big question mark into write amplification? I read a bunch of the techreport stuff. Now, I've got another concern, unpowered data retention. I have laptops that sit in the drawer unused for a year. It appears that the electrons are leaking out of the memory cells. This seems to suggest that the effect is cumulative. I can't find any mention of it, but if half the electrons required to maintain data integrity have leaked out, how do they get refreshed? It appears that they don't unless you erase/write the block. That leaves a big question mark regarding data integrity of lightly used systems. And it's not clear that it's any better for heavily used systems for blocks infrequently written. Stated another way, is wear leveling always adequate to keep the data error free even when you have lots of unwritten blocks available? The more I learn, the less I like SSD. So far, the only thing that looks better for my usage than a HDD is boot time. Great for battery-powered devices, but not much help for devices that don't get powered down. start to worry when it's in the petabyte range. https://techreport.com/review/27436/...eriment-two-fr eaking-petabytes/2 Flash failures started piling up after 600TB of writes. Apart from a few undulations, the retirement rate has been fairly consistent since. The 840 Pro is now up to 5591 reallocated sectors, which translates to over 8GB of flash. That may sound like a lot, but it's only 3% of the drive's 256GB total. The SMART data indicates that we're only 61% into the used block reserve. https://techreport.com/r.x/endurance-2pb/vitals-840pro-reallocated.gif https://techreport.com/r.x/endurance-2pb/vitals-hyperxcomp-life.gif |
#11
|
|||
|
|||
SSD and surveillance camera?
In article , mike
wrote: If my math is correct, that's about 63GB/year times write amplification. Seems like a lot of unnecessary small writes to the SSD. Multiply that by all the other old programs that had no concept of SSD limitations and it seems worth looking into. 63 gig is nothing these days. Can't argue with that. BUT...if you have an older drive that's near full, how does that 63GB relate to actual TBW? that info should be in smart data as lba written, which is the number of sectors (normally 512 bytes). however, it's not worth worrying about because in typical use (and 63gb/yr is *way* under that), the ssd will last a *really* long time. One of the drives I'd like to use is a 256GB Kingston SSDNOW 200V. I don't know the history of this drive, but the SMART data seems OK. It was made near the transition when SSDs went from crap to marginally usable. then get a new one. 256g ssds are cheap because 512gb and 1tb are more cost effective. a very quick check shows samsung around $50 and other brands for less. Virtually all the Kingston data relates to the SSDNOW 300V series. Best info I can get out of Kingston is, "some of our older drives don't support TRIM". trim is helpful but not required. modern ssds have their own garbage collection. I put win10 on it and ran TRIM from the optimization dialog. It claimed it was trimming. But, I also read that TRIM is merely a suggestion to the drive and there's no feedback whether it actually TRIMmed. Is there a test to confirm this? probably, but i never looked. it's not worth worrying about. Without TRIM support, doesn't that put a big question mark into write amplification? no. I read a bunch of the techreport stuff. Now, I've got another concern, unpowered data retention. I have laptops that sit in the drawer unused for a year. It appears that the electrons are leaking out of the memory cells. This seems to suggest that the effect is cumulative. I can't find any mention of it, but if half the electrons required to maintain data integrity have leaked out, how do they get refreshed? It appears that they don't unless you erase/write the block. That leaves a big question mark regarding data integrity of lightly used systems. And it's not clear that it's any better for heavily used systems for blocks infrequently written. Stated another way, is wear leveling always adequate to keep the data error free even when you have lots of unwritten blocks available? ssds are not ideal for very long term storage, especially as they approach end of life, but from what you describe, that won't be for a very long time. in any event, no matter what type of storage you use, make multiple backups of anything important. if it fails (and eventually they all will), there are other copies. The more I learn, the less I like SSD. nothing is perfect, but the benefits *greatly* outweigh the drawbacks, notably speed and reliability. So far, the only thing that looks better for my usage than a HDD is boot time. Great for battery-powered devices, but not much help for devices that don't get powered down. anything disk-bound will see an improvement, sometimes a very dramatic one, including app launch time, copying files and overall performance, particularly for apps that use a lot of scratch files or read & write large document files. that said, a surveillance camera doesn't really need an ssd. |
#12
|
|||
|
|||
SSD and surveillance camera?
On 12/14/2018 5:45 PM, nospam wrote:
In article , mike wrote: I'd be interested to hear about an IPCAM viewer that can display a compact window without a lot of overhead. I can't use anything that captures images from a live stream. That uses way too much local network bandwidth. most output an mpeg stream, which uses very little bandwidth, however, there are cameras that will upload snapshots at a user-defined interval to a user-defined server (or public) if that's what you want. I really don't want to download (as in store the image somewhere) anything. I want to read the image data and render it to the screen...period. The only reason I care about it being saved is because of disk clutter and SSD life. I run bleachbit about once a week. It typically throws away about half a gigabyte of stuff, mostly in .ie5. As for the mpeg stream, my network can easily handle the data rate, but it obscures my ability to watch internet access going on in the background. |
#13
|
|||
|
|||
SSD and surveillance camera?
In article , mike
wrote: I'd be interested to hear about an IPCAM viewer that can display a compact window without a lot of overhead. I can't use anything that captures images from a live stream. That uses way too much local network bandwidth. most output an mpeg stream, which uses very little bandwidth, however, there are cameras that will upload snapshots at a user-defined interval to a user-defined server (or public) if that's what you want. I really don't want to download (as in store the image somewhere) anything. I want to read the image data and render it to the screen...period. The only reason I care about it being saved is because of disk clutter and SSD life. I run bleachbit about once a week. It typically throws away about half a gigabyte of stuff, mostly in .ie5. As for the mpeg stream, my network can easily handle the data rate, but it obscures my ability to watch internet access going on in the background. it should have no effect on internet access since it's entirely local. |
#14
|
|||
|
|||
SSD and surveillance camera?
On Fri, 14 Dec 2018 17:35:25 -0800, mike wrote:
On 12/14/2018 12:19 PM, Pat wrote: One option would be to use a RAM disk. Then just periodically wipe the files if not needed. I use a RAM disk for camera images and periodically write them to the SSD. There must be a better way for you, though. If you don't want or need the files and your software can't be told to throw the images away, find new software. There are plenty of camera handling packages available for free. Good luck. I'd be interested in particular recommendations for free viewers that don't write to the C: drive, don't put up big windows with lots of frames and buttons and..., and don't require capturing a stream. That wastes way too much local network bandwidth. Do you need video to be displayed or will a periodic "snapshot" do? As nospam said, your camera has the ability to send snapshots when requested to do so. (I also have a few Panasonic cameras). I wrote a program in C# that requests a snapshot from the camera once per second and displays it in a borderless window with no other buttons or controls. In another post you said all you want is "an IPCAM viewer". By coincidence, I named my program IPCam. If you can program in C#, you could write something similar (the tools are free). Mine wouldn't work for you because I get sloppy when writing software for my own use and I embed all sorts of hard coded network address, passwords, and such). Pat |
#15
|
|||
|
|||
SSD and surveillance camera?
mike wrote:
SSD and surveillance camera? On the system I'm migrating from win7 to win10 I have a sidebar gadget called "coconutview" which monitors an IP surveillance camera and displays the image. It creates a 22KByte file every second. If my math is correct, that's about 63GB/year times write amplification. Seems like a lot of unnecessary small writes to the SSD. Multiply that by all the other old programs that had no concept of SSD limitations and it seems worth looking into. MicroSD are standard equipment on surveillance cameras. |
Thread Tools | |
Display Modes | Rate This Thread |
|
|