If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below. |
|
|
Thread Tools | Rate Thread | Display Modes |
#46
|
|||
|
|||
Sort files by aspect ratio?
"Terry Pinnell" wrote
| MEP sometimes introduces inexplicably long delays between commands that | should be 'immediate'. But also, of course, merely simulating the steps | I would perform to accomplish is a major factor. In case you missed my | earlier post, here's a paste of how I planned to do it (and which I | implemented): | | -------------------- | Failing anything simpler (and especially faster), I may do it the | long-winded way with Macro Express Pro: | https://www.dropbox.com/s/tejhxmfa6a...pect.jpg?raw=1 | | 1. Open file in IrfanView | 2. Tap the 'i' key to display its properties | 3. Select the 'Current size' field | 4. Edit that to get a prefix like '1.333', 1.778, etc. (Need to avoid a | semi-colon. And a decimal value has the advantage of showing me at a | glance if AR is close to a standard. Or I could add that to my macro | after providing a percentage tolerance.) | 5. Still in IrfanView, use F2 to change file name, e.g to | '1.333-Pugia-006.jpg' in the example shown. | 6. Save that to a new folder. | 7. Repeat for all files in the source folder. | Thanks. If you pasted that earlier I missed it. I can see why it takes so long. For each image you load IrfanView and display the file, which means not only parsing the whole file and getting the bitmap, but also waiting for Windows to paint it onscreen. Regardless of the JPG size, it's holding a 24-bit-color bitmap, so of you have an image 4000x3000 that's going to be 12 million pixels x 3 bytes, or 36 million bytes, even if the image has been compacted to a low-res 2 MB. It has to be decompressed. Then that has to be resized to fit it onscreen, which is a very work intensive operation. Then Windows needs to be called to paint it. Then all the editing steps have to go through several intermediaries. It's interesting how many ways there can be to do things. In one case we run next door to borrow a cup of sugar. In another we call a friend to go get it. In a third we hire an 18-wheeler to drive to Cincinatti, get a cup of sugar, and ship it back via Fedex. But on the surface, the difference is often not apparent. A BMP is an especially dramatic example. To get w/h all you need is to read in the first 26 bytes and convert two groups of 4 bytes to numbers. That probably take a couple of milliseconds. Here are the first 26 bytes for a BMP I checked that's 600x400: 42 4D B6 FC 0A 00 00 00 00 00 36 00 00 00 28 00 00 00 58 02 00 00 [width: 600, hex 258] 90 01 00 00 [height: 400, hex 190] But to use MEP you have to load IrfanView, with it's libraries. Then IV has to read in the whole file, parse it, set up the device-independent bitmap, send that to Windows, and Windows has to paint it. IV also has to update it's GUI, then MEP has to use clunky API methods to get the text from the size field in IV and get the numeric values from that. A JPG is similar to a BMP, though slightly more involved. But it's negligible compared to the amount of work done by the other methods. My code is calling the neighbor to get the cup of sugar, and Macro Express is hiring the 18-wheeler. I tried out Dimensions2Folders. It looks like someone wrote exactly the software you want, including GUI options for variables like ratio. Can't beat that! |
Ads |
#47
|
|||
|
|||
Sort files by aspect ratio?
Paul wrote:
Terry Pinnell wrote: Paul wrote: Terry Pinnell wrote: "B. R. 'BeAr' Ederson" wrote: On Tue, 07 Aug 2018 12:27:21 +0100, Terry Pinnell wrote: Anyone know of a tool or hack that will do something that Win 10 File Explorer unfortunately cannot: sort a folder of files into aspect ratio (width/height)? Although I still think, Dimensions2Folder is best suited for your needs, http://www.dcmembers.com/skwire/down...ions-2-folders maybe you like Arti from the same author even better: http://www.dcmembers.com/skwire/download/arti Be sure to (really) try both of them. They are no-install portable. F-Up set to acf. BeAr Thanks BeAr, Dimensions2Folder is a nice tool and it does indeed do what I want. And on brief acquaintance it has the advantage of being over three times faster than Mayayana's VBS script. For example, my test folder of 100 files took 4 s. Did you try Arti? It appears to be unstable here in Version 1803 (build 17134.167). A few seconds after dragging a folder or multiple files, it closes unceremoniously. Terry, East Grinstead, UK A faster way to get the picture information, would be to consult Windows.edb instead. This is a tech demo, not a complete application. It generates a CSV file which shows all the images on the path in the SCOPE. Adjust the SCOPE string to reflect where your image collection is on disk. If you set the scope to "C:\" then you'll pick up every 1x1 pixel picture ever invented :-) What this is doing, is leveraging the three hours spent by the Windows Search indexer, on your image collection already. Yes, you've seen this script before, in a previous question. I just tweaked it a little bit. You run this from a "regular user" Powershell. cd to the directory containing the script, then run it. Powershell will not run a file from the current working directory, unless the dot is in front, as shown. I'm not a Powershell expert, so am not going to be able to dig you out of trouble all that easily. I'm a "copy/paste expert" :-/ If I can do it with copy/paste, I will. ########### ./little_bobby_tables2.ps1 ################################ # # Inspiration: https://www.petri.com/how-to-query-t...and-powershell # # (Commented out text, is either outright failed stuff, or variants you can test) # # At the bottom line, pick a more imaginative filename... # # Hint - the less formatting you do in PowerShell, the faster the script runs. # # The CSV can be pulled into Excel, LibreOffice Calc, or Notepad. # ################################################## #################### $sql = "SELECT System.ItemFolderPathDisplay, ` System.ItemName, ` System.Image.HorizontalSize, ` System.Image.VerticalSize FROM SYSTEMINDEX ` WHERE System.Image.HorizontalSize0 AND ` System.Image.VerticalSize0 AND ` SCOPE='C:\users\Terry Pinnell\Downloads'" $provider = "provider=search.collatordso;extended properties=’application=windows’;" $connector = new-object system.data.oledb.oledbdataadapter -argument $sql, $provider $dataset = new-object system.data.dataset if ($connector.fill($dataset)) { $dataset.tables[0] | Export-CSV little.bobby.tables.4.csv } ########### end of ./little_bobby_tables2.ps1 ########################## Paul Thanks, but I'm having trouble at the first hurdle. Haven't enough time right now to plunge into learning the basics of Powershell. But as a start I copy pasted your entire text to my editor and saved it in C:\Users\terry\Dropbox\Scripts as ./little_bobby_tables2.ps1 (I assumed that's a lower case letter 'l', not the number '1'? But I've tried both. Neither look like legitimate filenames... I opened a PS window. What should I now type on the command line to run that script file? Or how should I rename it? Terry, East Grinstead, UK You can rename it to "powertst.ps1" if you want. That's Pee Esse One as a file extension. I didn't invent that extension. Microsoft did. 1) Open a Powershell window. 2) .\powertst.ps1 Then open the output file that's been dumped into the same directory. You could change the name in the script from "little.bobby.tables.4.csv" if you want. You could use "powertst.csv" if you wanted, to make the name shorter. Paul A very useful learning exercise, thanks. I now know how to run a copy/pasted PS1 script. But not a practical route to a solution, as it delivers only a CSV file, not a folder of selected or renamed files. BeAr's recommended solution, Dimensions2Folders, is the one I've started using. But it will also be interesting to see the AWK/GAWK script Reinhard is developing. Terry, East Grinstead, UK |
#48
|
|||
|
|||
Sort files by aspect ratio?
Terry Pinnell wrote:
A very useful learning exercise, thanks. I now know how to run a copy/pasted PS1 script. But not a practical route to a solution, as it delivers only a CSV file, not a folder of selected or renamed files. BeAr's recommended solution, Dimensions2Folders, is the one I've started using. But it will also be interesting to see the AWK/GAWK script Reinhard is developing. Terry, East Grinstead, UK What you were supposed to learn from the exercise, is you can "scan" the entire C: in a couple of seconds and get a CSV with all the image sizes. Now, feed the CSV into some other scripting language. This is intended to reduce the initial scan time, not the curated copy time. The fun part, would be seeing if gawk can fork a powershell subshell to run that script. The file copy time would be a fixed overhead for all participants and their code. You can't make that part go faster. There are some optimizations you could attempt, but they would be expensive in programming time, and non-scalable. If your tree of things to copy was too large, maybe an optimization wouldn't work right. If you have an optimization in time, it should work properly no matter what the total quantity of files is, to be considered a "win". (For example, using a RAMDisk to hold a temporary copy, would be defeated if the file set was larger than the RAMDisk. As an example of an attempt at an optimization.) ******* The only purpose of showing you that demo, is to demonstrate that the three hours you waited for Search Indexer to keep Windows.edb up to date wasn't wasted. For any image format that defined a Width and Height, you can tap into Windows.edb and get the information. Instead of "walking" the file tree and doing it manually. The Search Indexer also keeps the index up to date (mostly) in real time. It's not like doing a scan of the file tree has an advantage of giving the most advanced view of the tree. Search Indexer is working on that, constantly. If you dump a million files onto C: and then run that script, well obviously, the indexing won't be done by then. This concept is mainly intended for "stable data silos" where the content you're searching against, is collected a little bit at a time. Then later you want to search your corpus of work and find just the right item. If you're tossing crap onto a scratch disk, then wanting to search the scratch disk, one of the other programs would work better for that. Paul |
#49
|
|||
|
|||
Sort files by aspect ratio?
Terry Pinnell wrote:
Paul wrote: Terry Pinnell wrote: Paul wrote: Terry Pinnell wrote: "B. R. 'BeAr' Ederson" wrote: On Tue, 07 Aug 2018 12:27:21 +0100, Terry Pinnell wrote: Anyone know of a tool or hack that will do something that Win 10 File Explorer unfortunately cannot: sort a folder of files into aspect ratio (width/height)? Although I still think, Dimensions2Folder is best suited for your needs, http://www.dcmembers.com/skwire/down...ions-2-folders maybe you like Arti from the same author even better: http://www.dcmembers.com/skwire/download/arti Be sure to (really) try both of them. They are no-install portable. F-Up set to acf. BeAr Thanks BeAr, Dimensions2Folder is a nice tool and it does indeed do what I want. And on brief acquaintance it has the advantage of being over three times faster than Mayayana's VBS script. For example, my test folder of 100 files took 4 s. Did you try Arti? It appears to be unstable here in Version 1803 (build 17134.167). A few seconds after dragging a folder or multiple files, it closes unceremoniously. Terry, East Grinstead, UK A faster way to get the picture information, would be to consult Windows.edb instead. This is a tech demo, not a complete application. It generates a CSV file which shows all the images on the path in the SCOPE. Adjust the SCOPE string to reflect where your image collection is on disk. If you set the scope to "C:\" then you'll pick up every 1x1 pixel picture ever invented :-) What this is doing, is leveraging the three hours spent by the Windows Search indexer, on your image collection already. Yes, you've seen this script before, in a previous question. I just tweaked it a little bit. You run this from a "regular user" Powershell. cd to the directory containing the script, then run it. Powershell will not run a file from the current working directory, unless the dot is in front, as shown. I'm not a Powershell expert, so am not going to be able to dig you out of trouble all that easily. I'm a "copy/paste expert" :-/ If I can do it with copy/paste, I will. ########### ./little_bobby_tables2.ps1 ################################ # # Inspiration: https://www.petri.com/how-to-query-t...and-powershell # # (Commented out text, is either outright failed stuff, or variants you can test) # # At the bottom line, pick a more imaginative filename... # # Hint - the less formatting you do in PowerShell, the faster the script runs. # # The CSV can be pulled into Excel, LibreOffice Calc, or Notepad. # ################################################## #################### $sql = "SELECT System.ItemFolderPathDisplay, ` System.ItemName, ` System.Image.HorizontalSize, ` System.Image.VerticalSize FROM SYSTEMINDEX ` WHERE System.Image.HorizontalSize0 AND ` System.Image.VerticalSize0 AND ` SCOPE='C:\users\Terry Pinnell\Downloads'" $provider = "provider=search.collatordso;extended properties=’application=windows’;" $connector = new-object system.data.oledb.oledbdataadapter -argument $sql, $provider $dataset = new-object system.data.dataset if ($connector.fill($dataset)) { $dataset.tables[0] | Export-CSV little.bobby.tables.4.csv } ########### end of ./little_bobby_tables2.ps1 ########################## Paul Thanks, but I'm having trouble at the first hurdle. Haven't enough time right now to plunge into learning the basics of Powershell. But as a start I copy pasted your entire text to my editor and saved it in C:\Users\terry\Dropbox\Scripts as ./little_bobby_tables2.ps1 (I assumed that's a lower case letter 'l', not the number '1'? But I've tried both. Neither look like legitimate filenames... I opened a PS window. What should I now type on the command line to run that script file? Or how should I rename it? Terry, East Grinstead, UK You can rename it to "powertst.ps1" if you want. That's Pee Esse One as a file extension. I didn't invent that extension. Microsoft did. 1) Open a Powershell window. 2) .\powertst.ps1 Then open the output file that's been dumped into the same directory. You could change the name in the script from "little.bobby.tables.4.csv" if you want. You could use "powertst.csv" if you wanted, to make the name shorter. Paul A very useful learning exercise, thanks. I now know how to run a copy/pasted PS1 script. But not a practical route to a solution, as it delivers only a CSV file, not a folder of selected or renamed files. BeAr's recommended solution, Dimensions2Folders, is the one I've started using. But it will also be interesting to see the AWK/GAWK script Reinhard is developing. Terry, East Grinstead, UK Here's my project. Two files. The awk file uses Gawk 3. The binaries link from this page, has the executable. http://gnuwin32.sourceforge.net/packages/gawk.htm Gawk calls powershell, to do the database query. Powershell has a set-executionpolicy, which would normally be set to "UnRestricted" for home usage. I had trouble calling Powershell, using the normal technique. However, to my shock and horror, the shell allows override being passed to it directly. powershell -executionpolicy bypass -file query.ps1 It's only running as a regular user, so there's no "elevation attack" so far by doing this. In any case, mines running :-/ **************** Helper script "query.ps1" ******************** # powershell -file query.ps1 -TREEDIR "'C:\'" param([string]$TREEDIR="'C:\'") $sql = "SELECT System.ItemFolderPathDisplay, ` System.ItemName, ` System.Image.HorizontalSize, ` System.Image.VerticalSize FROM SYSTEMINDEX ` WHERE System.Image.HorizontalSize0 AND ` System.Image.VerticalSize0 AND ` SCOPE=$TREEDIR" $provider = "provider=search.collatordso;extended properties=’application=windows’;" $connector = new-object system.data.oledb.oledbdataadapter -argument $sql, $provider $dataset = new-object system.data.dataset if ($connector.fill($dataset)) { $dataset.tables[0] | Export-CSV query.csv } **************** end of Helper script "query.ps1" ************** **************** "skimmer.awk" ******************** # gawk -f skimmer.awk width height percent scan_path out_dir # # gawk -f skimmer.awk 16 9 1 "C:\\" "C:\users\user name\downloads\outdir" NUL # # 0 1 2 3 4 5 (no input file) # # ARGC = 6 ARGV[0] .. ARGV[5] # # query.csv looks like this, skip the first two lines. There can be commas in the filename! # # #TYPE System.Data.DataRow # "SYSTEM.ITEMFOLDERPATHDISPLAY","SYSTEM.ITEMNAME"," SYSTEM.IMAGE.HORIZONTALSIZE","SYSTEM.IMAGE.VERTICA LSIZE" # "C:\Users\user name\Downloads\JPG2","0000014994_1.jpg","669","600 " # "C:\Users\user name\Downloads\JPG2","04.jpg","500","375" # ################################################## ######################### # This is a first cut script, with no error handling or disaster proofing! # No warranty expressed or implied. Paul. BEGIN { if (ARGC != 6) { print "Usage: width height percent_tol source_tree destdir" print "gawk -f skimmer.awk 16 9 1 " "\"C:\\\\\" " "\"C:\\users\\user name\\downloads\\outdir\" NUL" print "" print "The program needs five arguments." print "In some cases, two backslashes may be required on the end of a path, to work." print "This proof print will then show one backslash as having made it through." print "" for (i = 1; i ARGC; i++) print ARGV[i] exit 0 } else { print "Called with" print "" for (i = 1; i ARGC; i++) print ARGV[i] print "" width = ARGV[1]+0 height = ARGV[2]+0 percent = ARGV[3]+0 outdir = ARGV[5] } # houseclean before run - no collision protection, run one copy only! cmd = "\"del query.csv\"" system( cmd ) cmd = "\"powershell -executionpolicy bypass -file query.ps1 -TREEDIR \"'" ARGV[4] "'\"\"" print "Query: " cmd print "" system( cmd ) # You can redirect stderr ("2") output to clean the output a bit. # Here, I'm hiding the warning that the directory already exists. cmd = "\"md \"" outdir "\" 2NUL\"" print "Cmd: " cmd print "" system( cmd ) high = width/height * (1 + percent/100) low = width/height * (1 - percent/100) if ( (high 0) || (low 0) ) exit 0 i=0 j=0 # No FPAT in Gawk3 FS="\"" while ( (getline "query.csv") 0 ) { if ( i = 2 ) { # "C:\Users\xxxx yyyyyyy\Downloads\JPG2","04.jpg","500","375" # 2 3 4 5 6 7 8 aspect = $6/$8 if ( (high = aspect) && (low = aspect) ) { print $4 " = " $6 " " $8 cmd = "\"copy \"" $2 "\\" $4 "\" \"" outdir "\"\"" print "Cmd: " cmd system( cmd ) j++ } } i++ } print i " images in tree, of which " j " got copied" close( "query.csv" ) # The following jazz not required now, as the program has no other clauses delete ARGV[5] delete ARGV[4] delete ARGV[3] delete ARGV[2] delete ARGV[1] } #END { # print "Processed " i " lines" #} ************* end of "skimmer.awk" **************** HTH, Paul |
#50
|
|||
|
|||
Sort files by aspect ratio?
Paul wrote:
[i] Terry Pinnell wrote: Paul wrote: Terry Pinnell wrote: Paul wrote: Terry Pinnell wrote: "B. R. 'BeAr' Ederson" wrote: On Tue, 07 Aug 2018 12:27:21 +0100, Terry Pinnell wrote: Anyone know of a tool or hack that will do something that Win 10 File Explorer unfortunately cannot: sort a folder of files into aspect ratio (width/height)? Although I still think, Dimensions2Folder is best suited for your needs, http://www.dcmembers.com/skwire/down...ions-2-folders maybe you like Arti from the same author even better: http://www.dcmembers.com/skwire/download/arti Be sure to (really) try both of them. They are no-install portable. F-Up set to acf. BeAr Thanks BeAr, Dimensions2Folder is a nice tool and it does indeed do what I want. And on brief acquaintance it has the advantage of being over three times faster than Mayayana's VBS script. For example, my test folder of 100 files took 4 s. Did you try Arti? It appears to be unstable here in Version 1803 (build 17134.167). A few seconds after dragging a folder or multiple files, it closes unceremoniously. Terry, East Grinstead, UK A faster way to get the picture information, would be to consult Windows.edb instead. This is a tech demo, not a complete application. It generates a CSV file which shows all the images on the path in the SCOPE. Adjust the SCOPE string to reflect where your image collection is on disk. If you set the scope to "C:\" then you'll pick up every 1x1 pixel picture ever invented :-) What this is doing, is leveraging the three hours spent by the Windows Search indexer, on your image collection already. Yes, you've seen this script before, in a previous question. I just tweaked it a little bit. You run this from a "regular user" Powershell. cd to the directory containing the script, then run it. Powershell will not run a file from the current working directory, unless the dot is in front, as shown. I'm not a Powershell expert, so am not going to be able to dig you out of trouble all that easily. I'm a "copy/paste expert" :-/ If I can do it with copy/paste, I will. ########### ./little_bobby_tables2.ps1 ################################ # # Inspiration: https://www.petri.com/how-to-query-t...and-powershell # # (Commented out text, is either outright failed stuff, or variants you can test) # # At the bottom line, pick a more imaginative filename... # # Hint - the less formatting you do in PowerShell, the faster the script runs. # # The CSV can be pulled into Excel, LibreOffice Calc, or Notepad. # ################################################## #################### $sql = "SELECT System.ItemFolderPathDisplay, ` System.ItemName, ` System.Image.HorizontalSize, ` System.Image.VerticalSize FROM SYSTEMINDEX ` WHERE System.Image.HorizontalSize0 AND ` System.Image.VerticalSize0 AND ` SCOPE='C:\users\Terry Pinnell\Downloads'" $provider = "provider=search.collatordso;extended properties=’application=windows’;" $connector = new-object system.data.oledb.oledbdataadapter -argument $sql, $provider $dataset = new-object system.data.dataset if ($connector.fill($dataset)) { $dataset.tables[0] | Export-CSV little.bobby.tables.4.csv } ########### end of ./little_bobby_tables2.ps1 ########################## Paul Thanks, but I'm having trouble at the first hurdle. Haven't enough time right now to plunge into learning the basics of Powershell. But as a start I copy pasted your entire text to my editor and saved it in C:\Users\terry\Dropbox\Scripts as ./little_bobby_tables2.ps1 (I assumed that's a lower case letter 'l', not the number '1'? But I've tried both. Neither look like legitimate filenames... I opened a PS window. What should I now type on the command line to run that script file? Or how should I rename it? Terry, East Grinstead, UK You can rename it to "powertst.ps1" if you want. That's Pee Esse One as a file extension. I didn't invent that extension. Microsoft did. 1) Open a Powershell window. 2) .\powertst.ps1 Then open the output file that's been dumped into the same directory. You could change the name in the script from "little.bobby.tables.4.csv" if you want. You could use "powertst.csv" if you wanted, to make the name shorter. Paul A very useful learning exercise, thanks. I now know how to run a copy/pasted PS1 script. But not a practical route to a solution, as it delivers only a CSV file, not a folder of selected or renamed files. BeAr's recommended solution, Dimensions2Folders, is the one I've started using. But it will also be interesting to see the AWK/GAWK script Reinhard is developing. Terry, East Grinstead, UK Here's my project. Two files. The awk file uses Gawk 3. The binaries link from this page, has the executable. http://gnuwin32.sourceforge.net/packages/gawk.htm Gawk calls powershell, to do the database query. Powershell has a set-executionpolicy, which would normally be set to "UnRestricted" for home usage. I had trouble calling Powershell, using the normal technique. However, to my shock and horror, the shell allows override being passed to it directly. powershell -executionpolicy bypass -file query.ps1 It's only running as a regular user, so there's no "elevation attack" so far by doing this. In any case, mines running :-/ **************** Helper script "query.ps1" ******************** # powershell -file query.ps1 -TREEDIR "'C:\'" param([string]$TREEDIR="'C:\'") $sql = "SELECT System.ItemFolderPathDisplay, ` System.ItemName, ` System.Image.HorizontalSize, ` System.Image.VerticalSize FROM SYSTEMINDEX ` WHERE System.Image.HorizontalSize0 AND ` System.Image.VerticalSize0 AND ` SCOPE=$TREEDIR" $provider = "provider=search.collatordso;extended properties=’application=windows’;" $connector = new-object system.data.oledb.oledbdataadapter -argument $sql, $provider $dataset = new-object system.data.dataset if ($connector.fill($dataset)) { $dataset.tables[0] | Export-CSV query.csv } **************** end of Helper script "query.ps1" ************** **************** "skimmer.awk" ******************** # gawk -f skimmer.awk width height percent scan_path out_dir # # gawk -f skimmer.awk 16 9 1 "C:\\" "C:\users\user name\downloads\outdir" NUL # # 0 1 2 3 4 5 (no input file) # # ARGC = 6 ARGV[0] .. ARGV[5] # # query.csv looks like this, skip the first two lines. There can be commas in the filename! # # #TYPE System.Data.DataRow # "SYSTEM.ITEMFOLDERPATHDISPLAY","SYSTEM.ITEMNAME"," SYSTEM.IMAGE.HORIZONTALSIZE","SYSTEM.IMAGE.VERTICA LSIZE" # "C:\Users\user name\Downloads\JPG2","0000014994_1.jpg","669","600 " # "C:\Users\user name\Downloads\JPG2","04.jpg","500","375" # ################################################# ########################## # This is a first cut script, with no error handling or disaster proofing! # No warranty expressed or implied. Paul. BEGIN { if (ARGC != 6) { print "Usage: width height percent_tol source_tree destdir" print "gawk -f skimmer.awk 16 9 1 " "\"C:\\\\\" " "\"C:\\users\\user name\\downloads\\outdir\" NUL" print "" print "The program needs five arguments." print "In some cases, two backslashes may be required on the end of a path, to work." print "This proof print will then show one backslash as having made it through." print "" for (i = 1; i ARGC; i++) print ARGV[i] exit 0 } else { print "Called with" print "" for (i = 1; i ARGC; i++) print ARGV print "" width = ARGV[1]+0 height = ARGV[2]+0 percent = ARGV[3]+0 outdir = ARGV[5] } # houseclean before run - no collision protection, run one copy only! cmd = "\"del query.csv\"" system( cmd ) cmd = "\"powershell -executionpolicy bypass -file query.ps1 -TREEDIR \"'" ARGV[4] "'\"\"" print "Query: " cmd print "" system( cmd ) # You can redirect stderr ("2") output to clean the output a bit. # Here, I'm hiding the warning that the directory already exists. cmd = "\"md \"" outdir "\" 2NUL\"" print "Cmd: " cmd print "" system( cmd ) high = width/height * (1 + percent/100) low = width/height * (1 - percent/100) if ( (high 0) || (low 0) ) exit 0 i=0 j=0 # No FPAT in Gawk3 FS="\"" while ( (getline "query.csv") 0 ) { if ( i = 2 ) { # "C:\Users\xxxx yyyyyyy\Downloads\JPG2","04.jpg","500","375" # 2 3 4 5 6 7 8 aspect = $6/$8 if ( (high = aspect) && (low = aspect) ) { print $4 " = " $6 " " $8 cmd = "\"copy \"" $2 "\\" $4 "\" \"" outdir "\"\"" print "Cmd: " cmd system( cmd ) j++ } } i++ } print i " images in tree, of which " j " got copied" close( "query.csv" ) # The following jazz not required now, as the program has no other clauses delete ARGV[5] delete ARGV[4] delete ARGV[3] delete ARGV[2] delete ARGV[1] } #END { # print "Processed " i " lines" #} ************* end of "skimmer.awk" **************** HTH, Paul Thanks, I'd love to try it. But can you spell out exactly how I proceed to do that please? As mentioned, I have no PowerShell, GAWK or VBS experience. I'm not a programmer or 'techie'. I see at least three scripts you've taken the trouble to develop. How do I run them? Let's say on this test folder of 100 JPGs: D:\Pictures\Misc-Graphics\Test - Various AR (100) If you'd like to experiment first, here's my test folder of 10 JPGs, which includes six with AR's of 16:9. https://www.dropbox.com/s/3nnyah68iq...r169.zip?raw=1 Terry, East Grinstead, UK |
#51
|
|||
|
|||
Sort files by aspect ratio?
Paul wrote:
[i] Terry Pinnell wrote: Paul wrote: Terry Pinnell wrote: Paul wrote: Terry Pinnell wrote: "B. R. 'BeAr' Ederson" wrote: On Tue, 07 Aug 2018 12:27:21 +0100, Terry Pinnell wrote: Anyone know of a tool or hack that will do something that Win 10 File Explorer unfortunately cannot: sort a folder of files into aspect ratio (width/height)? Although I still think, Dimensions2Folder is best suited for your needs, http://www.dcmembers.com/skwire/down...ions-2-folders maybe you like Arti from the same author even better: http://www.dcmembers.com/skwire/download/arti Be sure to (really) try both of them. They are no-install portable. F-Up set to acf. BeAr Thanks BeAr, Dimensions2Folder is a nice tool and it does indeed do what I want. And on brief acquaintance it has the advantage of being over three times faster than Mayayana's VBS script. For example, my test folder of 100 files took 4 s. Did you try Arti? It appears to be unstable here in Version 1803 (build 17134.167). A few seconds after dragging a folder or multiple files, it closes unceremoniously. Terry, East Grinstead, UK A faster way to get the picture information, would be to consult Windows.edb instead. This is a tech demo, not a complete application. It generates a CSV file which shows all the images on the path in the SCOPE. Adjust the SCOPE string to reflect where your image collection is on disk. If you set the scope to "C:\" then you'll pick up every 1x1 pixel picture ever invented :-) What this is doing, is leveraging the three hours spent by the Windows Search indexer, on your image collection already. Yes, you've seen this script before, in a previous question. I just tweaked it a little bit. You run this from a "regular user" Powershell. cd to the directory containing the script, then run it. Powershell will not run a file from the current working directory, unless the dot is in front, as shown. I'm not a Powershell expert, so am not going to be able to dig you out of trouble all that easily. I'm a "copy/paste expert" :-/ If I can do it with copy/paste, I will. ########### ./little_bobby_tables2.ps1 ################################ # # Inspiration: https://www.petri.com/how-to-query-t...and-powershell # # (Commented out text, is either outright failed stuff, or variants you can test) # # At the bottom line, pick a more imaginative filename... # # Hint - the less formatting you do in PowerShell, the faster the script runs. # # The CSV can be pulled into Excel, LibreOffice Calc, or Notepad. # ################################################## #################### $sql = "SELECT System.ItemFolderPathDisplay, ` System.ItemName, ` System.Image.HorizontalSize, ` System.Image.VerticalSize FROM SYSTEMINDEX ` WHERE System.Image.HorizontalSize0 AND ` System.Image.VerticalSize0 AND ` SCOPE='C:\users\Terry Pinnell\Downloads'" $provider = "provider=search.collatordso;extended properties=’application=windows’;" $connector = new-object system.data.oledb.oledbdataadapter -argument $sql, $provider $dataset = new-object system.data.dataset if ($connector.fill($dataset)) { $dataset.tables[0] | Export-CSV little.bobby.tables.4.csv } ########### end of ./little_bobby_tables2.ps1 ########################## Paul Thanks, but I'm having trouble at the first hurdle. Haven't enough time right now to plunge into learning the basics of Powershell. But as a start I copy pasted your entire text to my editor and saved it in C:\Users\terry\Dropbox\Scripts as ./little_bobby_tables2.ps1 (I assumed that's a lower case letter 'l', not the number '1'? But I've tried both. Neither look like legitimate filenames... I opened a PS window. What should I now type on the command line to run that script file? Or how should I rename it? Terry, East Grinstead, UK You can rename it to "powertst.ps1" if you want. That's Pee Esse One as a file extension. I didn't invent that extension. Microsoft did. 1) Open a Powershell window. 2) .\powertst.ps1 Then open the output file that's been dumped into the same directory. You could change the name in the script from "little.bobby.tables.4.csv" if you want. You could use "powertst.csv" if you wanted, to make the name shorter. Paul A very useful learning exercise, thanks. I now know how to run a copy/pasted PS1 script. But not a practical route to a solution, as it delivers only a CSV file, not a folder of selected or renamed files. BeAr's recommended solution, Dimensions2Folders, is the one I've started using. But it will also be interesting to see the AWK/GAWK script Reinhard is developing. Terry, East Grinstead, UK Here's my project. Two files. The awk file uses Gawk 3. The binaries link from this page, has the executable. http://gnuwin32.sourceforge.net/packages/gawk.htm Gawk calls powershell, to do the database query. Powershell has a set-executionpolicy, which would normally be set to "UnRestricted" for home usage. I had trouble calling Powershell, using the normal technique. However, to my shock and horror, the shell allows override being passed to it directly. powershell -executionpolicy bypass -file query.ps1 It's only running as a regular user, so there's no "elevation attack" so far by doing this. In any case, mines running :-/ **************** Helper script "query.ps1" ******************** # powershell -file query.ps1 -TREEDIR "'C:\'" param([string]$TREEDIR="'C:\'") $sql = "SELECT System.ItemFolderPathDisplay, ` System.ItemName, ` System.Image.HorizontalSize, ` System.Image.VerticalSize FROM SYSTEMINDEX ` WHERE System.Image.HorizontalSize0 AND ` System.Image.VerticalSize0 AND ` SCOPE=$TREEDIR" $provider = "provider=search.collatordso;extended properties=’application=windows’;" $connector = new-object system.data.oledb.oledbdataadapter -argument $sql, $provider $dataset = new-object system.data.dataset if ($connector.fill($dataset)) { $dataset.tables[0] | Export-CSV query.csv } **************** end of Helper script "query.ps1" ************** **************** "skimmer.awk" ******************** # gawk -f skimmer.awk width height percent scan_path out_dir # # gawk -f skimmer.awk 16 9 1 "C:\\" "C:\users\user name\downloads\outdir" NUL # # 0 1 2 3 4 5 (no input file) # # ARGC = 6 ARGV[0] .. ARGV[5] # # query.csv looks like this, skip the first two lines. There can be commas in the filename! # # #TYPE System.Data.DataRow # "SYSTEM.ITEMFOLDERPATHDISPLAY","SYSTEM.ITEMNAME"," SYSTEM.IMAGE.HORIZONTALSIZE","SYSTEM.IMAGE.VERTICA LSIZE" # "C:\Users\user name\Downloads\JPG2","0000014994_1.jpg","669","600 " # "C:\Users\user name\Downloads\JPG2","04.jpg","500","375" # ################################################# ########################## # This is a first cut script, with no error handling or disaster proofing! # No warranty expressed or implied. Paul. BEGIN { if (ARGC != 6) { print "Usage: width height percent_tol source_tree destdir" print "gawk -f skimmer.awk 16 9 1 " "\"C:\\\\\" " "\"C:\\users\\user name\\downloads\\outdir\" NUL" print "" print "The program needs five arguments." print "In some cases, two backslashes may be required on the end of a path, to work." print "This proof print will then show one backslash as having made it through." print "" for (i = 1; i ARGC; i++) print ARGV[i] exit 0 } else { print "Called with" print "" for (i = 1; i ARGC; i++) print ARGV print "" width = ARGV[1]+0 height = ARGV[2]+0 percent = ARGV[3]+0 outdir = ARGV[5] } # houseclean before run - no collision protection, run one copy only! cmd = "\"del query.csv\"" system( cmd ) cmd = "\"powershell -executionpolicy bypass -file query.ps1 -TREEDIR \"'" ARGV[4] "'\"\"" print "Query: " cmd print "" system( cmd ) # You can redirect stderr ("2") output to clean the output a bit. # Here, I'm hiding the warning that the directory already exists. cmd = "\"md \"" outdir "\" 2NUL\"" print "Cmd: " cmd print "" system( cmd ) high = width/height * (1 + percent/100) low = width/height * (1 - percent/100) if ( (high 0) || (low 0) ) exit 0 i=0 j=0 # No FPAT in Gawk3 FS="\"" while ( (getline "query.csv") 0 ) { if ( i = 2 ) { # "C:\Users\xxxx yyyyyyy\Downloads\JPG2","04.jpg","500","375" # 2 3 4 5 6 7 8 aspect = $6/$8 if ( (high = aspect) && (low = aspect) ) { print $4 " = " $6 " " $8 cmd = "\"copy \"" $2 "\\" $4 "\" \"" outdir "\"\"" print "Cmd: " cmd system( cmd ) j++ } } i++ } print i " images in tree, of which " j " got copied" close( "query.csv" ) # The following jazz not required now, as the program has no other clauses delete ARGV[5] delete ARGV[4] delete ARGV[3] delete ARGV[2] delete ARGV[1] } #END { # print "Processed " i " lines" #} ************* end of "skimmer.awk" **************** HTH, Paul Meanwhile these are the steps I tried. 1. Unwilling to test on my very large \Downloads folder I made a global replacement of 'downloads' by 'Test 10 for 169' in skimmer.awk and copied my test folder to C:\Users\terry\Test 10 for 169. But I don't know what to do to these lines: # "C:\Users\terry\Test 10 for 169\JPG2","0000014994_1.jpg","669","600" # "C:\Users\terry\Test 10 for 169\JPG2","04.jpg","500","375" 2. I ran skimmer.awk anyway but could see no obvious result. What *should* I see? Terry, East Grinstead, UK |
#52
|
|||
|
|||
Sort files by aspect ratio?
Terry Pinnell wrote:
Thanks, I'd love to try it. But can you spell out exactly how I proceed to do that please? As mentioned, I have no PowerShell, GAWK or VBS experience. I'm not a programmer or 'techie'. I see at least three scripts you've taken the trouble to develop. How do I run them? Let's say on this test folder of 100 JPGs: D:\Pictures\Misc-Graphics\Test - Various AR (100) If you'd like to experiment first, here's my test folder of 10 JPGs, which includes six with AR's of 16:9. https://www.dropbox.com/s/3nnyah68iq...r169.zip?raw=1 Terry, East Grinstead, UK 1) Execute control.exe and bring up the Win10 Control Panels. Find the "Indexing Options". Make sure D: is included in your index. https://s33.postimg.cc/vpyj17idr/che...g_settings.gif 2) Prepare two text files. You can associate the .awk extension with Notepad if you want. That's how I make it easy to edit. Editing a Powershell script, sometimes brings up a kind of IDE for Powershell, which isn't really needed in this case. It's possible I've associated .ps1 with Notepad too. skimmer.awk query.ps1 3) Click the "binaries" link here to get the ZIP with "gawk.exe" inside it. http://gnuwin32.sourceforge.net/packages/gawk.htm gawk-3.1.6-1-bin.zip 1,448,542 bytes In the bin folder, you'll find "gawk.exe" 352768 bytes. It's a 32 bit executable. Move it next to the items in (2). 4) Open a regular Command Prompt. admin isn't needed. Change directory to where-ever the three files are stored. In the command itself, you can change the "X:" to the drive and folder where you want the output placed. I don't want to make the command too long for an example. The "cd" command doesn't need quotes on the path spec, but many other Command Prompt activities do need double quotes, to isolate paths from one another and pass properly as tokens to the command. cd /d C:\users\Terry Pinnell\Downloads\threefiles gawk -f skimmer.awk 16 9 1 "D:\Pictures\Misc-Graphics\Test - Various AR (100)\\" "X:\outdir" NUL The "NUL" on the very end of the command, ensures that the text input stream to gawk is replaced by an empty file. This causes gawk to properly terminate after a run, and the Command Prompt prompt will come back when it's done. 5) The command has two main phases. a) Gawk script echoes your parameters, for visual verification later. b) A powershell task is forked, running an SQL query. That's the first delay. Temp file is "query.csv". c) After the delay, the copy phase begins, with some text output during each file copy. Gawk reads the "query.csv" file, and does the aspect ratio math. d) At the end, a progress statement is emitted. "28616 images in tree, of which 127 got copied" e) Your curated files are now copied to "X:\outdir" f) If you run the command again, query.csv is deleted before powershell is run. This allows query.csv to sit in the folder, if you want to look at it. Now, if you look in the working directory, you'll see a new query.csv file. That lists the 28616 images obtained during (b). I didn't bother making a logfile, as that's a "detail" for later. Things in the script that are "easy", I didn't bother with for an alpha run. The purpose of the alpha run at the moment, is to see if this shaves any time off the run or not (by consulting Windows Search). In the "query.ps1" file, the width0 and height0 are intended to find *all* images in the path. If you know, as an artist, that no works worth including will be smaller than 640x480, you can adjust the parameters in query.ps1 with a text editor, and make "width640" and "height480" and smaller rubbish images will be ignored. This will cause the "28616 images" to drop to some smaller number. You have on purpose, used a very small tree for your source tree, so making adjustments at the moment isn't necessary. The SQL query delay, will be representative of real usage cases, and tuning width and height isn't going to help. It's only if your disk had a million images on it, the program might scale a bit better if width and height were restricted to reasonable artistic values. My disk had some 40x40 images I didn't want included, since they just clog up the query.csv file for nothing. The above process is entirely portable, and nothing was installed as such, neither were any registry edits done. Only powershell-related crap might have causes a state change on your system. I can't predict ahead of time, what Powershell "get-executionpolicy" or "set-executionpolicy" changes might be needed. If you see red-text spewed by Powershell, it is most likely to be caused by ExecutionPolicy settings. I did the best I could, by passing the "Bypass" option in skimmer.awk, but I cannot guarantee there will not be surprises on your system. I was surprised myself by what a damn nuisance Powershell made for me (the policy was set low enough for PowerShell to run, yet it would not run). Good luck, Paul |
#53
|
|||
|
|||
Sort files by aspect ratio?
Terry Pinnell wrote:
Anyone know of a tool or hack that will do something that Win 10 File Explorer unfortunately cannot: sort a folder of files into aspect ratio (width/height)? It's an operation I need quite frequently, such as when trying to isolate all files with say a 16:9 ratio (to some fine tolerance if necessary). Terry, East Grinstead, UK Reinhard has developed a great solution based on GAWK and EXIFTOOL, pulled together by a batch file. It's not only marginally faster than Destinations2Folders but also shows the originals sorted to aspect ratio, which I sometimes prefer before I split off into subfolders. (That's the approach I used in my own macro - which was about 150 times slower!) In practice the source folder, typically a copy of the originals, is selected and 'Ratio-Rename' selected from the right click context menu, a shortcut to that batch file having already been copied to the Send To folder. The files are then renamed with prefixes defining the AR and sorted by that name. My test folder of 100 JPGs and BMPs were processed in under 4 seconds on this PC under Win 10 Pro (i7, 4.0GHz, 32 GB). Excellent work, thanks Reinhard! Terry, East Grinstead, UK |
#54
|
|||
|
|||
Sort files by aspect ratio?
Terry Pinnell wrote:
Terry Pinnell wrote: Anyone know of a tool or hack that will do something that Win 10 File Explorer unfortunately cannot: sort a folder of files into aspect ratio (width/height)? It's an operation I need quite frequently, such as when trying to isolate all files with say a 16:9 ratio (to some fine tolerance if necessary). Terry, East Grinstead, UK Reinhard has developed a great solution based on GAWK and EXIFTOOL, pulled together by a batch file. It's not only marginally faster than Destinations2Folders but also shows the originals sorted to aspect ratio, which I sometimes prefer before I split off into subfolders. (That's the approach I used in my own macro - which was about 150 times slower!) In practice the source folder, typically a copy of the originals, is selected and 'Ratio-Rename' selected from the right click context menu, a shortcut to that batch file having already been copied to the Send To folder. The files are then renamed with prefixes defining the AR and sorted by that name. My test folder of 100 JPGs and BMPs were processed in under 4 seconds on this PC under Win 10 Pro (i7, 4.0GHz, 32 GB). Excellent work, thanks Reinhard! Terry, East Grinstead, UK Here's my test result. My test folder had 120 files, of which my criterion of 4:3 with a 1 percent tolerance caused 47 files to be copied. The timing function uses "timeit" from rktools. https://s15.postimg.cc/6z9oq7k7v/skimmer_test.gif I have problems with most of the timing tools I've used over the years, in that I can't believe what I'm seeing. I guess I'm not very good at guestimating time with the ole Mark One Eyeball. I modified skimmer.awk slightly, to remove some of the print statements in the output, so the screen wouldn't be quite such a mess. The debug statements were left in the version I published, to make it easier to figure out when something was going wrong. Paul |
#55
|
|||
|
|||
Sort files by aspect ratio?
Paul wrote:
Terry Pinnell wrote: Terry Pinnell wrote: Anyone know of a tool or hack that will do something that Win 10 File Explorer unfortunately cannot: sort a folder of files into aspect ratio (width/height)? It's an operation I need quite frequently, such as when trying to isolate all files with say a 16:9 ratio (to some fine tolerance if necessary). Terry, East Grinstead, UK Reinhard has developed a great solution based on GAWK and EXIFTOOL, pulled together by a batch file. It's not only marginally faster than Destinations2Folders but also shows the originals sorted to aspect ratio, which I sometimes prefer before I split off into subfolders. (That's the approach I used in my own macro - which was about 150 times slower!) In practice the source folder, typically a copy of the originals, is selected and 'Ratio-Rename' selected from the right click context menu, a shortcut to that batch file having already been copied to the Send To folder. The files are then renamed with prefixes defining the AR and sorted by that name. My test folder of 100 JPGs and BMPs were processed in under 4 seconds on this PC under Win 10 Pro (i7, 4.0GHz, 32 GB). Excellent work, thanks Reinhard! Terry, East Grinstead, UK Here's my test result. My test folder had 120 files, of which my criterion of 4:3 with a 1 percent tolerance caused 47 files to be copied. The timing function uses "timeit" from rktools. https://s15.postimg.cc/6z9oq7k7v/skimmer_test.gif I have problems with most of the timing tools I've used over the years, in that I can't believe what I'm seeing. I guess I'm not very good at guestimating time with the ole Mark One Eyeball. I modified skimmer.awk slightly, to remove some of the print statements in the output, so the screen wouldn't be quite such a mess. The debug statements were left in the version I published, to make it easier to figure out when something was going wrong. Paul Just under a tenth of a second? From clicking 'Go' to seeing your finished result? Terry, East Grinstead, UK |
#56
|
|||
|
|||
Sort files by aspect ratio?
Terry Pinnell wrote:
Paul wrote: Terry Pinnell wrote: Terry Pinnell wrote: Anyone know of a tool or hack that will do something that Win 10 File Explorer unfortunately cannot: sort a folder of files into aspect ratio (width/height)? It's an operation I need quite frequently, such as when trying to isolate all files with say a 16:9 ratio (to some fine tolerance if necessary). Terry, East Grinstead, UK Reinhard has developed a great solution based on GAWK and EXIFTOOL, pulled together by a batch file. It's not only marginally faster than Destinations2Folders but also shows the originals sorted to aspect ratio, which I sometimes prefer before I split off into subfolders. (That's the approach I used in my own macro - which was about 150 times slower!) In practice the source folder, typically a copy of the originals, is selected and 'Ratio-Rename' selected from the right click context menu, a shortcut to that batch file having already been copied to the Send To folder. The files are then renamed with prefixes defining the AR and sorted by that name. My test folder of 100 JPGs and BMPs were processed in under 4 seconds on this PC under Win 10 Pro (i7, 4.0GHz, 32 GB). Excellent work, thanks Reinhard! Terry, East Grinstead, UK Here's my test result. My test folder had 120 files, of which my criterion of 4:3 with a 1 percent tolerance caused 47 files to be copied. The timing function uses "timeit" from rktools. https://s15.postimg.cc/6z9oq7k7v/skimmer_test.gif I have problems with most of the timing tools I've used over the years, in that I can't believe what I'm seeing. I guess I'm not very good at guestimating time with the ole Mark One Eyeball. I modified skimmer.awk slightly, to remove some of the print statements in the output, so the screen wouldn't be quite such a mess. The debug statements were left in the version I published, to make it easier to figure out when something was going wrong. Paul Just under a tenth of a second? From clicking 'Go' to seeing your finished result? Terry, East Grinstead, UK OK, I was looking at 'Process Time', my mistake. But I see your Elapsed Time, is impressive at under 1.5 seconds. That's what I've been measuring with my stop watch. I'm curious about what goes on for the other 1.4 s on top of 'Process Time'? Terry, East Grinstead, UK |
#57
|
|||
|
|||
Sort files by aspect ratio?
Terry Pinnell wrote:
Terry Pinnell wrote: Paul wrote: Terry Pinnell wrote: Terry Pinnell wrote: Anyone know of a tool or hack that will do something that Win 10 File Explorer unfortunately cannot: sort a folder of files into aspect ratio (width/height)? It's an operation I need quite frequently, such as when trying to isolate all files with say a 16:9 ratio (to some fine tolerance if necessary). Terry, East Grinstead, UK Reinhard has developed a great solution based on GAWK and EXIFTOOL, pulled together by a batch file. It's not only marginally faster than Destinations2Folders but also shows the originals sorted to aspect ratio, which I sometimes prefer before I split off into subfolders. (That's the approach I used in my own macro - which was about 150 times slower!) In practice the source folder, typically a copy of the originals, is selected and 'Ratio-Rename' selected from the right click context menu, a shortcut to that batch file having already been copied to the Send To folder. The files are then renamed with prefixes defining the AR and sorted by that name. My test folder of 100 JPGs and BMPs were processed in under 4 seconds on this PC under Win 10 Pro (i7, 4.0GHz, 32 GB). Excellent work, thanks Reinhard! Terry, East Grinstead, UK Here's my test result. My test folder had 120 files, of which my criterion of 4:3 with a 1 percent tolerance caused 47 files to be copied. The timing function uses "timeit" from rktools. https://s15.postimg.cc/6z9oq7k7v/skimmer_test.gif I have problems with most of the timing tools I've used over the years, in that I can't believe what I'm seeing. I guess I'm not very good at guestimating time with the ole Mark One Eyeball. I modified skimmer.awk slightly, to remove some of the print statements in the output, so the screen wouldn't be quite such a mess. The debug statements were left in the version I published, to make it easier to figure out when something was going wrong. Paul Just under a tenth of a second? From clicking 'Go' to seeing your finished result? Terry, East Grinstead, UK OK, I was looking at 'Process Time', my mistake. But I see your Elapsed Time, is impressive at under 1.5 seconds. That's what I've been measuring with my stop watch. I'm curious about what goes on for the other 1.4 s on top of 'Process Time'? Terry, East Grinstead, UK I actually hate those little timer programs, because they never seem to match what I see going on the screen. In this case, it's possible Windows Defender scans the files before the run actually starts. I'm thinking maybe I should be shooting video or something :-) One possibility, is I could watch the run with Process Monitor, and use timestamps off that run as a metric. ******* The results show that the powershell call has more variation than the file copying. Here are three runs after booting up. The Powershell (database query) seems to have a variable time. The third run is after several repeated invocations, so is "fully warmed up". The Timeit time seems to match the gawk stamps. 3:44:57.533 gawk process start 3:44:57.603 powershell.exe process start 3:44:59.913 powershell.exe process exit 3:45:00.988 gawk process exit Timeit Elapsed Time 3.457 seconds Timeit Process Time 0.140 seconds ******* 3:57:05.562 gawk process start 3:57:05.623 powershell.exe process start 3:57:07.057 powershell.exe process exit 3:57:08.082 gawk process exit Timeit Elapsed Time 2.522 seconds Timeit Process Time 0.125 seconds ******* 4:02:49.400 gawk process start 4:02:49.456 powershell.exe process start 4:02:50.738 powershell.exe process exit 4:02:51.777 gawk process exit Timeit Elapsed Time 2.379 seconds Timeit Process Time 0.078 seconds And I'm not even getting close to yesterdays 1.416 second time. But some sort of update came in yesterday. Benchmarking using Windows 10 as a platform is largely a waste of time. You're not in control. Paul |
#58
|
|||
|
|||
Sort files by aspect ratio?
Paul wrote:
Terry Pinnell wrote: Terry Pinnell wrote: Paul wrote: Terry Pinnell wrote: Terry Pinnell wrote: Anyone know of a tool or hack that will do something that Win 10 File Explorer unfortunately cannot: sort a folder of files into aspect ratio (width/height)? It's an operation I need quite frequently, such as when trying to isolate all files with say a 16:9 ratio (to some fine tolerance if necessary). Terry, East Grinstead, UK Reinhard has developed a great solution based on GAWK and EXIFTOOL, pulled together by a batch file. It's not only marginally faster than Destinations2Folders but also shows the originals sorted to aspect ratio, which I sometimes prefer before I split off into subfolders. (That's the approach I used in my own macro - which was about 150 times slower!) In practice the source folder, typically a copy of the originals, is selected and 'Ratio-Rename' selected from the right click context menu, a shortcut to that batch file having already been copied to the Send To folder. The files are then renamed with prefixes defining the AR and sorted by that name. My test folder of 100 JPGs and BMPs were processed in under 4 seconds on this PC under Win 10 Pro (i7, 4.0GHz, 32 GB). Excellent work, thanks Reinhard! Terry, East Grinstead, UK Here's my test result. My test folder had 120 files, of which my criterion of 4:3 with a 1 percent tolerance caused 47 files to be copied. The timing function uses "timeit" from rktools. https://s15.postimg.cc/6z9oq7k7v/skimmer_test.gif I have problems with most of the timing tools I've used over the years, in that I can't believe what I'm seeing. I guess I'm not very good at guestimating time with the ole Mark One Eyeball. I modified skimmer.awk slightly, to remove some of the print statements in the output, so the screen wouldn't be quite such a mess. The debug statements were left in the version I published, to make it easier to figure out when something was going wrong. Paul Just under a tenth of a second? From clicking 'Go' to seeing your finished result? Terry, East Grinstead, UK OK, I was looking at 'Process Time', my mistake. But I see your Elapsed Time, is impressive at under 1.5 seconds. That's what I've been measuring with my stop watch. I'm curious about what goes on for the other 1.4 s on top of 'Process Time'? Terry, East Grinstead, UK I actually hate those little timer programs, because they never seem to match what I see going on the screen. In this case, it's possible Windows Defender scans the files before the run actually starts. I'm thinking maybe I should be shooting video or something :-) One possibility, is I could watch the run with Process Monitor, and use timestamps off that run as a metric. ******* The results show that the powershell call has more variation than the file copying. Here are three runs after booting up. The Powershell (database query) seems to have a variable time. The third run is after several repeated invocations, so is "fully warmed up". The Timeit time seems to match the gawk stamps. 3:44:57.533 gawk process start 3:44:57.603 powershell.exe process start 3:44:59.913 powershell.exe process exit 3:45:00.988 gawk process exit Timeit Elapsed Time 3.457 seconds Timeit Process Time 0.140 seconds ******* 3:57:05.562 gawk process start 3:57:05.623 powershell.exe process start 3:57:07.057 powershell.exe process exit 3:57:08.082 gawk process exit Timeit Elapsed Time 2.522 seconds Timeit Process Time 0.125 seconds ******* 4:02:49.400 gawk process start 4:02:49.456 powershell.exe process start 4:02:50.738 powershell.exe process exit 4:02:51.777 gawk process exit Timeit Elapsed Time 2.379 seconds Timeit Process Time 0.078 seconds And I'm not even getting close to yesterdays 1.416 second time. But some sort of update came in yesterday. Benchmarking using Windows 10 as a platform is largely a waste of time. You're not in control. Paul Sometimes benchmarking can teach you a lot. In the first new test case, I threw 50000 JPGs into my work folder, then had supper while the indexer indexed them all. In the first case, I selected aspect ratio files which do not match the majority of the files. The total time is 4.8 seconds or so. 50144 images in tree, of which 47 got copied 10:09:42.145 10:09:42.192 \__ database lookup time about 3.8 seconds 10:09:45.963 / \___ file copy time about 1 second 10:09:47.028 / 50144 images in tree, of which 47 got copied ******* In this case, I selected an aspect ratio of 1280x720 which matches all 50000 of the new files. The lookup of the files still takes 3.8 seconds (because my search makes no change to how the query is done). But the file copy took 10 minutes. It needs a speedup of around a factor of 50. I blame this on all the subshells getting forked. I would need to rewrite the code a tiny bit, to fix this. 10:10:28.879 10:10:28.926 10:10:32.736 .... dnc === computer *crashed* without finishing - nice timeit=10:00.498 === 3.8 seconds database, 596 seconds to copy files The computer crashed, because I was using procmon to monitor the experiment, and it exhausted system memory. All of it. Removing Procmon and rerunning with "timeit" completed the measurement for me. I did eventually get the 10 minute number. 50144 images in tree, of which 50010 got copied ******* So there is some value in testing "scaling" a bit. I don't consider my script taking 10 minutes to do that, to be very encouraging. I really wanted to use "robocopy" to transfer the files, but robocopy doesn't accept a copy_list. Robocopy would kick ass, given a chance. Using "copy" like I did, was a poor second choice. Paul |
#59
|
|||
|
|||
Sort files by aspect ratio?
On Fri, 17 Aug 2018 05:37:43 +0100, Terry Pinnell wrote:
[GAWK + EXIFTOOL script] It's not only marginally faster than Destinations2Folders but also shows the originals sorted to aspect ratio, which I sometimes prefer before I split off into subfolders. You wrote that you managed to register with the Donation Coder forum. But I saw no posting wrt your crashes with Arti on Win10. Maybe you contacted Jody by mail? If you did, you could ask him to add "move" functionality (in addition to the copy function already present). This way you could sort by (rounded or exact) ratio, select the ones you want to deal with, and move them wherever you like. ;-) I, myself, prefer Total Commander for such tasks (OT in acf). I add the columns necessary to user defined views (ImgSize and WDX for Images both support image ratio, for instance), sort by these columns and do whatever I like to do with the images. F-Up set to acf. BeAr -- ================================================== ========================= = What do you mean with: "Perfection is always an illusion"? = ================================================== =============--(Oops!)=== |
#60
|
|||
|
|||
Sort files by aspect ratio?
Paul wrote:
Paul wrote: Terry Pinnell wrote: Terry Pinnell wrote: Paul wrote: Terry Pinnell wrote: Terry Pinnell wrote: Anyone know of a tool or hack that will do something that Win 10 File Explorer unfortunately cannot: sort a folder of files into aspect ratio (width/height)? It's an operation I need quite frequently, such as when trying to isolate all files with say a 16:9 ratio (to some fine tolerance if necessary). Terry, East Grinstead, UK Reinhard has developed a great solution based on GAWK and EXIFTOOL, pulled together by a batch file. It's not only marginally faster than Destinations2Folders but also shows the originals sorted to aspect ratio, which I sometimes prefer before I split off into subfolders. (That's the approach I used in my own macro - which was about 150 times slower!) In practice the source folder, typically a copy of the originals, is selected and 'Ratio-Rename' selected from the right click context menu, a shortcut to that batch file having already been copied to the Send To folder. The files are then renamed with prefixes defining the AR and sorted by that name. My test folder of 100 JPGs and BMPs were processed in under 4 seconds on this PC under Win 10 Pro (i7, 4.0GHz, 32 GB). Excellent work, thanks Reinhard! Terry, East Grinstead, UK Here's my test result. My test folder had 120 files, of which my criterion of 4:3 with a 1 percent tolerance caused 47 files to be copied. The timing function uses "timeit" from rktools. https://s15.postimg.cc/6z9oq7k7v/skimmer_test.gif I have problems with most of the timing tools I've used over the years, in that I can't believe what I'm seeing. I guess I'm not very good at guestimating time with the ole Mark One Eyeball. I modified skimmer.awk slightly, to remove some of the print statements in the output, so the screen wouldn't be quite such a mess. The debug statements were left in the version I published, to make it easier to figure out when something was going wrong. Paul Just under a tenth of a second? From clicking 'Go' to seeing your finished result? Terry, East Grinstead, UK OK, I was looking at 'Process Time', my mistake. But I see your Elapsed Time, is impressive at under 1.5 seconds. That's what I've been measuring with my stop watch. I'm curious about what goes on for the other 1.4 s on top of 'Process Time'? Terry, East Grinstead, UK I actually hate those little timer programs, because they never seem to match what I see going on the screen. In this case, it's possible Windows Defender scans the files before the run actually starts. I'm thinking maybe I should be shooting video or something :-) One possibility, is I could watch the run with Process Monitor, and use timestamps off that run as a metric. ******* The results show that the powershell call has more variation than the file copying. Here are three runs after booting up. The Powershell (database query) seems to have a variable time. The third run is after several repeated invocations, so is "fully warmed up". The Timeit time seems to match the gawk stamps. 3:44:57.533 gawk process start 3:44:57.603 powershell.exe process start 3:44:59.913 powershell.exe process exit 3:45:00.988 gawk process exit Timeit Elapsed Time 3.457 seconds Timeit Process Time 0.140 seconds ******* 3:57:05.562 gawk process start 3:57:05.623 powershell.exe process start 3:57:07.057 powershell.exe process exit 3:57:08.082 gawk process exit Timeit Elapsed Time 2.522 seconds Timeit Process Time 0.125 seconds ******* 4:02:49.400 gawk process start 4:02:49.456 powershell.exe process start 4:02:50.738 powershell.exe process exit 4:02:51.777 gawk process exit Timeit Elapsed Time 2.379 seconds Timeit Process Time 0.078 seconds And I'm not even getting close to yesterdays 1.416 second time. But some sort of update came in yesterday. Benchmarking using Windows 10 as a platform is largely a waste of time. You're not in control. Paul Sometimes benchmarking can teach you a lot. In the first new test case, I threw 50000 JPGs into my work folder, then had supper while the indexer indexed them all. In the first case, I selected aspect ratio files which do not match the majority of the files. The total time is 4.8 seconds or so. 50144 images in tree, of which 47 got copied 10:09:42.145 10:09:42.192 \__ database lookup time about 3.8 seconds 10:09:45.963 / \___ file copy time about 1 second 10:09:47.028 / 50144 images in tree, of which 47 got copied ******* In this case, I selected an aspect ratio of 1280x720 which matches all 50000 of the new files. The lookup of the files still takes 3.8 seconds (because my search makes no change to how the query is done). But the file copy took 10 minutes. It needs a speedup of around a factor of 50. I blame this on all the subshells getting forked. I would need to rewrite the code a tiny bit, to fix this. 10:10:28.879 10:10:28.926 10:10:32.736 ... dnc === computer *crashed* without finishing - nice timeit=10:00.498 === 3.8 seconds database, 596 seconds to copy files The computer crashed, because I was using procmon to monitor the experiment, and it exhausted system memory. All of it. Removing Procmon and rerunning with "timeit" completed the measurement for me. I did eventually get the 10 minute number. 50144 images in tree, of which 50010 got copied ******* So there is some value in testing "scaling" a bit. I don't consider my script taking 10 minutes to do that, to be very encouraging. I really wanted to use "robocopy" to transfer the files, but robocopy doesn't accept a copy_list. Robocopy would kick ass, given a chance. Using "copy" like I did, was a poor second choice. Paul I baulked at 50,000 but I doubled up my 100 successively to 3,200. That 5 GB folder had a wide range of ARs, mostly JPGs, a few BMPs. The first attempt failed because AWK apparently dislikes filenames containing spaces, and FE created lots of those, like 20020302-125739-Ashdown6 - Copy - Copy - Copy - Copy - Copy.JPG But after renaming them simply in Bulk Renamer Utility (0001 to 3200), the elapsed time of Reinhard's AWK/BAT combo was 18 secs, by stop watch. If the relationship is roughly linear that would imply 4:40 for 50,000; close to five minutes. So, ProcMon, not something to leave running while you have a coffee then! Terry, East Grinstead, UK |
Thread Tools | |
Display Modes | Rate This Thread |
|
|