If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below. |
|
|
|
Thread Tools | Display Modes |
#1
|
|||
|
|||
Merge text files & then sort unique in DOS
Merging text files is easy in DOS
for %f in (*.txt) do type "%f" output.txt But how do you sort unique in DOS? |
Ads |
#2
|
|||
|
|||
Merge text files & then sort unique in DOS
On Tue, 4 Sep 2018 10:16:57 -0500, Grease Monkey wrote:
Merging text files is easy in DOS for %f in (*.txt) do type "%f" output.txt But how do you sort unique in DOS? You can sort the contents of a text file using the SORT program. e.g. type file.txt | sort But removing duplicate lines is not possible using SORT, because it simple doesn't have such capability built it. So, you'll have to check for duplicates manually from a batch file. e.g. @echo off setlocal enabledelayedexpansion :: sort contents type output.txt | sort sorted.txt set line= del unique.txt 2nul for /f "delims=" %%A in ('type sorted.txt') do ( if not "%%A" == "!line!" ( echo %%A unique.txt set line=%%A ) ) Note that this is problematic if the file contains double quote (") character. In this case, I'd suggest using JScript. |
#3
|
|||
|
|||
Merge text files & then sort unique in DOS
Replying to 09/04 JJ
You can sort the contents of a text file using the SORT program. e.g. type file.txt | sort Yep. If it were Linux, it would simply be piped to "sort -u". But removing duplicate lines is not possible using SORT, because it simple doesn't have such capability built it. Yep. That's why I asked the question. I can already sort albeit manually using regular expressions in gvim but I'm trying to batch merge text files unique in DOS outside of gvim. So, you'll have to check for duplicates manually from a batch file. e.g. @echo off setlocal enabledelayedexpansion :: sort contents type output.txt | sort sorted.txt set line= del unique.txt 2nul for /f "delims=" %%A in ('type sorted.txt') do ( if not "%%A" == "!line!" ( echo %%A unique.txt set line=%%A ) ) Note that this is problematic if the file contains double quote (") character. In this case, I'd suggest using JScript. Thanks for that suggestion. I'm manually using a regexp from here http://vim.wikia.com/wiki/Uniq_-_Rem...uplicate_lines https://www.regular-expressions.info...catelines.html But I was hoping for a DOS command that sorted uniquely. |
#4
|
|||
|
|||
Merge text files & then sort unique in DOS
Grease Monkey wrote:
Replying to 09/04 JJ You can sort the contents of a text file using the SORT program. e.g. type file.txt | sort Yep. If it were Linux, it would simply be piped to "sort -u". But removing duplicate lines is not possible using SORT, because it simple doesn't have such capability built it. Yep. That's why I asked the question. I can already sort albeit manually using regular expressions in gvim but I'm trying to batch merge text files unique in DOS outside of gvim. So, you'll have to check for duplicates manually from a batch file. e.g. @echo off setlocal enabledelayedexpansion :: sort contents type output.txt | sort sorted.txt set line= del unique.txt 2nul for /f "delims=" %%A in ('type sorted.txt') do ( if not "%%A" == "!line!" ( echo %%A unique.txt set line=%%A ) ) Note that this is problematic if the file contains double quote (") character. In this case, I'd suggest using JScript. Thanks for that suggestion. I'm manually using a regexp from here http://vim.wikia.com/wiki/Uniq_-_Rem...uplicate_lines https://www.regular-expressions.info...catelines.html But I was hoping for a DOS command that sorted uniquely. http://gnuwin32.sourceforge.net/packages/coreutils.htm You want the binaries download and the dependencies download. Put these three files in the same directory. The latter two come from the dep zip file. sort.exe libiconv2.dll libintl3.dll Check it's working: sort --help Run a test case. The input file is delta bravo alpha delta bravo alpha sort -u in.txt out.txt alpha bravo delta You should also be able to pipe it. type in.txt | sort -u HTH, Paul |
#5
|
|||
|
|||
Merge text files & then sort unique in DOS
Grease Monkey,
Merging text files is easy in DOS for %f in (*.txt) do type "%f" output.txt Another way, likely faster: copy *.txt output.txt -or, for safity- copy /a *.txt output.txt But how do you sort unique in DOS? You don't. There is no such (buildin) command available. But Google is your friend: "xp batch remove duplicate lines" (without the quotes). First result ? Looks to be a keeper: https://stackoverflow.com/questions/...from-text-file Regards, Rudy Wieser |
#6
|
|||
|
|||
Merge text files & then sort unique in DOS
Grease Monkey wrote:
Replying to 09/04 JJ You can sort the contents of a text file using the SORT program. e.g. type file.txt | sort Yep. If it were Linux, it would simply be piped to "sort -u". But removing duplicate lines is not possible using SORT, because it simple doesn't have such capability built it. Yep. That's why I asked the question. [...] But I was hoping for a DOS command that sorted uniquely. Paul already gave a solution for a Linux-like (GNU) sort(1). Since you seem to be used to Linux, I strongly recommend to install Cygwin [1] and have all the Linux-like commands you like, in a 'DOS' Command Prompt window or/and in a bash(1) window. That way, you have the best of both worlds. FWIW, I have a - three and a half decade - (professional) UNIX background and have always used UNIX tools under MS-DOS/MS-Windows. I've been using Cygwin for some 15-20 years now (see my User-Agent: header). [1] http://cygwin.com |
#7
|
|||
|
|||
Merge text files & then sort unique in DOS
On 9/4/2018 11:16 PM, Grease Monkey wrote:
Merging text files is easy in DOS for %f in (*.txt) do type "%f" output.txt But how do you sort unique in DOS? MS DOS has a SORT.EXE if I remember correctly... it's such a long time ago.... -- @~@ Remain silent! Drink, Blink, Stretch! Live long and prosper!! / v \ Simplicity is Beauty! /( _ )\ May the Force and farces be with you! ^ ^ (x86_64 Ubuntu 9.10) Linux 2.6.39.3 不借貸! 不詐騙! 不*錢! 不援交! 不打交! 不打劫! 不自殺! 不求神! 請考慮綜援 (CSSA): http://www.swd.gov.hk/tc/index/site_...sub_addressesa |
#8
|
|||
|
|||
Merge text files & then sort unique in DOS
Mr. Man-wai Chang wrote:
On 9/4/2018 11:16 PM, Grease Monkey wrote: Merging text files is easy in DOS for %f in (*.txt) do type "%f" output.txt But how do you sort unique in DOS? MS DOS has a SORT.EXE if I remember correctly... it's such a long time ago.... Why all the additional crossposted groups ? ******* There is no UNIQUE option in MSDOS sort. http://www.vfrazee.com/ms-dos/6.22/help/sort.htm Paul |
#9
|
|||
|
|||
Merge text files & then sort unique in DOS
"Grease Monkey" wrote in message
news Merging text files is easy in DOS for %f in (*.txt) do type "%f" output.txt But how do you sort unique in DOS? unique ? sort Descending or Accenting ? Use http://bing.com Search For "command prompt file list to text" You Get 34,600,000 Results Any time Command Prompt file list to text It's very, very easy in the Windows Command-Line Interpreter (all Windows OSes): Open a command prompt (Start - Run - cmd "Enter" ). Navigate (cd) to the directory whose files you want to list. Enter dir output_file_name (e.g., dir C:\dir.txt) and press "Enter". See More... How to copy a list of file names to text file? - Super User superuser.com/questions/395836/how-to-copy-a-list-of-file-names-to-text-file How to copy a list of file names to text file? - Super User https://superuser.com/questions/3958...a-list-of-file... Open a command prompt (Start - Run - cmd Enter) Navigate (cd) to the directory whose files you want to list. Enter dir output_file_name (e.g., dir C:\dir.txt) and press Enter. Open the newly created text file (C:\dir.txt) and you'll have the complete output of the dir command in that directory. |
#10
|
|||
|
|||
Merge text files & then sort unique in DOS
On Wed, 5 Sep 2018 17:07:05 -0500, "Mynews"
wrote: "Grease Monkey" wrote in message news Merging text files is easy in DOS for %f in (*.txt) do type "%f" output.txt But how do you sort unique in DOS? unique ? sort Descending or Accenting ? Use http://bing.com Search For "command prompt file list to text" You Get 34,600,000 Results Any time Command Prompt file list to text It's very, very easy in the Windows Command-Line Interpreter (all Windows OSes): Open a command prompt (Start - Run - cmd "Enter" ). Navigate (cd) to the directory whose files you want to list. Enter dir output_file_name (e.g., dir C:\dir.txt) and press "Enter". See More... How to copy a list of file names to text file? - Super User superuser.com/questions/395836/how-to-copy-a-list-of-file-names-to-text-file How to copy a list of file names to text file? - Super User https://superuser.com/questions/3958...a-list-of-file... Open a command prompt (Start - Run - cmd Enter) Navigate (cd) to the directory whose files you want to list. Enter dir output_file_name (e.g., dir C:\dir.txt) and press Enter. Open the newly created text file (C:\dir.txt) and you'll have the complete output of the dir command in that directory. I suppose we really need to figure out what kind of data we are talking about here. Is this several files of column data that you want to reduce to one? Once you get that into a single file there are other tools you can use for that but some are not in a basic DOS load. I would use dBase but only because I have it and am comfortable with it. |
#12
|
|||
|
|||
Merge text files & then sort unique in DOS
|
#13
|
|||
|
|||
cross-posting
On 9/6/2018 1:33 AM, Paul wrote:
Why all the additional crossposted groups ? To reach out for more people and hence more help and answers. Anyway, I always avoid over-doing that. -- @~@ Remain silent! Drink, Blink, Stretch! Live long and prosper!! / v \ Simplicity is Beauty! /( _ )\ May the Force and farces be with you! ^ ^ (x86_64 Ubuntu 9.10) Linux 2.6.39.3 不借貸! 不詐騙! 不*錢! 不援交! 不打交! 不打劫! 不自殺! 不求神! 請考慮綜援 (CSSA): http://www.swd.gov.hk/tc/index/site_...sub_addressesa |
#14
|
|||
|
|||
cross-posting
Mr. Man-wai Chang wrote:
On 9/6/2018 1:33 AM, Paul wrote: Why all the additional crossposted groups ? To reach out for more people and hence more help and answers. Anyway, I always avoid over-doing that. alt.comp.hardware.pc-homebuilt === a hardware group, over-doing it! Paul |
#15
|
|||
|
|||
cross-posting
On 9/6/2018 8:35 PM, Paul wrote:
alt.comp.hardware.pc-homebuilt === a hardware group, ************************* *********** over-doing it! Sorry, SIR! -- @~@ Remain silent! Drink, Blink, Stretch! Live long and prosper!! / v \ Simplicity is Beauty! /( _ )\ May the Force and farces be with you! ^ ^ (x86_64 Ubuntu 9.10) Linux 2.6.39.3 不借貸! 不詐騙! 不*錢! 不援交! 不打交! 不打劫! 不自殺! 不求神! 請考慮綜援 (CSSA): http://www.swd.gov.hk/tc/index/site_...sub_addressesa |
|
Thread Tools | |
Display Modes | |
|
|