Merge text files & then sort unique in DOS

Merging text files is easy in DOS
for %f in (*.txt) do type "%f" output.txt

But how do you sort unique in DOS?

On Tue, 4 Sep 2018 10:16:57 -0500, Grease Monkey wrote:
Merging text files is easy in DOS
for %f in (*.txt) do type "%f" output.txt

But how do you sort unique in DOS?

You can sort the contents of a text file using the SORT program. e.g.

type file.txt | sort

But removing duplicate lines is not possible using SORT, because it simple
doesn't have such capability built it.

So, you'll have to check for duplicates manually from a batch file. e.g.

@echo off
setlocal enabledelayedexpansion
:: sort contents
type output.txt | sort sorted.txt
set line=
del unique.txt 2nul
for /f "delims=" %%A in ('type sorted.txt') do (
if not "%%A" == "!line!" (
echo %%A unique.txt
set line=%%A
)
)

Note that this is problematic if the file contains double quote (")
character. In this case, I'd suggest using JScript.

Replying to 09/04 JJ

You can sort the contents of a text file using the SORT program. e.g.

type file.txt | sort

Yep. If it were Linux, it would simply be piped to "sort -u".

But removing duplicate lines is not possible using SORT, because it simple
doesn't have such capability built it.

Yep. That's why I asked the question.

I can already sort albeit manually using regular expressions in gvim but
I'm trying to batch merge text files unique in DOS outside of gvim.

So, you'll have to check for duplicates manually from a batch file. e.g.

@echo off
setlocal enabledelayedexpansion
:: sort contents
type output.txt | sort sorted.txt
set line=
del unique.txt 2nul
for /f "delims=" %%A in ('type sorted.txt') do (
if not "%%A" == "!line!" (
echo %%A unique.txt
set line=%%A
)
)

Note that this is problematic if the file contains double quote (")
character. In this case, I'd suggest using JScript.

Thanks for that suggestion. I'm manually using a regexp from here
http://vim.wikia.com/wiki/Uniq_-_Rem...uplicate_lines
https://www.regular-expressions.info...catelines.html

But I was hoping for a DOS command that sorted uniquely.

Grease Monkey wrote:
Replying to 09/04 JJ

You can sort the contents of a text file using the SORT program. e.g.

type file.txt | sort

Yep. If it were Linux, it would simply be piped to "sort -u".

But removing duplicate lines is not possible using SORT, because it simple
doesn't have such capability built it.

Yep. That's why I asked the question.

I can already sort albeit manually using regular expressions in gvim but
I'm trying to batch merge text files unique in DOS outside of gvim.

So, you'll have to check for duplicates manually from a batch file. e.g.

@echo off
setlocal enabledelayedexpansion
:: sort contents
type output.txt | sort sorted.txt
set line=
del unique.txt 2nul
for /f "delims=" %%A in ('type sorted.txt') do (
if not "%%A" == "!line!" (
echo %%A unique.txt
set line=%%A
)
)

Note that this is problematic if the file contains double quote (")
character. In this case, I'd suggest using JScript.

Thanks for that suggestion. I'm manually using a regexp from here
http://vim.wikia.com/wiki/Uniq_-_Rem...uplicate_lines
https://www.regular-expressions.info...catelines.html

But I was hoping for a DOS command that sorted uniquely.

http://gnuwin32.sourceforge.net/packages/coreutils.htm

You want the binaries download and the dependencies download.
Put these three files in the same directory. The latter two
come from the dep zip file.

sort.exe
libiconv2.dll
libintl3.dll

Check it's working:

sort --help

Run a test case. The input file is

delta
bravo
alpha
delta
bravo
alpha

sort -u in.txt out.txt

alpha
bravo
delta

You should also be able to pipe it.

type in.txt | sort -u

HTH,
Paul

Grease Monkey,

Merging text files is easy in DOS
for %f in (*.txt) do type "%f" output.txt

Another way, likely faster:
copy *.txt output.txt
-or, for safity-
copy /a *.txt output.txt

But how do you sort unique in DOS?

You don't. There is no such (buildin) command available.

But Google is your friend: "xp batch remove duplicate lines" (without the
quotes).

First result ? Looks to be a keeper:
https://stackoverflow.com/questions/...from-text-file

Regards,
Rudy Wieser

Grease Monkey wrote:
Replying to 09/04 JJ

You can sort the contents of a text file using the SORT program. e.g.

type file.txt | sort

Yep. If it were Linux, it would simply be piped to "sort -u".

But removing duplicate lines is not possible using SORT, because it simple
doesn't have such capability built it.

Yep. That's why I asked the question.
[...]
But I was hoping for a DOS command that sorted uniquely.

Paul already gave a solution for a Linux-like (GNU) sort(1).

Since you seem to be used to Linux, I strongly recommend to install
Cygwin [1] and have all the Linux-like commands you like, in a 'DOS'
Command Prompt window or/and in a bash(1) window.

That way, you have the best of both worlds.

FWIW, I have a - three and a half decade - (professional) UNIX
background and have always used UNIX tools under MS-DOS/MS-Windows. I've
been using Cygwin for some 15-20 years now (see my User-Agent: header).

[1] http://cygwin.com

On 9/4/2018 11:16 PM, Grease Monkey wrote:
Merging text files is easy in DOS
for %f in (*.txt) do type "%f" output.txt

But how do you sort unique in DOS?


MS DOS has a SORT.EXE if I remember correctly... it's such a long time
ago....

--
@~@ Remain silent! Drink, Blink, Stretch! Live long and prosper!!
/ v \ Simplicity is Beauty!
/( _ )\ May the Force and farces be with you!
^ ^ (x86_64 Ubuntu 9.10) Linux 2.6.39.3
ä¸�借貸! ä¸�è©�騙! ä¸�è³*錢! ä¸�æ�´äº¤! ä¸�打交! ä¸�打劫! ä¸�自殺! ä¸�求神! 請考慮綜æ�´
(CSSA):
http://www.swd.gov.hk/tc/index/site_...sub_addressesa

Mr. Man-wai Chang wrote:
On 9/4/2018 11:16 PM, Grease Monkey wrote:
Merging text files is easy in DOS
for %f in (*.txt) do type "%f" output.txt

But how do you sort unique in DOS?


MS DOS has a SORT.EXE if I remember correctly... it's such a long time
ago....

Why all the additional crossposted groups ?

*******

There is no UNIQUE option in MSDOS sort.

http://www.vfrazee.com/ms-dos/6.22/help/sort.htm

Paul

"Grease Monkey" wrote in message
news
Merging text files is easy in DOS
for %f in (*.txt) do type "%f" output.txt

But how do you sort unique in DOS?
unique ? sort Descending or Accenting ?




Use http://bing.com Search For
"command prompt file list to text"


You Get

34,600,000 Results Any time

Command Prompt file list to text
It's very, very easy in the Windows Command-Line Interpreter (all Windows
OSes):

Open a command prompt (Start - Run - cmd "Enter" ).
Navigate (cd) to the directory whose files you want to list.
Enter dir output_file_name (e.g., dir C:\dir.txt) and press "Enter".
See More...
How to copy a list of file names to text file? - Super User
superuser.com/questions/395836/how-to-copy-a-list-of-file-names-to-text-file

How to copy a list of file names to text file? - Super User
https://superuser.com/questions/3958...a-list-of-file...
Open a command prompt (Start - Run - cmd Enter) Navigate (cd) to the
directory whose files you want to list.
Enter dir output_file_name (e.g., dir C:\dir.txt) and press Enter.
Open the newly created text file (C:\dir.txt) and
you'll have the complete output of the dir command in that directory.

On Wed, 5 Sep 2018 17:07:05 -0500, "Mynews"
wrote:

"Grease Monkey" wrote in message
news
Merging text files is easy in DOS
for %f in (*.txt) do type "%f" output.txt

But how do you sort unique in DOS?
unique ? sort Descending or Accenting ?




Use http://bing.com Search For
"command prompt file list to text"


You Get

34,600,000 Results Any time

Command Prompt file list to text
It's very, very easy in the Windows Command-Line Interpreter (all Windows
OSes):

Open a command prompt (Start - Run - cmd "Enter" ).
Navigate (cd) to the directory whose files you want to list.
Enter dir output_file_name (e.g., dir C:\dir.txt) and press "Enter".
See More...
How to copy a list of file names to text file? - Super User
superuser.com/questions/395836/how-to-copy-a-list-of-file-names-to-text-file

How to copy a list of file names to text file? - Super User
https://superuser.com/questions/3958...a-list-of-file...
Open a command prompt (Start - Run - cmd Enter) Navigate (cd) to the
directory whose files you want to list.
Enter dir output_file_name (e.g., dir C:\dir.txt) and press Enter.
Open the newly created text file (C:\dir.txt) and
you'll have the complete output of the dir command in that directory.

I suppose we really need to figure out what kind of data we are
talking about here. Is this several files of column data that you want
to reduce to one? Once you get that into a single file there are other
tools you can use for that but some are not in a basic DOS load.
I would use dBase but only because I have it and am comfortable with
it.

In article , says...

"Grease Monkey" wrote in message
news
Merging text files is easy in DOS
for %f in (*.txt) do type "%f" output.txt

But how do you sort unique in DOS?
unique ? sort Descending or Accenting ?




Use http://bing.com Search For
"command prompt file list to text"


You Get

34,600,000 Results Any time

Command Prompt file list to text
It's very, very easy in the Windows Command-Line Interpreter (all Windows
OSes):

Open a command prompt (Start - Run - cmd "Enter" ).
Navigate (cd) to the directory whose files you want to list.
Enter dir output_file_name (e.g., dir C:\dir.txt) and press "Enter".
See More...
How to copy a list of file names to text file? - Super User
superuser.com/questions/395836/how-to-copy-a-list-of-file-names-to-text-file

How to copy a list of file names to text file? - Super User
https://superuser.com/questions/3958...a-list-of-file...
Open a command prompt (Start - Run - cmd Enter) Navigate (cd) to the
directory whose files you want to list.
Enter dir output_file_name (e.g., dir C:\dir.txt) and press Enter.
Open the newly created text file (C:\dir.txt) and
you'll have the complete output of the dir command in that directory.

I do that regularily for one specific music folder I maintain. As I burn
the backup off to disk when it reaches 2.5 Gb I also create a text file
of the contents of the disk and save that also. It's easy enough to do.
You can even easily load the text file into Word or whatever you use and
format it as you like etc.

In article ,
says...

On 9/4/2018 11:16 PM, Grease Monkey wrote:
Merging text files is easy in DOS
for %f in (*.txt) do type "%f" output.txt

But how do you sort unique in DOS?


MS DOS has a SORT.EXE if I remember correctly... it's such a long time
ago....

Yes, but it doesn't have a /unique option.

Powershell can do this, as can the Bash shell if you install that.

--

Phil, London

On 9/6/2018 1:33 AM, Paul wrote:

Why all the additional crossposted groups ?


To reach out for more people and hence more help and answers.

Anyway, I always avoid over-doing that.
--
@~@ Remain silent! Drink, Blink, Stretch! Live long and prosper!!
/ v \ Simplicity is Beauty!
/( _ )\ May the Force and farces be with you!
^ ^ (x86_64 Ubuntu 9.10) Linux 2.6.39.3
ä¸�借貸! ä¸�è©�騙! ä¸�è³*錢! ä¸�æ�´äº¤! ä¸�打交! ä¸�打劫! ä¸�自殺! ä¸�求神! 請考慮綜æ�´
(CSSA):
http://www.swd.gov.hk/tc/index/site_...sub_addressesa

Mr. Man-wai Chang wrote:
On 9/6/2018 1:33 AM, Paul wrote:

Why all the additional crossposted groups ?


To reach out for more people and hence more help and answers.

Anyway, I always avoid over-doing that.

alt.comp.hardware.pc-homebuilt === a hardware group,
over-doing it!

Paul

On 9/6/2018 8:35 PM, Paul wrote:

alt.comp.hardware.pc-homebuilt === a hardware group,
Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â* Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â* over-doing it!

Sorry, SIR!

--
@~@ Remain silent! Drink, Blink, Stretch! Live long and prosper!!
/ v \ Simplicity is Beauty!
/( _ )\ May the Force and farces be with you!
^ ^ (x86_64 Ubuntu 9.10) Linux 2.6.39.3
ä¸�借貸! ä¸�è©�騙! ä¸�è³*錢! ä¸�æ�´äº¤! ä¸�打交! ä¸�打劫! ä¸�自殺! ä¸�求神! 請考慮綜æ�´
(CSSA):
http://www.swd.gov.hk/tc/index/site_...sub_addressesa