A Windows XP help forum. PCbanter

If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

Go Back   Home » PCbanter forum » Microsoft Windows XP » General XP issues or comments
Site Map Home Register Authors List Search Today's Posts Mark Forums Read Web Partners

Merge text files & then sort unique in DOS



 
 
Thread Tools Display Modes
  #1  
Old September 4th 18, 04:16 PM posted to alt.windows7.general,microsoft.public.windowsxp.general
Grease Monkey
external usenet poster
 
Posts: 10
Default Merge text files & then sort unique in DOS

Merging text files is easy in DOS
for %f in (*.txt) do type "%f" output.txt

But how do you sort unique in DOS?
Ads
  #2  
Old September 4th 18, 04:31 PM posted to alt.windows7.general,microsoft.public.windowsxp.general
JJ[_11_]
external usenet poster
 
Posts: 611
Default Merge text files & then sort unique in DOS

On Tue, 4 Sep 2018 10:16:57 -0500, Grease Monkey wrote:
Merging text files is easy in DOS
for %f in (*.txt) do type "%f" output.txt

But how do you sort unique in DOS?


You can sort the contents of a text file using the SORT program. e.g.

type file.txt | sort

But removing duplicate lines is not possible using SORT, because it simple
doesn't have such capability built it.

So, you'll have to check for duplicates manually from a batch file. e.g.

@echo off
setlocal enabledelayedexpansion
:: sort contents
type output.txt | sort sorted.txt
set line=
del unique.txt 2nul
for /f "delims=" %%A in ('type sorted.txt') do (
if not "%%A" == "!line!" (
echo %%A unique.txt
set line=%%A
)
)

Note that this is problematic if the file contains double quote (")
character. In this case, I'd suggest using JScript.
  #3  
Old September 4th 18, 04:54 PM posted to alt.windows7.general,microsoft.public.windowsxp.general
Grease Monkey
external usenet poster
 
Posts: 10
Default Merge text files & then sort unique in DOS

Replying to 09/04 JJ

You can sort the contents of a text file using the SORT program. e.g.

type file.txt | sort


Yep. If it were Linux, it would simply be piped to "sort -u".

But removing duplicate lines is not possible using SORT, because it simple
doesn't have such capability built it.


Yep. That's why I asked the question.

I can already sort albeit manually using regular expressions in gvim but
I'm trying to batch merge text files unique in DOS outside of gvim.

So, you'll have to check for duplicates manually from a batch file. e.g.

@echo off
setlocal enabledelayedexpansion
:: sort contents
type output.txt | sort sorted.txt
set line=
del unique.txt 2nul
for /f "delims=" %%A in ('type sorted.txt') do (
if not "%%A" == "!line!" (
echo %%A unique.txt
set line=%%A
)
)

Note that this is problematic if the file contains double quote (")
character. In this case, I'd suggest using JScript.


Thanks for that suggestion. I'm manually using a regexp from here
http://vim.wikia.com/wiki/Uniq_-_Rem...uplicate_lines
https://www.regular-expressions.info...catelines.html

But I was hoping for a DOS command that sorted uniquely.
  #4  
Old September 4th 18, 05:23 PM posted to alt.windows7.general,microsoft.public.windowsxp.general
Paul[_32_]
external usenet poster
 
Posts: 7,313
Default Merge text files & then sort unique in DOS

Grease Monkey wrote:
Replying to 09/04 JJ

You can sort the contents of a text file using the SORT program. e.g.

type file.txt | sort


Yep. If it were Linux, it would simply be piped to "sort -u".

But removing duplicate lines is not possible using SORT, because it simple
doesn't have such capability built it.


Yep. That's why I asked the question.

I can already sort albeit manually using regular expressions in gvim but
I'm trying to batch merge text files unique in DOS outside of gvim.

So, you'll have to check for duplicates manually from a batch file. e.g.

@echo off
setlocal enabledelayedexpansion
:: sort contents
type output.txt | sort sorted.txt
set line=
del unique.txt 2nul
for /f "delims=" %%A in ('type sorted.txt') do (
if not "%%A" == "!line!" (
echo %%A unique.txt
set line=%%A
)
)

Note that this is problematic if the file contains double quote (")
character. In this case, I'd suggest using JScript.


Thanks for that suggestion. I'm manually using a regexp from here
http://vim.wikia.com/wiki/Uniq_-_Rem...uplicate_lines
https://www.regular-expressions.info...catelines.html

But I was hoping for a DOS command that sorted uniquely.


http://gnuwin32.sourceforge.net/packages/coreutils.htm

You want the binaries download and the dependencies download.
Put these three files in the same directory. The latter two
come from the dep zip file.

sort.exe
libiconv2.dll
libintl3.dll

Check it's working:

sort --help

Run a test case. The input file is

delta
bravo
alpha
delta
bravo
alpha

sort -u in.txt out.txt

alpha
bravo
delta

You should also be able to pipe it.

type in.txt | sort -u

HTH,
Paul

  #5  
Old September 4th 18, 05:34 PM posted to alt.windows7.general,microsoft.public.windowsxp.general
R.Wieser
external usenet poster
 
Posts: 757
Default Merge text files & then sort unique in DOS

Grease Monkey,

Merging text files is easy in DOS
for %f in (*.txt) do type "%f" output.txt


Another way, likely faster:
copy *.txt output.txt
-or, for safity-
copy /a *.txt output.txt

But how do you sort unique in DOS?


You don't. There is no such (buildin) command available.

But Google is your friend: "xp batch remove duplicate lines" (without the
quotes).

First result ? Looks to be a keeper:
https://stackoverflow.com/questions/...from-text-file

Regards,
Rudy Wieser


  #6  
Old September 4th 18, 06:55 PM posted to alt.windows7.general,microsoft.public.windowsxp.general
Frank Slootweg
external usenet poster
 
Posts: 615
Default Merge text files & then sort unique in DOS

Grease Monkey wrote:
Replying to 09/04 JJ

You can sort the contents of a text file using the SORT program. e.g.

type file.txt | sort


Yep. If it were Linux, it would simply be piped to "sort -u".

But removing duplicate lines is not possible using SORT, because it simple
doesn't have such capability built it.


Yep. That's why I asked the question.

[...]
But I was hoping for a DOS command that sorted uniquely.


Paul already gave a solution for a Linux-like (GNU) sort(1).

Since you seem to be used to Linux, I strongly recommend to install
Cygwin [1] and have all the Linux-like commands you like, in a 'DOS'
Command Prompt window or/and in a bash(1) window.

That way, you have the best of both worlds.

FWIW, I have a - three and a half decade - (professional) UNIX
background and have always used UNIX tools under MS-DOS/MS-Windows. I've
been using Cygwin for some 15-20 years now (see my User-Agent: header).

[1] http://cygwin.com
  #7  
Old September 5th 18, 06:15 PM posted to alt.windows7.general,microsoft.public.windowsxp.general,alt.comp.os.windows-10,alt.comp.freeware,alt.comp.hardware.pc-homebuilt
Mr. Man-wai Chang
external usenet poster
 
Posts: 1,740
Default Merge text files & then sort unique in DOS

On 9/4/2018 11:16 PM, Grease Monkey wrote:
Merging text files is easy in DOS
for %f in (*.txt) do type "%f" output.txt

But how do you sort unique in DOS?


MS DOS has a SORT.EXE if I remember correctly... it's such a long time
ago....

--
@[email protected] Remain silent! Drink, Blink, Stretch! Live long and prosper!!
/ v \ Simplicity is Beauty!
/( _ )\ May the Force and farces be with you!
^ ^ (x86_64 Ubuntu 9.10) Linux 2.6.39.3
不借貸! 不詐騙! 不*錢! 不援交! 不打交! 不打劫! 不自殺! 不求神! 請考慮綜援
(CSSA):
http://www.swd.gov.hk/tc/index/site_...sub_addressesa
  #8  
Old September 5th 18, 06:33 PM posted to alt.windows7.general,microsoft.public.windowsxp.general
Paul[_32_]
external usenet poster
 
Posts: 7,313
Default Merge text files & then sort unique in DOS

Mr. Man-wai Chang wrote:
On 9/4/2018 11:16 PM, Grease Monkey wrote:
Merging text files is easy in DOS
for %f in (*.txt) do type "%f" output.txt

But how do you sort unique in DOS?


MS DOS has a SORT.EXE if I remember correctly... it's such a long time
ago....


Why all the additional crossposted groups ?

*******

There is no UNIQUE option in MSDOS sort.

http://www.vfrazee.com/ms-dos/6.22/help/sort.htm

Paul

  #9  
Old September 5th 18, 11:07 PM posted to alt.windows7.general,microsoft.public.windowsxp.general
Mynews
external usenet poster
 
Posts: 5
Default Merge text files & then sort unique in DOS

"Grease Monkey" wrote in message
news
Merging text files is easy in DOS
for %f in (*.txt) do type "%f" output.txt


But how do you sort unique in DOS?

unique ? sort Descending or Accenting ?




Use http://bing.com Search For
"command prompt file list to text"


You Get

34,600,000 Results Any time

Command Prompt file list to text
It's very, very easy in the Windows Command-Line Interpreter (all Windows
OSes):

Open a command prompt (Start - Run - cmd "Enter" ).
Navigate (cd) to the directory whose files you want to list.
Enter dir output_file_name (e.g., dir C:\dir.txt) and press "Enter".
See More...
How to copy a list of file names to text file? - Super User
superuser.com/questions/395836/how-to-copy-a-list-of-file-names-to-text-file

How to copy a list of file names to text file? - Super User
https://superuser.com/questions/3958...a-list-of-file...
Open a command prompt (Start - Run - cmd Enter) Navigate (cd) to the
directory whose files you want to list.
Enter dir output_file_name (e.g., dir C:\dir.txt) and press Enter.
Open the newly created text file (C:\dir.txt) and
you'll have the complete output of the dir command in that directory.

  #10  
Old September 6th 18, 02:40 AM posted to microsoft.public.windowsxp.general
No_Name
external usenet poster
 
Posts: 524
Default Merge text files & then sort unique in DOS

On Wed, 5 Sep 2018 17:07:05 -0500, "Mynews"
wrote:

"Grease Monkey" wrote in message
news
Merging text files is easy in DOS
for %f in (*.txt) do type "%f" output.txt


But how do you sort unique in DOS?

unique ? sort Descending or Accenting ?




Use http://bing.com Search For
"command prompt file list to text"


You Get

34,600,000 Results Any time

Command Prompt file list to text
It's very, very easy in the Windows Command-Line Interpreter (all Windows
OSes):

Open a command prompt (Start - Run - cmd "Enter" ).
Navigate (cd) to the directory whose files you want to list.
Enter dir output_file_name (e.g., dir C:\dir.txt) and press "Enter".
See More...
How to copy a list of file names to text file? - Super User
superuser.com/questions/395836/how-to-copy-a-list-of-file-names-to-text-file

How to copy a list of file names to text file? - Super User
https://superuser.com/questions/3958...a-list-of-file...
Open a command prompt (Start - Run - cmd Enter) Navigate (cd) to the
directory whose files you want to list.
Enter dir output_file_name (e.g., dir C:\dir.txt) and press Enter.
Open the newly created text file (C:\dir.txt) and
you'll have the complete output of the dir command in that directory.


I suppose we really need to figure out what kind of data we are
talking about here. Is this several files of column data that you want
to reduce to one? Once you get that into a single file there are other
tools you can use for that but some are not in a basic DOS load.
I would use dBase but only because I have it and am comfortable with
it.
  #11  
Old September 6th 18, 02:41 AM posted to alt.windows7.general,microsoft.public.windowsxp.general
pjp[_10_]
external usenet poster
 
Posts: 1,058
Default Merge text files & then sort unique in DOS

In article , says...

"Grease Monkey" wrote in message
news
Merging text files is easy in DOS
for %f in (*.txt) do type "%f" output.txt


But how do you sort unique in DOS?

unique ? sort Descending or Accenting ?




Use
http://bing.com Search For
"command prompt file list to text"


You Get

34,600,000 Results Any time

Command Prompt file list to text
It's very, very easy in the Windows Command-Line Interpreter (all Windows
OSes):

Open a command prompt (Start - Run - cmd "Enter" ).
Navigate (cd) to the directory whose files you want to list.
Enter dir output_file_name (e.g., dir C:\dir.txt) and press "Enter".
See More...
How to copy a list of file names to text file? - Super User
superuser.com/questions/395836/how-to-copy-a-list-of-file-names-to-text-file

How to copy a list of file names to text file? - Super User
https://superuser.com/questions/3958...a-list-of-file...
Open a command prompt (Start - Run - cmd Enter) Navigate (cd) to the
directory whose files you want to list.
Enter dir output_file_name (e.g., dir C:\dir.txt) and press Enter.
Open the newly created text file (C:\dir.txt) and
you'll have the complete output of the dir command in that directory.


I do that regularily for one specific music folder I maintain. As I burn
the backup off to disk when it reaches 2.5 Gb I also create a text file
of the contents of the disk and save that also. It's easy enough to do.
You can even easily load the text file into Word or whatever you use and
format it as you like etc.
  #13  
Old September 6th 18, 12:43 PM posted to alt.windows7.general,microsoft.public.windowsxp.general
Mr. Man-wai Chang
external usenet poster
 
Posts: 1,740
Default cross-posting

On 9/6/2018 1:33 AM, Paul wrote:

Why all the additional crossposted groups ?


To reach out for more people and hence more help and answers.

Anyway, I always avoid over-doing that.
--
@[email protected] Remain silent! Drink, Blink, Stretch! Live long and prosper!!
/ v \ Simplicity is Beauty!
/( _ )\ May the Force and farces be with you!
^ ^ (x86_64 Ubuntu 9.10) Linux 2.6.39.3
不借貸! 不詐騙! 不*錢! 不援交! 不打交! 不打劫! 不自殺! 不求神! 請考慮綜援
(CSSA):
http://www.swd.gov.hk/tc/index/site_...sub_addressesa
  #14  
Old September 6th 18, 01:35 PM posted to alt.windows7.general,microsoft.public.windowsxp.general
Paul[_32_]
external usenet poster
 
Posts: 7,313
Default cross-posting

Mr. Man-wai Chang wrote:
On 9/6/2018 1:33 AM, Paul wrote:

Why all the additional crossposted groups ?


To reach out for more people and hence more help and answers.

Anyway, I always avoid over-doing that.


alt.comp.hardware.pc-homebuilt === a hardware group,
over-doing it!

Paul
  #15  
Old September 6th 18, 04:17 PM posted to alt.windows7.general,microsoft.public.windowsxp.general
Mr. Man-wai Chang
external usenet poster
 
Posts: 1,740
Default cross-posting

On 9/6/2018 8:35 PM, Paul wrote:

alt.comp.hardware.pc-homebuilt === a hardware group,
************************* *********** over-doing it!


Sorry, SIR!

--
@[email protected] Remain silent! Drink, Blink, Stretch! Live long and prosper!!
/ v \ Simplicity is Beauty!
/( _ )\ May the Force and farces be with you!
^ ^ (x86_64 Ubuntu 9.10) Linux 2.6.39.3
不借貸! 不詐騙! 不*錢! 不援交! 不打交! 不打劫! 不自殺! 不求神! 請考慮綜援
(CSSA):
http://www.swd.gov.hk/tc/index/site_...sub_addressesa
 




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is Off
HTML code is Off






All times are GMT +1. The time now is 11:38 PM.


Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2018, Jelsoft Enterprises Ltd.
Copyright 2004-2018 PCbanter.
The comments are property of their posters.