View Single Post
  #4  
Old September 4th 18, 05:23 PM posted to alt.windows7.general,microsoft.public.windowsxp.general
Paul[_32_]
external usenet poster
 
Posts: 11,873
Default Merge text files & then sort unique in DOS

Grease Monkey wrote:
Replying to 09/04 JJ

You can sort the contents of a text file using the SORT program. e.g.

type file.txt | sort


Yep. If it were Linux, it would simply be piped to "sort -u".

But removing duplicate lines is not possible using SORT, because it simple
doesn't have such capability built it.


Yep. That's why I asked the question.

I can already sort albeit manually using regular expressions in gvim but
I'm trying to batch merge text files unique in DOS outside of gvim.

So, you'll have to check for duplicates manually from a batch file. e.g.

@echo off
setlocal enabledelayedexpansion
:: sort contents
type output.txt | sort sorted.txt
set line=
del unique.txt 2nul
for /f "delims=" %%A in ('type sorted.txt') do (
if not "%%A" == "!line!" (
echo %%A unique.txt
set line=%%A
)
)

Note that this is problematic if the file contains double quote (")
character. In this case, I'd suggest using JScript.


Thanks for that suggestion. I'm manually using a regexp from here
http://vim.wikia.com/wiki/Uniq_-_Rem...uplicate_lines
https://www.regular-expressions.info...catelines.html

But I was hoping for a DOS command that sorted uniquely.


http://gnuwin32.sourceforge.net/packages/coreutils.htm

You want the binaries download and the dependencies download.
Put these three files in the same directory. The latter two
come from the dep zip file.

sort.exe
libiconv2.dll
libintl3.dll

Check it's working:

sort --help

Run a test case. The input file is

delta
bravo
alpha
delta
bravo
alpha

sort -u in.txt out.txt

alpha
bravo
delta

You should also be able to pipe it.

type in.txt | sort -u

HTH,
Paul

Ads