View Single Post
  #3  
Old September 4th 18, 04:54 PM posted to alt.windows7.general,microsoft.public.windowsxp.general
Grease Monkey
external usenet poster
 
Posts: 19
Default Merge text files & then sort unique in DOS

Replying to 09/04 JJ

You can sort the contents of a text file using the SORT program. e.g.

type file.txt | sort


Yep. If it were Linux, it would simply be piped to "sort -u".

But removing duplicate lines is not possible using SORT, because it simple
doesn't have such capability built it.


Yep. That's why I asked the question.

I can already sort albeit manually using regular expressions in gvim but
I'm trying to batch merge text files unique in DOS outside of gvim.

So, you'll have to check for duplicates manually from a batch file. e.g.

@echo off
setlocal enabledelayedexpansion
:: sort contents
type output.txt | sort sorted.txt
set line=
del unique.txt 2nul
for /f "delims=" %%A in ('type sorted.txt') do (
if not "%%A" == "!line!" (
echo %%A unique.txt
set line=%%A
)
)

Note that this is problematic if the file contains double quote (")
character. In this case, I'd suggest using JScript.


Thanks for that suggestion. I'm manually using a regexp from here
http://vim.wikia.com/wiki/Uniq_-_Rem...uplicate_lines
https://www.regular-expressions.info...catelines.html

But I was hoping for a DOS command that sorted uniquely.
Ads