Convert those dastardly curly quotes to straight quotes on Windows?

He who is Char Jackson said on Mon, 09 Oct 2017 01:28:53 -0500:

yy10jp == this is a "quick edit" yanking a line to place it 10 lines lower

The actual keyboard presses would be different, but can't every text
editor do all of those things? I would say yes.

Thank you for the explanation, by the way!

Again you are correct that I erred in not being precise in my English.

I apologize as I had typed that explanation using faulty "brain memory"
instead of the more reliable "muscle memory" that innervates my fingers.

Running a quick test case in vi on Windows, the steps to yank a line to
place it ten lines below are ... oh ... I see ... they are what I said they
were with the exception of pressing the "escape" key (depending on what
editing mode I was in at the time).

So, this will *always* work:
Escapeyy10jp
But this will only work if I'm not in editing mode:
yy10jp

More to the point, what I want to do is globally replace those dotard curly
quotes with more politically correct keyboard quotes.

Assuming for our purpose the dastardly curly quotes are represented by the
keystroke "x", then the global search and replace sequence is:

Escape:%s/x/"/g

Which means, in vi speak...
Escape = Make sure you're in command mode (and not editing mode)
:% = run the following command on the entire file
s/x/ = search for "x" (where I need to know how to define a curly quote)
"/g = replace what you found with the double quote for all instances found

In reality I'd want to replace both "66" curly quotes (we'll call them "x")
and "99" curly quotes (we'll call them "y") with keyboard quotes, so the vi
command would be more like this (which itself can be simplified):

Escape:%s/x/"/g|%s/y/"/g
Where the bar "|" just separates the two editing commands on one line.

If I wish to confirm each change, I just add the "global confirm" gc:
Escape:%s/x/"/gc|%s/y/"/gc

Another way to search for "x" or "y" and replace with "z" is...
Escape:%s/[x,y]/z/g

Which, in vi-speak, interprets to:
Escape:%s/ = In command mode, search the whole file for
[x,y] = either "x" or "y"
/z/ = and replace what you found with "z"
g = doing so for every instance found in each line

If I just knew what to type for "x" and "y" to represent those dastardly
curly quotes, I'd be done since it doesn't seem that there is an easier
way.

I will, for the first time, search for how to represent curly quotes in a
vi command....(since this appears to be the simplest solution).

On 10/8/2017 11:55 PM, harry newton wrote:
He who is Mike S said on Sun, 8 Oct 2017 23:01:14 -0700:

Can you make a Word doc available for d/l somewhere that has all of
the characters you want to replace with one 'bad' character, space,
replacement character pair per line?

Yes, I can. And will. And just did.
http://wetakepic.com/images/2017/10/09/sample.jpg

1. This web page has the fundamental problem-pair set you ask about:
* https://practicaltypography.com/straight-and-curly-quotes.html
2. So I created this simple sample text out of that web page:
* https://pastebin.com/BFRYNs1c

2. I copied that sample text paste to this MS Word document:
* http://www.filedropper.com/fixquote

3. I then copied from MS Word to the vi text editor to create this file:
* https://www.sendspace.com/file/bhgsl8

All suffered from the same malady of having dotard curly quotes instead of
keyboard quotes. http://wetakepic.com/images/2017/10/09/sample.jpg


I used the VB6 Asc() function to see what character code the two curly
quotes were, it returned character codes 147 and 148. I believe Word has
a similar VBA function, so you can check the character codes yourself e.g.

?asc("”")
?asc("“")

https://msdn.microsoft.com/en-us/vba...s/asc-function

This worked when I pasted the text into a textbox
input:
Bad: (“)(”)
output:
Bad: (")(")

Dim s1 As String
s1 = Text1.Text
s1 = Replace(s1, Chr(147), Chr$(34))
s1 = Replace(s1, Chr(148), Chr$(34))
Text1.Text = s1

I believe you can you use the replace function to do character
substitution in an entire Word doc using VBA. This might be close to
what you need:

https://msdn.microsoft.com/en-us/lib...ffice.11).aspx

Is that approach useful?

On 10/9/2017 12:30 AM, Mike S wrote:
On 10/8/2017 11:55 PM, harry newton wrote:
He who is Mike S said on Sun, 8 Oct 2017 23:01:14 -0700:

Can you make a Word doc available for d/l somewhere that has all of
the characters you want to replace with one 'bad' character, space,
replacement character pair per line?

Yes, I can. And will. And just did.
http://wetakepic.com/images/2017/10/09/sample.jpg

1. This web page has the fundamental problem-pair set you ask about:
Â*Â* https://practicaltypography.com/straight-and-curly-quotes.html
2. So I created this simple sample text out of that web page:
Â*Â* https://pastebin.com/BFRYNs1c

2. I copied that sample text paste to this MS Word document:
Â*Â* http://www.filedropper.com/fixquote

3. I then copied from MS Word to the vi text editor to create this file:
Â*Â* https://www.sendspace.com/file/bhgsl8

All suffered from the same malady of having dotard curly quotes
instead of
keyboard quotes. http://wetakepic.com/images/2017/10/09/sample.jpg

If you are thinking of trying the VBA replacement function this might be
useful:

What's That Character?

If you knew the character's numeric code, you could search for it, but
this character falls way off the usual list. How can you find its
numeric code? Put the following macro in the template of your choice
[Hack #50] , select Tools→Macro→Macros, choose WhatCharacterCode from
the list, and click the Run button:

Sub WhatCharacterCode( )
MsgBox Asc(Selection.Text)
End Sub

This macro will display the ASCII character code for the first character
in the current selection; you can then search for it using the ^0 syntax.

If the macro reports a value of 63 and fails to match the character, you
may be facing a Unicode character. The following macro will report the
Unicode code of a character, which you can search for using the ^u syntax:

Sub WhatUnicodeCharacterCode( )
MsgBox AscW(Selection.Text)
End Sub

The result displayed will be the decimal version of the Unicode
character code, not the hexadecimal version used when inserting Unicode
characters.

https://www.safaribooksonline.com/li...1/ch04s05.html

He who is Mike S said on Sat, 7 Oct 2017 22:30:46 -0700:

Looks like you may be able to do this withing Word.
Does this approach do what you need?

Yes.
No.

Microsoft Office has the "smarts" to eliminate all the non-standard
typography characters - so if that's the only solution - that's the only
solution.

Since I'm already editing in text (vi in my case), I was hoping for a
simpler solution than bringing up a behemoth text editor - but - if MS
Office is the *only* solution - then that is the fact.

I'm hoping for a *simpler* solution than Microsoft Office though...

He who is J. P. Gilliver (John) said on Sun, 8 Oct 2017 10:17:22 +0100:

Of course, some would (and will) say why are you using a text editor
(probably inserting the word "still", to imply you're a dinosaur), but I
_do_ see the problem, and like you am surprised that no-one has written
a "simplify" utility that does what you want. (Or if they have, that
no-one has mentioned it.)

Thanks for understanding the problem set where it would be *fantastic* if a
simple solution can be found, since *everyone* (who cuts and pastes into
text files) would want this capability to remove black box unprintable
text.

He who is Wolf K said on Sat, 7 Oct 2017 20:58:20 -0400:

All I want is so simple that I can't believe anyone wouldn't want it.
[...]

Well, I don't want it. I guess I'm ab/subnormal. :-)

Fair enough.

A clarification is that that you likely don't copy and paste research
snippets into a text file.

But if you did ....

Then you'd want all that non-printible black-box stuff out of your text
files!

He who is Mayayana said on Sat, 7 Oct 2017 23:16:53 -0400:

Auric's solution is the most realistic. I know there
are different characters

Since my Windows default text editor is VIM, I guess I could define macros.
Like any VI user, I'm adept at global search-and-replace - but I'm not sure
how to handle the non-printing characters in that search.

I don't know what to look for but the search-and-replace command would look
something like this, where "x" below indicates the unknown character
sequence (which is probably a few keystrokes).

%s/x/"/g

Which means (offhand):
% = for the whole file
s = search each line
/ = for
x = (this is where I'll need the character sequence for a curly quote)
/ = and replace it with
" = literally the doublequote
/ = and do that
g = every time you see it in each line

Does anyone know offhand what the super secret character set is to type on
a US American English Keyboard for the curly quotes?

On 10/9/2017 12:52 AM, harry newton wrote:
He who is Mike S said on Sat, 7 Oct 2017 22:30:46 -0700:

Looks like you may be able to do this withing Word.
Does this approach do what you need?

Yes. No.

Microsoft Office has the "smarts" to eliminate all the non-standard
typography characters - so if that's the only solution - that's the only
solution.

Since I'm already editing in text (vi in my case), I was hoping for a
simpler solution than bringing up a behemoth text editor - but - if MS
Office is the *only* solution - then that is the fact.

I'm hoping for a *simpler* solution than Microsoft Office though...

the VBA editor is actually fast and easy to use.

Sub Demo()
Application.ScreenUpdating = False
Options.AutoFormatAsYouTypeReplaceQuotes = False
With ActiveDocument.Range
With .Find
.ClearFormatting
.Replacement.ClearFormatting
.Forward = True
.Wrap = wdFindContinue
.Format = False
.MatchWildcards = True
.Text = "[“”]"
.Replacement.Text = Chr(34)
.Execute Replace:=wdReplaceAll
.Text = "[‘’]"
.Replacement.Text = Chr(39)
.Execute Replace:=wdReplaceAll
End With
End With
Options.AutoFormatAsYouTypeReplaceQuotes = True
Application.ScreenUpdating = True
End Sub

I'll leave it to you to do the VSTO adaptation.

Cheers
Paul Edstein
[MS MVP - Word]

https://social.msdn.microsoft.com/Fo...ght-quotes?for

He who is Mike S said on Mon, 9 Oct 2017 00:30:10 -0700:

I used the VB6 Asc() function to see what character code the two curly
quotes were, it returned character codes 147 and 148. I believe Word has
a similar VBA function, so you can check the character codes yourself e.g.

?asc("+IB0-")
?asc("+IBw-")

Thanks for stating that the "character code" for the opening & closing
curly quotes is decimal [146] and decimal [148] respectively.

I don't use Word much so I'm not at all familiar with how you did that
(even though you explained it); but all I need, in the end, is a way to
tell vi how to recognize these two characters.

Once I have the two characters figured out, it's simple to replace them:
:%s/[x,y]/"/g
Which means, in my vi-speak...
:%s/ = Search the whole file for
[x,y] = either "x" or "y"
/"/g = and replace all instances with the straight double quote.

I just have to figure out how to tell vi that "x" and "y" are digital [146]
and [148] respectively.


https://msdn.microsoft.com/en-us/vba...s/asc-function

This worked when I pasted the text into a textbox
input:
Bad: (+IBw-)(+IB0-)
output:
Bad: (")(")

Dim s1 As String
s1 = Text1.Text
s1 = Replace(s1, Chr(147), Chr$(34))
s1 = Replace(s1, Chr(148), Chr$(34))
Text1.Text = s1

I believe you can you use the replace function to do character
substitution in an entire Word doc using VBA. This might be close to
what you need:

https://msdn.microsoft.com/en-us/lib...ffice.11).aspx

Is that approach useful?

Thank you for the suggested approach.
I admit you did a lot of work, which I appreciate.

The most important part of that work is the method for identifying the
characters, since there are admittedly more dastardly characters than just
the opening and closing curly doublequotes.

To be bluntly honest and open with you, I edit plain (pure?) text files all
day every day where MS Word is too ponderous to be useful except as a hack.

In this case, you proposed a valid hack, but when I look at the amount of
code, I have to compare it to the amount of code required to do the same
thing in vi, which is a comparison of this to this:

This is the pseudocode in Word to replace x & y with z globally:
Dim s1 As String
s1 = Text1.Text
s1 = Replace(s1, Chr(x), Chr$(z))
s1 = Replace(s1, Chr(y), Chr$(z))
Text1.Text = s1
This is the pseudocode in vi to replace x & y with z globally:
:%s/[x,y]/z/g

If I'm going to run a global search and replace, I may as well do it in the
text editor that is most efficient for such things.

I was originally hoping there would be a "switch" in Word, that just did it
for me, so that I could use Word as an intermediary; but if I'm going to
run a global search and replace in Word, I may as well run that same global
search and replace in vi.

So right now, I think the only viable solution is to figure out two things,
one of which I think you've figured out for me (which I appreciate).

1. How to figure out what the character is (e.g., decimal [146] & [148])
2. How to tell vi to search for that character (replacing is easy).

On Mon, 9 Oct 2017 07:52:53 +0000 (UTC), harry newton
wrote:

He who is J. P. Gilliver (John) said on Sun, 8 Oct 2017 10:17:22 +0100:

Of course, some would (and will) say why are you using a text editor
(probably inserting the word "still", to imply you're a dinosaur), but I
_do_ see the problem, and like you am surprised that no-one has written
a "simplify" utility that does what you want. (Or if they have, that
no-one has mentioned it.)

Thanks for understanding the problem set where it would be *fantastic* if a
simple solution can be found, since *everyone* (who cuts and pastes into
text files) would want this capability to remove black box unprintable
text.

Come on now, this far into the discussion and you're still talking about
black boxes, after multiple people have pointed out that just about
every other text editor doesn't have that problem? That doesn't seem
fair.

He who is Mayayana said on Mon, 9 Oct 2017 01:29:50 -0400:

If I remember correctly, that's also how the creator
of the Melissa virus got caught.

Yikes. Is my name in the MS Word document I already uploaded?
http://www.filedropper.com/fixquote

Anyway, I just now looked up how the Melissa-Virus guy got caught:
"Melissa virus turns 10"
https://www.cnet.com/news/melissa-virus-turns-10/

Salient points:
* March 26, 1999
* They scanned Usenet articles for viruses (they still do!)
* The Melissa-virus was posted as a zipped Word doc to alt.sex
* They tracked the NNTP posting host to an AOL account
* AOL provided the FBI the dial-in logs giving the phone number
* The account was compromised; the phone tracked to New Jersey
* The phone account owner was David L. Smith
* The article says "David L. Smith pleaded guilty" (plead? guilty)

It didn't say anything about the person's name being in the file.
Even so, your point is still valid that a name can be in the file.

Is my name in my Word file?

He who is Peter Moylan said on Mon, 9 Oct 2017 16:27:24 +1100:

There's nothing wrong with using old software; it doesn't go rusty, or
anything like that. You don't even have to dust it.

The other day I had to add a feature to my FTP server where a list of
users could be sorted by username, or user type, or timestamp of last
login. To do the sort I used a Quicksort module that I had written 21
years ago. I wasn't surprised that it worked without changes. If it
worked back then, it should work now.

There is a belief in some quarters that if there have been no bug fixes
for a program in the last year or so, then it is "abandonware". That is
annoying to those of us who choose not to insert the bugs in the first
place.

I'm with you on the tried-and-true software being just fine.
The text editor I use most is based on ed which is as old as the hills.]
It still works - and it's still extremely efficient at editing pure text.

BTW, as an aside on the sorting, the fact that Windows and Linux both have
a "sort" command makes sorting of lines trivially easy, as in the vi-speak
example below:

:%!sort

Which means in vi-speak:
: = begin a command
% = run that command on the entire file
! = run a MS DOS shell command (i.e., from *outside* the editor)
sort = run the Windows (or Linux) "sort" command on what it found

I use this all the time to efficiently sort selected lines alphabetically.

The Microsoft "sort" command has a lot of options that I haven't used yet:
https://technet.microsoft.com/en-us/library/bb491004.aspx

He who is Char Jackson said on Mon, 09 Oct 2017 03:26:56 -0500:

I would only point out that no one suggested you replace your editor
with Notepad. You know that, right?

At first, I was hoping that MS Word or Notepad could be used as an
"intermediary" temporary file - but even they didn't save the output in a
pure-text format, at least not in my tests of saving MS Word as
"unformatted" and my tests saving Notepad as "UTF-8" text.

As Mayayana said, you have to know a lot about encoding to figure out why,
so, I gave up on that approach once the obvious tests each failed.

The *efficient* answer is for me to simply concentrate on one thing and on
that one thing only, which is to learn how to tell vi how to recognize a
decimal [146] or decimal [148] which are apparently the encoding for the
opening and closing curly quotes.

As for the rest of it, I think what you're saying is that you've fully
memorized the keyboard commands to do the tasks that you routinely want
to do, and that's fine, but it's equally fine for the rest of us to
point out that your chosen text editor is far from perfect.

I don't think there is anyone on this planet who can safely assert that
common text-editing tasks aren't more efficient in the core text editors
(e.g., vi, emacs, nano, atom, etc.) than they are in the behemoth editors
(such as MS Word).

I just googled "most efficient text editors" and it seems this discussion
of which is the "most efficient" of text editors is an endless discussion -
but - the point is that Microsoft products will *never* be on those lists.

What is the most efficient TextEditor ...
https://www.quora.com/What-is-the-mo...-compiling-why

One reason, of course, is that the efficient editors are born on Linux:
https://www.maketecheasier.com/linux-text-editors/
5 Best Linux Text Editors
1. Vim
2. Emacs
3. Geany
4. Gedit
5. Sublime

While that's a Linux hit (and while I'd only consider the top two), the
most important point is their first sentence: "Debates on which one is the
best have been going on for years. Everyone has an opinion; everyone has a
favorite, a certain one they absolutely swear by."

So we will just have to agree to disagree in that I assert that vi is
generally on top of the short list of the most efficient "pure text"
editors extant on Windows.

If anyone else asserts that Notepad or MS Windows (or any other behemoth
editor) is more efficient, I'm not going to argue with them for more than a
post or two - since this is an age-old debate sort of along the lines of
whether iOS or Android or Mac or Windows is better.

You've shown
us that. So what's most interesting to me is that in many other aspects
of your computing life, you go on endless quests to find the best
freeware to do a specific task, but in this case you saddle yourself
with a tool that's clearly not up to the task, but a tool that you are
intimately familiar with. Very interesting.

You are astutely correct in that I consider myself a freeware expert.
That means I find the "best" freeware for the tasks that I do most.

For me, the definition of "best" pure text editor is an editor that most
efficiently edits pure text.

One thing about me is I'm a bare-bones kind of guy.
I'm KISS if nothing else.

If someone thinks Notepad is the most efficient freeware for making
constant and complex edits to hundreds of "puretext" files a day, then
maybe Notepad has something that I don't know about.

I don't know everything there is to know about freeware.
I'm always learning.

So maybe Notepad is far more efficient than vi is.
But then why can't I find Notepad yet on any list of the most efficient
text editors (at least not in my cursory search just now)?

He who is Mike S said on Mon, 9 Oct 2017 00:49:25 -0700:

If you are thinking of trying the VBA replacement function this might be
useful:

What's That Character?

I just found how to figure out the non-printable character in vi!

I found out how to do that by reading these references:
https://durgaprasad.wordpress.com/2007/09/25/find-replace-non-printable-characters-in-vim/
https://vi.stackexchange.com/questions/13379/how-to-interpret-ascii-codes-returned-by-ga-command

All I needed to do in vi to figure out what the non-printing characters
were was to sidle my cursor up to the non-printable character and just
press "ga" as shown in this screenshot below.
http://wetakepic.com/images/2017/10/09/hex.jpg

The "66" curly quotes showed up as ~S, ^S, [147], Hex 93, Octal 223
The "99" curly quotes showed up as ~T, ^T, [148], Hex 94, Octal 224

The funny thing is that this works fine individually:
:%s/\%x93/"/g (search for hex93 and replace it with straight quotes)
:%s/\%x94/"/g (search for hex94 and replace it with straight quotes)

But this isn't working for the whole file yet:
:%s/[\%x93,\%x94]/"/g

I'm working on why that multiple-search-and-replace syntax failed.

On 2017-10-09, J. P. Gilliver (John) wrote:
In message , Wolf K
writes:
On 2017-10-08 13:32, Char Jackson wrote:
On Sun, 8 Oct 2017 17:11:10 +0100, Andy Burns
wrote:

Mayayana wrote:

ASCII is standard in all uses

Except when UK users want a pound sign £ and get a hash symbol # (yes I
realise Americans may call that a pound sign)
I was working with a customer about a year ago, helping him edit the
config file for a piece of his networking gear. He wanted to add a
comment, which in that case is signified by a line starting with the "#"
symbol.
I asked him to type a pound sign. He paused, scanning his keyboard
unsuccessfully, so I helpfully added, "Shift-3". He said, "Oh! You mean
a hashtag!"
Millennials... Thanks, Twitter!

)-:


# as "pound sign" is engineering usage. Learned it 61 years ago....
Also used kip to mean 1,000 lbs.

BTW, robo-instrictions to "enter account number" usually continue with
"... and the pound sign."

Not here. "Press the hash key".

I think the normal UK name for the # character is just "hash". (I think
hashtag comes from that: it's a tag which consists of the hash character
followed by some other characters.)

The old state-owned British telephone company called the # sign 'gate'.

--
-- ^^^^^^^^^^
-- Whiskers
-- ~~~~~~~~~~