[Vanity]: About “ and other character problems on FR -- any chance for a solution?

1-6-16 | Moi

[Maceman]

I am just wondering whether these substitute characters, which render some paragraphs unreadable, are going to be with FR forever?

Can someone explain why these are happening, and whether there is a solution?

It's been a while since they started appearing (like maybe around September 2015?).

If we just have to suck it up and accept the situation because there is no viable solution, so be it.

But is that really the case? Can nothing be done to get rid of these character interlopers?

[ButThreeLeftsDo]

http://dan.hersam.com/tools/smart-quotes.html

[CatOwner]

http://dan.hersam.com/tools/smart-quotes.html

That link only deals with part of the punctuation corruption on FR. There are some punctuation marks, letter accents, etc., that the above web site doesn't fix.

[Iron Munro]

The problem sure makes posting articles laborious.

After doing a copy and paste just about every quote, apostrophe and dash have to be converted.

[ButThreeLeftsDo]

Dunno. I have had zero problems.

[MarchonDC09122009]

The alien character quote problem is an embarrassment for FR’s reputation.
A Internet presence such as ours should not have archived material that looks like we’re an amateur site.

Our website administration should provide us a detailed explanation and action plan to resolve this.

[Iron Munro]

But does anyone know what made it suddenly start happening?

[unlearner]

I can write some lines of code in PHP that will fix this issue as well as the problem of people posting the same thing multiple times.

If my services could be helpful, please email me.

[thackney]

http://www.freerepublic.com/focus/bloggers/3358512/posts?page=37#37

To: Yaelle; John Robinson; All

This started happening after a system software upgrade a couple weeks ago. It appears something in that upgrade affected our software’s ability to properly handle ms smart characters. John is working on finding a solution. Praying he finds it soon. It’s driving us all nuts.

Straight quotes and apostrophes from your keyboard usually work. Curly or slanted usually fail (especially on cut and paste). It’s a big pain to edit them out before posting, but can be done. “Smart” ellipses and long dashes also fail.

37 posted on 11/10/2015, 2:43:56 PM by Jim Robinson

[Gay State Conservative]

I've found that when I replace certain characters (’ and “ for example) that I've cut and pasted with the same characters from *my* keyboard there's no problem.Pain in the backside for sure but...

[TBP]

Why is it happening and how do we stop it?

[pax_et_bonum]

FReepers don’t have character problems.

:-)

[Maceman]

FReepers don’t have character problems.

Speak for yourself ;-)

[TomGuy]

I am trying out a little text editor program called BDV Notepad.

One feature I noticed was a conversion from Windows text to DOS text. That does seem to correct the problem.

While searching for a text editor, I also ran across references to a ‘code page’ which is a table of values that describes the character set.

The problem or resultion may be somewhere in either the Windows 2 DOS text conversion or the code page. How or why or what is beyond my paygrade.

[CatOwner]

For the basic word processing punctuation marks, the link works fine. But there are some more obscure punctuation marks that are still corrupted. Also, if you happen to post text with any accented characters, that link won’t help with those.

[TomGuy]

But does anyone know what made it suddenly start happening?

I would suspect some software upgrade. Occasionally, software developers are bad about changing items/features without making such changes known in advance. I have dropped several software packages over the years for that reason.

I have turned off Firefox/Cyberfox/Palemoon/IceDragon browser updates for that same reason -- new versions make changes that create more havoc than they are worth.

[dennisw]

I use the old MS program Front Page Express to solve this. Copy the HTML off it and you are good to go

[cynwoody]

I am just wondering whether these substitute characters, which render some paragraphs unreadable, are going to be with FR forever?

Maybe. Maybe not. The bug started appearing in late October, if memory serves. Google was black-listing FR, apparently because some thread linked to a malware site. Something was apparently done. I would think the diagnosis would start with the question, time-honored in software development, What did I change?

In any case, I've not been seeing the effects of the bug since November, after I coded up a client-side fix. Here is how the current thread looks to me:

To stop seeing the bug's effects, install this user script to clean up the garbage characters when viewing a page and this user script to allow you to post arbitrary Unicode without it appearing as garbage.

Can someone explain why these are happening, and whether there is a solution?

The bug is that the FR server mangles posted Unicode characters which fall outside the original 7-bit ASCII range. Such characters are uploaded in UTF-8, which encodes each such special character as two or more bytes in the hex 80-FF range. Instead of leaving such sequences unmodified, as it used to do, the server replaces each byte of each sequence with its corresponding HTML entity (as opposed to the entity for the sequence as a whole, which would be harmless but unnecessary). That results in a mess on the screen, but also allows the original content to be reconstructed. Which is what my user scripts do.

[Iron Munro]

It’s probably a worm implanted by the foul, villainous, dastardly libtards at the DNC.

It keeps FREEPERS tied up doing hunt-and-peck corrections of smart quotes and other punctuation instead of quickly posting, then moving on to share brilliant insight and make more pithy remards on other threads.

[CatOwner]

I have turned off Firefox/Cyberfox/Palemoon/IceDragon browser updates for that same reason -- new versions make changes that create more havoc than they are worth.

My only hesitation with doing that is the risk of missing out on some critical security patch/update.