Home > unix > How to remove “smart” quotes from a text file

How to remove “smart” quotes from a text file

If you’ve copied and pasted text from Microsoft Word, chances are there will be the so-called smart quotes in that text. Some programs don’t handle these characters very well. You can turn them off in Word but if you’re trying to remedy the problem after the fact, sed is your old friend.  I’ll show you how to replace these curly quotes with the traditional straight quote.

Recall that you can do global find/replace by using sed.

sed s/[”“]/'"'/g File.txt

This won’t actually change the contents of the File, but you can save the results to a new file

sed s/[”“]/'"'/g File.txt > WithoutSmartQuotes.txt

If you wish to save the files in place, overwriting the original contents, you would do

sed -i ".bk" s/[”“]/'"'/g File.txt

This tells the sed command to make the change “in place”, while backing up the original file to File.txt.bk in case anything goes wrong.

To fix the smart quotes in all the text files in a directory, do the following:

for i in *.txt; do sed -i ".bk" s/[”“]/'"'/g $i; done

At the conclusion of the command, you will have double the number of text files in the directory, due to all the backup files. When you’ve concluded that the changes are correct (do a diff File.txt File.txt.bk to see the difference), you can delete all the backup files with rm *.bk.

  1. December 20, 2011 at 12:00 pm

    But, what if I want to go the other direction?
    I have a .po file with all straight quotes, and, this is tricky, I want to replace straight quotes with curly quotes in the target segments, only, and only those that are not within html/xml like tags (ie., replace them in
    she said,”hello!”
    but not in
    click here for more information).
    See the link I’m posting (http://tazman.parlementum.net/doku.php?id=public:smartquotes) for more details on the problem, and how I’ve tried to work around it in vim, etc., but still not satisfied, looking for a quick, simple, efficient means of scripting the solution.


  2. February 29, 2012 at 10:25 pm

    Thanks — I hadn’t thought of replacing both open and close quotes at the same time. I can do that with a regular expression in my text editor. But you forgot about ‘smart apostrophes’ and/or ‘single quotes.’ They’re just as deadly to a lot of software, and will either need a second search/replace pass or a much more interesting search string.

    I am also much in agreement with the comment from tonybaldwin. Inserting smart quotes is even tougher! Personally, I use Word’s AutoFormat. It’s not infallible (it messes up an em-dash followed by a close quote, for instance), but it is the best tool I’ve found so far.

  3. i82much
    March 1, 2012 at 1:08 am

    @fung0 good point. You could probably do something like

    sed -i “.bk” -e s/[”“]/'”‘/g -e s/[`’]/’g File.txt

  1. March 16, 2015 at 12:53 am

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: