A bash script to convert Unicode (UTF-16) to UTF-8

If you wonder why this is tagged under hacks, consider for a moment how crappy this script is with error checking/etc. It is 100% a hack 🙂

Anyway, I wrote this short batch script to convert about 80 files from UTF-16 which MSSQL outputs to UTF-8 which is what our change control system will accept (by the way, it would be *really* nice if the error was more useful than “this file contains illegal characters”). Anyway, the idea is that you put this script in the directory that contains the files you want to change and then run it once (and only once). The “mv” line is crap I know, but I forgot the “right” way to overwrite a file you are also reading from in the same command.

#!/bin/bash
#This will corrupt any file that is not UTF-16, beware
#You can likely reverse any mistakes by switching the UTF-16
#and utf8 so that it converts back in the opposite direction.
#I’m not sure if that is universal though.
#You must be careful that your globbing statement doesn’t
#include the script itself or it will be eaten 😉
#My example is *.sql because I was fixing SQL files… obviously.
for file in *.sql
do
iconv -f UTF-16 -t utf8 $file > 1_$file
mv 1_$file $file
done

Anyway, I know this is lame, but I hope it helps at least one person. I can scarcely believe there are no free Windows utilities to do this by the way. I really ought to write one…

Edit: Use

 Preformatted code here. 

to insert horizontal linespace and have the automatic “code” font.