1CONVMV(1) CONVMV(1)
2
6 convmv - converts filenames from one encoding to another
7
9 convmv [options] FILE(S) ... DIRECTORY(S)
10
12 -f ENCODING
13 specify the current encoding of the filename(s) from which should
14 be converted
15 -t ENCODING
17 specify the encoding to which the filename(s) should be converted
18 -i interactive mode (ask y/n for each action)
20 -r recursively go through directories
22 --nfc
24 target files will be normalization form C for UTF-8 (Linux etc.)
25 --nfd
27 target files will be normalization form D for UTF-8 (OS X etc.).
28 --qfrom , --qto
30 be more quiet about the "from" or "to" of a rename (if it screws up
31 your terminal e.g.). This will in fact do nothing else than replace
32 any non-ASCII character (bytewise) with ? and any control character
33 with * on printout, this does not affect rename operation itself.
34 --exec command
36 execute the given command. You have to quote the command and #1
37 will be substituted by the old, #2 by the new filename. Using this
38 option link targets will stay untouched.
39 Example:
41 convmv -f latin1 -t utf-8 -r --exec "echo #1 should be renamed to
43 #2" path/to/files
44 --list
46 list all available encodings. To get support for more Chinese or
47 Japanese encodings install the Perl HanExtra or JIS2K Encode
48 packages.
49 --lowmem
51 keep memory footprint low by not creating a hash of all files. This
52 disables checking if symlink targets are in subtree. Symlink target
53 pointers will be converted regardlessly. If you convert multiple
54 hundredthousands or millions of files the memory usage of convmv
55 might grow quite high. This option would help you out in that case.
56 --nosmart
58 by default convmv will detect if a filename is already UTF8 encoded
59 and will skip this file if conversion from some charset to UTF8
60 should be performed. "--nosmart" will also force conversion to
61 UTF-8 for such files, which might result in "double encoded UTF-8"
62 (see section below).
63 --fixdouble
65 using the "--fixdouble" option convmv does only convert files which
66 will still be UTF-8 encoded after conversion. That's useful for
67 fixing double-encoded UTF-8 files. All files which are not UTF-8 or
68 will not result in UTF-8 after conversion will not be touched. Also
69 see chapter "How to undo double UTF-8 ..." below.
70 --notest
72 Needed to actually rename the files. By default convmv will just
73 print what it wants to do.
74 --parsable
76 This is an advanced option that people who want to write a GUI
77 front end will find useful (some others maybe, too). It will convmv
78 make print out what it would do in an easy parsable way. The first
79 column contains the action or some kind of information, the second
80 column mostly contains the file that is to be modified and if
81 appropriate the third column contains the modified value. Each
82 column is separated by \0\n (nullbyte newline). Each row (one
83 action) is separated by \0\0\n (nullbyte nullbyte newline).
84 --preserve-mtimes
86 modifying filenames usually causes the parent directory's mtime
87 being updated. This option allows to reset the mtime to the old
88 value. If your filesystem supports sub-second resolution the sub-
89 second part of the atime and mtime will be lost as Perl does not
90 yet support that.
91 --replace
93 if the file to which shall be renamed already exists, it will be
94 overwritten if the other file content is equal.
95 --unescape
97 this option will remove this ugly % hex sequences from filenames
98 and turn them into (hopefully) nicer 8-bit characters. After
99 --unescape you might want to do a charset conversion. This
100 sequences like %20 etc. are sometimes produced when downloading via
101 http or ftp.
102 --upper , --lower
104 turn filenames into all upper or all lower case. When the file is
105 not ASCII-encoded, convmv expects a charset to be entered via the
106 -f switch.
107 --dotlessi
109 care about the dotless i/I issue. A lowercase version of "I" will
110 also be dotless while an uppercase version of "i" will also be
111 dotted. This is an issue for Turkish and Azeri.
112 By the way: The superscript dot of the letter i was added in the
114 Middle Ages to distinguish the letter (in manuscripts) from
115 adjacent vertical strokes in such letters as u, m, and n. J is a
116 variant form of i which emerged at this time and subsequently
117 became a separate letter.
118 --help
120 print a short summary of available options
121 --dump-options
123 print a list of all available options
124
126 convmv is meant to help convert a single filename, a directory tree and
127 the contained files or a whole filesystem into a different encoding. It
128 just converts the filenames, not the content of the files. A special
129 feature of convmv is that it also takes care of symlinks, also converts
130 the symlink target pointer in case the symlink target is being
131 converted, too.
132 All this comes in very handy when one wants to switch over from old
134 8-bit locales to UTF-8 locales. It is also possible to convert
135 directories to UTF-8 which are already partly UTF-8 encoded. convmv is
136 able to detect if certain files are UTF-8 encoded and will skip them by
137 default. To turn this smartness off use the "--nosmart" switch.
138 Filesystem issues
140 Almost all POSIX filesystems do not care about how filenames are
141 encoded, here are some exceptions:
142 HFS+ on OS X / Darwin
144 Linux and (most?) other Unix-like operating systems use the so called
146 normalization form C (NFC) for its UTF-8 encoding by default but do not
147 enforce this. Darwin, the base of the Macintosh OS enforces
148 normalization form D (NFD), where a few characters are encoded in a
149 different way. On OS X it's not possible to create NFC UTF-8 filenames
150 because this is prevented at filesystem layer. On HFS+ filenames are
151 internally stored in UTF-16 and when converted back to UTF-8, for the
152 underlying BSD system to be handable, NFD is created. See
153 http://developer.apple.com/qa/qa2001/qa1173.html for defails. I think
154 it was a very bad idea and breaks many things under OS X which expect a
155 normal POSIX conforming system. Anywhere else convmv is able to convert
156 files from NFC to NFD or vice versa which makes interoperability with
157 such systems a lot easier.
158 JFS
160 If people mount JFS partitions with iocharset=utf8, there is a similar
162 problem, because JFS is designed to store filenames internally in
163 UTF-16, too; that is because Linux' JFS is really JFS2, which was a
164 rewrite of JFS for OS/2. JFS partitions should always be mounted with
165 iocharset=iso8859-1, which is also the default with recent 2.6.6
166 kernels. If this is not done, JFS does not behave like a POSIX
167 filesystem and it might happen that certain files cannot be created at
168 all, for example filenames in ISO-8859-1 encoding. Only when
169 interoperation with OS/2 is needed iocharset should be set according to
170 your used locale charmap.
171 NFS4
173 Despite other POSIX filesystems RFC3530 (NFS 4) mandates UTF-8 but also
175 says: "The nfs4_cs_prep profile does not specify a normalization form.
176 A later revision of this specification may specify a particular
177 normalization form." In other words, if you want to use NFS4 you might
178 find the conversion and normalization features of convmv quite useful.
179 FAT/VFAT and NTFS
181 NTFS and VFAT (for long filenames) use UTF-16 internally to store
183 filenames. You should not need to convert filenames if you mount one
184 of those filesystems. Use appropriate mount options instead!
185 How to undo double UTF-8 (or other) encoded filenames
187 Sometimes it might happen that you "double-encoded" certain filenames,
188 for example the file names already were UTF-8 encoded and you
189 accidently did another conversion from some charset to UTF-8. You can
190 simply undo that by converting that the other way round. The from-
191 charset has to be UTF-8 and the to-charset has to be the from-charset
192 you previously accidently used. If you use the "--fixdouble" option
193 convmv will make sure that only files will be processed that will still
194 be UTF-8 encoded after conversion and it will leave non-UTF-8 files
195 untouched. You should check to get the correct results by doing the
196 conversion without "--notest" before, also the "--qfrom" option might
197 be helpful, because the double utf-8 file names might screw up your
198 terminal if they are being printed - they often contain control
199 sequences which do funny things with your terminal window. If you are
200 not sure about the charset which was accidently converted from, using
201 "--qfrom" is a good way to fiddle out the required encoding without
202 destroying the file names finally.
203 How to repair Samba files
205 When in the smb.conf (of Samba 2.x) there hasn't been set a correct
206 "character set" variable, files which are created from Win* clients are
207 being created in the client's codepage, e.g. cp850 for western european
208 languages. As a result of that the files which contain non-ASCII
209 characters are screwed up if you "ls" them on the Unix server. If you
210 change the "character set" variable afterwards to iso8859-1, newly
211 created files are okay, but the old files are still screwed up in the
212 Windows encoding. In this case convmv can also be used to convert the
213 old Samba-shared files from cp850 to iso8859-1.
214 By the way: Samba 3.x finally maps to UTF-8 filenames by default, so
216 also when you migrate from Samba 2 to Samba 3 you might have to convert
217 your file names.
218 Netatalk interoperability issues
220 When Netatalk is being switched to UTF-8 which is supported in version
221 2 then it is NOT sufficient to rename the file names. There needs to be
222 done more. See
223 http://netatalk.sourceforge.net/2.0/htmldocs/upgrade.html#volumes-and-filenames
224 and the uniconv utility of Netatalk for details.
225
227 locale(1) utf-8(7) charsets(7)
228
230 no bugs or fleas known
231
233 Bjoern JACKE
234 Send mail to bjoern [at] j3e.de for bug reports and suggestions.
236perl v5.10.1 2011-08-20 CONVMV(1)