1hunspell(5) File Formats Manual hunspell(5)
2
6 hunspell - format of Hunspell dictionaries and affix files
7
9 Hunspell(1) Hunspell requires two files to define the way a language is
10 being spell checked: a dictionary file containing words and applicable
11 flags, and an affix file that specifies how these flags will control
12 spell checking. An optional file is the personal dictionary file.
13
16 A dictionary file (*.dic) contains a list of words, one per line. The
17 first line of the dictionaries (except personal dictionaries) contains
18 the approximate word count (for optimal hash memory size). Each word
19 may optionally be followed by a slash ("/") and one or more flags,
20 which represents the word attributes, for example affixes.
21 Note: Dictionary words can contain also slashes when escaped like ""
23 syntax.
24 It's worth to add not only words, but word pairs to the dictionary to
26 get correct suggestions for common misspellings with missing space, as
27 in the following example, for the bad "alot" and "inspite" (see also
28 "REP" and field "ph:" about correct suggestions for common mis‐
29 spellings):
30 3
33 word
34 a lot
35 in spite
36
38 Personal dictionaries are simple word lists. Asterisk at the first
39 character position signs prohibition. A second word separated by a
40 slash sets the affixation.
41 foo
44 Foo/Simpson
45 *bar
46 In this example, "foo" and "Foo" are personal words, plus Foo will be
48 recognized with affixes of Simpson (Foo's etc.) and bar is a forbidden
49 word.
50
53 Dictionary file:
54 3
56 hello
57 try/B
58 work/AB
59 The flags B and A specify attributes of these words.
61 Affix file:
63 SET UTF-8
66 TRY esianrtolcdugmphbyfvkwzESIANRTOLCDUGMPHBYFVKWZ'
67 REP 2
69 REP f ph
70 REP ph f
71 PFX A Y 1
73 PFX A 0 re .
74 SFX B Y 2
76 SFX B 0 ed [^y]
77 SFX B y ied y
78 In the affix file, prefix A and suffix B have been defined. Flag A
80 defines a `re-' prefix. Class B defines two `-ed' suffixes. First B
81 suffix can be added to a word if the last character of the word isn't
82 `y'. Second suffix can be added to the words terminated with an `y'.
83 All accepted words with this dictionary and affix combination are:
85 "hello", "try", "tried", "work", "worked", "rework", "reworked".
86
89 Hunspell source distribution contains more than 80 examples for option
90 usage.
91 SET encoding
94 Set character encoding of words and morphemes in affix and dic‐
95 tionary files. Possible values: UTF-8, ISO8859-1 - ISO8859-10,
96 ISO8859-13 - ISO8859-15, KOI8-R, KOI8-U, cp1251, ISCII-DEVANA‐
97 GARI.
98 SET UTF-8
100 FLAG value
102 Set flag type. Default type is the extended ASCII (8-bit) char‐
103 acter. `UTF-8' parameter sets UTF-8 encoded Unicode character
104 flags. The `long' value sets the double extended ASCII charac‐
105 ter flag type, the `num' sets the decimal number flag type. Dec‐
106 imal flags numbered from 1 to 65000, and in flag fields are sep‐
107 arated by comma. BUG: UTF-8 flag type doesn't work on ARM plat‐
108 form.
109 FLAG long
111 COMPLEXPREFIXES
113 Set twofold prefix stripping (but single suffix stripping) eg.
114 for morphologically complex languages with right-to-left writing
115 system.
116 LANG langcode
119 Set language code for language-specific functions of Hunspell.
120 Use it to activate special casing of Azeri (LANG az), Turkish
121 (LANG tr) and Crimean Tatar (LANG crh), also not generalized
122 syllable-counting compounding rules of Hungarian (LANG hu).
123 IGNORE characters
126 Sets characters to ignore dictionary words, affixes and input
127 words. Useful for optional characters, as Arabic (harakat) or
128 Hebrew (niqqud) diacritical marks (see tests/ignore.* test dic‐
129 tionary in Hunspell distribution).
130 AF number_of_flag_vector_aliases
133 AF flag_vector
135 Hunspell can substitute affix flag sets with ordinal numbers in
136 affix rules (alias compression, see makealias tool). First exam‐
137 ple with alias compression:
138 3
140 hello
141 try/1
142 work/2
143 AF definitions in the affix file:
145 AF 2
147 AF A
148 AF AB
149 It is equivalent of the following dic file:
151 3
153 hello
154 try/A
155 work/AB
156 See also tests/alias* examples of the source distribution.
158 Note I: If affix file contains the FLAG parameter, define it before the
160 AF definitions.
161 Note II: Use makealias utility in Hunspell distribution to compress aff
163 and dic files.
164 AM number_of_morphological_aliases
166 AM morphological_fields
168 Hunspell can substitute also morphological data with ordinal
169 numbers in affix rules (alias compression). See tests/alias*
170 examples.
171
173 Suggestion parameters can optimize the default n-gram (similarity
174 search in the dictionary words based on the common 1, 2, 3, 4-character
175 length common character-sequences), character swap and deletion sugges‐
176 tions of Hunspell. REP is suggested to fix the typical and especially
177 bad language specific bugs, because the REP suggestions have the high‐
178 est priority in the suggestion list. PHONE is for languages with not
179 pronunciation based orthography.
180 For short common misspellings, it's important to use the ph: field (see
182 later) to give the best suggestions.
183 KEY characters_separated_by_vertical_line_optionally
185 Hunspell searches and suggests words with one different charac‐
186 ter replaced by a neighbor KEY character. Not neighbor charac‐
187 ters in KEY string separated by vertical line characters. Sug‐
188 gested KEY parameters for QWERTY and Dvorak keyboard layouts:
189 KEY qwertyuiop|asdfghjkl|zxcvbnm
191 KEY pyfgcrl|aeouidhtns|qjkxbmwvz
192 Using the first QWERTY layout, Hunspell suggests "nude" and "node" for
194 "*nide". A character may have more neighbors, too:
195 KEY qwertzuop|yxcvbnm|qaw|say|wse|dsx|sy|edr|fdc|dx|rft|gfv|fc|tgz|hgb|gv|zhu|jhn|hb|uji|kjm|jn|iko|lkm
197 TRY characters
199 Hunspell can suggest right word forms, when they differ from the
200 bad input word by one TRY character. The parameter of TRY is
201 case sensitive.
202 NOSUGGEST flag
204 Words signed with NOSUGGEST flag are not suggested (but still
205 accepted when typed correctly). Proposed flag for vulgar and
206 obscene words (see also SUBSTANDARD).
207 MAXCPDSUGS num
209 Set max. number of suggested compound words generated by com‐
210 pound rules. The number of the suggested compound words may be
211 greater from the same 1-character distance type.
212 MAXNGRAMSUGS num
214 Set max. number of n-gram suggestions. Value 0 switches off the
215 n-gram suggestions (see also MAXDIFF).
216 MAXDIFF [0-10]
218 Set the similarity factor for the n-gram based suggestions (5 =
219 default value; 0 = fewer n-gram suggestions, but min. 1; 10 =
220 MAXNGRAMSUGS n-gram suggestions).
221 ONLYMAXDIFF
223 Remove all bad n-gram suggestions (default mode keeps one, see
224 MAXDIFF).
225 NOSPLITSUGS
227 Disable word suggestions with spaces.
228 SUGSWITHDOTS
230 Add dot(s) to suggestions, if input word terminates in dot(s).
231 (Not for LibreOffice dictionaries, because LibreOffice has an
232 automatic dot expansion mechanism.)
233 REP number_of_replacement_definitions
235 REP what replacement
237 This table specifies modifications to try first. First REP is
238 the header of this table and one or more REP data line are fol‐
239 lowing it. With this table, Hunspell can suggest the right
240 forms for the typical spelling mistakes when the incorrect form
241 differs by more than 1 letter from the right form (see also
242 "ph:"). The search string supports the regex boundary signs (^
243 and $). For example a possible English replacement table defi‐
244 nition to handle misspelled consonants:
245 REP 5
247 REP f ph
248 REP ph f
249 REP tion$ shun
250 REP ^cooccurr co-occurr
251 REP ^alot$ a_lot
252 Note I: It's very useful to define replacements for the most typical
254 one-character mistakes, too: with REP you can add higher priority to a
255 subset of the TRY suggestions (suggestion list begins with the REP sug‐
256 gestions).
257 Note II: Suggesting separated words, specify spaces with underlines:
259 REP 1
262 REP onetwothree one_two_three
263 Note III: Replacement table can be used for a stricter compound word
265 checking with the option CHECKCOMPOUNDREP.
266 MAP number_of_map_definitions
269 MAP string_of_related_chars_or_parenthesized_character_sequences
271 We can define language-dependent information on characters and
272 character sequences that should be considered related (i.e.
273 nearer than other chars not in the set) in the affix file (.aff)
274 by a map table. With this table, Hunspell can suggest the right
275 forms for words, which incorrectly choose the wrong letter or
276 letter groups from a related set more than once in a word (see
277 REP).
278 For example a possible mapping could be for the German umlauted
280 ü versus the regular u; the word Frühstück really should be
281 written with umlauted u's and not regular ones
282 MAP 1
284 MAP uü
285 Use parenthesized groups for character sequences (eg. for composed Uni‐
287 code characters):
288 MAP 3
290 MAP ß(ss) (character sequence)
291 MAP fi(fi) ("fi" compatibility characters for Unicode fi ligature)
292 MAP (ọ́)o (composed Unicode character: ó with bottom dot)
293 PHONE number_of_phone_definitions
295 PHONE what replacement
297 PHONE uses a table-driven phonetic transcription algorithm bor‐
298 rowed from Aspell. It is useful for languages with not pronunci‐
299 ation based orthography. You can add a full alphabet conversion
300 and other rules for conversion of special letter sequences. For
301 detailed documentation see http://aspell.net/man-html/Phonetic-
302 Code.html. Note: Multibyte UTF-8 characters have not worked
303 with bracket expression yet. Dash expression has signed bytes
304 and not UTF-8 characters yet.
305 WARN flag
307 This flag is for rare words, which are also often spelling mis‐
308 takes, see option -r of command line Hunspell and FORBIDWARN.
309 FORBIDWARN
311 Words with flag WARN aren't accepted by the spell checker using
312 this parameter.
313
315 BREAK number_of_break_definitions
316 BREAK character_or_character_sequence
318 Define new break points for breaking words and checking word
319 parts separately. Use ^ and $ to delete characters at end and
320 start of the word. Rationale: useful for compounding with join‐
321 ing character or strings (for example, hyphen in English and
322 German or hyphen and n-dash in Hungarian). Dashes are often bad
323 break points for tokenization, because compounds with dashes may
324 contain not valid parts, too.) With BREAK, Hunspell can check
325 both side of these compounds, breaking the words at dashes and
326 n-dashes:
327 BREAK 2
329 BREAK -
330 BREAK -- # n-dash
331 Breaking are recursive, so foo-bar, bar-foo and foo-foo--bar-bar would
333 be valid compounds. Note: The default word break of Hunspell is equiv‐
334 alent of the following BREAK definition:
335 BREAK 3
337 BREAK -
338 BREAK ^-
339 BREAK -$
340 Hunspell doesn't accept the "-word" and "word-" forms by this BREAK
342 definition:
343 BREAK 1
345 BREAK -
346 Switching off the default values:
348 BREAK 0
350 Note II: COMPOUNDRULE is better for handling dashes and other compound
352 joining characters or character strings. Use BREAK, if you want to
353 check words with dashes or other joining characters and there is no
354 time or possibility to describe precise compound rules with COM‐
355 POUNDRULE (COMPOUNDRULE handles only the suffixation of the last word
356 part of a compound word).
357 Note III: For command line spell checking of words with extra charac‐
359 ters, set WORDCHARS parameters: WORDCHARS --- (see tests/break.*) exam‐
360 ple
361 COMPOUNDRULE number_of_compound_definitions
363 COMPOUNDRULE compound_pattern
365 Define custom compound patterns with a regex-like syntax. The
366 first COMPOUNDRULE is a header with the number of the following
367 COMPOUNDRULE definitions. Compound patterns consist compound
368 flags, parentheses, star and question mark meta characters. A
369 flag followed by a `*' matches a word sequence of 0 or more
370 matches of words signed with this compound flag. A flag fol‐
371 lowed by a `?' matches a word sequence of 0 or 1 matches of a
372 word signed with this compound flag. See tests/compound*.*
373 examples.
374 Note: en_US dictionary of OpenOffice.org uses COMPOUNDRULE for
376 ordinal number recognition (1st, 2nd, 11th, 12th, 22nd, 112th,
377 1000122nd etc.).
378 Note II: In the case of long and numerical flag types use only
380 parenthesized flags: (1500)*(2000)?
381 Note III: COMPOUNDRULE flags work completely separately from the
383 compounding mechanisms using COMPOUNDFLAG, COMPOUNDBEGIN, etc.
384 compound flags. (Use these flags on different entries for
385 words).
386 COMPOUNDMIN num
389 Minimum length of words used for compounding. Default value is
390 3 letters.
391 COMPOUNDFLAG flag
393 Words signed with COMPOUNDFLAG may be in compound words (except
394 when word shorter than COMPOUNDMIN). Affixes with COMPOUNDFLAG
395 also permits compounding of affixed words.
396 COMPOUNDBEGIN flag
398 Words signed with COMPOUNDBEGIN (or with a signed affix) may be
399 first elements in compound words.
400 COMPOUNDLAST flag
402 Words signed with COMPOUNDLAST (or with a signed affix) may be
403 last elements in compound words.
404 COMPOUNDMIDDLE flag
406 Words signed with COMPOUNDMIDDLE (or with a signed affix) may be
407 middle elements in compound words.
408 ONLYINCOMPOUND flag
410 Suffixes signed with ONLYINCOMPOUND flag may be only inside of
411 compounds (Fuge-elements in German, fogemorphemes in Swedish).
412 ONLYINCOMPOUND flag works also with words (see tests/onlyincom‐
413 pound.*). Note: also valuable to flag compounding parts which
414 are not correct as a word by itself.
415 COMPOUNDPERMITFLAG flag
417 Prefixes are allowed at the beginning of compounds, suffixes are
418 allowed at the end of compounds by default. Affixes with COM‐
419 POUNDPERMITFLAG may be inside of compounds.
420 COMPOUNDFORBIDFLAG flag
422 Suffixes with this flag forbid compounding of the affixed word.
423 Dictionary words with this flag are removed from the beginning
424 and middle of compound words, overriding the effect of COMPOUND‐
425 PERMITFLAG.
426 COMPOUNDMORESUFFIXES
428 Allow twofold suffixes within compounds.
429 COMPOUNDROOT flag
431 COMPOUNDROOT flag signs the compounds in the dictionary (Now it
432 is used only in the Hungarian language specific code).
433 COMPOUNDWORDMAX number
435 Set maximum word count in a compound word. (Default is unlim‐
436 ited.)
437 CHECKCOMPOUNDDUP
439 Forbid word duplication in compounds (e.g. foofoo).
440 CHECKCOMPOUNDREP
442 Forbid compounding, if the (usually bad) compound word may be a
443 non-compound word with a REP fault. Useful for languages with
444 `compound friendly' orthography.
445 CHECKCOMPOUNDCASE
447 Forbid upper case characters at word boundaries in compounds.
448 CHECKCOMPOUNDTRIPLE
450 Forbid compounding, if compound word contains triple repeating
451 letters (e.g. foo|ox or xo|oof). Bug: missing multi-byte charac‐
452 ter support in UTF-8 encoding (works only for 7-bit ASCII char‐
453 acters).
454 SIMPLIFIEDTRIPLE
456 Allow simplified 2-letter forms of the compounds forbidden by
457 CHECKCOMPOUNDTRIPLE. It's useful for Swedish and Norwegian (and
458 for the old German orthography: Schiff|fahrt -> Schiffahrt).
459 CHECKCOMPOUNDPATTERN number_of_checkcompoundpattern_definitions
461 CHECKCOMPOUNDPATTERN endchars[/flag] beginchars[/flag] [replacement]
463 Forbid compounding, if the first word in the compound ends with
464 endchars, and next word begins with beginchars and (optionally)
465 they have the requested flags. The optional replacement parame‐
466 ter allows simplified compound form.
467 The special "endchars" pattern 0 (zero) limits the rule to the
469 unmodified stems (stems and stems with zero affixes):
470 CHECKCOMPOUNDPATTERN 0/x /y
472 Note: COMPOUNDMIN doesn't work correctly with the compound word alter‐
474 nation, so it may need to set COMPOUNDMIN to lower value.
475 FORCEUCASE flag
477 Last word part of a compound with flag FORCEUCASE forces capi‐
478 talization of the whole compound word. Eg. Dutch word "straat"
479 (street) with FORCEUCASE flags will allowed only in capitalized
480 compound forms, according to the Dutch spelling rules for proper
481 names.
482 COMPOUNDSYLLABLE max_syllable vowels
484 Need for special compounding rules in Hungarian. First parame‐
485 ter is the maximum syllable number, that may be in a compound,
486 if words in compounds are more than COMPOUNDWORDMAX. Second
487 parameter is the list of vowels (for calculating syllables).
488 SYLLABLENUM flags
490 Need for special compounding rules in Hungarian.
491
493 PFX flag cross_product number
494 PFX flag stripping prefix [condition [morphological_fields...]]
496 SFX flag cross_product number
498 SFX flag stripping suffix [condition [morphological_fields...]]
500 An affix is either a prefix or a suffix attached to root words
501 to make other words. We can define affix classes with arbitrary
502 number affix rules. Affix classes are signed with affix flags.
503 The first line of an affix class definition is the header. The
504 fields of an affix class header:
505 (0) Option name (PFX or SFX)
507 (1) Flag (name of the affix class)
509 (2) Cross product (permission to combine prefixes and suffixes).
511 Possible values: Y (yes) or N (no)
512 (3) Line count of the following rules.
514 Fields of an affix rules:
516 (0) Option name
518 (1) Flag
520 (2) stripping characters from beginning (at prefix rules) or end
522 (at suffix rules) of the word
523 (3) affix (optionally with flags of continuation classes, sepa‐
525 rated by a slash)
526 (4) condition.
528 Zero stripping or affix are indicated by zero. Zero condition is
530 indicated by dot. Condition is a simplified, regular expres‐
531 sion-like pattern, which must be met before the affix can be
532 applied. (Dot signs an arbitrary character. Characters in braces
533 sign an arbitrary character from the character subset. Dash
534 hasn't got special meaning, but circumflex (^) next the first
535 brace sets the complementer character set.)
536 (5) Optional morphological fields separated by spaces or tabula‐
538 tors.
539
542 CIRCUMFIX flag
543 Affixes signed with CIRCUMFIX flag may be on a word when this
544 word also has a prefix with CIRCUMFIX flag and vice versa (see
545 circumfix.* test files in the source distribution).
546 FORBIDDENWORD flag
548 This flag signs forbidden word form. Because affixed forms are
549 also forbidden, we can subtract a subset from set of the
550 accepted affixed and compound words. Note: usefull to forbid
551 erroneous words, generated by the compounding mechanism.
552 FULLSTRIP
554 With FULLSTRIP, affix rules can strip full words, not only one
555 less characters, before adding the affixes, see fullstrip.* test
556 files in the source distribution). Note: conditions may be word
557 length without FULLSTRIP, too.
558 KEEPCASE flag
560 Forbid uppercased and capitalized forms of words signed with
561 KEEPCASE flags. Useful for special orthographies (measurements
562 and currency often keep their case in uppercased texts) and
563 writing systems (e.g. keeping lower case of IPA characters).
564 Also valuable for words erroneously written in the wrong case.
565 Note: With CHECKSHARPS declaration, words with sharp s and KEEP‐
567 CASE flag may be capitalized and uppercased, but uppercased
568 forms of these words may not contain sharp s, only SS. See ger‐
569 mancompounding example in the tests directory of the Hunspell
570 distribution.
571 ICONV number_of_ICONV_definitions
574 ICONV pattern pattern2
576 Define input conversion table. Note: useful to convert one type
577 of quote to another one, or change ligature.
578 OCONV number_of_OCONV_definitions
580 OCONV pattern pattern2
582 Define output conversion table.
583 LEMMA_PRESENT flag
585 Deprecated. Use "st:" field instead of LEMMA_PRESENT.
586 NEEDAFFIX flag
588 This flag signs virtual stems in the dictionary, words only
589 valid when affixed. Except, if the dictionary word has a
590 homonym or a zero affix. NEEDAFFIX works also with prefixes and
591 prefix + suffix combinations (see tests/needaffix5.*).
592 PSEUDOROOT flag
594 Deprecated. (Former name of the NEEDAFFIX option.)
595 SUBSTANDARD flag
597 SUBSTANDARD flag signs affix rules and dictionary words (allo‐
598 morphs) not used in morphological generation and root words
599 removed from suggestion. See also NOSUGGEST.
600 WORDCHARS characters
602 WORDCHARS extends tokenizer of Hunspell command line interface
603 with additional word character. For example, dot, dash, n-dash,
604 numbers, percent sign are word character in Hungarian.
605 CHECKSHARPS
607 SS letter pair in uppercased (German) words may be upper case
608 sharp s (ß). Hunspell can handle this special casing with the
609 CHECKSHARPS declaration (see also KEEPCASE flag and tests/ger‐
610 mancompounding example) in both spelling and suggestion.
611
614 Hunspell's dictionary items and affix rules may have optional space or
615 tabulator separated morphological description fields, started with
616 3-character (two letters and a colon) field IDs:
617 word/flags po:noun is:nom
620 Example: We define a simple resource with morphological informations, a
622 derivative suffix (ds:) and a part of speech category (po:):
623 Affix file:
625 SFX X Y 1
628 SFX X 0 able . ds:able
629 Dictionary file:
631 drink/X po:verb
634 Test file:
636 drink
639 drinkable
640 Test:
642 $ analyze test.aff test.dic test.txt
645 > drink
646 analyze(drink) = po:verb
647 stem(drink) = po:verb
648 > drinkable
649 analyze(drinkable) = po:verb ds:able
650 stem(drinkable) = drinkable
651 You can see in the example, that the analyzer concatenates the morpho‐
653 logical fields in item and arrangement style.
654
657 Default morphological and other IDs (used in suggestion, stemming and
658 morphological generation):
659 ph: Alternative transliteration for better suggestions, ie. mis‐
661 spellings related to the special orthography and pronunciation
662 of the word. The best way to handle common misspellings, so it's
663 worth to add ph: field to the most affected few thousand dictio‐
664 nary words (or word pairs etc.) to get correct suggestions for
665 their misspellings.
666 For example:
669 Wednesday ph:wendsay ph:wensday
672 Marseille ph:maarsayl
673 Hunspell adds all ph: transliterations to the inner REP table, so it
675 will always suggest the correct word for the specified misspellings
676 with the highest priority.
677 The previous example is equivalent of the following REP definition:
679 REP 6
682 REP wendsay Wednesday
683 REP Wendsay Wednesday
684 REP wensday Wednesday
685 REP Wensday Wednesday
686 REP maarsayl Marseille
687 REP Maarsayl Marseille
688 The asterisk at the end of the ph: pattern means stripping the termi‐
690 nating character both from the pattern and the word in the associated
691 REP rule:
692 pretty ph:prity*
695 will result
697 REP 1
700 REP prit prett
701 REP rule, resulting the following correct suggestions
703 *prity -> pretty
706 *pritier -> prettier
707 *pritiest -> prettiest
708 Moreover, ph: fields can handle suggestions with more than two words,
710 also different suggestions for the same misspelling:
711 do not know ph:dunno
713 don't know ph:dunno
714 results
716 *dunno -> do not know, don't know
719 Note: if available, ph: is used in n-gram similarity, too.
721 The ASCII arrow "->" in a ph: pattern means a REP rule (see REP), cre‐
723 ating arbitrary replacement rule associated to the dictionary item:
724 happy/B ph:hepy ph:hepi->happi
726 results
728 *hepy -> happy
731 *hepiest -> happiest
732 st: Stem. Optional: default stem is the dictionary item in morpho‐
734 logical analysis. Stem field is useful for virtual stems (dic‐
735 tionary words with NEEDAFFIX flag) and morphological exceptions
736 instead of new, single used morphological rules.
737 feet st:foot is:plural
739 mice st:mouse is:plural
740 teeth st:tooth is:plural
741 Word forms with multiple stems need multiple dictionary items:
743 lay po:verb st:lie is:past_2
746 lay po:verb is:present
747 lay po:noun
748 al: Allomorph(s). A dictionary item is the stem of its allomorphs.
750 Morphological generation needs stem, allomorph and affix fields.
751 sing al:sang al:sung
753 sang st:sing
754 sung st:sing
755 po: Part of speech category.
757 ds: Derivational suffix(es). Stemming doesn't remove derivational
759 suffixes. Morphological generation depends on the order of the
760 suffix fields.
761 In affix rules:
763 SFX Y Y 1
766 SFX Y 0 ly . ds:ly_adj
767 In the dictionary:
769 ably st:able ds:ly_adj
772 able al:ably
773 is: Inflectional suffix(es). All inflectional suffixes are removed
775 by stemming. Morphological generation depends on the order of
776 the suffix fields.
777 feet st:foot is:plural
780 ts: Terminal suffix(es). Terminal suffix fields are inflectional
782 suffix fields "removed" by additional (not terminal) suffixes.
783 Useful for zero morphemes and affixes removed by splitting
785 rules.
786 work/D ts:present
789 SFX D Y 2
791 SFX D 0 ed . is:past_1
792 SFX D 0 ed . is:past_2
793 Typical example of the terminal suffix is the zero morpheme of the nom‐
795 inative case.
796 sp: Surface prefix. Temporary solution for adding prefixes to the
799 stems and generated word forms. See tests/morph.* example.
800 pa: Parts of the compound words. Output fields of morphological
803 analysis for stemming.
804 dp: Planned: derivational prefix.
806 ip: Planned: inflectional prefix.
808 tp: Planned: terminal prefix.
810
813 Ispell's original algorithm strips only one suffix. Hunspell can strip
814 another one yet (or a plus prefix in COMPLEXPREFIXES mode).
815 The twofold suffix stripping is a significant improvement in handling
817 of immense number of suffixes, that characterize agglutinative lan‐
818 guages.
819 A second `s' suffix (affix class Y) will be the continuation class of
821 the suffix `able' in the following example:
822 SFX Y Y 1
825 SFX Y 0 s .
826 SFX X Y 1
828 SFX X 0 able/Y .
829 Dictionary file:
831 drink/X
834 Test file:
836 drink
839 drinkable
840 drinkables
841 Test:
843 $ hunspell -m -d test <test.txt
846 drink st:drink
847 drinkable st:drink fl:X
848 drinkables st:drink fl:X fl:Y
849 Theoretically with the twofold suffix stripping needs only the square
851 root of the number of suffix rules, compared with a Hunspell implemen‐
852 tation. In our practice, we could have elaborated the Hungarian inflec‐
853 tional morphology with twofold suffix stripping.
854
857 Hunspell can handle more than 65000 affix classes. There are three new
858 syntax for giving flags in affix and dictionary files.
859 FLAG long command sets 2-character flags:
861 FLAG long
864 SFX Y1 Y 1
865 SFX Y1 0 s 1
866 Dictionary record with the Y1, Z3, F? flags:
868 foo/Y1Z3F?
871 FLAG num command sets numerical flags separated by comma:
873 FLAG num
876 SFX 65000 Y 1
877 SFX 65000 0 s 1
878 Dictionary example:
880 foo/65000,12,2756
883 The third one is the Unicode character flags.
885
888 Hunspell's dictionary can contain repeating elements that are homonyms:
889 work/A po:verb
892 work/B po:noun
893 An affix file:
895 SFX A Y 1
898 SFX A 0 s . sf:sg3
899 SFX B Y 1
901 SFX B 0 s . is:plur
902 Test file:
904 works
907 Test:
909 $ hunspell -d test -m <testwords
912 work st:work po:verb is:sg3
913 work st:work po:noun is:plur
914 This feature also gives a way to forbid illegal prefix/suffix combina‐
916 tions.
917
920 An interesting side-effect of multi-step stripping is, that the appro‐
921 priate treatment of circumfixes now comes for free. For instance, in
922 Hungarian, superlatives are formed by simultaneous prefixation of leg-
923 and suffixation of -bb to the adjective base. A problem with the one-
924 level architecture is that there is no way to render lexical licensing
925 of particular prefixes and suffixes interdependent, and therefore
926 incorrect forms are recognized as valid, i.e. *legvén = leg + vén
927 `old'. Until the introduction of clusters, a special treatment of the
928 superlative had to be hardwired in the earlier HunSpell code. This may
929 have been legitimate for a single case, but in fact prefix--suffix
930 dependences are ubiquitous in category-changing derivational patterns
931 (cf. English payable, non-payable but *non-pay or drinkable, undrink‐
932 able but *undrink). In simple words, here, the prefix un- is legitimate
933 only if the base drink is suffixed with -able. If both these patters
934 are handled by on-line affix rules and affix rules are checked against
935 the base only, there is no way to express this dependency and the sys‐
936 tem will necessarily over- or undergenerate.
937 In next example, suffix class R have got a prefix `continuation' class
939 (class P).
940 PFX P Y 1
943 PFX P 0 un . [prefix_un]+
944 SFX S Y 1
946 SFX S 0 s . +PL
947 SFX Q Y 1
949 SFX Q 0 s . +3SGV
950 SFX R Y 1
952 SFX R 0 able/PS . +DER_V_ADJ_ABLE
953 Dictionary:
955 2
958 drink/RQ [verb]
959 drink/S [noun]
960 Morphological analysis:
962 > drink
965 drink[verb]
966 drink[noun]
967 > drinks
968 drink[verb]+3SGV
969 drink[noun]+PL
970 > drinkable
971 drink[verb]+DER_V_ADJ_ABLE
972 > drinkables
973 drink[verb]+DER_V_ADJ_ABLE+PL
974 > undrinkable
975 [prefix_un]+drink[verb]+DER_V_ADJ_ABLE
976 > undrinkables
977 [prefix_un]+drink[verb]+DER_V_ADJ_ABLE+PL
978 > undrink
979 Unknown word.
980 > undrinks
981 Unknown word.
982
984 Conditional affixes implemented by a continuation class are not enough
985 for circumfixes, because a circumfix is one affix in morphology. We
986 also need CIRCUMFIX option for correct morphological analysis.
987 # circumfixes: ~ obligate prefix/suffix combinations
990 # superlative in Hungarian: leg- (prefix) AND -bb (suffix)
991 # nagy, nagyobb, legnagyobb, legeslegnagyobb
992 # (great, greater, greatest, most greatest)
993 CIRCUMFIX X
995 PFX A Y 1
997 PFX A 0 leg/X .
998 PFX B Y 1
1000 PFX B 0 legesleg/X .
1001 SFX C Y 3
1003 SFX C 0 obb . +COMPARATIVE
1004 SFX C 0 obb/AX . +SUPERLATIVE
1005 SFX C 0 obb/BX . +SUPERSUPERLATIVE
1006 Dictionary:
1008 1
1011 nagy/C [MN]
1012 Analysis:
1014 > nagy
1017 nagy[MN]
1018 > nagyobb
1019 nagy[MN]+COMPARATIVE
1020 > legnagyobb
1021 nagy[MN]+SUPERLATIVE
1022 > legeslegnagyobb
1023 nagy[MN]+SUPERSUPERLATIVE
1024
1026 Allowing free compounding yields decrease in precision of recognition,
1027 not to mention stemming and morphological analysis. Although lexical
1028 switches are introduced to license compounding of bases by Ispell, this
1029 proves not to be restrictive enough. For example:
1030 # affix file
1033 COMPOUNDFLAG X
1034 2
1036 foo/X
1037 bar/X
1038 With this resource, foobar and barfoo also are accepted words.
1040 This has been improved upon with the introduction of direction-sensi‐
1042 tive compounding, i.e., lexical features can specify separately whether
1043 a base can occur as leftmost or rightmost constituent in compounds.
1044 This, however, is still insufficient to handle the intricate patterns
1045 of compounding, not to mention idiosyncratic (and language specific)
1046 norms of hyphenation.
1047 The Hunspell algorithm currently allows any affixed form of words,
1049 which are lexically marked as potential members of compounds. Hunspell
1050 improved this, and its recursive compound checking rules makes it pos‐
1051 sible to implement the intricate spelling conventions of Hungarian com‐
1052 pounds. For example, using COMPOUNDWORDMAX, COMPOUNDSYLLABLE, COMPOUND‐
1053 ROOT, SYLLABLENUM options can be set the noteworthy Hungarian `6-3'
1054 rule. Further example in Hungarian, derivate suffixes often modify
1055 compounding properties. Hunspell allows the compounding flags on the
1056 affixes, and there are two special flags (COMPOUNDPERMITFLAG and (COM‐
1057 POUNDFORBIDFLAG) to permit or prohibit compounding of the derivations.
1058 Suffixes with this flag forbid compounding of the affixed word.
1060 We also need several Hunspell features for handling German compounding:
1062 # German compounding
1065 # set language to handle special casing of German sharp s
1067 LANG de_DE
1069 # compound flags
1071 COMPOUNDBEGIN U
1073 COMPOUNDMIDDLE V
1074 COMPOUNDEND W
1075 # Prefixes are allowed at the beginning of compounds,
1077 # suffixes are allowed at the end of compounds by default:
1078 # (prefix)?(root)+(affix)?
1079 # Affixes with COMPOUNDPERMITFLAG may be inside of compounds.
1080 COMPOUNDPERMITFLAG P
1081 # for German fogemorphemes (Fuge-element)
1083 # Hint: ONLYINCOMPOUND is not required everywhere, but the
1084 # checking will be a little faster with it.
1085 ONLYINCOMPOUND X
1087 # forbid uppercase characters at compound word bounds
1089 CHECKCOMPOUNDCASE
1090 # for handling Fuge-elements with dashes (Arbeits-)
1092 # dash will be a special word
1093 COMPOUNDMIN 1
1095 WORDCHARS -
1096 # compound settings and fogemorpheme for `Arbeit'
1098 SFX A Y 3
1100 SFX A 0 s/UPX .
1101 SFX A 0 s/VPDX .
1102 SFX A 0 0/WXD .
1103 SFX B Y 2
1105 SFX B 0 0/UPX .
1106 SFX B 0 0/VWXDP .
1107 # a suffix for `Computer'
1109 SFX C Y 1
1111 SFX C 0 n/WD .
1112 # for forbid exceptions (*Arbeitsnehmer)
1114 FORBIDDENWORD Z
1116 # dash prefix for compounds with dash (Arbeits-Computer)
1118 PFX - Y 1
1120 PFX - 0 -/P .
1121 # decapitalizing prefix
1123 # circumfix for positioning in compounds
1124 PFX D Y 29
1126 PFX D A a/PX A
1127 PFX D Ä ä/PX Ä
1128 .
1129 .
1130 PFX D Y y/PX Y
1131 PFX D Z z/PX Z
1132 Example dictionary:
1134 4
1137 Arbeit/A-
1138 Computer/BC-
1139 -/W
1140 Arbeitsnehmer/Z
1141 Accepted compound compound words with the previous resource:
1143 Computer
1146 Computern
1147 Arbeit
1148 Arbeits-
1149 Computerarbeit
1150 Computerarbeits-
1151 Arbeitscomputer
1152 Arbeitscomputern
1153 Computerarbeitscomputer
1154 Computerarbeitscomputern
1155 Arbeitscomputerarbeit
1156 Computerarbeits-Computer
1157 Computerarbeits-Computern
1158 Not accepted compoundings:
1160 computer
1163 arbeit
1164 Arbeits
1165 arbeits
1166 ComputerArbeit
1167 ComputerArbeits
1168 Arbeitcomputer
1169 ArbeitsComputer
1170 Computerarbeitcomputer
1171 ComputerArbeitcomputer
1172 ComputerArbeitscomputer
1173 Arbeitscomputerarbeits
1174 Computerarbeits-computer
1175 Arbeitsnehmer
1176 This solution is still not ideal, however, and will be replaced by a
1178 pattern-based compound-checking algorithm which is closely integrated
1179 with input buffer tokenization. Patterns describing compounds come as a
1180 separate input resource that can refer to high-level properties of con‐
1181 stituent parts (e.g. the number of syllables, affix flags, and contain‐
1182 ment of hyphens). The patterns are matched against potential segmenta‐
1183 tions of compounds to assess wellformedness.
1184
1187 Both Ispell and Myspell use 8-bit ASCII character encoding, which is a
1188 major deficiency when it comes to scalability. Although a language
1189 like Hungarian has a standard ASCII character set (ISO 8859-2), it
1190 fails to allow a full implementation of Hungarian orthographic conven‐
1191 tions. For instance, the '--' symbol (n-dash) is missing from this
1192 character set contrary to the fact that it is not only the official
1193 symbol to delimit parenthetic clauses in the language, but it can be in
1194 compound words as a special 'big' hyphen.
1195 MySpell has got some 8-bit encoding tables, but there are languages
1197 without standard 8-bit encoding, too. For example, a lot of African
1198 languages have non-latin or extended latin characters.
1199 Similarly, using the original spelling of certain foreign names like
1201 Ångström or Molière is encouraged by the Hungarian spelling norm, and,
1202 since characters 'Å' and 'è' are not part of ISO 8859-2, when they com‐
1203 bine with inflections containing characters only in ISO 8859-2 (like
1204 elative -ből, allative -től or delative -ről with double acute), these
1205 result in words (like Ångströmről or Molière-től.) that can not be
1206 encoded using any single ASCII encoding scheme.
1207 The problems raised in relation to 8-bit ASCII encoding have long been
1209 recognized by proponents of Unicode. It is clear that trading effi‐
1210 ciency for encoding-independence has its advantages when it comes a
1211 truly multi-lingual application. There is implemented a memory and time
1212 efficient Unicode handling in Hunspell. In non-UTF-8 character encod‐
1213 ings Hunspell works with the original 8-bit strings. In UTF-8 encoding,
1214 affixes and words are stored in UTF-8, during the analysis are handled
1215 in mostly UTF-8, under condition checking and suggestion are converted
1216 to UTF-16. Unicode text analysis and spell checking have a minimal
1217 (0-20%) time overhead and minimal or reasonable memory overhead depends
1218 from the language (its UTF-8 encoding and affixation).
1219
1222 Aspell dictionaries can be easily converted into hunspell. Conversion
1223 steps:
1224 dictionary (xx.cwl -> xx.wl):
1226 preunzip xx.cwl
1228 wc -l < xx.wl > xx.dic
1229 cat xx.wl >> xx.dic
1230 affix file
1232 If the affix file exists, copy it:
1234 cp xx_affix.dat xx.aff
1235 If not, create it with the suitable character encoding (see xx.dat)
1236 echo "SET ISO8859-x" > xx.aff
1237 or
1238 echo "SET UTF-8" > xx.aff
1239 It's useful to add a TRY option with the characters of the dictionary
1241 with frequency order to set edit distance suggestions:
1242 echo "TRY qwertzuiopasdfghjklyxcvbnmQWERTZUIOPASDFGHJKLYXCVBNM" >>xx.aff
1243
1246 hunspell (1), ispell (1), ispell (4)
1247 2017-09-20 hunspell(5)