1LucyX::Index::LongFieldUSsiemr(3C)ontributed Perl DocumeLnutcaytXi:o:nIndex::LongFieldSim(3)
2
3
4

NAME

6       LucyX::Index::LongFieldSim - Similarity optimized for long fields.
7

SYNOPSIS

9           package MySchema::body;
10           use base qw( Lucy::Plan::FullTextType );
11           use LucyX::Index::LongFieldSim;
12           sub make_similarity { LucyX::Index::LongFieldSim->new }
13

DESCRIPTION

15       Apache Lucy's default Similarity implmentation produces a bias towards
16       extremely short fields.
17
18           Lucy::Index::Similarity
19
20           | more weight
21           | *
22           |  **
23           |    ***
24           |       **********
25           |                 ********************
26           |                                     *******************************
27           | less weight                                                        ****
28           |------------------------------------------------------------------------
29             fewer tokens                                              more tokens
30
31       LongFieldSim eliminates this bias.
32
33           LucyX::Index::LongFieldSim
34
35           | more weight
36           |
37           |
38           |
39           |*****************
40           |                 ********************
41           |                                     *******************************
42           | less weight                                                        ****
43           |------------------------------------------------------------------------
44             fewer tokens                                              more tokens
45
46       In most cases, the default bias towards short fields is desirable.  For
47       instance, say you have two documents:
48
49       •   "George Washington"
50
51       •   "George Washington Carver"
52
53       If a user searches for "george washington", we want the exact title
54       match to appear first.  Under the default Similarity implementation it
55       will, because the "Carver" in "George Washington Carver" dilutes the
56       impact of the other two tokens.
57
58       However, under LongFieldSim, the two titles will yield equal scores.
59       That would be bad in this particular case, but it could be good in
60       another.
61
62            "George Washington Carver is cool."
63
64            "George Washington Carver was born on the eve of the US Civil War, in
65            1864.  His exact date of birth is unknown... Carver's research in crop
66            rotation revolutionized agriculture..."
67
68       The first document is succinct, but useless.  Unfortunately, the
69       default similarity will assess it as extremely relevant to a query of
70       "george washington carver".  However, under LongFieldSim, the short-
71       field bias is eliminated, and the addition of other mentions of
72       Carver's name in the second document yield a higher score and a higher
73       rank.
74
75
76
77perl v5.34.0                      2022-01-21     LucyX::Index::LongFieldSim(3)
Impressum