1mlpack_nca(1) User Commands mlpack_nca(1)
2
6 mlpack_nca - neighborhood components analysis (nca)
7
9 mlpack_nca -i string [-A double] [-b int] [-l string] [-L bool] [-n int] [-T int] [-M double] [-m double] [-N bool] [-B int] [-O string] [-s int] [-a double] [-t double] [-V bool] [-w double] [-o string] [-h -v]
10
12 This program implements Neighborhood Components Analysis, both a linear
13 dimensionality reduction technique and a distance learning technique.
14 The method seeks to improve k-nearest-neighbor classification on a
15 dataset by scaling the dimensions. The method is nonparametric, and
16 does not require a value of k. It works by using stochastic ("soft")
17 neighbor assignments and using optimization techniques over the gradi‐
18 ent of the accuracy of the neighbor assignments.
19 To work, this algorithm needs labeled data. It can be given as the last
21 row of the input dataset (specified with '--input_file (-i)'), or
22 alternatively as a separate matrix (specified with '--labels_file
23 (-l)').
24 This implementation of NCA uses stochastic gradient descent, mini-batch
26 stochastic gradient descent, or the L_BFGS optimizer. These optimizers
27 do not guarantee global convergence for a nonconvex objective function
28 (NCA's objective function is nonconvex), so the final results could
29 depend on the random seed or other optimizer parameters.
30 Stochastic gradient descent, specified by the value 'sgd' for the
32 parameter ’--optimizer (-O)', depends primarily on three parameters:
33 the step size (specified with '--step_size (-a)'), the batch size
34 (specified with ’--batch_size (-b)'), and the maximum number of itera‐
35 tions (specified with ’--max_iterations (-n)'). In addition, a normal‐
36 ized starting point can be used by specifying the '--normalize (-N)'
37 parameter, which is necessary if many warnings of the form 'Denominator
38 of p_i is 0!' are given. Tuning the step size can be a tedious affair.
39 In general, the step size is too large if the objective is not mostly
40 uniformly decreasing, or if zero-valued denominator warnings are being
41 issued. The step size is too small if the objective is changing very
42 slowly. Setting the termination condition can be done easily once a
43 good step size parameter is found; either increase the maximum itera‐
44 tions to a large number and allow SGD to find a minimum, or set the
45 maximum iterations to 0 (allowing infinite iterations) and set the tol‐
46 erance (specified by '--tolerance (-t)') to define the maximum allowed
47 difference between objectives for SGD to terminate. Be careful---set‐
48 ting the tolerance instead of the maximum iterations can take a very
49 long time and may actually never converge due to the properties of the
50 SGD optimizer. Note that a single iteration of SGD refers to a single
51 point, so to take a single pass over the dataset, set the value of the
52 '--max_iterations (-n)' parameter equal to the number of points in the
53 dataset.
54 The L-BFGS optimizer, specified by the value 'lbfgs' for the parameter
56 ’--optimizer (-O)', uses a back-tracking line search algorithm to mini‐
57 mize a function. The following parameters are used by L-BFGS:
58 '--num_basis (-B)' (specifies the number of memory points used by L-
59 BFGS), '--max_iterations (-n)', '--armijo_constant (-A)', '--wolfe
60 (-w)', '--tolerance (-t)' (the optimization is terminated when the gra‐
61 dient norm is below this value), ’--max_line_search_trials (-T)',
62 '--min_step (-m)', and '--max_step (-M)' (which both refer to the line
63 search routine). For more details on the L-BFGS optimizer, consult
64 either the mlpack L-BFGS documentation (in lbfgs.hpp) or the vast set
65 of published literature on L-BFGS.
66 By default, the SGD optimizer is used.
68
70 --input_file (-i) [string]
71 Input dataset to run NCA on.
72
74 --armijo_constant (-A) [double]
75 Armijo constant for L-BFGS. Default value 0.0001.
76 --batch_size (-b) [int]
78 Batch size for mini-batch SGD. Default value 50.
79 --help (-h) [bool]
81 Default help info.
82 --info [string]
84 Get help on a specific module or option. Default value ''.
85 --labels_file (-l) [string]
87 Labels for input dataset. Default value ''.
88 --linear_scan (-L) [bool]
90 Don't shuffle the order in which data points are visited for SGD
91 or mini-batch SGD.
92 --max_iterations (-n) [int]
94 Maximum number of iterations for SGD or L-BFGS (0 indicates no
95 limit). Default value 500000.
96 --max_line_search_trials (-T) [int]
98 Maximum number of line search trials for L-BFGS. Default value
99 50.
100 --max_step (-M) [double]
102 Maximum step of line search for L-BFGS. Default value 1e+20.
103 --min_step (-m) [double]
105 Minimum step of line search for L-BFGS. Default value 1e-20.
106 --normalize (-N) [bool]
108 Use a normalized starting point for optimization. This is useful
109 for when points are far apart, or when SGD is returning NaN.
110 --num_basis (-B) [int]
112 Number of memory points to be stored for L-BFGS. Default value
113 5.
114 --optimizer (-O) [string]
116 Optimizer to use; 'sgd' or 'lbfgs'. Default value 'sgd'.
117 --seed (-s) [int]
119 Random seed. If 0, 'std::time(NULL)' is used. Default value 0.
120 --step_size (-a) [double]
122 Step size for stochastic gradient descent (alpha). Default value
123 0.01.
124 --tolerance (-t) [double]
126 Maximum tolerance for termination of SGD or L-BFGS. Default
127 value 1e-07.
128 --verbose (-v) [bool]
130 Display informational messages and the full list of parameters
131 and timers at the end of execution.
132 --version (-V) [bool]
134 Display the version of mlpack.
135 --wolfe (-w) [double]
137 Wolfe condition parameter for L-BFGS. Default value 0.9.
138
140 --output_file (-o) [string]
141 Output matrix for learned distance matrix. Default value ''.
142
144 For further information, including relevant papers, citations, and the‐
145 ory, consult the documentation found at http://www.mlpack.org or
146 included with your distribution of mlpack.
147mlpack-3.0.4 21 February 2019 mlpack_nca(1)