1Search::Elasticsearch::UCsleirenCto:n:t7r_i0b:u:tSBeeudalrkPc(eh3r:)l:EDloacsutmiecnsteaatricohn::Client::7_0::Bulk(3)
2
6 Search::Elasticsearch::Client::7_0::Bulk - A helper module for the Bulk
7 API
8
10 version 7.30
11
13 use Search::Elasticsearch;
14 my $es = Search::Elasticsearch->new;
16 my $bulk = $es->bulk_helper(
17 index => 'my_index',
18 type => 'my_type'
19 );
20 # Index docs:
22 $bulk->index({ id => 1, source => { foo => 'bar' }});
23 $bulk->add_action( index => { id => 1, source => { foo=> 'bar' }});
24 # Create docs:
26 $bulk->create({ id => 1, source => { foo => 'bar' }});
27 $bulk->add_action( create => { id => 1, source => { foo=> 'bar' }});
28 $bulk->create_docs({ foo => 'bar' })
29 # Delete docs:
31 $bulk->delete({ id => 1});
32 $bulk->add_action( delete => { id => 1 });
33 $bulk->delete_ids(1,2,3)
34 # Update docs:
36 $bulk->update({ id => 1, script => '...' });
37 $bulk->add_action( update => { id => 1, script => '...' });
38 # Manual flush
40 $bulk->flush;
41
43 This module provides a wrapper for the "bulk()" in
44 Search::Elasticsearch::Client::7_0::Direct method which makes it easier
45 to run multiple create, index, update or delete actions in a single
46 request.
47 The Search::Elasticsearch::Client::7_0::Bulk module acts as a queue,
49 buffering up actions until it reaches a maximum count of actions, or a
50 maximum size of JSON request body, at which point it issues a "bulk()"
51 request.
52 Once you have finished adding actions, call "flush()" to force the
54 final "bulk()" request on the items left in the queue.
55 This class does Search::Elasticsearch::Client::7_0::Role::Bulk and
57 Search::Elasticsearch::Role::Is_Sync.
58
60 "new()"
61 my $bulk = $es->bulk_helper(
62 index => 'default_index', # optional
64 type => 'default_type', # optional
65 %other_bulk_params # optional
66 max_count => 1_000, # optional
68 max_size => 1_000_000, # optional
69 max_time => 6, # optional
70 verbose => 0 | 1, # optional
72 on_success => sub {...}, # optional
74 on_error => sub {...}, # optional
75 on_conflict => sub {...}, # optional
76 );
79 The "new()" method returns a new $bulk object. You must pass your
81 Search::Elasticsearch client as the "es" argument.
82 The "index" and "type" parameters provide default values for "index"
84 and "type", which can be overridden in each action. You can also pass
85 any other values which are accepted by the bulk() method.
86 See "flush()" for more information about the other parameters.
88
90 "flush()"
91 $result = $bulk->flush;
92 The "flush()" method sends all buffered actions to Elasticsearch using
94 a bulk() request.
95 Auto-flushing
97 An automatic "flush()" is triggered whenever the "max_count",
98 "max_size", or "max_time" threshold is breached. This causes all
99 actions in the buffer to be sent to Elasticsearch.
100 • "max_count"
102 The maximum number of actions to allow before triggering a
104 "flush()". This can be disabled by setting "max_count" to 0.
105 Defaults to "1,000".
106 • "max_size"
108 The maximum size of JSON request body to allow before triggering a
110 "flush()". This can be disabled by setting "max_size" to 0.
111 Defaults to "1_000,000" bytes.
112 • "max_time"
114 The maximum number of seconds to wait before triggering a flush.
116 Defaults to 0 seconds, which means that it is disabled. Note: This
117 timeout is only triggered when new items are added to the queue,
118 not in the background.
119 Errors when flushing
121 There are two types of error which can be thrown when "flush()" is
122 called, either manually or automatically.
123 • Temporary Elasticsearch errors
125 A "Cxn" error like a "NoNodes" error which indicates that your
127 cluster is down. These errors do not clear the buffer, as they can
128 be retried later on.
129 • Action errors
131 Individual actions may fail. For instance, a "create" action will
133 fail if a document with the same "index", "type" and "id" already
134 exists. These action errors are reported via callbacks.
135 Using callbacks
137 By default, any Action errors (see above) cause warnings to be written
138 to "STDERR". However, you can use the "on_error", "on_conflict" and
139 "on_success" callbacks for more fine-grained control.
140 All callbacks receive the following arguments:
142 $action
144 The name of the action, ie "index", "create", "update" or "delete".
145 $response
147 The response that Elasticsearch returned for this action.
148 $i The index of the action, ie the first action in the flush request
150 will have $i set to 0, the second will have $i set to 1 etc.
151 "on_success"
153 my $bulk = $es->bulk_helper(
155 on_success => sub {
156 my ($action,$response,$i) = @_;
157 # do something
158 },
159 );
160 The "on_success" callback is called for every action that has a
162 successful response.
163 "on_conflict"
165 my $bulk = $es->bulk_helper(
167 on_conflict => sub {
168 my ($action,$response,$i,$version) = @_;
169 # do something
170 },
171 );
172 The "on_conflict" callback is called for actions that have triggered a
174 "Conflict" error, eg trying to "create" a document which already
175 exists. The $version argument will contain the version number of the
176 document currently stored in Elasticsearch (if found).
177 "on_error"
179 my $bulk = $es->bulk_helper(
181 on_error => sub {
182 my ($action,$response,$i) = @_;
183 # do something
184 },
185 );
186 The "on_error" callback is called for any error (unless the
188 "on_conflict") callback has already been called).
189 Disabling callbacks and autoflush
191 If you want to be in control of flushing, and you just want to receive
192 the raw response that Elasticsearch sends instead of using callbacks,
193 then you can do so as follows:
194 my $bulk = $es->bulk_helper(
196 max_count => 0,
197 max_size => 0,
198 on_error => undef
199 );
200 $bulk->add_actions(....);
202 $response = $bulk->flush;
203
205 "add_action()"
206 $bulk->add_action(
207 create => { ...params... },
208 index => { ...params... },
209 update => { ...params... },
210 delete => { ...params... }
211 );
212 The "add_action()" method allows you to add multiple "create", "index",
214 "update" and "delete" actions to the queue. The first value is the
215 action type, and the second value is the parameters that describe that
216 action. See the individual helper methods below for details.
217 Note: Parameters like "index" or "type" can be specified as "index" or
219 as "_index", so the following two lines are equivalent:
220 index => { index => 'index', type => 'type', id => 1, source => {...}},
222 index => { _index => 'index', _type => 'type', _id => 1, source => {...}},
223 Note: The "index" and "type" parameters can be specified in the params
225 for any action, but if not specified, will default to the "index" and
226 "type" values specified in "new()". These are required parameters:
227 they must be specified either in "new()" or in every action.
228 "create()"
230 $bulk->create(
231 { index => 'custom_index', source => { doc body }},
232 { type => 'custom_type', id => 1, source => { doc body }},
233 ...
234 );
235 The "create()" helper method allows you to add multiple "create"
237 actions. It accepts the same parameters as "create()" in
238 Search::Elasticsearch::Client::7_0::Direct except that the document
239 body should be passed as the "source" or "_source" parameter, instead
240 of as "body".
241 "create_docs()"
243 $bulk->create_docs(
244 { doc body },
245 { doc body },
246 ...
247 );
248 The "create_docs()" helper is a shorter form of "create()" which can be
250 used when you are using the default "index" and "type" as set in
251 "new()" and you are not specifying a custom "id" per document. In this
252 case, you can just pass the individual document bodies.
253 "index()"
255 $bulk->index(
256 { index => 'custom_index', source => { doc body }},
257 { type => 'custom_type', id => 1, source => { doc body }},
258 ...
259 );
260 The "index()" helper method allows you to add multiple "index" actions.
262 It accepts the same parameters as "index()" in
263 Search::Elasticsearch::Client::7_0::Direct except that the document
264 body should be passed as the "source" or "_source" parameter, instead
265 of as "body".
266 "delete()"
268 $bulk->delete(
269 { index => 'custom_index', id => 1},
270 { type => 'custom_type', id => 2},
271 ...
272 );
273 The "delete()" helper method allows you to add multiple "delete"
275 actions. It accepts the same parameters as "delete()" in
276 Search::Elasticsearch::Client::7_0::Direct.
277 "delete_ids()"
279 $bulk->delete_ids(1,2,3...)
280 The "delete_ids()" helper method can be used when all of the documents
282 you want to delete have the default "index" and "type" as set in
283 "new()". In this case, all you have to do is to pass in a list of IDs.
284 "update()"
286 $bulk->update(
287 { id => 1,
288 doc => { partial doc },
289 doc_as_upsert => 1
290 },
291 { id => 2,
292 script => { script }
293 upsert => { upsert doc }
294 },
295 ...
296 );
297 The "update()" helper method allows you to add multiple "update"
299 actions. It accepts the same parameters as "update()" in
300 Search::Elasticsearch::Client::7_0::Direct. An update can either use a
301 partial doc which gets merged with an existing doc (example 1 above),
302 or can use a "script" to update an existing doc (example 2 above). More
303 information on "script" can be found here: "update()" in
304 Search::Elasticsearch::Client::7_0::Direct.
305
307 Enrico Zimuel <enrico.zimuel@elastic.co>
308
310 This software is Copyright (c) 2020 by Elasticsearch BV.
311 This is free software, licensed under:
313 The Apache License, Version 2.0, January 2004
315perl v5.34.0 20S2e1a-r0c7h-:2:2Elasticsearch::Client::7_0::Bulk(3)