1DBD::Gofer(3)         User Contributed Perl Documentation        DBD::Gofer(3)
2
3
4

NAME

6       DBD::Gofer - A stateless-proxy driver for communicating with a remote
7       DBI
8

SYNOPSIS

10         use DBI;
11
12         $original_dsn = "dbi:..."; # your original DBI Data Source Name
13
14         $dbh = DBI->connect("dbi:Gofer:transport=$transport;...;dsn=$original_dsn",
15                             $user, $passwd, \%attributes);
16
17         ... use $dbh as if it was connected to $original_dsn ...
18
19       The "transport=$transport" part specifies the name of the module to use
20       to transport the requests to the remote DBI. If $transport doesn't
21       contain any double colons then it's prefixed with
22       "DBD::Gofer::Transport::".
23
24       The "dsn=$original_dsn" part must be the last element of the DSN
25       because everything after "dsn=" is assumed to be the DSN that the
26       remote DBI should use.
27
28       The "..." represents attributes that influence the operation of the
29       Gofer driver or transport. These are described below or in the
30       documentation of the transport module being used.
31

DESCRIPTION

33       DBD::Gofer is a DBI database driver that forwards requests to another
34       DBI driver, usually in a separate process, often on a separate machine.
35       It tries to be as transparent as possible so it appears that you are
36       using the remote driver directly.
37
38       DBD::Gofer is very similar to DBD::Proxy. The major difference is that
39       with DBD::Gofer no state is maintained on the remote end. That means
40       every request contains all the information needed to create the
41       required state. (So, for example, every request includes the DSN to
42       connect to.) Each request can be sent to any available server. The
43       server executes the request and returns a single response that includes
44       all the data.
45
46       This is very similar to the way http works as a stateless protocol for
47       the web.  Each request from your web browser can be handled by a
48       different web server process.
49
50   Use Cases
51       This may seem like pointless overhead but there are situations where
52       this is a very good thing. Let's consider a specific case.
53
54       Imagine using DBD::Gofer with an http transport. Your application calls
55       connect(), prepare("select * from table where foo=?"), bind_param(),
56       and execute().  At this point DBD::Gofer builds a request containing
57       all the information about the method calls. It then uses the httpd
58       transport to send that request to an apache web server.
59
60       This 'dbi execute' web server executes the request (using
61       DBI::Gofer::Execute and related modules) and builds a response that
62       contains all the rows of data, if the statement returned any, along
63       with all the attributes that describe the results, such as
64       $sth->{NAME}. This response is sent back to DBD::Gofer which unpacks it
65       and presents it to the application as if it had executed the statement
66       itself.
67
68   Advantages
69       Okay, but you still don't see the point? Well let's consider what we've
70       gained:
71
72       Connection Pooling and Throttling
73
74       The 'dbi execute' web server leverages all the functionality of web
75       infrastructure in terms of load balancing, high-availability,
76       firewalls, access management, proxying, caching.
77
78       At its most basic level you get a configurable pool of persistent
79       database connections.
80
81       Simple Scaling
82
83       Got thousands of processes all trying to connect to the database? You
84       can use DBD::Gofer to connect them to your smaller pool of 'dbi
85       execute' web servers instead.
86
87       Caching
88
89       Client-side caching is as simple as adding ""cache=1"" to the DSN.
90       This feature alone can be worth using DBD::Gofer for.
91
92       Fewer Network Round-trips
93
94       DBD::Gofer sends as few requests as possible (dependent on the policy
95       being used).
96
97       Thin Clients / Unsupported Platforms
98
99       You no longer need drivers for your database on every system.
100       DBD::Gofer is pure perl.
101

CONSTRAINTS

103       There are some natural constraints imposed by the DBD::Gofer
104       'stateless' approach.  But not many:
105
106   You can't change database handle attributes after connect()
107       You can't change database handle attributes after you've connected.
108       Use the connect() call to specify all the attribute settings you want.
109
110       This is because it's critical that when a request is complete the
111       database handle is left in the same state it was when first connected.
112
113       An exception is made for attributes with names starting ""private_"":
114       They can be set after connect() but the change is only applied locally.
115
116   You can't change statement handle attributes after prepare()
117       You can't change statement handle attributes after prepare.
118
119       An exception is made for attributes with names starting ""private_"":
120       They can be set after prepare() but the change is only applied locally.
121
122   You can't use transactions
123       AutoCommit only. Transactions aren't supported.
124
125       (In theory transactions could be supported when using a transport that
126       maintains a connection, like "stream" does. If you're interested in
127       this please get in touch via dbi-dev@perl.org)
128
129   You can't call driver-private sth methods
130       But that's rarely needed anyway.
131

GENERAL CAVEATS

133       A few important things to keep in mind when using DBD::Gofer:
134
135   Temporary tables, locks, and other per-connection persistent state
136       You shouldn't expect any per-session state to persist between requests.
137       This includes locks and temporary tables.
138
139       Because the server-side may execute your requests via a different
140       database connections, you can't rely on any per-connection persistent
141       state, such as temporary tables, being available from one request to
142       the next.
143
144       This is an easy trap to fall into. A good way to check for this is to
145       test your code with a Gofer policy package that sets the
146       "connect_method" policy to 'connect' to force a new connection for each
147       request. The "pedantic" policy does this.
148
149   Driver-private Database Handle Attributes
150       Some driver-private dbh attributes may not be available if the driver
151       has not implemented the private_attribute_info() method (added in DBI
152       1.54).
153
154   Driver-private Statement Handle Attributes
155       Driver-private sth attributes can be set in the prepare() call. TODO
156
157       Some driver-private sth attributes may not be available if the driver
158       has not implemented the private_attribute_info() method (added in DBI
159       1.54).
160
161   Multiple Resultsets
162       Multiple resultsets are supported only if the driver supports the
163       more_results() method (an exception is made for DBD::Sybase).
164
165   Statement activity that also updates dbh attributes
166       Some drivers may update one or more dbh attributes after performing
167       activity on a child sth.  For example, DBD::mysql provides
168       $dbh->{mysql_insertid} in addition to $sth->{mysql_insertid}. Currently
169       mysql_insertid is supported via a hack but a more general mechanism is
170       needed for other drivers to use.
171
172   Methods that report an error always return undef
173       With DBD::Gofer, a method that sets an error always return an undef or
174       empty list.  That shouldn't be a problem in practice because the DBI
175       doesn't define any methods that return meaningful values while also
176       reporting an error.
177
178   Subclassing only applies to client-side
179       The RootClass and DbTypeSubclass attributes are not passed to the Gofer
180       server.
181

CAVEATS FOR SPECIFIC METHODS

183   last_insert_id
184       To enable use of last_insert_id you need to indicate to DBD::Gofer that
185       you'd like to use it.  You do that my adding a "go_last_insert_id_args"
186       attribute to the do() or prepare() method calls. For example:
187
188           $dbh->do($sql, { go_last_insert_id_args => [...] });
189
190       or
191
192           $sth = $dbh->prepare($sql, { go_last_insert_id_args => [...] });
193
194       The array reference should contains the args that you want passed to
195       the last_insert_id() method.
196
197   execute_for_fetch
198       The array methods bind_param_array() and execute_array() are supported.
199       When execute_array() is called the data is serialized and executed in a
200       single round-trip to the Gofer server. This makes it very fast, but
201       requires enough memory to store all the serialized data.
202
203       The execute_for_fetch() method currently isn't optimised, it uses the
204       DBI fallback behaviour of executing each tuple individually.  (It could
205       be implemented as a wrapper for execute_array() - patches welcome.)
206

TRANSPORTS

208       DBD::Gofer doesn't concern itself with transporting requests and
209       responses to and fro.  For that it uses special Gofer transport
210       modules.
211
212       Gofer transport modules usually come in pairs: one for the 'client'
213       DBD::Gofer driver to use and one for the remote 'server' end. They have
214       very similar names:
215
216           DBD::Gofer::Transport::<foo>
217           DBI::Gofer::Transport::<foo>
218
219       Sometimes the transports on the DBD and DBI sides may have different
220       names. For example DBD::Gofer::Transport::http is typically used with
221       DBI::Gofer::Transport::mod_perl (DBD::Gofer::Transport::http and
222       DBI::Gofer::Transport::mod_perl modules are part of the GoferTransport-
223       http distribution).
224
225   Bundled Transports
226       Several transport modules are provided with DBD::Gofer:
227
228       null
229
230       The null transport is the simplest of them all. It doesn't actually
231       transport the request anywhere.  It just serializes (freezes) the
232       request into a string, then thaws it back into a data structure before
233       passing it to DBI::Gofer::Execute to execute. The same freeze and thaw
234       is applied to the results.
235
236       The null transport is the best way to test if your application will
237       work with Gofer.  Just set the DBI_AUTOPROXY environment variable to
238       ""dbi:Gofer:transport=null;policy=pedantic"" (see "Using DBI_AUTOPROXY"
239       below) and run your application, or ideally its test suite, as usual.
240
241       It doesn't take any parameters.
242
243       pipeone
244
245       The pipeone transport launches a subprocess for each request. It passes
246       in the request and reads the response.
247
248       The fact that a new subprocess is started for each request ensures that
249       the server side is truly stateless. While this does make the transport
250       very slow, it is useful as a way to test that your application doesn't
251       depend on per-connection state, such as temporary tables, persisting
252       between requests.
253
254       It's also useful both as a proof of concept and as a base class for the
255       stream driver.
256
257       stream
258
259       The stream driver also launches a subprocess and writes requests and
260       reads responses, like the pipeone transport.  In this case, however,
261       the subprocess is expected to handle more that one request. (Though it
262       will be automatically restarted if it exits.)
263
264       This is the first transport that is truly useful because it can launch
265       the subprocess on a remote machine using "ssh". This means you can now
266       use DBD::Gofer to easily access any databases that's accessible from
267       any system you can login to.  You also get all the benefits of ssh,
268       including encryption and optional compression.
269
270       See "Using DBI_AUTOPROXY" below for an example.
271
272   Other Transports
273       Implementing a Gofer transport is very simple, and more transports are
274       very welcome.  Just take a look at any existing transports that are
275       similar to your needs.
276
277       http
278
279       See the GoferTransport-http distribution on CPAN:
280       http://search.cpan.org/dist/GoferTransport-http/
281
282       Gearman
283
284       I know Ask Bjørn Hansen has implemented a transport for the "gearman"
285       distributed job system, though it's not on CPAN at the time of writing
286       this.
287

CONNECTING

289       Simply prefix your existing DSN with
290       ""dbi:Gofer:transport=$transport;dsn="" where $transport is the name of
291       the Gofer transport you want to use (see "TRANSPORTS").  The
292       "transport" and "dsn" attributes must be specified and the "dsn"
293       attributes must be last.
294
295       Other attributes can be specified in the DSN to configure DBD::Gofer
296       and/or the Gofer transport module being used. The main attributes after
297       "transport", are "url" and "policy". These and other attributes are
298       described below.
299
300   Using DBI_AUTOPROXY
301       The simplest way to try out DBD::Gofer is to set the DBI_AUTOPROXY
302       environment variable.  In this case you don't include the "dsn=" part.
303       For example:
304
305           export DBI_AUTOPROXY="dbi:Gofer:transport=null"
306
307       or, for a more useful example, try:
308
309           export DBI_AUTOPROXY="dbi:Gofer:transport=stream;url=ssh:user@example.com"
310
311   Connection Attributes
312       These attributes can be specified in the DSN. They can also be passed
313       in the \%attr parameter of the DBI connect method by adding a ""go_""
314       prefix to the name.
315
316       transport
317
318       Specifies the Gofer transport class to use. Required. See "TRANSPORTS"
319       above.
320
321       If the value does not include "::" then ""DBD::Gofer::Transport::"" is
322       prefixed.
323
324       The transport object can be accessed via $h->{go_transport}.
325
326       dsn
327
328       Specifies the DSN for the remote side to connect to. Required, and must
329       be last.
330
331       url
332
333       Used to tell the transport where to connect to. The exact form of the
334       value depends on the transport used.
335
336       policy
337
338       Specifies the policy to use. See "CONFIGURING BEHAVIOUR POLICY".
339
340       If the value does not include "::" then ""DBD::Gofer::Policy"" is
341       prefixed.
342
343       The policy object can be accessed via $h->{go_policy}.
344
345       timeout
346
347       Specifies a timeout, in seconds, to use when waiting for responses from
348       the server side.
349
350       retry_limit
351
352       Specifies the number of times a failed request will be retried. Default
353       is 0.
354
355       retry_hook
356
357       Specifies a code reference to be called to decide if a failed request
358       should be retried.  The code reference is called like this:
359
360         $transport = $h->{go_transport};
361         $retry = $transport->go_retry_hook->($request, $response, $transport);
362
363       If it returns true then the request will be retried, up to the
364       "retry_limit".  If it returns a false but defined value then the
365       request will not be retried.  If it returns undef then the default
366       behaviour will be used, as if "retry_hook" had not been specified.
367
368       The default behaviour is to retry requests where
369       $request->is_idempotent is true, or the error message matches "/induced
370       by DBI_GOFER_RANDOM/".
371
372       cache
373
374       Specifies that client-side caching should be performed.  The value is
375       the name of a cache class to use.
376
377       Any class implementing get($key) and set($key, $value) methods can be
378       used.  That includes a great many powerful caching classes on CPAN,
379       including the Cache and Cache::Cache distributions.
380
381       You can use ""cache=1"" is a shortcut for
382       ""cache=DBI::Util::CacheMemory"".  See DBI::Util::CacheMemory for a
383       description of this simple fast default cache.
384
385       The cache object can be accessed via $h->go_cache. For example:
386
387           $dbh->go_cache->clear; # free up memory being used by the cache
388
389       The cache keys are the frozen (serialized) requests, and the values are
390       the frozen responses.
391
392       The default behaviour is to only use the cache for requests where
393       $request->is_idempotent is true (i.e., the dbh has the ReadOnly
394       attribute set or the SQL statement is obviously a SELECT without a FOR
395       UPDATE clause.)
396
397       For even more control you can use the "go_cache" attribute to pass in
398       an instantiated cache object. Individual methods, including prepare(),
399       can also specify alternative caches via the "go_cache" attribute. For
400       example, to specify no caching for a particular query, you could use
401
402           $sth = $dbh->prepare( $sql, { go_cache => 0 } );
403
404       This can be used to implement different caching policies for different
405       statements.
406
407       It's interesting to note that DBD::Gofer can be used to add client-side
408       caching to any (gofer compatible) application, with no code changes and
409       no need for a gofer server.  Just set the DBI_AUTOPROXY environment
410       variable like this:
411
412           DBI_AUTOPROXY='dbi:Gofer:transport=null;cache=1'
413

CONFIGURING BEHAVIOUR POLICY

415       DBD::Gofer supports a 'policy' mechanism that allows you to fine-tune
416       the number of round-trips to the Gofer server.  The policies are
417       grouped into classes (which may be subclassed) and referenced by the
418       name of the class.
419
420       The DBD::Gofer::Policy::Base class is the base class for all the policy
421       packages and describes all the available policies.
422
423       Three policy packages are supplied with DBD::Gofer:
424
425       DBD::Gofer::Policy::pedantic is most 'transparent' but slowest because
426       it makes more  round-trips to the Gofer server.
427
428       DBD::Gofer::Policy::classic is a reasonable compromise - it's the
429       default policy.
430
431       DBD::Gofer::Policy::rush is fastest, but may require code changes in
432       your applications.
433
434       Generally the default "classic" policy is fine. When first testing an
435       existing application with Gofer it is a good idea to start with the
436       "pedantic" policy first and then switch to "classic" or a custom
437       policy, for final testing.
438

AUTHOR

440       Tim Bunce, <http://www.tim.bunce.name>
441
443       Copyright (c) 2007, Tim Bunce, Ireland. All rights reserved.
444
445       This module is free software; you can redistribute it and/or modify it
446       under the same terms as Perl itself. See perlartistic.
447

ACKNOWLEDGEMENTS

449       The development of DBD::Gofer and related modules was sponsored by
450       Shopzilla.com (<http://Shopzilla.com>), where I currently work.
451

SEE ALSO

453       DBI::Gofer::Request, DBI::Gofer::Response, DBI::Gofer::Execute.
454
455       DBI::Gofer::Transport::Base, DBD::Gofer::Policy::Base.
456
457       DBI
458

Caveats for specific drivers

460       This section aims to record issues to be aware of when using Gofer with
461       specific drivers.  It usually only documents issues that are not
462       natural consequences of the limitations of the Gofer approach - as
463       documented above.
464

TODO

466       This is just a random brain dump... (There's more in the source of the
467       Changes file, not the pod)
468
469       Document policy mechanism
470
471       Add mechanism for transports to list config params and for Gofer to
472       apply any that match (and warn if any left over?)
473
474       Driver-private sth attributes - set via prepare() - change DBI spec
475
476       add hooks into transport base class for checking & updating a result
477       set cache
478          ie via a standard cache interface such as:
479          http://search.cpan.org/~robm/Cache-FastMmap/FastMmap.pm
480          http://search.cpan.org/~bradfitz/Cache-Memcached/lib/Cache/Memcached.pm
481          http://search.cpan.org/~dclinton/Cache-Cache/
482          http://search.cpan.org/~cleishman/Cache/ Also caching instructions
483       could be passed through the httpd transport layer in such a way that
484       appropriate http cache headers are added to the results so that web
485       caches (squid etc) could be used to implement the caching.  (MUST
486       require the use of GET rather than POST requests.)
487
488       Rework handling of installed_methods to not piggyback on
489       dbh_attributes?
490
491       Perhaps support transactions for transports where it's possible (ie
492       null and stream)?  Would make stream transport (ie ssh) more useful to
493       more people.
494
495       Make sth_result_attr more like dbh_attributes (using '*' etc)
496
497       Add @val = FETCH_many(@names) to DBI in C and use in Gofer/Execute?
498
499       Implement _new_sth in C.
500
501
502
503perl v5.32.0                      2020-07-28                     DBD::Gofer(3)
Impressum