1DBD::Gofer(3) User Contributed Perl Documentation DBD::Gofer(3)
2
6 DBD::Gofer - A stateless-proxy driver for communicating with a remote
7 DBI
8
10 use DBI;
11 $original_dsn = "dbi:..."; # your original DBI Data Source Name
13 $dbh = DBI->connect("dbi:Gofer:transport=$transport;...;dsn=$original_dsn",
15 $user, $passwd, \%attributes);
16 ... use $dbh as if it was connected to $original_dsn ...
18 The "transport=$transport" part specifies the name of the module to use
20 to transport the requests to the remote DBI. If $transport doesn't
21 contain any double colons then it's prefixed with
22 "DBD::Gofer::Transport::".
23 The "dsn=$original_dsn" part must be the last element of the DSN
25 because everything after "dsn=" is assumed to be the DSN that the
26 remote DBI should use.
27 The "..." represents attributes that influence the operation of the
29 Gofer driver or transport. These are described below or in the
30 documentation of the transport module being used.
31
33 DBD::Gofer is a DBI database driver that forwards requests to another
34 DBI driver, usually in a seperate process, often on a separate machine.
35 It tries to be as transparent as possible so it appears that you are
36 using the remote driver directly.
37 DBD::Gofer is very similar to DBD::Proxy. The major difference is that
39 with DBD::Gofer no state is maintained on the remote end. That means
40 every request contains all the information needed to create the
41 required state. (So, for example, every request includes the DSN to
42 connect to.) Each request can be sent to any available server. The
43 server executes the request and returns a single response that includes
44 all the data.
45 This is very similar to the way http works as a stateless protocol for
47 the web. Each request from your web browser can be handled by a
48 different web server process.
49 Use Cases
51 This may seem like pointless overhead but there are situations where
52 this is a very good thing. Let's consider a specific case.
53 Imagine using DBD::Gofer with an http transport. Your application calls
55 connect(), prepare("select * from table where foo=?"), bind_param(),
56 and execute(). At this point DBD::Gofer builds a request containing
57 all the information about the method calls. It then uses the httpd
58 transport to send that request to an apache web server.
59 This 'dbi execute' web server executes the request (using
61 DBI::Gofer::Execute and related modules) and builds a response that
62 contains all the rows of data, if the statement returned any, along
63 with all the attributes that describe the results, such as
64 $sth->{NAME}. This response is sent back to DBD::Gofer which unpacks it
65 and presents it to the application as if it had executed the statement
66 itself.
67 Advantages
69 Okay, but you still don't see the point? Well let's consider what we've
70 gained:
71 Connection Pooling and Throttling
73 The 'dbi execute' web server leverages all the functionality of web
75 infrastructure in terms of load balancing, high-availability,
76 firewalls, access management, proxying, caching.
77 At its most basic level you get a configurable pool of persistent
79 database connections.
80 Simple Scaling
82 Got thousands of processes all trying to connect to the database? You
84 can use DBD::Gofer to connect them to your smaller pool of 'dbi
85 execute' web servers instead.
86 Caching
88 Client-side caching is as simple as adding ""cache=1"" to the DSN.
90 This feature alone can be worth using DBD::Gofer for.
91 Fewer Network Round-trips
93 DBD::Gofer sends as few requests as possible (dependent on the policy
95 being used).
96 Thin Clients / Unsupported Platforms
98 You no longer need drivers for your database on every system.
100 DBD::Gofer is pure perl.
101
103 There are some natural constraints imposed by the DBD::Gofer
104 'stateless' approach. But not many:
105 You can't change database handle attributes after connect()
107 You can't change database handle attributes after you've connected.
108 Use the connect() call to specify all the attribute settings you want.
109 This is because it's critical that when a request is complete the
111 database handle is left in the same state it was when first connected.
112 An exception is made for attributes with names starting ""private_"":
114 They can be set after connect() but the change is only applied locally.
115 You can't change statement handle attributes after prepare()
117 You can't change statment handle attributes after prepare.
118 An exception is made for attributes with names starting ""private_"":
120 They can be set after prepare() but the change is only applied locally.
121 You can't use transactions
123 AutoCommit only. Transactions aren't supported.
124 (In theory transactions could be supported when using a transport that
126 maintains a connection, like "stream" does. If you're interested in
127 this please get in touch via dbi-dev@perl.org)
128 You can't call driver-private sth methods
130 But that's rarely needed anyway.
131
133 A few important things to keep in mind when using DBD::Gofer:
134 Temporary tables, locks, and other per-connection persistent state
136 You shouldn't expect any per-session state to persist between requests.
137 This includes locks and temporary tables.
138 Because the server-side may execute your requests via a different
140 database connections, you can't rely on any per-connection persistent
141 state, such as temporary tables, being available from one request to
142 the next.
143 This is an easy trap to fall into. A good way to check for this is to
145 test your code with a Gofer policy package that sets the
146 "connect_method" policy to 'connect' to force a new connection for each
147 request. The "pedantic" policy does this.
148 Driver-private Database Handle Attributes
150 Some driver-private dbh attributes may not be available if the driver
151 has not implemented the private_attribute_info() method (added in DBI
152 1.54).
153 Driver-private Statement Handle Attributes
155 Driver-private sth attributes can be set in the prepare() call. TODO
156 Some driver-private dbh attributes may not be available if the driver
158 has not implemented the private_attribute_info() method (added in DBI
159 1.54).
160 Multiple Resultsets
162 Multiple resultsets are supported only if the driver supports the
163 more_results() method (an exception is made for DBD::Sybase).
164 Statement activity that also updates dbh attributes
166 Some drivers may update one or more dbh attributes after performing
167 activity on a child sth. For example, DBD::mysql provides
168 $dbh->{mysql_insertid} in addition to $sth->{mysql_insertid}. Currently
169 mysql_insertid is supported via a hack but a more general mechanism is
170 needed for other drivers to use.
171 Methods that report an error always return undef
173 With DBD::Gofer, a method that sets an error always return an undef or
174 empty list. That shouldn't be a problem in practice because the DBI
175 doesn't define any methods that return meaningful values while also
176 reporting an error.
177 Subclassing only applies to client-side
179 The RootClass and DbTypeSubclass attributes are not passed to the Gofer
180 server.
181
183 last_insert_id
184 To enable use of last_insert_id you need to indicate to DBD::Gofer that
185 you'd like to use it. You do that my adding a "go_last_insert_id_args"
186 attribute to the do() or prepare() method calls. For example:
187 $dbh->do($sql, { go_last_insert_id_args => [...] });
189 or
191 $sth = $dbh->prepare($sql, { go_last_insert_id_args => [...] });
193 The array reference should contains the args that you want passed to
195 the last_insert_id() method.
196 execute_for_fetch
198 The array methods bind_param_array() and execute_array() are supported.
199 When execute_array() is called the data is serialized and executed in a
200 single round-trip to the Gofer server. This makes it very fast, but
201 requires enough memory to store all the serialized data.
202 The execute_for_fetch() method currently isn't optimised, it uses the
204 DBI fallback behaviour of executing each tuple individually. (It could
205 be implemented as a wrapper for execute_array() - patches welcome.)
206
208 DBD::Gofer doesn't concern itself with transporting requests and
209 responses to and fro. For that it uses special Gofer transport
210 modules.
211 Gofer transport modules usually come in pairs: one for the 'client'
213 DBD::Gofer driver to use and one for the remote 'server' end. They have
214 very similar names:
215 DBD::Gofer::Transport::<foo>
217 DBI::Gofer::Transport::<foo>
218 Sometimes the transports on the DBD and DBI sides may have different
220 names. For example DBD::Gofer::Transport::http is typically used with
221 DBI::Gofer::Transport::mod_perl (DBD::Gofer::Transport::http and
222 DBI::Gofer::Transport::mod_perl modules are part of the GoferTransport-
223 http distribution).
224 Bundled Transports
226 Several transport modules are provided with DBD::Gofer:
227 null
229 The null transport is the simplest of them all. It doesn't actually
231 transport the request anywhere. It just serializes (freezes) the
232 request into a string, then thaws it back into a data structure before
233 passing it to DBI::Gofer::Execute to execute. The same freeze and thaw
234 is applied to the results.
235 The null transport is the best way to test if your application will
237 work with Gofer. Just set the DBI_AUTOPROXY environment variable to
238 ""dbi:Gofer:transport=null;policy=pedantic"" (see "Using DBI_AUTOPROXY"
239 below) and run your application, or ideally its test suite, as usual.
240 It doesn't take any parameters.
242 pipeone
244 The pipeone transport launches a subprocess for each request. It passes
246 in the request and reads the response.
247 The fact that a new subprocess is started for each request ensures that
249 the server side is truly stateless. While this does make the transport
250 very slow, it is useful as a way to test that your application doesn't
251 depend on per-connection state, such as temporary tables, persisting
252 between requests.
253 It's also useful both as a proof of concept and as a base class for the
255 stream driver.
256 stream
258 The stream driver also launches a subprocess and writes requests and
260 reads responses, like the pipeone transport. In this case, however,
261 the subprocess is expected to handle more that one request. (Though it
262 will be automitically restarted if it exits.)
263 This is the first transport that is truly useful because it can launch
265 the subprocess on a remote machine using "ssh". This means you can now
266 use DBD::Gofer to easily access any databases that's accessible from
267 any system you can login to. You also get all the benefits of ssh,
268 including encryption and optional compression.
269 See "Using DBI_AUTOPROXY" below for an example.
271 Other Transports
273 Implementing a Gofer transport is very simple, and more transports are
274 very welcome. Just take a look at any existing transports that are
275 similar to your needs.
276 http
278 See the GoferTransport-http distribution on CPAN:
280 http://search.cpan.org/dist/GoferTransport-http/
281 Gearman
283 I know Ask BjA~Xrn Hansen has implemented a transport for the "gearman"
285 distributed job system, though it's not on CPAN at the time of writing
286 this.
287
289 Simply prefix your existing DSN with
290 ""dbi:Gofer:transport=$transport;dsn="" where $transport is the name of
291 the Gofer transport you want to use (see "TRANSPORTS"). The
292 "transport" and "dsn" attributes must be specified and the "dsn"
293 attributes must be last.
294 Other attributes can be specified in the DSN to configure DBD::Gofer
296 and/or the Gofer transport module being used. The main attributes after
297 "transport", are "url" and "policy". These and other attributes are
298 described below.
299 Using DBI_AUTOPROXY
301 The simplest way to try out DBD::Gofer is to set the DBI_AUTOPROXY
302 environment variable. In this case you don't include the "dsn=" part.
303 For example:
304 export DBI_AUTOPROXY="dbi:Gofer:transport=null"
306 or, for a more useful example, try:
308 export DBI_AUTOPROXY="dbi:Gofer:transport=stream;url=ssh:user@example.com"
310 Connection Attributes
312 These attributes can be specified in the DSN. They can also be passed
313 in the \%attr parameter of the DBI connect method by adding a ""go_""
314 prefix to the name.
315 transport
317 Specifies the Gofer transport class to use. Required. See "TRANSPORTS"
319 above.
320 If the value does not include "::" then ""DBD::Gofer::Transport::"" is
322 prefixed.
323 The transport object can be accessed via $h->{go_transport}.
325 dsn
327 Specifies the DSN for the remote side to connect to. Required, and must
329 be last.
330 url
332 Used to tell the transport where to connect to. The exact form of the
334 value depends on the transport used.
335 policy
337 Specifies the policy to use. See "CONFIGURING BEHAVIOUR POLICY".
339 If the value does not include "::" then ""DBD::Gofer::Policy"" is
341 prefixed.
342 The policy object can be accessed via $h->{go_policy}.
344 timeout
346 Specifies a timeout, in seconds, to use when waiting for responses from
348 the server side.
349 retry_limit
351 Specifies the number of times a failed request will be retried. Default
353 is 0.
354 retry_hook
356 Specifies a code reference to be called to decide if a failed request
358 should be retried. The code reference is called like this:
359 $transport = $h->{go_transport};
361 $retry = $transport->go_retry_hook->($request, $response, $transport);
362 If it returns true then the request will be retried, upto the
364 "retry_limit". If it returns a false but defined value then the
365 request will not be retried. If it returns undef then the default
366 behaviour will be used, as if "retry_hook" had not been specified.
367 The default behaviour is to retry requests where
369 $request->is_idempotent is true, or the error message matches "/induced
370 by DBI_GOFER_RANDOM/".
371 cache
373 Specifies that client-side caching should be performed. The value is
375 the name of a cache class to use.
376 Any class implementing get($key) and set($key, $value) methods can be
378 used. That includes a great many powerful caching classes on CPAN,
379 including the Cache and Cache::Cache distributions.
380 You can use ""cache=1"" is a shortcut for
382 ""cache=DBI::Util::CacheMemory"". See DBI::Util::CacheMemory for a
383 description of this simple fast default cache.
384 The cache object can be accessed via $h->go_cache. For example:
386 $dbh->go_cache->clear; # free up memory being used by the cache
388 The cache keys are the frozen (serialized) requests, and the values are
390 the frozen responses.
391 The default behaviour is to only use the cache for requests where
393 $request->is_idempotent is true (i.e., the dbh has the ReadOnly
394 attribute set or the SQL statement is obviously a SELECT without a FOR
395 UPDATE clause.)
396 For even more control you can use the "go_cache" attribute to pass in
398 an instanciated cache object. Individual methods, including prepare(),
399 can also specify alternative caches via the "go_cache" attribute. For
400 example, to specify no caching for a particular query, you could use
401 $sth = $dbh->prepare( $sql, { go_cache => 0 } );
403 This can be used to implement different caching policies for different
405 statements.
406 It's interesting to note that DBD::Gofer can be used to add client-side
408 caching to any (gofer compatible) application, with no code changes and
409 no need for a gofer server. Just set the DBI_AUTOPROXY environment
410 variable like this:
411 DBI_AUTOPROXY='dbi:Gofer:transport=null;cache=1'
413
415 DBD::Gofer supports a 'policy' mechanism that allows you to fine-tune
416 the number of round-trips to the Gofer server. The policies are
417 grouped into classes (which may be subclassed) and referenced by the
418 name of the class.
419 The DBD::Gofer::Policy::Base class is the base class for all the policy
421 packages and describes all the available policies.
422 Three policy packages are supplied with DBD::Gofer:
424 DBD::Gofer::Policy::pedantic is most 'transparent' but slowest because
426 it makes more round-trips to the Gofer server.
427 DBD::Gofer::Policy::classic is a reasonable compromise - it's the
429 default policy.
430 DBD::Gofer::Policy::rush is fastest, but may require code changes in
432 your applications.
433 Generally the default "classic" policy is fine. When first testing an
435 existing application with Gofer it is a good idea to start with the
436 "pedantic" policy first and then switch to "classic" or a custom
437 policy, for final testing.
438
440 Tim Bunce, <http://www.tim.bunce.name>
441
443 Copyright (c) 2007, Tim Bunce, Ireland. All rights reserved.
444 This module is free software; you can redistribute it and/or modify it
446 under the same terms as Perl itself. See perlartistic.
447
449 The development of DBD::Gofer and related modules was sponsored by
450 Shopzilla.com (<http://Shopzilla.com>), where I currently work.
451
453 DBI::Gofer::Request, DBI::Gofer::Response, DBI::Gofer::Execute.
454 DBI::Gofer::Transport::Base, DBD::Gofer::Policy::Base.
456 DBI
458
460 This section aims to record issues to be aware of when using Gofer with
461 specific drivers. It usually only documents issues that are not
462 natural consequences of the limitations of the Gofer approach - as
463 documented avove.
464
466 This is just a random brain dump... (There's more in the source of the
467 Changes file, not the pod)
468 Document policy mechanism
470 Add mechanism for transports to list config params and for Gofer to
472 apply any that match (and warn if any left over?)
473 Driver-private sth attributes - set via prepare() - change DBI spec
475 add hooks into transport base class for checking & updating a result
477 set cache
478 ie via a standard cache interface such as:
479 http://search.cpan.org/~robm/Cache-FastMmap/FastMmap.pm
480 http://search.cpan.org/~bradfitz/Cache-Memcached/lib/Cache/Memcached.pm
481 http://search.cpan.org/~dclinton/Cache-Cache/
482 http://search.cpan.org/~cleishman/Cache/ Also caching instructions
483 could be passed through the httpd transport layer in such a way that
484 appropriate http cache headers are added to the results so that web
485 caches (squid etc) could be used to implement the caching. (MUST
486 require the use of GET rather than POST requests.)
487 Rework handling of installed_methods to not piggback on dbh_attributes?
489 Perhaps support transactions for transports where it's possible (ie
491 null and stream)? Would make stream transport (ie ssh) more useful to
492 more people.
493 Make sth_result_attr more like dbh_attributes (using '*' etc)
495 Add @val = FETCH_many(@names) to DBI in C and use in Gofer/Execute?
497 Implement _new_sth in C.
499perl v5.10.1 2010-08-20 DBD::Gofer(3)