1LWP::ConsoleLogger(3) User Contributed Perl DocumentationLWP::ConsoleLogger(3)
2
6 LWP::ConsoleLogger - LWP tracing and debugging
7
9 version 1.000001
10
12 The simplest way to get started is by adding
13 LWP::ConsoleLogger::Everywhere to your code and then just watching your
14 output.
15 use LWP::ConsoleLogger::Everywhere ();
17 If you need more control, look at LWP::ConsoleLogger::Easy.
19 use LWP::ConsoleLogger::Easy qw( debug_ua );
21 use WWW::Mechanize;
22 my $mech = WWW::Mechanize->new; # or LWP::UserAgent->new() etc
24 my $console_logger = debug_ua( $mech );
25 $mech->get( 'https://metacpan.org' );
26 # now watch the console for debugging output
28 # turn off header dumps
29 $console_logger->dump_headers( 0 );
30 $mech->get( $some_other_url );
32 To get down to the lowest level, use LWP::ConsoleLogger directly.
34 my $ua = LWP::UserAgent->new( cookie_jar => {} );
36 my $console_logger = LWP::ConsoleLogger->new(
37 dump_content => 1,
38 dump_text => 1,
39 content_pre_filter => sub {
40 my $content = shift;
41 my $content_type = shift;
42 # mangle content here
44 # ...
45 return $content;
47 },
48 );
49 $ua->default_header(
51 'Accept-Encoding' => scalar HTTP::Message::decodable() );
52 $ua->add_handler( 'response_done',
54 sub { $console_logger->response_callback( @_ ) } );
55 $ua->add_handler( 'request_send',
56 sub { $console_logger->request_callback( @_ ) } );
57 # now watch debugging output to your screen
59 $ua->get( 'http://nytimes.com/' );
60 Sample output might look like this.
62 GET http://www.nytimes.com/2014/04/24/technology/fcc-new-net-neutrality-rules.html
64 GET params:
66 .-----+-------.
67 | Key | Value |
68 +-----+-------+
69 | _r | 1 |
70 | hp | |
71 '-----+-------'
72 .-----------------+--------------------------------.
74 | Request Header | Value |
75 +-----------------+--------------------------------+
76 | Accept-Encoding | gzip |
77 | Cookie2 | $Version="1" |
78 | Referer | http://www.nytimes.com?foo=bar |
79 | User-Agent | WWW-Mechanize/1.73 |
80 '-----------------+--------------------------------'
81 ==> 200 OK
83 Title: The New York Times - Breaking News, World News & Multimedia
85 .--------------------------+-------------------------------.
87 | Response Header | Value |
88 +--------------------------+-------------------------------+
89 | Accept-Ranges | bytes |
90 | Age | 176 |
91 | Cache-Control | no-cache |
92 | Channels | NytNow |
93 | Client-Date | Fri, 30 May 2014 22:37:42 GMT |
94 | Client-Peer | 170.149.172.130:80 |
95 | Client-Response-Num | 1 |
96 | Client-Transfer-Encoding | chunked |
97 | Connection | keep-alive |
98 | Content-Encoding | gzip |
99 | Content-Type | text/html; charset=utf-8 |
100 | Date | Fri, 30 May 2014 22:37:41 GMT |
101 | NtCoent-Length | 65951 |
102 | Server | Apache |
103 | Via | 1.1 varnish |
104 | X-Cache | HIT |
105 | X-Varnish | 1142859770 1142854917 |
106 '--------------------------+-------------------------------'
107 .--------------------------+-------------------------------.
109 | Text |
110 +--------------------------+-------------------------------+
111 | F.C.C., in a Shift, Backs Fast Lanes for Web Traffic... |
112 '--------------------------+-------------------------------'
113
115 It can be hard (or at least tedious) to debug mechanize scripts.
116 LWP::Debug is deprecated. It suggests you write your own debugging
117 handlers, set up a proxy or install Wireshark. Those are all workable
118 solutions, but this module exists to save you some of that work. The
119 guts of this module are stolen from Plack::Middleware::DebugLogging,
120 which in turn stole most of its internals from Catalyst. If you're new
121 to LWP::ConsoleLogger, I suggest getting started with the
122 LWP::ConsoleLogger::Easy wrapper. This will get you up and running in
123 minutes. If you need to tweak the settings that
124 LWP::ConsoleLogger::Easy chooses for you (or if you just want to be
125 fancy), please read on.
126 Since this is a debugging library, I've left as much mutable state as
128 possible, so that you can easily toggle output on and off and otherwise
129 adjust how you deal with the output.
130
132 new()
133 The following arguments can be passed to new(), although none are
134 required. They can also be called as methods on an instantiated
135 object. I'll list them here and discuss them in detail below.
136 • "dump_content => 0|1"
138 • "dump_cookies => 0|1"
140 • "dump_headers => 0|1"
142 • "dump_params => 0|1"
144 • "dump_status => 0|1"
146 • "dump_text => 0|1"
148 • "dump_title => 0|1"
150 • "dump_text => 0|1"
152 • "dump_uri => 0|1"
154 • "content_pre_filter => sub { ... }"
156 • "headers_to_redact => ['Authentication', 'Foo']"
158 • "params_to_redact => ['token', 'password']"
160 • "text_pre_filter => sub { ... }"
162 • "html_restrict => HTML::Restrict->new( ... )"
164 • "logger => Log::Dispatch->new( ... )"
166 • "pretty => 0|1"
168 • "term_width => $integer"
170
172 dump_content( 0|1 )
173 Boolean value. If true, the actual content of your response (HTML,
174 JSON, etc) will be dumped to your screen. Defaults to false.
175 dump_cookies( 0|1 )
177 Boolean value. If true, the content of your cookies will be dumped to
178 your screen. Defaults to false.
179 dump_headers( 0|1 )
181 Boolean value. If true, both request and response headers will be
182 dumped to your screen. Defaults to true.
183 Headers are dumped in alphabetical order.
185 dump_params( 0|1 )
187 Boolean value. If true, both GET and POST params will be dumped to your
188 screen. Defaults to true.
189 Params are dumped in alphabetical order.
191 dump_status( 0|1 )
193 Boolean value. If true, dumps the HTTP response code for each page
194 being visited. Defaults to true.
195 dump_text( 0|1 )
197 Boolean value. If true, dumps the text of your page after both the
198 content_pre_filter and text_pre_filters have been applied. Defaults to
199 true.
200 dump_title( 0|1 )
202 Boolean value. If true, dumps the titles of HTML pages if your
203 UserAgent has a "title" method and if it returns something useful.
204 Defaults to true.
205 dump_uri( 0|1 )
207 Boolean value. If true, dumps the URI of each page being visited.
208 Defaults to true.
209 pretty ( 0|1 )
211 Boolean value. If disabled, request headers, response headers, content
212 and text sections will be dumped without using tables. Handy for
213 copy/pasting JSON etc for faking responses later. Defaults to true.
214 content_pre_filter( sub { ... } )
216 Subroutine reference. This allows you to manipulate content before it
217 is dumped. A common use case might be stripping headers and footers
218 away from HTML content to make it easier to detect changes in the body
219 of the page.
220 $easy_logger->content_pre_filter(
222 sub {
223 my $content = shift;
224 my $content_type = shift; # the value of the Content-Type header
225 if ( $content_type =~ m{html}i
226 && $content =~ m{<!--\scontent\s-->(.*)<!--\sfooter}msx ) {
227 return $1;
228 }
229 return $content;
230 }
231 );
232 Try to make sure that your content mangling doesn't return broken HTML
234 as that may not play well with HTML::Restrict.
235 request_callback
237 Use this handler to set up console logging on your requests.
238 my $ua = LWP::UserAgent->new;
240 $ua->add_handler(
241 'request_send',
242 sub { $console_logger->request_callback(@_) }
243 );
244 This is done for you by default if you set up your logging via
246 LWP::ConsoleLogger::Easy.
247 response_callback
249 Use this handler to set up console logging on your responses.
250 my $ua = LWP::UserAgent->new;
252 $ua->add_handler(
253 'response_done',
254 sub { $console_logger->response_callback(@_) }
255 );
256 This is done for you by default if you set up your logging via
258 LWP::ConsoleLogger::Easy.
259 text_pre_filter( sub { ... } )
261 Subroutine reference. This allows you to manipulate text before it is
262 dumped. A common use case might be stripping away duplicate whitespace
263 and/or newlines in order to improve formatting. Keep in mind that the
264 "content_pre_filter" will have been applied to the content which is
265 passed to the text_pre_filter. The idea is that you can strip away an
266 HTML you don't care about in the content_pre_filter phase and then
267 process the remainder of the content in the text_pre_filter.
268 $easy_logger->text_pre_filter(
270 sub {
271 my $content = shift;
272 my $content_type = shift; # the value of the Content-Type header
273 my $base_url = shift;
274 # do something with the content
276 # ...
277 return ( $content, $new_content_type );
279 }
280 );
281 If your text_pre_filter() converts from HTML to plain text, be sure to
283 return the new content type (text/plain) when you exit the sub. If you
284 do not do this, HTML formatting will then be applied to your plain text
285 as is explained below.
286 If this is HTML content, HTML::Restrict will be applied after the
288 text_pre_filter has been run. LWP::ConsoleLogger will then strip away
289 some whitespace and newlines from processed HTML in its own opinionated
290 way, in order to present you with more readable text.
291 html_restrict( HTML::Restrict->new( ... ) )
293 If the content_type indicates HTML then HTML::Restrict will be used to
294 strip tags from your content in the text rendering process. You may
295 pass your own HTML::Restrict object, if you like. This would be
296 helpful in situations where you still do want to have some tags in your
297 text.
298 logger( Log::Dispatch->new( ... ) )
300 By default all data will be dumped to your console (as the name of this
301 module implies) using Log::Dispatch. However, you may use your own
302 Log::Dispatch module in order to facilitate logging to files or any
303 other output which Log::Dispatch supports.
304 term_width( $integer )
306 By default this module will try to find the maximum width of your
307 terminal and use all available space when displaying tabular data. You
308 may use this parameter to constrain the tables to an arbitrary width.
309
311 I've written this to suit my needs and there are a lot of things I
312 haven't considered. For example, I'm mostly assuming that the content
313 will be text, HTML, JSON or XML.
314 The test suite is not very robust either. If you'd like to contribute
316 to this module and you can't find an appropriate test, do add something
317 to the example folder (either a new script or alter an existing one),
318 so that I can see what your patch does.
319
321 Olaf Alders <olaf@wundercounter.com>
322
324 This software is Copyright (c) 2014 by MaxMind, Inc.
325 This is free software, licensed under:
327 The Artistic License 2.0 (GPL Compatible)
329perl v5.36.1 2023-06-22 LWP::ConsoleLogger(3)