1LWP::ConsoleLogger(3) User Contributed Perl DocumentationLWP::ConsoleLogger(3)
2
6 LWP::ConsoleLogger - LWP tracing and debugging
7
9 version 0.000043
10
12 The simplest way to get started is by adding
13 LWP::ConsoleLogger::Everywhere to your code and then just watching your
14 output.
15 use LWP::ConsoleLogger::Everywhere ();
17 If you need more control, look at LWP::ConsoleLogger::Easy.
19 use LWP::ConsoleLogger::Easy qw( debug_ua );
21 use WWW::Mechanize;
22 my $mech = WWW::Mechanize->new; # or LWP::UserAgent->new() etc
24 my $console_logger = debug_ua( $mech );
25 $mech->get( 'https://metacpan.org' );
26 # now watch the console for debugging output
28 # turn off header dumps
29 $console_logger->dump_headers( 0 );
30 $mech->get( $some_other_url );
32 To get down to the lowest level, use LWP::ConsoleLogger directly.
34 my $ua = LWP::UserAgent->new( cookie_jar => {} );
36 my $console_logger = LWP::ConsoleLogger->new(
37 dump_content => 1,
38 dump_text => 1,
39 content_pre_filter => sub {
40 my $content = shift;
41 my $content_type = shift;
42 # mangle content here
44 # ...
45 return $content;
47 },
48 );
49 $ua->default_header(
51 'Accept-Encoding' => scalar HTTP::Message::decodable() );
52 $ua->add_handler( 'response_done',
54 sub { $console_logger->response_callback( @_ ) } );
55 $ua->add_handler( 'request_send',
56 sub { $console_logger->request_callback( @_ ) } );
57 # now watch debugging output to your screen
59 $ua->get( 'http://nytimes.com/' );
60 Sample output might look like this.
62 GET http://www.nytimes.com/2014/04/24/technology/fcc-new-net-neutrality-rules.html
64 GET params:
66 .-----+-------.
67 | Key | Value |
68 +-----+-------+
69 | _r | 1 |
70 | hp | |
71 '-----+-------'
72 .-----------------+--------------------------------.
74 | Request Header | Value |
75 +-----------------+--------------------------------+
76 | Accept-Encoding | gzip |
77 | Cookie2 | $Version="1" |
78 | Referer | http://www.nytimes.com?foo=bar |
79 | User-Agent | WWW-Mechanize/1.73 |
80 '-----------------+--------------------------------'
81 ==> 200 OK
83 Title: The New York Times - Breaking News, World News & Multimedia
85 .--------------------------+-------------------------------.
87 | Response Header | Value |
88 +--------------------------+-------------------------------+
89 | Accept-Ranges | bytes |
90 | Age | 176 |
91 | Cache-Control | no-cache |
92 | Channels | NytNow |
93 | Client-Date | Fri, 30 May 2014 22:37:42 GMT |
94 | Client-Peer | 170.149.172.130:80 |
95 | Client-Response-Num | 1 |
96 | Client-Transfer-Encoding | chunked |
97 | Connection | keep-alive |
98 | Content-Encoding | gzip |
99 | Content-Type | text/html; charset=utf-8 |
100 | Date | Fri, 30 May 2014 22:37:41 GMT |
101 | NtCoent-Length | 65951 |
102 | Server | Apache |
103 | Via | 1.1 varnish |
104 | X-Cache | HIT |
105 | X-Varnish | 1142859770 1142854917 |
106 '--------------------------+-------------------------------'
107 .--------------------------+-------------------------------.
109 | Text |
110 +--------------------------+-------------------------------+
111 | F.C.C., in a Shift, Backs Fast Lanes for Web Traffic... |
112 '--------------------------+-------------------------------'
113
115 BETA BETA BETA. This is currently an experiment. Things could change.
116 Please adjust accordingly.
117 It can be hard (or at least tedious) to debug mechanize scripts.
119 LWP::Debug is deprecated. It suggests you write your own debugging
120 handlers, set up a proxy or install Wireshark. Those are all workable
121 solutions, but this module exists to save you some of that work. The
122 guts of this module are stolen from Plack::Middleware::DebugLogging,
123 which in turn stole most of its internals from Catalyst. If you're new
124 to LWP::ConsoleLogger, I suggest getting started with the
125 LWP::ConsoleLogger::Easy wrapper. This will get you up and running in
126 minutes. If you need to tweak the settings that
127 LWP::ConsoleLogger::Easy chooses for you (or if you just want to be
128 fancy), please read on.
129 Since this is a debugging library, I've left as much mutable state as
131 possible, so that you can easily toggle output on and off and otherwise
132 adjust how you deal with the output.
133
135 new()
136 The following arguments can be passed to new(), although none are
137 required. They can also be called as methods on an instantiated
138 object. I'll list them here and discuss them in detail below.
139 • "dump_content => 0|1"
141 • "dump_cookies => 0|1"
143 • "dump_headers => 0|1"
145 • "dump_params => 0|1"
147 • "dump_status => 0|1"
149 • "dump_text => 0|1"
151 • "dump_title => 0|1"
153 • "dump_text => 0|1"
155 • "dump_uri => 0|1"
157 • "content_pre_filter => sub { ... }"
159 • "headers_to_redact => ['Authentication', 'Foo']"
161 • "params_to_redact => ['token', 'password']"
163 • "text_pre_filter => sub { ... }"
165 • "html_restrict => HTML::Restrict->new( ... )"
167 • "logger => Log::Dispatch->new( ... )"
169 • "pretty => 0|1"
171 • "term_width => $integer"
173
175 dump_content( 0|1 )
176 Boolean value. If true, the actual content of your response (HTML,
177 JSON, etc) will be dumped to your screen. Defaults to false.
178 dump_cookies( 0|1 )
180 Boolean value. If true, the content of your cookies will be dumped to
181 your screen. Defaults to false.
182 dump_headers( 0|1 )
184 Boolean value. If true, both request and response headers will be
185 dumped to your screen. Defaults to true.
186 Headers are dumped in alphabetical order.
188 dump_params( 0|1 )
190 Boolean value. If true, both GET and POST params will be dumped to your
191 screen. Defaults to true.
192 Params are dumped in alphabetical order.
194 dump_status( 0|1 )
196 Boolean value. If true, dumps the HTTP response code for each page
197 being visited. Defaults to true.
198 dump_text( 0|1 )
200 Boolean value. If true, dumps the text of your page after both the
201 content_pre_filter and text_pre_filters have been applied. Defaults to
202 true.
203 dump_title( 0|1 )
205 Boolean value. If true, dumps the titles of HTML pages if your
206 UserAgent has a "title" method and if it returns something useful.
207 Defaults to true.
208 dump_uri( 0|1 )
210 Boolean value. If true, dumps the URI of each page being visited.
211 Defaults to true.
212 pretty ( 0|1 )
214 Boolean value. If disabled, request headers, response headers, content
215 and text sections will be dumped without using tables. Handy for
216 copy/pasting JSON etc for faking responses later. Defaults to true.
217 content_pre_filter( sub { ... } )
219 Subroutine reference. This allows you to manipulate content before it
220 is dumped. A common use case might be stripping headers and footers
221 away from HTML content to make it easier to detect changes in the body
222 of the page.
223 $easy_logger->content_pre_filter(
225 sub {
226 my $content = shift;
227 my $content_type = shift; # the value of the Content-Type header
228 if ( $content_type =~ m{html}i
229 && $content =~ m{<!--\scontent\s-->(.*)<!--\sfooter}msx ) {
230 return $1;
231 }
232 return $content;
233 }
234 );
235 Try to make sure that your content mangling doesn't return broken HTML
237 as that may not play well with HTML::Restrict.
238 request_callback
240 Use this handler to set up console logging on your requests.
241 my $ua = LWP::UserAgent->new;
243 $ua->add_handler(
244 'request_send',
245 sub { $console_logger->request_callback(@_) }
246 );
247 This is done for you by default if you set up your logging via
249 LWP::ConsoleLogger::Easy.
250 response_callback
252 Use this handler to set up console logging on your responses.
253 my $ua = LWP::UserAgent->new;
255 $ua->add_handler(
256 'response_done',
257 sub { $console_logger->response_callback(@_) }
258 );
259 This is done for you by default if you set up your logging via
261 LWP::ConsoleLogger::Easy.
262 text_pre_filter( sub { ... } )
264 Subroutine reference. This allows you to manipulate text before it is
265 dumped. A common use case might be stripping away duplicate whitespace
266 and/or newlines in order to improve formatting. Keep in mind that the
267 "content_pre_filter" will have been applied to the content which is
268 passed to the text_pre_filter. The idea is that you can strip away an
269 HTML you don't care about in the content_pre_filter phase and then
270 process the remainder of the content in the text_pre_filter.
271 $easy_logger->text_pre_filter(
273 sub {
274 my $content = shift;
275 my $content_type = shift; # the value of the Content-Type header
276 my $base_url = shift;
277 # do something with the content
279 # ...
280 return ( $content, $new_content_type );
282 }
283 );
284 If your "text_pre_filter()" converts from HTML to plain text, be sure
286 to return the new content type (text/plain) when you exit the sub. If
287 you do not do this, HTML formatting will then be applied to your plain
288 text as is explained below.
289 If this is HTML content, HTML::Restrict will be applied after the
291 text_pre_filter has been run. LWP::ConsoleLogger will then strip away
292 some whitespace and newlines from processed HTML in its own opinionated
293 way, in order to present you with more readable text.
294 html_restrict( HTML::Restrict->new( ... ) )
296 If the content_type indicates HTML then HTML::Restrict will be used to
297 strip tags from your content in the text rendering process. You may
298 pass your own HTML::Restrict object, if you like. This would be
299 helpful in situations where you still do want to have some tags in your
300 text.
301 logger( Log::Dispatch->new( ... ) )
303 By default all data will be dumped to your console (as the name of this
304 module implies) using Log::Dispatch. However, you may use your own
305 Log::Dispatch module in order to facilitate logging to files or any
306 other output which Log::Dispatch supports.
307 term_width( $integer )
309 By default this module will try to find the maximum width of your
310 terminal and use all available space when displaying tabular data. You
311 may use this parameter to constrain the tables to an arbitrary width.
312
314 Aside from the BETA warnings, I should say that I've written this to
315 suit my needs and there are a lot of things I haven't considered. For
316 example, I'm mostly assuming that the content will be text, HTML, JSON
317 or XML.
318 The test suite is not very robust either. If you'd like to contribute
320 to this module and you can't find an appropriate test, do add something
321 to the example folder (either a new script or alter an existing one),
322 so that I can see what your patch does.
323
325 Olaf Alders <olaf@wundercounter.com>
326
328 This software is Copyright (c) 2014-2019 by MaxMind, Inc.
329 This is free software, licensed under:
331 The Artistic License 2.0 (GPL Compatible)
333perl v5.34.0 2021-07-22 LWP::ConsoleLogger(3)