1CALAMARIS(1) Calamaris Manual CALAMARIS(1)
2
6 calamaris - generate statistic out of log files from Proxy-Cache-
7 Servers
8
11 cat log | calamaris [ calamaris_options ]
12
15 Calamaris is used to produce statistical output from Squid, NetCache,
16 Inktomi Traffic Server, Oops! proxy server, Compaq Tasksmart, Cisco
17 Content Engines or related Proxy log files. The resulting output can
18 be ascii or html. It is possible to cache calculated data in a file to
19 use them in later runs.
20 This manual page describes the options of Calamaris and gives a few
22 examples.
23
26 Reports
27 -a all: extracts all reports available, -a equals -D 10 -d 20 -P 60
28 -r 20 -s -t 20.
29 -d n domain report: switches the top level and the second level
32 report on. The data is derived from the URL. The output is
33 limited by n. (-1 is unlimited)
34 -p type
37 peak report: Measures the peaks of the Proxy usage in requests
38 per second, per minute and per hour. It reports peaks for TCP,
39 UDP and ALL requests. If set to 'old' these values were calcu‐
40 lated with the old slow method, if set to 'new' the new faster
41 (but still slow) method is used.
42 -P n performance report: Shows the throughput of TCP requests for
45 every n minutes.
46 -r n requester report: Switches the UDP and TCP requester reports on.
49 The output is limited by n. (-1 is unlimited)
50 -R n Requested report: adds to each line of the requester report the
53 requested URLs. The output is limited by n. (-1 is unlimited,
54 and can result in very very long reports.)
55 Warning: Using this option breaks the privacy of your users!
57 (see PRIVACY-Section below)
58 -s status report: alters the default behaviour of Calamaris and
61 makes the status reports more verbose.
62 -t n type report: switches the content type and the file extension
65 report on. The output is limited by n. (-1 is unlimited)
66 -D n Distribution Histogram: shows size-based distribution of
69 requested objects, smaller numbers result in more verbose
70 reports. (choose 2, 10 or 100 for useful output.)
71 Allocation report: shows how different sized objects perform on
73 your Proxy.
74 Input Format
77 -f type
78 format: sets the type of input logfiles. If set to
79 'auto' Calamaris tries to guess the input file format. This is
81 the Default.
82 Note: If the first line of your input file is corrupted, Cala‐
84 maris will stop with an error.
85 'squid' Calamaris expects native logfile derived from Squid
87 V1.1.beta26-V2.x or OOPS.
88 'squid-extended' Calamaris expects native logfile derived from
90 Squid V1.1.alpha1-V2.x with log_mime_hdrs enabled or Squid with
91 Smartfilter-Patch. or squid-style logfiles out of Cisco Content
92 Engines. (This only enables parsing of these kind of logfile,
93 the additional data will be ignored.)
94 (Logging of MIME-Headers breaks the privacy of your users! (see
96 PRIVACY-Section below)
97 'squid-old' Calamaris expects native logfile derived from Squid
99 V1.1.alpha1-V1.1.beta25.
100 'nc' Calamaris expects Logfiles from NetCache up to V4.x.
102 (Please see the README on this.)
103 'its' Calamaris expects Logfiles from Inktomi Traffic Server.
105 'elff' Calamaris expects Logfiles in Extended Logfile Format
107 (i.e. from Compaq Tasksmart, Novell Internet Caching System or
108 NetCache V5.x )
109 'nse' Calamaris expects Logfiles in Netscape Extended-1 or Net‐
111 scape Extended-2 Logfile Format (from Netscape/ipoint Proxy-
112 Server )
113 Output Format
116 Standard output format is plain ascii with 80 chars width.
117 -F type[,type[,type[,...]]]
120 Format: sets the format of the output-report. If set to
121 'mail' adds a subject header to the beginning of the report.
123 'html' all output is given in html with tables. Can be combined
125 with 'mail' to send html mails.
126 'html-embed' all output is given in html with tables without
128 HTML-Headers. Useful for Server-Side-Includes.
129 'unformatted' gives out the raw numbers seperated by spaces.
131 Useful for re-using the output in other scripts. If you use this
132 along with -U, the byte values are calculated in the given Unit,
133 and displayed without indication along with the numbers. the
134 indication moves up to the header of the report.
135 -M string
138 Meta: adds a custom string into the <HEAD> of a HTML-Report.
139 Useful if you want to add Stylesheets or something to the
140 Report.
141 Note: -M works only in combination with -F html.
143 -l string
146 logo: add a custom string to a HTML-Report. It'll be added to a
147 table on the top of the output. -l '<A
148 HREF="http://cord.de/"><IMG BORDER=0
149 SRC="http://cord.de/Images/cord/cordlog2n.gif" ALT="Cord"></A>'
150 will add my logo with a link to the Report.
151 Note: -l works only in combination with -F html.
153 -U string
156 Unit: You can define this to K(ilo), M(ega), G(iga) or T(era)
157 for the Byte-units.
158 -S n[,n[,n[,...]]]
161 Show: Shows only the defined reports in the specified order.
162 Default is to display the reports as they are defined through
163 the report-switches above. The following numbers are defined:
164 0 Summary
166 1 Incoming request peak per protocol
167 2 Incoming transfer volume per protocol
168 3 Incoming requests by method
169 4 Incoming UDP-requests by status
170 5 Incoming TCP-requests by status
171 6 Outgoing requests by status
172 7 Outgoing requests by destination
173 8 Request-destinations by 2ndlevel-domain
174 9 Request-destinations by toplevel-domain
175 10 TCP-Request-protocol
176 11 Requested content-type
177 12 Requested extensions
178 13 Incoming UDP-requests by host
179 14 Incoming TCP-requests by host
180 15 Distribution Histogram
181 16 Performance in minute steps
182 Note: Using this doesn't make Calamaris any faster, the internal
184 calculations will be done as the report-switches were set (see
185 above).
186 Caching
189 -i file
190 input file: You can reuse a cache file generated with -o file to
191 add old data to a new report. Several files can be seperated
192 with a ':'.
193 Note: if you use more than one cache file, make sure they are
195 chronologicaly ordered (oldest first).
196 Note: if you reuse cache-files, which were not created with -d
198 -1 -r -1 -t -1 -R -1 the number of 'others' is likely wrong
199 everywhere.
200 -o file
203 output file: Calamaris stores a summary of the computed informa‐
204 tion in file and you can reuse it at a later time with -i.
205 Note: The output file can be the same as the input file: it is
207 simply overwritten after reading the data. It is not recom‐
208 mended to change the options between different runs if you
209 include older data as this can result in strange measurements.
210 Misc
213 -H name
214 Host name: The name for the title or subject of the output. If
215 set to 'lookup' Calamaris looks up the host name of the system
216 its been run on.
217 -n no lookup: Switches the IP lookup for the requesters off.
220 -O Order: Changes the sort order in the reports to request size,
223 default is sorting by number of requests.
224 -I t-t Interval: defines which time-interval should be parsed. t has
227 to be the format yyyymmddhhmmss
228 Note: omitting the beginning or ending date is allowed.
230 -N N-level: All URL-Host reports will be switched from 2nd-level to
233 N-level-reports. (-1 shows a full urlhost-report)
234 Note: This option is only useful with activated domain-report.
236 -T n Time: Adjust the Performancereport in minutes for non GMT-Time‐
239 zoners.
240 -c case: Switch to case-insensitive. This is useful for the
243 'Requested extensions' report.
244 -u user: Switches the usage of eventually available ident informa‐
247 tion for requester reports on.
248 Warning: This breaks the privacy of your users! (see PRIVACY-
250 Section below)
251 -z zero: Switches reading from standard input off. You can use
254 this to merge many cache files to one (see -i and -o) or to gen‐
255 erate a report out of cache files.
256 -b n benchmark: A switch for the impatient as it prints a '#' for
259 every n parsed lines.
260 -v verbose: print more information about what is Calamaris is doing
263 and believing.
264 -L Loop: prints the internal loop to STDERR. (for Debugging)
267 -C Copyright: Prints the copyright information of Calamaris.
270 -h help: Prints a brief description of the command line options.
273 -V Version: Prints out the Version-Number.
276
279 This example mails the daily statistics to root:
280 cat /var/log/squid/access.log | nice -39 calamaris -aH 'daily
281 worf' -F mail | mail root
282 This one only caches a summary for later use:
285 cat /var/log/squid/access.log | calamaris -a -o daily.`date
286 +"%w"` > /dev/null
287 You can then use the caches to have weekly statistics:
290 if [ $DAYOFWEEK = "0" ]; then calamaris -a -i
292 daily.1:daily.2:daily.3:daily.4:daily.5:daily.6:daily.0 -z -F
293 mail -H "weekly worf" | mail root ; fi
294
297 If you have a problem with Calamaris, please make sure that you use the
298 recent version of Calamaris (see VERSION below). Also check that your
299 proxy works correctly and doesn't produce invalid Logfiles. (see the
300 README for buglist and pointers.)
301 If you're sure that you've encountered a bug in Calamaris, please
303 report it to Calamaris-bug@cord.de. This also applies if Calamaris
304 itself says 'please report this'.
305
308 Calamaris can be (mis-)used to track what users are requesting.
309 So please read the following and think about it, before using Calamaris
311 to be the Big Brother.
312 - If you don't trust your users than there is something more wrong
315 than the loss of productivity.
316 - Squid has some nice acl-mechanisms. If you think that your users
319 don't use the net properly, don't let them use it. (You can also
320 open the net at specific times or to specific sites, if you
321 want.)
322 - If you still want to use Calamaris that way, let your
325 vict^Wusers know, that they'll be monitored. (in Germany you
326 have to let them know!)
327
330 squid(8)
331
334 Cord Beermann <Cord@Wunder-Nett.org>. There are also a lot of people
335 who contributed code, gave ideas or requested features. Look them up in
336 the executable.
337 This man page was written by Philipp Frauenfelder
339 <pfrauenf@debian.org>, maintainer of the Debian package. Maintenance
340 is now taken over by Cord Beermann.
341
344 Version of this manpage: $Id: calamaris.1,v 1.28 2003/05/14 21:22:37
345 cord Exp $
346 It describes the usage of Calamaris V2.58 and later.
348 Information about new releases, mailing lists, and other related issues
350 can be found from the Calamaris home page at http://Calamaris.Cord.de/
351
354 Calamaris comes with "absolutely no warranty".
355
358 Copyright (C) 1997, 1998, 1999, 2000, 2001, 2002, 2003 Cord Beermann
359 This program is free software; you can redistribute it and/or modify it
361 under the terms of the GNU General Public License as published by the
362 Free Software Foundation; either version 2 of the License, or (at your
363 option) any later version.
364 (If you modify and want to publish it under the name Calamaris, please
366 ask me. I don't want to confuse the 'audience' with many different
367 versions of the same name and/or Version number. (This is not part of
368 the license, it is only a favour i asked of you.))
369 This program is distributed in the hope that it will be useful, but
371 WITHOUT ANY WARRANTY; without even the implied warranty of MER‐
372 CHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General
373 Public License for more details.
374 You should have received a copy of the GNU General Public License along
376 with this program; if not, write to the Free Software Foundation, Inc.,
377 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
378Handmade $Date: 2003/05/14 21:22:37 $ CALAMARIS(1)
