1CALAMARIS(1)                   Calamaris Manual                   CALAMARIS(1)
2
3
4

NAME

6       calamaris  -  generate  statistic  out  of  log files from Proxy-Cache-
7       Servers
8
9

SYNOPSIS

11       cat log | calamaris [ calamaris_options ]
12
13

DESCRIPTION

15       Calamaris is used to produce statistical output from  Squid,  NetCache,
16       Inktomi  Traffic  Server,  Oops!  proxy server, Compaq Tasksmart, Cisco
17       Content Engines or related Proxy log files.  The resulting  output  can
18       be ascii or html.  It is possible to cache calculated data in a file to
19       use them in later runs.
20
21       This manual page describes the options of Calamaris  and  gives  a  few
22       examples.
23
24

OPTIONS

26   Reports
27       -a     all: extracts all reports available, -a equals -D 10 -d 20 -P 60
28              -r 20 -s -t 20.
29
30
31       -d n   domain report: switches the  top  level  and  the  second  level
32              report  on.   The  data  is derived from the URL.  The output is
33              limited by n.  (-1 is unlimited)
34
35
36       -p type
37              peak report: Measures the peaks of the Proxy usage  in  requests
38              per  second, per minute and per hour.  It reports peaks for TCP,
39              UDP and ALL requests. If set to 'old' these values  were  calcu‐
40              lated  with  the old slow method, if set to 'new' the new faster
41              (but still slow) method is used.
42
43
44       -P n   performance report: Shows the throughput  of  TCP  requests  for
45              every n minutes.
46
47
48       -r n   requester report: Switches the UDP and TCP requester reports on.
49              The output is limited by n.  (-1 is unlimited)
50
51
52       -R n   Requested report: adds to each line of the requester report  the
53              requested  URLs.  The output is limited by n.  (-1 is unlimited,
54              and can result in very very long reports.)
55
56              Warning: Using this option breaks the  privacy  of  your  users!
57              (see PRIVACY-Section below)
58
59
60       -s     status  report:  alters  the  default behaviour of Calamaris and
61              makes the status reports more verbose.
62
63
64       -t n   type report: switches the content type and  the  file  extension
65              report on.  The output is limited by n.  (-1 is unlimited)
66
67
68       -D n   Distribution   Histogram:   shows   size-based  distribution  of
69              requested  objects,  smaller  numbers  result  in  more  verbose
70              reports. (choose 2, 10 or 100 for useful output.)
71
72              Allocation  report: shows how different sized objects perform on
73              your Proxy.
74
75
76   Input Format
77       -f type
78              format: sets the type of input logfiles. If set to
79
80              'auto' Calamaris tries to guess the input file format.  This  is
81              the Default.
82
83              Note:  If  the first line of your input file is corrupted, Cala‐
84              maris will stop with an error.
85
86              'squid' Calamaris expects  native  logfile  derived  from  Squid
87              V1.1.beta26-V2.x or OOPS.
88
89              'squid-extended'  Calamaris  expects native logfile derived from
90              Squid V1.1.alpha1-V2.x with log_mime_hdrs enabled or Squid  with
91              Smartfilter-Patch.  or squid-style logfiles out of Cisco Content
92              Engines.  (This only enables parsing of these kind  of  logfile,
93              the additional data will be ignored.)
94
95              (Logging  of MIME-Headers breaks the privacy of your users! (see
96              PRIVACY-Section below)
97
98              'squid-old' Calamaris expects native logfile derived from  Squid
99              V1.1.alpha1-V1.1.beta25.
100
101              'nc'  Calamaris  expects  Logfiles  from  NetCache  up  to V4.x.
102              (Please see the README on this.)
103
104              'its' Calamaris expects Logfiles from Inktomi Traffic Server.
105
106              'elff' Calamaris expects Logfiles  in  Extended  Logfile  Format
107              (i.e.  from  Compaq Tasksmart, Novell Internet Caching System or
108              NetCache V5.x )
109
110              'nse' Calamaris expects Logfiles in Netscape Extended-1 or  Net‐
111              scape  Extended-2  Logfile  Format  (from Netscape/ipoint Proxy-
112              Server )
113
114
115   Output Format
116       Standard output format is plain ascii with 80 chars width.
117
118
119       -F type[,type[,type[,...]]]
120              Format: sets the format of the output-report. If set to
121
122              'mail' adds a subject header to the beginning of the report.
123
124              'html' all output is given in html with tables.  Can be combined
125              with 'mail' to send html mails.
126
127              'html-embed'  all  output  is  given in html with tables without
128              HTML-Headers. Useful for Server-Side-Includes.
129
130              'unformatted' gives out the raw  numbers  seperated  by  spaces.
131              Useful for re-using the output in other scripts. If you use this
132              along with -U, the byte values are calculated in the given Unit,
133              and  displayed  without  indication  along with the numbers. the
134              indication moves up to the header of the report.
135
136
137       -M string
138              Meta: adds a custom string into the  <HEAD>  of  a  HTML-Report.
139              Useful  if  you  want  to  add  Stylesheets  or something to the
140              Report.
141
142              Note: -M works only in combination with -F html.
143
144
145       -l string
146              logo: add a custom string to a HTML-Report. It'll be added to  a
147              table     on    the    top    of    the    output.     -l    '<A
148              HREF="http://cord.de/"><IMG                             BORDER=0
149              SRC="http://cord.de/Images/cord/cordlog2n.gif"  ALT="Cord"></A>'
150              will add my logo with a link to the Report.
151
152              Note: -l works only in combination with -F html.
153
154
155       -U string
156              Unit: You can define this to K(ilo), M(ega),  G(iga)  or  T(era)
157              for the Byte-units.
158
159
160       -S n[,n[,n[,...]]]
161              Show:  Shows  only  the  defined reports in the specified order.
162              Default is to display the reports as they  are  defined  through
163              the report-switches above. The following numbers are defined:
164
165              0  Summary
166              1  Incoming request peak per protocol
167              2  Incoming transfer volume per protocol
168              3  Incoming requests by method
169              4  Incoming UDP-requests by status
170              5  Incoming TCP-requests by status
171              6  Outgoing requests by status
172              7  Outgoing requests by destination
173              8  Request-destinations by 2ndlevel-domain
174              9  Request-destinations by toplevel-domain
175              10 TCP-Request-protocol
176              11 Requested content-type
177              12 Requested extensions
178              13 Incoming UDP-requests by host
179              14 Incoming TCP-requests by host
180              15 Distribution Histogram
181              16 Performance in  minute steps
182
183              Note: Using this doesn't make Calamaris any faster, the internal
184              calculations will be done as the report-switches were  set  (see
185              above).
186
187
188   Caching
189       -i file
190              input file: You can reuse a cache file generated with -o file to
191              add old data to a new report.  Several files  can  be  seperated
192              with a ':'.
193
194              Note:  if  you  use more than one cache file, make sure they are
195              chronologicaly ordered (oldest first).
196
197              Note: if you reuse cache-files, which were not created  with  -d
198              -1  -r  -1  -t  -1  -R -1 the number of 'others' is likely wrong
199              everywhere.
200
201
202       -o file
203              output file: Calamaris stores a summary of the computed informa‐
204              tion in file and you can reuse it at a later time with -i.
205
206              Note:  The  output file can be the same as the input file: it is
207              simply overwritten after reading the data.   It  is  not  recom‐
208              mended  to  change  the  options  between  different runs if you
209              include older data as this can result in strange measurements.
210
211
212   Misc
213       -H name
214              Host name: The name for the title or subject of the output.   If
215              set  to  'lookup' Calamaris looks up the host name of the system
216              its been run on.
217
218
219       -n     no lookup: Switches the IP lookup for the requesters off.
220
221
222       -O     Order: Changes the sort order in the reports  to  request  size,
223              default is sorting by number of requests.
224
225
226       -I t-t Interval:  defines  which time-interval should be parsed.  t has
227              to be the format yyyymmddhhmmss
228
229              Note: omitting the beginning or ending date is allowed.
230
231
232       -N     N-level: All URL-Host reports will be switched from 2nd-level to
233              N-level-reports. (-1 shows a full urlhost-report)
234
235              Note: This option is only useful with activated domain-report.
236
237
238       -T n   Time:  Adjust the Performancereport in minutes for non GMT-Time‐
239              zoners.
240
241
242       -c     case:  Switch  to  case-insensitive.  This  is  useful  for  the
243              'Requested extensions' report.
244
245
246       -u     user:  Switches the usage of eventually available ident informa‐
247              tion for requester reports on.
248
249              Warning: This breaks the privacy of your  users!  (see  PRIVACY-
250              Section below)
251
252
253       -z     zero:  Switches  reading  from  standard input off.  You can use
254              this to merge many cache files to one (see -i and -o) or to gen‐
255              erate a report out of cache files.
256
257
258       -b n   benchmark:  A  switch  for  the impatient as it prints a '#' for
259              every n parsed lines.
260
261
262       -v     verbose: print more information about what is Calamaris is doing
263              and believing.
264
265
266       -L     Loop: prints the internal loop to STDERR. (for Debugging)
267
268
269       -C     Copyright: Prints the copyright information of Calamaris.
270
271
272       -h     help: Prints a brief description of the command line options.
273
274
275       -V     Version: Prints out the Version-Number.
276
277

EXAMPLES

279       This example mails the daily statistics to root:
280              cat  /var/log/squid/access.log  |  nice -39 calamaris -aH 'daily
281              worf' -F mail | mail root
282
283
284       This one only caches a summary for later use:
285              cat /var/log/squid/access.log  |  calamaris  -a  -o  daily.`date
286              +"%w"` > /dev/null
287
288
289       You can then use the caches to have weekly statistics:
290
291              if    [   $DAYOFWEEK   =   "0"   ];   then   calamaris   -a   -i
292              daily.1:daily.2:daily.3:daily.4:daily.5:daily.6:daily.0  -z   -F
293              mail -H "weekly worf" | mail root ; fi
294
295

BUGS

297       If you have a problem with Calamaris, please make sure that you use the
298       recent version of Calamaris (see VERSION below). Also check  that  your
299       proxy  works  correctly  and doesn't produce invalid Logfiles. (see the
300       README for buglist and pointers.)
301
302       If you're sure that you've  encountered  a  bug  in  Calamaris,  please
303       report  it  to  Calamaris-bug@cord.de.  This  also applies if Calamaris
304       itself says 'please report this'.
305
306

PRIVACY

308       Calamaris can be (mis-)used to track what users are requesting.
309
310       So please read the following and think about it, before using Calamaris
311       to be the Big Brother.
312
313
314       -      If you don't trust your users than there is something more wrong
315              than the loss of productivity.
316
317
318       -      Squid has some nice acl-mechanisms. If you think that your users
319              don't use the net properly, don't let them use it. (You can also
320              open the net at specific times or  to  specific  sites,  if  you
321              want.)
322
323
324       -      If   you  still  want  to  use  Calamaris  that  way,  let  your
325              vict^Wusers know, that they'll be  monitored.  (in  Germany  you
326              have to let them know!)
327
328

SEE ALSO

330       squid(8)
331
332

AUTHOR

334       Cord  Beermann  <Cord@Wunder-Nett.org>.  There are also a lot of people
335       who contributed code, gave ideas or requested features. Look them up in
336       the executable.
337
338       This     man    page    was    written    by    Philipp    Frauenfelder
339       <pfrauenf@debian.org>, maintainer of the Debian  package.   Maintenance
340       is now taken over by Cord Beermann.
341
342

VERSION

344       Version  of  this  manpage: $Id: calamaris.1,v 1.28 2003/05/14 21:22:37
345       cord Exp $
346
347       It describes the usage of Calamaris V2.58 and later.
348
349       Information about new releases, mailing lists, and other related issues
350       can be found from the Calamaris home page at http://Calamaris.Cord.de/
351
352

WARRANTY

354       Calamaris comes with "absolutely no warranty".
355
356
358       Copyright (C) 1997, 1998, 1999, 2000, 2001, 2002, 2003 Cord Beermann
359
360       This program is free software; you can redistribute it and/or modify it
361       under the terms of the GNU General Public License as published  by  the
362       Free  Software Foundation; either version 2 of the License, or (at your
363       option) any later version.
364
365       (If you modify and want to publish it under the name Calamaris,  please
366       ask  me.   I  don't  want to confuse the 'audience' with many different
367       versions of the same name and/or Version number.  (This is not part  of
368       the license, it is only a favour i asked of you.))
369
370       This  program  is  distributed  in the hope that it will be useful, but
371       WITHOUT ANY  WARRANTY;  without  even  the  implied  warranty  of  MER‐
372       CHANTABILITY  or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General
373       Public License for more details.
374
375       You should have received a copy of the GNU General Public License along
376       with this program; if not, write to the Free Software Foundation, Inc.,
377       59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
378
379
380
381Handmade                 $Date: 2003/05/14 21:22:37 $             CALAMARIS(1)
Impressum