1PMDACACHE(3)               Library Functions Manual               PMDACACHE(3)
2
3
4

NAME

6       pmdaCacheStore,  pmdaCacheStoreKey,  pmdaCacheLookup,  pmdaCacheLookup‐
7       Name, pmdaCacheLookupKey, pmdaCacheOp, pmdaCachePurge,  pmdaCachePurge‐
8       Callback,  pmdaCacheResize - manage a cache of instance domain informa‐
9       tion for a PMDA
10

C SYNOPSIS

12       #include <pcp/pmapi.h>
13       #include <pcp/pmda.h>
14
15       int pmdaCacheStore(pmInDom indom, int flags, const char *name,
16               void *private);
17       int pmdaCacheStoreKey(pmInDom indom, int flags, const char *name,
18               int keylen, const void *key, void *private);
19       int pmdaCacheLookup(pmInDom indom, int inst, char **name,
20               void **private);
21       int pmdaCacheLookupName(pmInDom indom, const char *name, int *inst,
22               void **private);
23       int pmdaCacheLookupKey(pmInDom indom, const char *name, int keylen,
24               const void *key, char **oname, int *inst, void **private);
25       int pmdaCacheOp(pmInDom indom, int op);
26       int pmdaCachePurge(pmInDom indom, time_t recent);
27       int pmdaCachePurgeCallback(pmInDom indom, time_t recent, void
28               (*callback)(void *));
29       int pmdaCacheResize(pmInDom indom, int maximum);
30
31       cc ... -lpcp_pmda -lpcp
32

DESCRIPTION

34       The pmdaCache family of routines provide services to support the  main‐
35       tenance  of  complex  instance  domains for Performance Co-Pilot PMDAs.
36       There is potentially one cache of information for each instance domain,
37       and for each instance the cache maintains:
38       - external instance name (supplied by the PMDA)
39       - internal instance identifier (assigned by pmdaCacheStore or calculat‐
40         ed from a ``hint'' by pmdaCacheStoreKey)
41       - state, where active instances are visible and part of the current in‐
42         stance  domain, and inactive instances are hidden, but not forgotten;
43         pmdaCacheStore or pmdaCacheStoreKey may be used to change  the  state
44         of an instance
45       - an  optional  opaque  pointer to data that is associated with the in‐
46         stance, but maintained by the PMDA
47       - an optional opaque key that is  used  as  a  ``hint''  to  pmdaCache‐
48         StoreKey when guessing the initial internal instance identifier
49       - the last time the cache was saved and the instance had been marked as
50         active at some point since the previous cache load or save operation
51
52       The semantics of a PCP instance domain require a number of rules to  be
53       followed, namely:
54       1. Each  internal instance identifier must be unique and in the range 0
55          to 2^31 - 1.  This rule is enforced by the pmdaCache family of  rou‐
56          tines.
57       2. The  external  instance name must be unique.  When the instance name
58          contains a space, it is further constrained such that  the  name  to
59          the  left  of  the first space (the short name) must also be unique.
60          Refer to the INSTANCE NAME MATCHING section below.   The  PMDA  must
61          honor  this  rule,  the pmdaCache family of routines will detect at‐
62          tempts to violate this rule.
63       3. Where an external instance name corresponds to some object or  enti‐
64          ty,  there  is  an expectation that the association between the name
65          and the object is fixed, e.g. ``/dev/hda'' is always the name of the
66          same  disk on a particular system.  This rule is perhaps the respon‐
67          sibility of the PMDA, but is often a characteristic of the  environ‐
68          ment in which the PMDA runs.
69       4. It  is  preferable,  although not mandatory, for the association be‐
70          tween an external instance name and an internal instance  identifier
71          to be persistent.  This rule is supported by the pmdaCache family of
72          routines.
73       5. When opaque keys are used, the values of the  keys  must  be  unique
74          across  all  instances  within an instance domain.  This rule is en‐
75          forced by the pmdaCache family of routines.
76
77       The visible interface to the cache is oriented towards the PMDA  devel‐
78       oper  who is most concerned about the names of instances, while the de‐
79       tails of how the rest of the PCP infrastructure  expects  the  internal
80       instance identifiers to be managed is not relevant.
81
82       Instances are updated in the cache for instance domain indom by calling
83       pmdaCacheStore or pmdaCacheStoreKey with the external name of  the  in‐
84       stance  passed via name.  The opaque pointer private may be used to as‐
85       sociate additional data with the entry in the cache; if no such data is
86       required,  private  should be NULL.  Any manipulation of the additional
87       data (including allocation or freeing) is the responsibility of the PM‐
88       DA  caller,  as  the  cache  simply  maintains  the pointer to the data
89       (passed via private).
90
91       The upper bound for identifiers allocated for any given indom cache can
92       be  optionally reduced from the default (2^31 - 1) to some lesser maxi‐
93       mum, using pmdaCacheResize.  This maximum will then  be  persisted  and
94       restored  in  the  usual manner, and can thus be associated permanently
95       with a cache once set.  This has applications when using  these  inter‐
96       faces as general purpose identifier caches, and is less applicable when
97       using them for instance domain caching.
98
99       For cases where the PMDA developer wishes to influence  the  allocation
100       of  internal  instance identifiers, e.g. for instance domains with more
101       than one natural dimension, or where there is a desire to allocate  the
102       same instance identifier each time the PMDA is started, even on differ‐
103       ent hosts, pmdaCacheStoreKey may be used.  In  this  case,  an  initial
104       ``hint''  for  the instance identifier is provided as an opaque key via
105       the first keylen bytes in key (which could be any sort of data, includ‐
106       ing  binary  values)  else if keylen is less than 1 or key is NULL then
107       name is used as the ``hint''.  The ``hint'' is  hashed  to  produce  an
108       initial instance identifier in the range 0 to 2^31 - 1 (or lesser maxi‐
109       mum, if set).  If this instance identifier is already  allocated,  then
110       the value is rehashed.  This procedure is repeated until an unallocated
111       instance identifier is found, or pmdaCacheStoreKey gives up and returns
112       PM_ERR_GENERIC.   For each instance domain, the ``hint'' must be unique
113       across all instances, else pmdaCacheStoreKey returns PM_ERR_INST.
114
115       The flags argument controls how the instance should be processed in the
116       cache as follows:
117
118       PMDA_CACHE_ADD
119              Insert  the  entry into the cache if it is not already there and
120              mark it active.  If the entry is already in the  cache  mark  it
121              active.
122
123       PMDA_CACHE_HIDE
124              Mark  the  entry  in the cache as inactive, but remember the de‐
125              tails of the association between the external instance name  and
126              the  internal  instance  identifier.   Entries that are inactive
127              will be hidden from cache traversal via PMDA_CACHE_WALK_NEXT op‐
128              erations,  but  remain  visible  to  pmdaCacheLookup, pmdaCache‐
129              LookupName and pmdaCacheLookupKey requests.
130
131       PMDA_CACHE_CULL
132              Remove the entry from the cache.
133
134       On success pmdaCacheStore or pmdaCacheStoreKey will return the internal
135       instance  identifier  of  the  associated  cache entry.  Valid instance
136       identifiers are guaranteed to be unique and non-negative.  Failure will
137       be   indicated   by  a  negative  value  (suitable  for  decoding  with
138       pmErrStr(3)) and most likely PM_ERR_INST to indicate the requested  in‐
139       stance  is  not in the cache, or -EINVAL to indicate a potential viola‐
140       tion of the short name  uniqueness  property  (see  the  INSTANCE  NAME
141       MATCHING section below).
142
143       pmdaCacheLookup is used to search the entries in the cache based on the
144       internal instance identifier inst.
145
146       On success the return value will be PMDA_CACHE_ACTIVE or PMDA_CACHE_IN‐
147       ACTIVE  (depending on the active or inactive state of the cache entry),
148       name (if not NULL) and private (if not NULL) will be set to the  exter‐
149       nal  instance  name  and the associate additional data area as provided
150       when the instance was last activated via pmdaCacheStore  or  pmdaCache‐
151       StoreKey.
152
153       pmdaCacheLookup  failure  is indicated by a negative return value suit‐
154       able for decoding with pmErrStr(3).
155
156       The pmdaCacheLookup interface is required by the PMDA's fetch  callback
157       that  is registered via pmdaSetFetchCallBack(3).  Here the internal in‐
158       stance identifier is passed to the fetch  callback  to  identifier  for
159       which instance a value is required.  Typical usage is shown in the code
160       fragment below.
161
162         static int
163         foo_callback(pmdaMetric *mdesc, unsigned int inst, pmAtomValue *atom)
164         {
165             mydata   *mdp;
166             char     *name;
167             int      sts;
168
169             sts = pmdaCacheLookup(mdesc->m_desc.indom, inst, &name, (void **)&mdp);
170             /*
171              * expect sts == PMDA_CACHE_ACTIVE except for cataclysmic events
172              * use mdp as required, name may be useful for diagnostics
173              */
174             ...
175
176
177       pmdaCacheLookupName is used to search the entries in the cache based on
178       the external instance name name.
179
180       On success the return value will be PMDA_CACHE_ACTIVE or PMDA_CACHE_IN‐
181       ACTIVE (depending on the active or inactive state of the cache  entry),
182       inst  (if not NULL) and private (if not NULL) will be set to the inter‐
183       nal instance identifier and the associate additional data area as  pro‐
184       vided  when the instance was last activated via pmdaCacheStore or pmda‐
185       CacheStoreKey.
186
187       pmdaCacheLookupName failure is indicated by  a  negative  return  value
188       suitable for decoding with pmErrStr(3).
189
190       The pmdaCacheLookupName interface is useful for PMDAs wishing to update
191       an instance domain based on the external instance names.
192
193       pmdaCacheLookupKey is used to search the entries in the cache based  on
194       an  opaque  key  (or  ``hint'') previously used in a call to pmdaCache‐
195       StoreKey.  The ``hint'' is provided via the first keylen bytes in  key.
196       For symmetry with pmdaCacheStoreKey, if keylen is less than 1 or key is
197       NULL then name is used as the ``hint'' (although the  results  will  be
198       the same as calling pmdaCacheLookupName in this case).
199
200       On success the return value will be PMDA_CACHE_ACTIVE or PMDA_CACHE_IN‐
201       ACTIVE (depending on the active or inactive state of the cache  entry),
202       oname  (if not NULL), inst (if not NULL) and private (if not NULL) will
203       be set to the external instance name, the internal instance  identifier
204       and  the  associate  additional data area as provided when the instance
205       was last activated via pmdaCacheStore or pmdaCacheStoreKey.
206
207       pmdaCacheLookupKey failure is indicated  by  a  negative  return  value
208       suitable for decoding with pmErrStr(3).
209
210       To  avoid  a persistent cache growing without bound, pmdaCachePurge can
211       be used to cull all entries that have not been active in the  last  re‐
212       cent  seconds.   For performance reasons, the time accounting is impre‐
213       cise and the entries are timestamped at the time of the next cache save
214       operation after the entry has been added or marked active (refer to PM‐
215       DA_CACHE_SAVE and PMDA_CACHE_SYNC below).   On  success  pmdaCachePurge
216       returns  the number of culled entries, else in the case of an error the
217       return value is negative (and suitable for decoding with pmErrStr(3)).
218
219       The pmdaCachePurgeCallback function is similar to pmdaCachePurge except
220       that  a  callback  function  will  also be called with the private data
221       pointer associated with the cache entry to be culled.  The callback  is
222       not  made if private is NULL.  This would typically be used to free the
223       private data when the associated entry is purged in PMDAs that  do  not
224       separately maintain any references to the private data.
225
226       pmdaCacheOp  may  be used to perform additional operations on the cache
227       as follows:
228
229       PMDA_CACHE_LOAD
230              The cache can optionally be maintained as a persistent  external
231              file,  so that the mapping of instance names to instance identi‐
232              fiers is persistent across executions of a PMDA.  This operation
233              loads  the  cache from the external file, and then all new cache
234              entries are marked inactive, and the additional data pointer  is
235              set  to NULL.  Entries loaded from the external file are checked
236              against the current cache contents and if the instance name  and
237              instance  identifiers  match then the state in the cache (active
238              or inactive) is not changed. Should a mismatch  be  found  (same
239              instance  name  and  different  instance identifier, or same in‐
240              stance identifier and different instance name, or some  but  not
241              all  of  the  instance  identifier,  the  instance  name and the
242              ``hint'' match) then the entry from the external file is ignored
243              and  a warning is issued on stderr.  Typically a PMDA would only
244              perform this operation once per execution.
245
246       PMDA_CACHE_SAVE
247              If any instance has been added to, or deleted from, the instance
248              domain  since  the  last PMDA_CACHE_LOAD, PMDA_CACHE_SAVE or PM‐
249              DA_CACHE_SYNC operation, the entire cache is written to the  ex‐
250              ternal file as a bulk operation.  This operation is provided for
251              PMDAs that are not interested in using pmdaCachePurge and simply
252              want  the  external  file  to reflect the set of known instances
253              without accurate details of when they were last marked active.
254
255              Returns the number of instances saved to the external file, else
256              0 if the external file was already up to date.
257
258       PMDA_CACHE_STRINGS
259              Annotates  this  cache as being a special-purpose cache used for
260              string de-duplication in PMDAs exporting large numbers of string
261              valued metrics.  This can be used to reduce the memory footprint
262              of the PMDA (duplicate strings hash to the same bucket, and  are
263              stored in memory once only).  Key comparisons are not terminated
264              at the first space but rather the  entire  string  is  used  for
265              matching.   These  are specialised caches not useful for general
266              purpose instance domain handling.
267
268       PMDA_CACHE_SYNC
269              Within an instance domain, if any instance has been added to, or
270              deleted  from,  or marked active since the last PMDA_CACHE_LOAD,
271              PMDA_CACHE_SAVE or PMDA_CACHE_SYNC operation, the  entire  cache
272              is written to the external file as a bulk operation.  This oper‐
273              ation is similar to PMDA_CACHE_SAVE, but will save the  instance
274              domain  more  frequently so the timestamps more accurately match
275              the semantics expected by pmdaCachePurge.
276
277              Returns the number of instances saved to the external file, else
278              0 if the external file was already synchronized.
279
280       PMDA_CACHE_CHECK
281              Returns  1  if a cache exists for the specified instance domain,
282              else 0.
283
284       PMDA_CACHE_REUSE
285              When a new instance is added to the cache, the default  strategy
286              is to assign instance identifiers in a monotonic increasing man‐
287              ner.  Once the maximum possible instance  identifier  value  has
288              been  assigned,  the strategy changes to one where starting from
289              0, the next available unused instance identifier will  be  used.
290              Calling pmdaCacheOp with PMDA_CACHE_REUSE forces an irreversible
291              change to a second (reuse) strategy where the  next  unallocated
292              instance  identifier  will be used.  This may be useful in cases
293              where there is a desire to restrict the allocated instance iden‐
294              tifiers  to  smaller  values.   The  prevailing strategy will be
295              saved and restored across  PMDA_CACHE_SAVE  and  PMDA_CACHE_LOAD
296              operations.   If  pmdaCacheStoreKey is ever used, the associated
297              instance domain will be changed to PMDA_CACHE_REUSE mode.
298
299       PMDA_CACHE_REORG
300              Reorganize the cache to allow faster  retrieval  of  active  en‐
301              tries, at the cost of slower retrieval for inactive entries, and
302              reclaim any culled entries.  The cache may be internally  re-or‐
303              ganized  as entries are added, so this operation is not required
304              for most PMDAs.
305
306       PMDA_CACHE_WALK_REWIND
307              Prepares for a traversal of  the  cache  in  ascending  instance
308              identifier sequence.
309
310       PMDA_CACHE_WALK_NEXT
311              Fetch  the  next active instance identifier from the cache.  Re‐
312              quires a prior call using PMDA_CACHE_WALK_REWIND and will return
313              -1 when all instances have been processed.
314
315              Only  one  cache  walk  can be active at any given time, nesting
316              calls to PMDA_CACHE_WALK and  PMDA_CACHE_REWIND  will  interfere
317              with each other.
318
319       PMDA_CACHE_ACTIVE
320              Changes every inactive entry in the cache to be marked active.
321
322       PMDA_CACHE_INACTIVE
323              Changes every active entry in the cache to be marked inactive.
324
325       PMDA_CACHE_CULL
326              Remove every entry from the cache.
327
328       PMDA_CACHE_SIZE
329              Return  the number of entries in the cache (includes active, in‐
330              active and any culled entries that have not yet been reclaimed).
331
332       PMDA_CACHE_SIZE_ACTIVE
333              Return the number of active entries in the cache.
334
335       PMDA_CACHE_SIZE_INACTIVE
336              Return the number of inactive entries in the cache.
337
338       PMDA_CACHE_DUMP
339              Dump the current state of the cache on stderr.
340
341       PMDA_CACHE_DUMP_ALL
342              Like PMDA_CACHE_DUMP, but also dump the internal hashing  struc‐
343              tures  used  to  support  lookup by instance name, lookup by in‐
344              stance identifier and  the  collision  statistics  for  ``hint''
345              hashing from pmdaCacheStoreKey.
346
347       pmdaCacheOp returns a non-negative value on success, and failure is in‐
348       dicated  by  a  negative  return  value  (suitable  for  decoding  with
349       pmErrStr(3)).
350

OTHER CONSIDERATIONS

352       When  the  pmdaCache  routines are used for particular instance domain,
353       pmdaInstance(3) and the instance domain enumeration behind pmdaFetch(3)
354       will  attempt  to  extract  instance domain information from the cache,
355       thereby avoiding reference to the pmdaIndom data structures  that  have
356       historically  been used to define instance domains and service instance
357       requests.  A PMDA can adopt a hybrid approach and choose  to  implement
358       some  instance domains via the traditional pmdaIndom method, and others
359       via the pmdaCache approach, however attempts to  manage  the  same  in‐
360       stance  domain  by  both  methods  will  result in the pmdaCache method
361       silently prevailing.
362
363       If all instances in a PMDA are to be serviced from a pmdaCache  then  a
364       pmdaIndom is not required, and the pmdaInit(3) call becomes
365
366             pmdaInit(dp, NULL, 0, metrictab, nmetrics);
367
368       However, the PMDA will need to explicitly initialize the indom field of
369       the pmDesc in the metrictab entries, as this  cannot  be  done  by  pm‐
370       daInit(3) if indomtab is missing entries for the instance domains main‐
371       tained in the cache.
372
373       Independent of how the instance domain is being maintained, to  refresh
374       an  instance  domain  prior to a fetch or an instance domain operation,
375       the standard methods  of  a  ``wrapper''  to  the  pmdaInstance(3)  and
376       pmdaFetch(3) methods should be used.
377
378       Refer  to  the  simple PMDA source code for an example use of the pmda‐
379       Cache routines.
380
381       When using pmdaCacheStoreKey, if there is a desire to ensure the  given
382       ``hint''  generates  the  same  initial  instance identifier across all
383       platforms, then the caller should ensure the endian and  word-size  is‐
384       sues  are  considered,  e.g. if the natural data structure used for the
385       key is an array of 32-bit integers, then htonl(3)  should  be  used  on
386       each element of the array before calling pmdaCacheStoreKey or pmdaCach‐
387       eLookupKey.
388

INSTANCE NAME MATCHING

390       The following table summarizes the ``short  name''  matching  semantics
391       for an instance domain (caches other than PMDA_CACHE_STRINGS style).
392
393       ┌────────┬─────────────────┬───────────────────────────────────────────┐
394       │name in │ pmdaCacheLookup │ result                                    │
395       │cache   │ name            │                                           │
396       ├────────┼─────────────────┼───────────────────────────────────────────┤
397       │foodle  │ foo             │ no match (PM_ERR_INST)                    │
398       │foo     │ foodle          │ no match (PM_ERR_INST)                    │
399       │foo     │ foo             │ match                                     │
400       │foo bar │ foo             │ match on short name (instance identifier) │
401       │foo bar │ foo bar         │ match on full name (instance identifier)  │
402       │foo     │ foo bar         │ bad match (-EDOM)                         │
403       │foo bar │ foo blah        │ bad match (-EDOM)                         │
404       └────────┴─────────────────┴───────────────────────────────────────────┘

FILES

406       Cache  persistence  uses  files  with  names constructed from the indom
407       within the $PCP_VAR_DIR/config/pmda directory.
408

SEE ALSO

410       BYTEORDER(3),   PMAPI(3),   PMDA(3),   pmdaInit(3),    pmdaInstance(3),
411       pmdaFetch(3), pmdaLabel(3), pmErrStr(3) and pmGetInDom(3).
412
413
414
415Performance Co-Pilot                  PCP                         PMDACACHE(3)
Impressum