1PMDACACHE(3)               Library Functions Manual               PMDACACHE(3)
2
3
4

NAME

6       pmdaCacheStore,  pmdaCacheStoreKey,  pmdaCacheLookup,  pmdaCacheLookup‐
7       Name, pmdaCacheLookupKey, pmdaCacheOp, pmdaCachePurge,  pmdaCacheResize
8       - manage a cache of instance domain information for a PMDA
9

C SYNOPSIS

11       #include <pcp/pmapi.h>
12       #include <pcp/pmda.h>
13
14       int pmdaCacheStore(pmInDom indom, int flags, const char *name,
15               void *private);
16       int pmdaCacheStoreKey(pmInDom indom, int flags, const char *name,
17               int keylen, const void *key, void *private);
18       int pmdaCacheLookup(pmInDom indom, int inst, char **name,
19               void **private);
20       int pmdaCacheLookupName(pmInDom indom, const char *name, int *inst,
21               void **private);
22       int pmdaCacheLookupKey(pmInDom indom, const char *name, int keylen,
23               const void *key, char **oname, int *inst, void **private);
24       int pmdaCacheOp(pmInDom indom, int op);
25       int pmdaCachePurge(pmInDom indom, time_t recent);
26       int pmdaCacheResize(pmInDom indom, int maximum);
27
28       cc ... -lpcp_pmda -lpcp
29

DESCRIPTION

31       The pmdaCache family of routines provide services to support the  main‐
32       tenance  of  complex  instance  domains for Performance Co-Pilot PMDAs.
33       There is potentially one cache of information for each instance domain,
34       and for each instance the cache maintains:
35       - external instance name (supplied by the PMDA)
36       - internal instance identifier (assigned by pmdaCacheStore or calculat‐
37         ed from a ``hint'' by pmdaCacheStoreKey)
38       - state, where active instances are visible and part of the current in‐
39         stance  domain, and inactive instances are hidden, but not forgotten;
40         pmdaCacheStore or pmdaCacheStoreKey may be used to change  the  state
41         of an instance
42       - an  optional  opaque  pointer to data that is associated with the in‐
43         stance, but maintained by the PMDA
44       - an optional opaque key that is  used  as  a  ``hint''  to  pmdaCache‐
45         StoreKey when guessing the initial internal instance identifier
46       - the last time the cache was saved and the instance had been marked as
47         active at some point since the previous cache load or save operation
48
49       The semantics of a PCP instance domain require a number of rules to  be
50       followed, namely:
51       1. Each  internal instance identifier must be unique and in the range 0
52          to 2^31 - 1.  This rule is enforced by the pmdaCache family of  rou‐
53          tines.
54       2. The  external  instance name must be unique.  When the instance name
55          contains a space, it is further constrained such that  the  name  to
56          the  left  of  the first space (the short name) must also be unique.
57          Refer to the INSTANCE NAME MATCHING section below.   The  PMDA  must
58          honor  this  rule,  the pmdaCache family of routines will detect at‐
59          tempts to violate this rule.
60       3. Where an external instance name corresponds to some object or  enti‐
61          ty,  there  is  an expectation that the association between the name
62          and the object is fixed, e.g. ``/dev/hda'' is always the name of the
63          same  disk on a particular system.  This rule is perhaps the respon‐
64          sibility of the PMDA, but is often a characteristic of the  environ‐
65          ment in which the PMDA runs.
66       4. It  is  preferable,  although not mandatory, for the association be‐
67          tween an external instance name and an internal instance  identifier
68          to be persistent.  This rule is supported by the pmdaCache family of
69          routines.
70       5. When opaque keys are used, the values of the  keys  must  be  unique
71          across  all  instances  within an instance domain.  This rule is en‐
72          forced by the pmdaCache family of routines.
73
74       The visible interface to the cache is oriented towards the PMDA  devel‐
75       oper  who is most concerned about the names of instances, while the de‐
76       tails of how the rest of the PCP infrastructure  expects  the  internal
77       instance identifiers to be managed is not relevant.
78
79       Instances are updated in the cache for instance domain indom by calling
80       pmdaCacheStore or pmdaCacheStoreKey with the external name of  the  in‐
81       stance  passed via name.  The opaque pointer private may be used to as‐
82       sociate additional data with the entry in the cache; if no such data is
83       required,  private  should be NULL.  Any manipulation of the additional
84       data (including allocation or freeing) is the responsibility of the PM‐
85       DA  caller,  as  the  cache  simply  maintains  the pointer to the data
86       (passed via private).
87
88       The upper bound for identifiers allocated for any given indom cache can
89       be  optionally reduced from the default (2^31 - 1) to some lesser maxi‐
90       mum, using pmdaCacheResize.  This maximum will then  be  persisted  and
91       restored  in  the  usual manner, and can thus be associated permanently
92       with a cache once set.  This has applications when using  these  inter‐
93       faces as general purpose identifier caches, and is less applicable when
94       using them for instance domain caching.
95
96       For cases where the PMDA developer wishes to influence  the  allocation
97       of  internal  instance identifiers, e.g. for instance domains with more
98       than one natural dimension, or where there is a desire to allocate  the
99       same instance identifier each time the PMDA is started, even on differ‐
100       ent hosts, pmdaCacheStoreKey may be used.  In  this  case,  an  initial
101       ``hint''  for  the instance identifier is provided as an opaque key via
102       the first keylen bytes in key (which could be any sort of data, includ‐
103       ing  binary  values)  else if keylen is less than 1 or key is NULL then
104       name is used as the ``hint''.  The ``hint'' is  hashed  to  produce  an
105       initial instance identifier in the range 0 to 2^31 - 1 (or lesser maxi‐
106       mum, if set).  If this instance identifier is already  allocated,  then
107       the value is rehashed.  This procedure is repeated until an unallocated
108       instance identifier is found, or pmdaCacheStoreKey gives up and returns
109       PM_ERR_GENERIC.   For each instance domain, the ``hint'' must be unique
110       across all instances, else pmdaCacheStoreKey returns PM_ERR_INST.
111
112       The flags argument controls how the instance should be processed in the
113       cache as follows:
114
115       PMDA_CACHE_ADD
116              Insert  the  entry into the cache if it is not already there and
117              mark it active.  If the entry is already in the  cache  mark  it
118              active.
119
120       PMDA_CACHE_HIDE
121              Mark  the  entry  in the cache as inactive, but remember the de‐
122              tails of the association between the external instance name  and
123              the  internal  instance  identifier.   Entries that are inactive
124              will be hidden from cache traversal via PMDA_CACHE_WALK_NEXT op‐
125              erations,  but  remain  visible  to  pmdaCacheLookup, pmdaCache‐
126              LookupName and pmdaCacheLookupKey requests.
127
128       PMDA_CACHE_CULL
129              Remove the entry from the cache.
130
131       On success pmdaCacheStore or pmdaCacheStoreKey will return the internal
132       instance  identifier  of  the  associated  cache entry.  Valid instance
133       identifiers are guaranteed to be unique and non-negative.  Failure will
134       be   indicated   by  a  negative  value  (suitable  for  decoding  with
135       pmErrStr(3)) and most likely PM_ERR_INST to indicate the requested  in‐
136       stance  is  not in the cache, or -EINVAL to indicate a potential viola‐
137       tion of the short name  uniqueness  property  (see  the  INSTANCE  NAME
138       MATCHING section below).
139
140       pmdaCacheLookup is used to search the entries in the cache based on the
141       internal instance identifier inst.
142
143       On success the return value will be PMDA_CACHE_ACTIVE or PMDA_CACHE_IN‐
144       ACTIVE  (depending on the active or inactive state of the cache entry),
145       name (if not NULL) and private (if not NULL) will be set to the  exter‐
146       nal  instance  name  and the associate additional data area as provided
147       when the instance was last activated via pmdaCacheStore  or  pmdaCache‐
148       StoreKey.
149
150       pmdaCacheLookup  failure  is indicated by a negative return value suit‐
151       able for decoding with pmErrStr(3).
152
153       The pmdaCacheLookup interface is required by the PMDA's fetch  callback
154       that  is registered via pmdaSetFetchCallBack(3).  Here the internal in‐
155       stance identifier is passed to the fetch  callback  to  identifier  for
156       which instance a value is required.  Typical usage is shown in the code
157       fragment below.
158
159         static int
160         foo_callback(pmdaMetric *mdesc, unsigned int inst, pmAtomValue *atom)
161         {
162             mydata   *mdp;
163             char     *name;
164             int      sts;
165
166             sts = pmdaCacheLookup(mdesc->m_desc.indom, inst, &name, (void **)&mdp);
167             /*
168              * expect sts == PMDA_CACHE_ACTIVE except for cataclysmic events
169              * use mdp as required, name may be useful for diagnostics
170              */
171             ...
172
173
174       pmdaCacheLookupName is used to search the entries in the cache based on
175       the external instance name name.
176
177       On success the return value will be PMDA_CACHE_ACTIVE or PMDA_CACHE_IN‐
178       ACTIVE (depending on the active or inactive state of the cache  entry),
179       inst  (if not NULL) and private (if not NULL) will be set to the inter‐
180       nal instance identifier and the associate additional data area as  pro‐
181       vided  when the instance was last activated via pmdaCacheStore or pmda‐
182       CacheStoreKey.
183
184       pmdaCacheLookupName failure is indicated by  a  negative  return  value
185       suitable for decoding with pmErrStr(3).
186
187       The pmdaCacheLookupName interface is useful for PMDAs wishing to update
188       an instance domain based on the external instance names.
189
190       pmdaCacheLookupKey is used to search the entries in the cache based  on
191       an  opaque  key  (or  ``hint'') previously used in a call to pmdaCache‐
192       StoreKey.  The ``hint'' is provided via the first keylen bytes in  key.
193       For symmetry with pmdaCacheStoreKey, if keylen is less than 1 or key is
194       NULL then name is used as the ``hint'' (although the  results  will  be
195       the same as calling pmdaCacheLookupName in this case).
196
197       On success the return value will be PMDA_CACHE_ACTIVE or PMDA_CACHE_IN‐
198       ACTIVE (depending on the active or inactive state of the cache  entry),
199       oname  (if not NULL), inst (if not NULL) and private (if not NULL) will
200       be set to the external instance name, the internal instance  identifier
201       and  the  associate  additional data area as provided when the instance
202       was last activated via pmdaCacheStore or pmdaCacheStoreKey.
203
204       pmdaCacheLookupKey failure is indicated  by  a  negative  return  value
205       suitable for decoding with pmErrStr(3).
206
207       To  avoid  a persistent cache growing without bound, pmdaCachePurge can
208       be used to cull all entries that have not been active in the  last  re‐
209       cent  seconds.   For performance reasons, the time accounting is impre‐
210       cise and the entries are timestamped at the time of the next cache save
211       operation after the entry has been added or marked active (refer to PM‐
212       DA_CACHE_SAVE and PMDA_CACHE_SYNC below).   On  success  pmdaCachePurge
213       returns  the number of culled entries, else in the case of an error the
214       return value is negative (and suitable for decoding with pmErrStr(3)).
215
216       pmdaCacheOp may be used to perform additional operations on  the  cache
217       as follows:
218
219       PMDA_CACHE_LOAD
220              The  cache can optionally be maintained as a persistent external
221              file, so that the mapping of instance names to instance  identi‐
222              fiers is persistent across executions of a PMDA.  This operation
223              loads the cache from the external file, and then all  new  cache
224              entries  are marked inactive, and the additional data pointer is
225              set to NULL.  Entries loaded from the external file are  checked
226              against  the current cache contents and if the instance name and
227              instance identifiers match then the state in the  cache  (active
228              or  inactive)  is  not changed. Should a mismatch be found (same
229              instance name and different instance  identifier,  or  same  in‐
230              stance  identifier  and different instance name, or some but not
231              all of the  instance  identifier,  the  instance  name  and  the
232              ``hint'' match) then the entry from the external file is ignored
233              and a warning is issued on stderr.  Typically a PMDA would  only
234              perform this operation once per execution.
235
236       PMDA_CACHE_SAVE
237              If any instance has been added to, or deleted from, the instance
238              domain since the last PMDA_CACHE_LOAD,  PMDA_CACHE_SAVE  or  PM‐
239              DA_CACHE_SYNC  operation, the entire cache is written to the ex‐
240              ternal file as a bulk operation.  This operation is provided for
241              PMDAs that are not interested in using pmdaCachePurge and simply
242              want the external file to reflect the  set  of  known  instances
243              without accurate details of when they were last marked active.
244
245              Returns the number of instances saved to the external file, else
246              0 if the external file was already up to date.
247
248       PMDA_CACHE_STRINGS
249              Annotates this cache as being a special-purpose cache  used  for
250              string de-duplication in PMDAs exporting large numbers of string
251              valued metrics.  This can be used to reduce the memory footprint
252              of  the PMDA (duplicate strings hash to the same bucket, and are
253              stored in memory once only).  Key comparisons are not terminated
254              at  the  first  space  but  rather the entire string is used for
255              matching.  These are specialised caches not useful  for  general
256              purpose instance domain handling.
257
258       PMDA_CACHE_SYNC
259              Within an instance domain, if any instance has been added to, or
260              deleted from, or marked active since the  last  PMDA_CACHE_LOAD,
261              PMDA_CACHE_SAVE  or  PMDA_CACHE_SYNC operation, the entire cache
262              is written to the external file as a bulk operation.  This oper‐
263              ation  is similar to PMDA_CACHE_SAVE, but will save the instance
264              domain more frequently so the timestamps more  accurately  match
265              the semantics expected by pmdaCachePurge.
266
267              Returns the number of instances saved to the external file, else
268              0 if the external file was already synchronized.
269
270       PMDA_CACHE_CHECK
271              Returns 1 if a cache exists for the specified  instance  domain,
272              else 0.
273
274       PMDA_CACHE_REUSE
275              When  a new instance is added to the cache, the default strategy
276              is to assign instance identifiers in a monotonic increasing man‐
277              ner.   Once  the  maximum possible instance identifier value has
278              been assigned, the strategy changes to one where  starting  from
279              0,  the  next available unused instance identifier will be used.
280              Calling pmdaCacheOp with PMDA_CACHE_REUSE forces an irreversible
281              change  to  a second (reuse) strategy where the next unallocated
282              instance identifier will be used.  This may be useful  in  cases
283              where there is a desire to restrict the allocated instance iden‐
284              tifiers to smaller values.   The  prevailing  strategy  will  be
285              saved  and  restored  across PMDA_CACHE_SAVE and PMDA_CACHE_LOAD
286              operations.  If pmdaCacheStoreKey is ever used,  the  associated
287              instance domain will be changed to PMDA_CACHE_REUSE mode.
288
289       PMDA_CACHE_REORG
290              Reorganize  the  cache  to  allow faster retrieval of active en‐
291              tries, at the cost of slower retrieval for inactive entries, and
292              reclaim  any culled entries.  The cache may be internally re-or‐
293              ganized as entries are added, so this operation is not  required
294              for most PMDAs.
295
296       PMDA_CACHE_WALK_REWIND
297              Prepares  for  a  traversal  of  the cache in ascending instance
298              identifier sequence.
299
300       PMDA_CACHE_WALK_NEXT
301              Fetch the next active instance identifier from the  cache.   Re‐
302              quires a prior call using PMDA_CACHE_WALK_REWIND and will return
303              -1 when all instances have been processed.
304
305              Only one cache walk can be active at  any  given  time,  nesting
306              calls  to  PMDA_CACHE_WALK  and PMDA_CACHE_REWIND will interfere
307              with each other.
308
309       PMDA_CACHE_ACTIVE
310              Changes every inactive entry in the cache to be marked active.
311
312       PMDA_CACHE_INACTIVE
313              Changes every active entry in the cache to be marked inactive.
314
315       PMDA_CACHE_CULL
316              Remove every entry from the cache.
317
318       PMDA_CACHE_SIZE
319              Return the number of entries in the cache (includes active,  in‐
320              active and any culled entries that have not yet been reclaimed).
321
322       PMDA_CACHE_SIZE_ACTIVE
323              Return the number of active entries in the cache.
324
325       PMDA_CACHE_SIZE_INACTIVE
326              Return the number of inactive entries in the cache.
327
328       PMDA_CACHE_DUMP
329              Dump the current state of the cache on stderr.
330
331       PMDA_CACHE_DUMP_ALL
332              Like  PMDA_CACHE_DUMP, but also dump the internal hashing struc‐
333              tures used to support lookup by instance  name,  lookup  by  in‐
334              stance  identifier  and  the  collision  statistics for ``hint''
335              hashing from pmdaCacheStoreKey.
336
337       pmdaCacheOp returns a non-negative value on success, and failure is in‐
338       dicated  by  a  negative  return  value  (suitable  for  decoding  with
339       pmErrStr(3)).
340

OTHER CONSIDERATIONS

342       When the pmdaCache routines are used for  particular  instance  domain,
343       pmdaInstance(3) and the instance domain enumeration behind pmdaFetch(3)
344       will attempt to extract instance domain  information  from  the  cache,
345       thereby  avoiding  reference to the pmdaIndom data structures that have
346       historically been used to define instance domains and service  instance
347       requests.   A  PMDA can adopt a hybrid approach and choose to implement
348       some instance domains via the traditional pmdaIndom method, and  others
349       via  the  pmdaCache  approach,  however attempts to manage the same in‐
350       stance domain by both methods  will  result  in  the  pmdaCache  method
351       silently prevailing.
352
353       If  all  instances in a PMDA are to be serviced from a pmdaCache then a
354       pmdaIndom is not required, and the pmdaInit(3) call becomes
355
356             pmdaInit(dp, NULL, 0, metrictab, nmetrics);
357
358       However, the PMDA will need to explicitly initialize the indom field of
359       the  pmDesc  in  the  metrictab  entries, as this cannot be done by pm‐
360       daInit(3) if indomtab is missing entries for the instance domains main‐
361       tained in the cache.
362
363       Independent  of how the instance domain is being maintained, to refresh
364       an instance domain prior to a fetch or an  instance  domain  operation,
365       the  standard  methods  of  a  ``wrapper''  to  the pmdaInstance(3) and
366       pmdaFetch(3) methods should be used.
367
368       Refer to the simple PMDA source code for an example use  of  the  pmda‐
369       Cache routines.
370
371       When  using pmdaCacheStoreKey, if there is a desire to ensure the given
372       ``hint'' generates the same  initial  instance  identifier  across  all
373       platforms,  then  the caller should ensure the endian and word-size is‐
374       sues are considered, e.g. if the natural data structure  used  for  the
375       key  is  an  array  of 32-bit integers, then htonl(3) should be used on
376       each element of the array before calling pmdaCacheStoreKey or pmdaCach‐
377       eLookupKey.
378

INSTANCE NAME MATCHING

380       The  following  table  summarizes the ``short name'' matching semantics
381       for an instance domain (caches other than PMDA_CACHE_STRINGS style).
382
383       ┌────────┬─────────────────┬───────────────────────────────────────────┐
384       │name in │ pmdaCacheLookup │ result                                    │
385       │cache   │ name            │                                           │
386       ├────────┼─────────────────┼───────────────────────────────────────────┤
387       │foodle  │ foo             │ no match (PM_ERR_INST)                    │
388       │foo     │ foodle          │ no match (PM_ERR_INST)                    │
389       │foo     │ foo             │ match                                     │
390       │foo bar │ foo             │ match on short name (instance identifier) │
391       │foo bar │ foo bar         │ match on full name (instance identifier)  │
392       │foo     │ foo bar         │ bad match (-EDOM)                         │
393       │foo bar │ foo blah        │ bad match (-EDOM)                         │
394       └────────┴─────────────────┴───────────────────────────────────────────┘

FILES

396       Cache persistence uses files with  names  constructed  from  the  indom
397       within the $PCP_VAR_DIR/config/pmda directory.
398

SEE ALSO

400       BYTEORDER(3),    PMAPI(3),   PMDA(3),   pmdaInit(3),   pmdaInstance(3),
401       pmdaFetch(3), pmdaLabel(3), pmErrStr(3) and pmGetInDom(3).
402
403
404
405Performance Co-Pilot                  PCP                         PMDACACHE(3)
Impressum