1PMDACACHE(3) Library Functions Manual PMDACACHE(3)
2
6 pmdaCacheStore, pmdaCacheStoreKey, pmdaCacheLookup, pmdaCacheLookup‐
7 Name, pmdaCacheLookupKey, pmdaCacheOp, pmdaCachePurge, pmdaCachePurge‐
8 Callback, pmdaCacheResize - manage a cache of instance domain informa‐
9 tion for a PMDA
10
12 #include <pcp/pmapi.h>
13 #include <pcp/pmda.h>
14 int pmdaCacheStore(pmInDom indom, int flags, const char *name,
16 void *private);
17 int pmdaCacheStoreKey(pmInDom indom, int flags, const char *name,
18 int keylen, const void *key, void *private);
19 int pmdaCacheLookup(pmInDom indom, int inst, char **name,
20 void **private);
21 int pmdaCacheLookupName(pmInDom indom, const char *name, int *inst,
22 void **private);
23 int pmdaCacheLookupKey(pmInDom indom, const char *name, int keylen,
24 const void *key, char **oname, int *inst, void **private);
25 int pmdaCacheOp(pmInDom indom, int op);
26 int pmdaCachePurge(pmInDom indom, time_t recent);
27 int pmdaCachePurgeCallback(pmInDom indom, time_t recent, void
28 (*callback)(void *));
29 int pmdaCacheResize(pmInDom indom, int maximum);
30 cc ... -lpcp_pmda -lpcp
32
34 The pmdaCache family of routines provide services to support the main‐
35 tenance of complex instance domains for Performance Co-Pilot PMDAs.
36 There is potentially one cache of information for each instance domain,
37 and for each instance the cache maintains:
38 - external instance name (supplied by the PMDA)
39 - internal instance identifier (assigned by pmdaCacheStore or calculat‐
40 ed from a ``hint'' by pmdaCacheStoreKey)
41 - state, where active instances are visible and part of the current in‐
42 stance domain, and inactive instances are hidden, but not forgotten;
43 pmdaCacheStore or pmdaCacheStoreKey may be used to change the state
44 of an instance
45 - an optional opaque pointer to data that is associated with the in‐
46 stance, but maintained by the PMDA
47 - an optional opaque key that is used as a ``hint'' to pmdaCache‐
48 StoreKey when guessing the initial internal instance identifier
49 - the last time the cache was saved and the instance had been marked as
50 active at some point since the previous cache load or save operation
51 The semantics of a PCP instance domain require a number of rules to be
53 followed, namely:
54 1. Each internal instance identifier must be unique and in the range 0
55 to 2^31 - 1. This rule is enforced by the pmdaCache family of rou‐
56 tines.
57 2. The external instance name must be unique. When the instance name
58 contains a space, it is further constrained such that the name to
59 the left of the first space (the short name) must also be unique.
60 Refer to the INSTANCE NAME MATCHING section below. The PMDA must
61 honor this rule, the pmdaCache family of routines will detect at‐
62 tempts to violate this rule.
63 3. Where an external instance name corresponds to some object or enti‐
64 ty, there is an expectation that the association between the name
65 and the object is fixed, e.g. ``/dev/hda'' is always the name of the
66 same disk on a particular system. This rule is perhaps the respon‐
67 sibility of the PMDA, but is often a characteristic of the environ‐
68 ment in which the PMDA runs.
69 4. It is preferable, although not mandatory, for the association be‐
70 tween an external instance name and an internal instance identifier
71 to be persistent. This rule is supported by the pmdaCache family of
72 routines.
73 5. When opaque keys are used, the values of the keys must be unique
74 across all instances within an instance domain. This rule is en‐
75 forced by the pmdaCache family of routines.
76 The visible interface to the cache is oriented towards the PMDA devel‐
78 oper who is most concerned about the names of instances, while the de‐
79 tails of how the rest of the PCP infrastructure expects the internal
80 instance identifiers to be managed is not relevant.
81 Instances are updated in the cache for instance domain indom by calling
83 pmdaCacheStore or pmdaCacheStoreKey with the external name of the in‐
84 stance passed via name. The opaque pointer private may be used to as‐
85 sociate additional data with the entry in the cache; if no such data is
86 required, private should be NULL. Any manipulation of the additional
87 data (including allocation or freeing) is the responsibility of the PM‐
88 DA caller, as the cache simply maintains the pointer to the data
89 (passed via private).
90 The upper bound for identifiers allocated for any given indom cache can
92 be optionally reduced from the default (2^31 - 1) to some lesser maxi‐
93 mum, using pmdaCacheResize. This maximum will then be persisted and
94 restored in the usual manner, and can thus be associated permanently
95 with a cache once set. This has applications when using these inter‐
96 faces as general purpose identifier caches, and is less applicable when
97 using them for instance domain caching.
98 For cases where the PMDA developer wishes to influence the allocation
100 of internal instance identifiers, e.g. for instance domains with more
101 than one natural dimension, or where there is a desire to allocate the
102 same instance identifier each time the PMDA is started, even on differ‐
103 ent hosts, pmdaCacheStoreKey may be used. In this case, an initial
104 ``hint'' for the instance identifier is provided as an opaque key via
105 the first keylen bytes in key (which could be any sort of data, includ‐
106 ing binary values) else if keylen is less than 1 or key is NULL then
107 name is used as the ``hint''. The ``hint'' is hashed to produce an
108 initial instance identifier in the range 0 to 2^31 - 1 (or lesser maxi‐
109 mum, if set). If this instance identifier is already allocated, then
110 the value is rehashed. This procedure is repeated until an unallocated
111 instance identifier is found, or pmdaCacheStoreKey gives up and returns
112 PM_ERR_GENERIC. For each instance domain, the ``hint'' must be unique
113 across all instances, else pmdaCacheStoreKey returns PM_ERR_INST.
114 The flags argument controls how the instance should be processed in the
116 cache as follows:
117 PMDA_CACHE_ADD
119 Insert the entry into the cache if it is not already there and
120 mark it active. If the entry is already in the cache mark it
121 active.
122 PMDA_CACHE_HIDE
124 Mark the entry in the cache as inactive, but remember the de‐
125 tails of the association between the external instance name and
126 the internal instance identifier. Entries that are inactive
127 will be hidden from cache traversal via PMDA_CACHE_WALK_NEXT op‐
128 erations, but remain visible to pmdaCacheLookup, pmdaCache‐
129 LookupName and pmdaCacheLookupKey requests.
130 PMDA_CACHE_CULL
132 Remove the entry from the cache.
133 On success pmdaCacheStore or pmdaCacheStoreKey will return the internal
135 instance identifier of the associated cache entry. Valid instance
136 identifiers are guaranteed to be unique and non-negative. Failure will
137 be indicated by a negative value (suitable for decoding with
138 pmErrStr(3)) and most likely PM_ERR_INST to indicate the requested in‐
139 stance is not in the cache, or -EINVAL to indicate a potential viola‐
140 tion of the short name uniqueness property (see the INSTANCE NAME
141 MATCHING section below).
142 pmdaCacheLookup is used to search the entries in the cache based on the
144 internal instance identifier inst.
145 On success the return value will be PMDA_CACHE_ACTIVE or PMDA_CACHE_IN‐
147 ACTIVE (depending on the active or inactive state of the cache entry),
148 name (if not NULL) and private (if not NULL) will be set to the exter‐
149 nal instance name and the associate additional data area as provided
150 when the instance was last activated via pmdaCacheStore or pmdaCache‐
151 StoreKey.
152 pmdaCacheLookup failure is indicated by a negative return value suit‐
154 able for decoding with pmErrStr(3).
155 The pmdaCacheLookup interface is required by the PMDA's fetch callback
157 that is registered via pmdaSetFetchCallBack(3). Here the internal in‐
158 stance identifier is passed to the fetch callback to identifier for
159 which instance a value is required. Typical usage is shown in the code
160 fragment below.
161 static int
163 foo_callback(pmdaMetric *mdesc, unsigned int inst, pmAtomValue *atom)
164 {
165 mydata *mdp;
166 char *name;
167 int sts;
168 sts = pmdaCacheLookup(mdesc->m_desc.indom, inst, &name, (void **)&mdp);
170 /*
171 * expect sts == PMDA_CACHE_ACTIVE except for cataclysmic events
172 * use mdp as required, name may be useful for diagnostics
173 */
174 ...
175 pmdaCacheLookupName is used to search the entries in the cache based on
178 the external instance name name.
179 On success the return value will be PMDA_CACHE_ACTIVE or PMDA_CACHE_IN‐
181 ACTIVE (depending on the active or inactive state of the cache entry),
182 inst (if not NULL) and private (if not NULL) will be set to the inter‐
183 nal instance identifier and the associate additional data area as pro‐
184 vided when the instance was last activated via pmdaCacheStore or pmda‐
185 CacheStoreKey.
186 pmdaCacheLookupName failure is indicated by a negative return value
188 suitable for decoding with pmErrStr(3).
189 The pmdaCacheLookupName interface is useful for PMDAs wishing to update
191 an instance domain based on the external instance names.
192 pmdaCacheLookupKey is used to search the entries in the cache based on
194 an opaque key (or ``hint'') previously used in a call to pmdaCache‐
195 StoreKey. The ``hint'' is provided via the first keylen bytes in key.
196 For symmetry with pmdaCacheStoreKey, if keylen is less than 1 or key is
197 NULL then name is used as the ``hint'' (although the results will be
198 the same as calling pmdaCacheLookupName in this case).
199 On success the return value will be PMDA_CACHE_ACTIVE or PMDA_CACHE_IN‐
201 ACTIVE (depending on the active or inactive state of the cache entry),
202 oname (if not NULL), inst (if not NULL) and private (if not NULL) will
203 be set to the external instance name, the internal instance identifier
204 and the associate additional data area as provided when the instance
205 was last activated via pmdaCacheStore or pmdaCacheStoreKey.
206 pmdaCacheLookupKey failure is indicated by a negative return value
208 suitable for decoding with pmErrStr(3).
209 To avoid a persistent cache growing without bound, pmdaCachePurge can
211 be used to cull all entries that have not been active in the last re‐
212 cent seconds. For performance reasons, the time accounting is impre‐
213 cise and the entries are timestamped at the time of the next cache save
214 operation after the entry has been added or marked active (refer to PM‐
215 DA_CACHE_SAVE and PMDA_CACHE_SYNC below). On success pmdaCachePurge
216 returns the number of culled entries, else in the case of an error the
217 return value is negative (and suitable for decoding with pmErrStr(3)).
218 The pmdaCachePurgeCallback function is similar to pmdaCachePurge except
220 that a callback function will also be called with the private data
221 pointer associated with the cache entry to be culled. The callback is
222 not made if private is NULL. This would typically be used to free the
223 private data when the associated entry is purged in PMDAs that do not
224 separately maintain any references to the private data.
225 pmdaCacheOp may be used to perform additional operations on the cache
227 as follows:
228 PMDA_CACHE_LOAD
230 The cache can optionally be maintained as a persistent external
231 file, so that the mapping of instance names to instance identi‐
232 fiers is persistent across executions of a PMDA. This operation
233 loads the cache from the external file, and then all new cache
234 entries are marked inactive, and the additional data pointer is
235 set to NULL. Entries loaded from the external file are checked
236 against the current cache contents and if the instance name and
237 instance identifiers match then the state in the cache (active
238 or inactive) is not changed. Should a mismatch be found (same
239 instance name and different instance identifier, or same in‐
240 stance identifier and different instance name, or some but not
241 all of the instance identifier, the instance name and the
242 ``hint'' match) then the entry from the external file is ignored
243 and a warning is issued on stderr. Typically a PMDA would only
244 perform this operation once per execution.
245 PMDA_CACHE_SAVE
247 If any instance has been added to, or deleted from, the instance
248 domain since the last PMDA_CACHE_LOAD, PMDA_CACHE_SAVE or PM‐
249 DA_CACHE_SYNC operation, the entire cache is written to the ex‐
250 ternal file as a bulk operation. This operation is provided for
251 PMDAs that are not interested in using pmdaCachePurge and simply
252 want the external file to reflect the set of known instances
253 without accurate details of when they were last marked active.
254 Returns the number of instances saved to the external file, else
256 0 if the external file was already up to date.
257 PMDA_CACHE_STRINGS
259 Annotates this cache as being a special-purpose cache used for
260 string de-duplication in PMDAs exporting large numbers of string
261 valued metrics. This can be used to reduce the memory footprint
262 of the PMDA (duplicate strings hash to the same bucket, and are
263 stored in memory once only). Key comparisons are not terminated
264 at the first space but rather the entire string is used for
265 matching. These are specialised caches not useful for general
266 purpose instance domain handling.
267 PMDA_CACHE_SYNC
269 Within an instance domain, if any instance has been added to, or
270 deleted from, or marked active since the last PMDA_CACHE_LOAD,
271 PMDA_CACHE_SAVE or PMDA_CACHE_SYNC operation, the entire cache
272 is written to the external file as a bulk operation. This oper‐
273 ation is similar to PMDA_CACHE_SAVE, but will save the instance
274 domain more frequently so the timestamps more accurately match
275 the semantics expected by pmdaCachePurge.
276 Returns the number of instances saved to the external file, else
278 0 if the external file was already synchronized.
279 PMDA_CACHE_CHECK
281 Returns 1 if a cache exists for the specified instance domain,
282 else 0.
283 PMDA_CACHE_REUSE
285 When a new instance is added to the cache, the default strategy
286 is to assign instance identifiers in a monotonic increasing man‐
287 ner. Once the maximum possible instance identifier value has
288 been assigned, the strategy changes to one where starting from
289 0, the next available unused instance identifier will be used.
290 Calling pmdaCacheOp with PMDA_CACHE_REUSE forces an irreversible
291 change to a second (reuse) strategy where the next unallocated
292 instance identifier will be used. This may be useful in cases
293 where there is a desire to restrict the allocated instance iden‐
294 tifiers to smaller values. The prevailing strategy will be
295 saved and restored across PMDA_CACHE_SAVE and PMDA_CACHE_LOAD
296 operations. If pmdaCacheStoreKey is ever used, the associated
297 instance domain will be changed to PMDA_CACHE_REUSE mode.
298 PMDA_CACHE_REORG
300 Reorganize the cache to allow faster retrieval of active en‐
301 tries, at the cost of slower retrieval for inactive entries, and
302 reclaim any culled entries. The cache may be internally re-or‐
303 ganized as entries are added, so this operation is not required
304 for most PMDAs.
305 PMDA_CACHE_WALK_REWIND
307 Prepares for a traversal of the cache in ascending instance
308 identifier sequence.
309 PMDA_CACHE_WALK_NEXT
311 Fetch the next active instance identifier from the cache. Re‐
312 quires a prior call using PMDA_CACHE_WALK_REWIND and will return
313 -1 when all instances have been processed.
314 Only one cache walk can be active at any given time, nesting
316 calls to PMDA_CACHE_WALK and PMDA_CACHE_REWIND will interfere
317 with each other.
318 PMDA_CACHE_ACTIVE
320 Changes every inactive entry in the cache to be marked active.
321 PMDA_CACHE_INACTIVE
323 Changes every active entry in the cache to be marked inactive.
324 PMDA_CACHE_CULL
326 Remove every entry from the cache.
327 PMDA_CACHE_SIZE
329 Return the number of entries in the cache (includes active, in‐
330 active and any culled entries that have not yet been reclaimed).
331 PMDA_CACHE_SIZE_ACTIVE
333 Return the number of active entries in the cache.
334 PMDA_CACHE_SIZE_INACTIVE
336 Return the number of inactive entries in the cache.
337 PMDA_CACHE_DUMP
339 Dump the current state of the cache on stderr.
340 PMDA_CACHE_DUMP_ALL
342 Like PMDA_CACHE_DUMP, but also dump the internal hashing struc‐
343 tures used to support lookup by instance name, lookup by in‐
344 stance identifier and the collision statistics for ``hint''
345 hashing from pmdaCacheStoreKey.
346 pmdaCacheOp returns a non-negative value on success, and failure is in‐
348 dicated by a negative return value (suitable for decoding with
349 pmErrStr(3)).
350
352 When the pmdaCache routines are used for particular instance domain,
353 pmdaInstance(3) and the instance domain enumeration behind pmdaFetch(3)
354 will attempt to extract instance domain information from the cache,
355 thereby avoiding reference to the pmdaIndom data structures that have
356 historically been used to define instance domains and service instance
357 requests. A PMDA can adopt a hybrid approach and choose to implement
358 some instance domains via the traditional pmdaIndom method, and others
359 via the pmdaCache approach, however attempts to manage the same in‐
360 stance domain by both methods will result in the pmdaCache method
361 silently prevailing.
362 If all instances in a PMDA are to be serviced from a pmdaCache then a
364 pmdaIndom is not required, and the pmdaInit(3) call becomes
365 pmdaInit(dp, NULL, 0, metrictab, nmetrics);
367 However, the PMDA will need to explicitly initialize the indom field of
369 the pmDesc in the metrictab entries, as this cannot be done by pm‐
370 daInit(3) if indomtab is missing entries for the instance domains main‐
371 tained in the cache.
372 Independent of how the instance domain is being maintained, to refresh
374 an instance domain prior to a fetch or an instance domain operation,
375 the standard methods of a ``wrapper'' to the pmdaInstance(3) and
376 pmdaFetch(3) methods should be used.
377 Refer to the simple PMDA source code for an example use of the pmda‐
379 Cache routines.
380 When using pmdaCacheStoreKey, if there is a desire to ensure the given
382 ``hint'' generates the same initial instance identifier across all
383 platforms, then the caller should ensure the endian and word-size is‐
384 sues are considered, e.g. if the natural data structure used for the
385 key is an array of 32-bit integers, then htonl(3) should be used on
386 each element of the array before calling pmdaCacheStoreKey or pmdaCach‐
387 eLookupKey.
388
390 The following table summarizes the ``short name'' matching semantics
391 for an instance domain (caches other than PMDA_CACHE_STRINGS style).
392 ┌────────┬─────────────────┬───────────────────────────────────────────┐
394 │name in │ pmdaCacheLookup │ result │
395 │cache │ name │ │
396 ├────────┼─────────────────┼───────────────────────────────────────────┤
397 │foodle │ foo │ no match (PM_ERR_INST) │
398 │foo │ foodle │ no match (PM_ERR_INST) │
399 │foo │ foo │ match │
400 │foo bar │ foo │ match on short name (instance identifier) │
401 │foo bar │ foo bar │ match on full name (instance identifier) │
402 │foo │ foo bar │ bad match (-EDOM) │
403 │foo bar │ foo blah │ bad match (-EDOM) │
404 └────────┴─────────────────┴───────────────────────────────────────────┘
406 Cache persistence uses files with names constructed from the indom
407 within the $PCP_VAR_DIR/config/pmda directory.
408
410 BYTEORDER(3), PMAPI(3), PMDA(3), pmdaInit(3), pmdaInstance(3),
411 pmdaFetch(3), pmdaLabel(3), pmErrStr(3) and pmGetInDom(3).
412Performance Co-Pilot PCP PMDACACHE(3)