1PMDACACHE(3) Library Functions Manual PMDACACHE(3)
2
6 pmdaCacheStore, pmdaCacheStoreKey, pmdaCacheLookup, pmdaCacheLookup‐
7 Name, pmdaCacheLookupKey, pmdaCacheOp, pmdaCachePurge, pmdaCacheResize
8 - manage a cache of instance domain information for a PMDA
9
11 #include <pcp/pmapi.h>
12 #include <pcp/pmda.h>
13 int pmdaCacheStore(pmInDom indom, int flags, const char *name,
15 void *private);
16 int pmdaCacheStoreKey(pmInDom indom, int flags, const char *name,
17 int keylen, const void *key, void *private);
18 int pmdaCacheLookup(pmInDom indom, int inst, char **name,
19 void **private);
20 int pmdaCacheLookupName(pmInDom indom, const char *name, int *inst,
21 void **private);
22 int pmdaCacheLookupKey(pmInDom indom, const char *name, int keylen,
23 const void *key, char **oname, int *inst, void **private);
24 int pmdaCacheOp(pmInDom indom, int op);
25 int pmdaCachePurge(pmInDom indom, time_t recent);
26 int pmdaCacheResize(pmInDom indom, int maximum);
27 cc ... -lpcp_pmda -lpcp
29
31 The pmdaCache family of routines provide services to support the main‐
32 tenance of complex instance domains for Performance Co-Pilot PMDAs.
33 There is potentially one cache of information for each instance domain,
34 and for each instance the cache maintains:
35 - external instance name (supplied by the PMDA)
36 - internal instance identifier (assigned by pmdaCacheStore or calculat‐
37 ed from a ``hint'' by pmdaCacheStoreKey)
38 - state, where active instances are visible and part of the current in‐
39 stance domain, and inactive instances are hidden, but not forgotten;
40 pmdaCacheStore or pmdaCacheStoreKey may be used to change the state
41 of an instance
42 - an optional opaque pointer to data that is associated with the in‐
43 stance, but maintained by the PMDA
44 - an optional opaque key that is used as a ``hint'' to pmdaCache‐
45 StoreKey when guessing the initial internal instance identifier
46 - the last time the cache was saved and the instance had been marked as
47 active at some point since the previous cache load or save operation
48 The semantics of a PCP instance domain require a number of rules to be
50 followed, namely:
51 1. Each internal instance identifier must be unique and in the range 0
52 to 2^31 - 1. This rule is enforced by the pmdaCache family of rou‐
53 tines.
54 2. The external instance name must be unique. When the instance name
55 contains a space, it is further constrained such that the name to
56 the left of the first space (the short name) must also be unique.
57 Refer to the INSTANCE NAME MATCHING section below. The PMDA must
58 honor this rule, the pmdaCache family of routines will detect at‐
59 tempts to violate this rule.
60 3. Where an external instance name corresponds to some object or enti‐
61 ty, there is an expectation that the association between the name
62 and the object is fixed, e.g. ``/dev/hda'' is always the name of the
63 same disk on a particular system. This rule is perhaps the respon‐
64 sibility of the PMDA, but is often a characteristic of the environ‐
65 ment in which the PMDA runs.
66 4. It is preferable, although not mandatory, for the association be‐
67 tween and external instance name and an internal instance identifier
68 to be persistent. This rule is supported by the pmdaCache family of
69 routines.
70 5. When opaque keys are used, the values of the keys must be unique
71 across all instances within an instance domain. This rule is en‐
72 forced by the pmdaCache family of routines.
73 The visible interface to the cache is oriented towards the PMDA devel‐
75 oper who is most concerned about the names of instances, while the de‐
76 tails of how the rest of the PCP infrastructure expects the internal
77 instance identifiers to be managed is not relevant.
78 Instances are updated in the cache for instance domain indom by calling
80 pmdaCacheStore or pmdaCacheStoreKey with the external name of the in‐
81 stance passed via name. The opaque pointer private may be used to as‐
82 sociate additional data with the entry in the cache; if no such data is
83 required, private should be NULL. Any manipulation of the additional
84 data (including allocation or freeing) is the responsibility of the PM‐
85 DA caller, as the cache simply maintains the pointer to the data
86 (passed via private).
87 The upper bound for identifiers allocated for any given indom cache can
89 be optionally reduced from the default (2^31 - 1) to some lesser maxi‐
90 mum, using pmdaCacheResize. This maximum will then be persisted and
91 restored in the usual manner, and can thus be associated permanently
92 with a cache once set. This has applications when using these inter‐
93 faces as general purpose identifier caches, and is less applicable when
94 using them for instance domain caching.
95 For cases where the PMDA developer wishes to influence the allocation
97 of internal instance identifiers, e.g. for instance domains with more
98 than one natural dimension, or where there is a desire to allocate the
99 same instance identifier each time the PMDA is started, even on differ‐
100 ent hosts, pmdaCacheStoreKey may be used. In this case, an initial
101 ``hint'' for the instance identifier is provided as an opaque key via
102 the first keylen bytes in key (which could be any sort of data, includ‐
103 ing binary values) else if keylen is less than 1 or key is NULL then
104 name is used as the ``hint''. The ``hint'' is hashed to produce an
105 initial instance identifier in the range 0 to 2^31 - 1 (or lesser maxi‐
106 mum, if set). If this instance identifier is already allocated, then
107 the value is rehashed. This procedure is repeated until an unallocated
108 instance identifier is found, or pmdaCacheStoreKey gives up and returns
109 PM_ERR_GENERIC. For each instance domain, the ``hint'' must be unique
110 across all instances, else pmdaCacheStoreKey returns PM_ERR_INST.
111 The flags argument controls how the instance should be processed in the
113 cache as follows:
114 PMDA_CACHE_ADD
116 Insert the entry into the cache if it is not already there and
117 mark it active. If the entry is already in the cache mark it
118 active.
119 PMDA_CACHE_HIDE
121 Mark the entry in the cache as inactive, but remember the de‐
122 tails of the association between the external instance name and
123 the internal instance identifier. Entries that are inactive
124 will be hidden from cache traversal via PMDA_CACHE_WALK_NEXT op‐
125 erations, but remain visible to pmdaCacheLookup, pmdaCache‐
126 LookupName and pmdaCacheLookupKey requests.
127 PMDA_CACHE_CULL
129 Remove the entry from the cache.
130 On success pmdaCacheStore or pmdaCacheStoreKey will return the internal
132 instance identifier of the associated cache entry. Valid instance
133 identifiers are guaranteed to be unique and non-negative. Failure will
134 be indicated by a negative value (suitable for decoding with
135 pmErrStr(3)) and most likely PM_ERR_INST to indicate the requested in‐
136 stance is not in the cache, or -EINVAL to indicate a potential viola‐
137 tion of the short name uniqueness property (see the INSTANCE NAME
138 MATCHING section below).
139 pmdaCacheLookup is used to search the entries in the cache based on the
141 internal instance identifier inst.
142 On success the return value will be PMDA_CACHE_ACTIVE or PMDA_CACHE_IN‐
144 ACTIVE (depending on the active or inactive state of the cache entry),
145 name (if not NULL) and private (if not NULL) will be set to the exter‐
146 nal instance name and the associate additional data area as provided
147 when the instance was last activated via pmdaCacheStore or pmdaCache‐
148 StoreKey.
149 pmdaCacheLookup failure is indicated by a negative return value suit‐
151 able for decoding with pmErrStr(3).
152 The pmdaCacheLookup interface is required by the PMDA's fetch callback
154 that is registered via pmdaSetFetchCallback(3). Here the internal in‐
155 stance identifier is passed to the fetch callback to identifier for
156 which instance a value is required. Typical usage is shown in the code
157 fragment below.
158 static int
160 foo_callback(pmdaMetric *mdesc, unsigned int inst, pmAtomValue *atom)
161 {
162 mydata *mdp;
163 char *name;
164 int sts;
165 sts = pmdaCacheLookup(mdesc->m_desc.indom, inst, &name, (void **)&mdp);
167 /*
168 * expect sts == PMDA_CACHE_ACTIVE except for cataclysmic events
169 * use mdp as required, name may be useful for diagnostics
170 */
171 ...
172 pmdaCacheLookupName is used to search the entries in the cache based on
175 the external instance name name.
176 On success the return value will be PMDA_CACHE_ACTIVE or PMDA_CACHE_IN‐
178 ACTIVE (depending on the active or inactive state of the cache entry),
179 inst (if not NULL) and private (if not NULL) will be set to the inter‐
180 nal instance identifier and the associate additional data area as pro‐
181 vided when the instance was last activated via pmdaCacheStore or pmda‐
182 CacheStoreKey.
183 pmdaCacheLookupName failure is indicated by a negative return value
185 suitable for decoding with pmErrStr(3).
186 The pmdaCacheLookupName interface is useful for PMDAs wishing to update
188 an instance domain based on the external instance names.
189 pmdaCacheLookupKey is used to search the entries in the cache based on
191 an opaque key (or ``hint'') previously used in a call to pmdaCache‐
192 StoreKey. The ``hint'' is provided via the first keylen bytes in key.
193 For symmetry with pmdaCacheStoreKey, if keylen is less than 1 or key is
194 NULL then name is used as the ``hint'' (although the results will be
195 the same as calling pmdaCacheLookupName in this case).
196 On success the return value will be PMDA_CACHE_ACTIVE or PMDA_CACHE_IN‐
198 ACTIVE (depending on the active or inactive state of the cache entry),
199 oname (if not NULL), inst (if not NULL) and private (if not NULL) will
200 be set to the external instance name, the internal instance identifier
201 and the associate additional data area as provided when the instance
202 was last activated via pmdaCacheStore or pmdaCacheStoreKey.
203 pmdaCacheLookupKey failure is indicated by a negative return value
205 suitable for decoding with pmErrStr(3).
206 To avoid a persistent cache growing without bound, pmdaCachePurge can
208 be used to cull all entries that have not been active in the last re‐
209 cent seconds. For performance reasons, the time accounting is impre‐
210 cise and the entries are timestamped at the time of the next cache save
211 operation after the entry has been added or marked active (refer to PM‐
212 DA_CACHE_SAVE and PMDA_CACHE_SYNC below). On success pmdaCachePurge
213 returns the number of culled entries, else in the case of an error the
214 return value is negative (and suitable for decoding with pmErrStr(3)).
215 pmdaCacheOp may be used to perform additional operations on the cache
217 as follows:
218 PMDA_CACHE_LOAD
220 The cache can optionally be maintained as a persistent external
221 file, so that the mapping of instance names to instance identi‐
222 fiers is persistent across executions of a PMDA. This operation
223 loads the cache from the external file, and then all new cache
224 entries are marked inactive, and the additional data pointer is
225 set to NULL. Entries loaded from the external file are checked
226 against the current cache contents and if the instance name and
227 instance identifiers match then the state in the cache (active
228 or inactive) is not changed. Should a mismatch be found (same
229 instance name and different instance identifier, or same in‐
230 stance identifier and different instance name, or some but not
231 all of the instance identifier, the instance name and the
232 ``hint'' match) then the entry from the external file is ignored
233 and a warning is issued on stderr. Typically a PMDA would only
234 perform this operation once per execution.
235 PMDA_CACHE_SAVE
237 If any instance has been added to, or deleted from, the instance
238 domain since the last PMDA_CACHE_LOAD, PMDA_CACHE_SAVE or PM‐
239 DA_CACHE_SYNC operation, the entire cache is written to the ex‐
240 ternal file as a bulk operation. This operation is provided for
241 PMDAs that are not interested in using pmdaCachePurge and simply
242 want the external file to reflect the set of known instances
243 without accurate details of when they were last marked active.
244 Returns the number of instances saved to the external file, else
246 0 if the external file was already up to date.
247 PMDA_CACHE_STRINGS
249 Annotates this cache as being a special-purpose cache used for
250 string de-duplication in PMDAs exporting large numbers of string
251 valued metrics. This can be used to reduce the memory footprint
252 of the PMDA (duplicate strings hash to the same bucket, and are
253 stored in memory once only). Key comparisons are not terminated
254 at the first space but rather the entire string is used for
255 matching. These are specialised caches not useful for general
256 purpose instance domain handling.
257 PMDA_CACHE_SYNC
259 Within an instance domain, if any instance has been added to, or
260 deleted from, or marked active since the last PMDA_CACHE_LOAD,
261 PMDA_CACHE_SAVE or PMDA_CACHE_SYNC operation, the entire cache
262 is written to the external file as a bulk operation. This oper‐
263 ation is similar to PMDA_CACHE_SAVE, but will save the instance
264 domain more frequently so the timestamps more accurately match
265 the semantics expected by pmdaCachePurge.
266 Returns the number of instances saved to the external file, else
268 0 if the external file was already synchronized.
269 PMDA_CACHE_CHECK
271 Returns 1 if a cache exists for the specified instance domain,
272 else 0.
273 PMDA_CACHE_REUSE
275 When a new instance is added to the cache, the default strategy
276 is to assign instance identifiers in a monotonic increasing man‐
277 ner. Once the maximum possible instance identifier value has
278 been assigned, the strategy changes to one where starting from
279 0, the next available unused instance identifier will be used.
280 Calling pmdaCacheOp with PMDA_CACHE_REUSE forces an irreversible
281 change to a second (reuse) strategy where the next unallocated
282 instance identifier will be used. This may be useful in cases
283 where there is a desire to restrict the allocated instance iden‐
284 tifiers to smaller values. The prevailing strategy will be
285 saved and restored across PMDA_CACHE_SAVE and PMDA_CACHE_LOAD
286 operations. If pmdaCacheStoreKey is ever used, the associated
287 instance domain will be changed to PMDA_CACHE_REUSE mode.
288 PMDA_CACHE_REORG
290 Reorganize the cache to allow faster retrieval of active en‐
291 tries, and the cost of slower retrieval for inactive entries,
292 and reclaim any culled entries. The cache may be internally re-
293 organized as entries are added, so this operation is not re‐
294 quired for most PMDAs.
295 PMDA_CACHE_WALK_REWIND
297 Prepares for a traversal of the cache in ascending instance
298 identifier sequence.
299 PMDA_CACHE_WALK_NEXT
301 Fetch the next active instance identifier from the cache. Re‐
302 quires a prior call using PMDA_CACHE_WALK_REWIND and will return
303 -1 when all instances have been processed.
304 Only one cache walk can be active at any given time, nesting
306 calls to PMDA_CACHE_WALK and PMDA_CACHE_REWIND will interfere
307 with each other.
308 PMDA_CACHE_ACTIVE
310 Changes every inactive entry in the cache to be marked active.
311 PMDA_CACHE_INACTIVE
313 Changes every active entry in the cache to be marked inactive.
314 PMDA_CACHE_CULL
316 Remove every entry from the cache.
317 PMDA_CACHE_SIZE
319 Return the number of entries in the cache (includes active, in‐
320 active and any culled entries that have not yet been reclaimed).
321 PMDA_CACHE_SIZE_ACTIVE
323 Return the number of active entries in the cache.
324 PMDA_CACHE_SIZE_INACTIVE
326 Return the number of inactive entries in the cache.
327 PMDA_CACHE_DUMP
329 Dump the current state of the cache on stderr.
330 PMDA_CACHE_DUMP_ALL
332 Like PMDA_CACHE_DUMP, but also dump the internal hashing struc‐
333 tures used to support lookup by instance name, lookup by in‐
334 stance identifier and the collision statistics for ``hint''
335 hashing from pmdaCacheStoreKey.
336 pmdaCacheOp returns a non-negative value on success, and failure is in‐
338 dicated by a negative return value (suitable for decoding with
339 pmErrStr(3)).
340
342 When the pmdaCache routines are used for particular instance domain,
343 pmdaInstance (3) and the instance domain enumeration behind
344 pmdaFetch(3) will attempt to extract instance domain information from
345 the cache, thereby avoiding reference to the pmdaIndom data structures
346 that have historically been used to define instance domains and service
347 instance requests. A PMDA can adopt a hybrid approach and choose to
348 implement some instance domains via the traditional pmdaIndom method,
349 and others via the pmdaCache approach, however attempts to manage the
350 same instance domain by both methods will result in the pmdaCache
351 method silently prevailing.
352 If all instances in a PMDA are to be serviced from a pmdaCache then a
354 pmdaIndom is not required, and the pmdaInit (3) call becomes
355 pmdaInit(dp, NULL, 0, metrictab, nmetrics);
357 However, the PMDA will need to explicitly initialize the indom field of
359 the pmDesc in the metrictab entries, as this cannot be done by pm‐
360 daInit(3) if indomtab is missing entries for the instance domains main‐
361 tained in the cache.
362 Independent of how the instance domain is being maintained, to refresh
364 an instance domain prior to a fetch or an instance domain operation,
365 the standard methods of a ``wrapper'' to the pmdaInstance (3) and
366 pmdaFetch (3) methods should be used.
367 Refer to the simple PMDA source code for an example use of the pmda‐
369 Cache routines.
370 When using pmdaCacheStoreKey, if there is a desire to ensure the given
372 ``hint'' generates the same initial instance identifier across all
373 platforms, then the caller should ensure the endian and word-size is‐
374 sues are considered, e.g. if the natural data structure used for the
375 key is an array of 32-bit integers, then htonl(3) should be used on
376 each element of the array before calling pmdaCacheStoreKey or pmdaCach‐
377 eLookupKey.
378
380 The following table summarizes the ``short name'' matching semantics
381 for an instance domain (caches other than PMDA_CACHE_STRINGS style).
382 ┌────────┬─────────────────┬───────────────────────────────────────────┐
384 │name in │ pmdaCacheLookup │ result │
385 │cache │ name │ │
386 ├────────┼─────────────────┼───────────────────────────────────────────┤
387 │foodle │ foo │ no match (PM_ERR_INST) │
388 │foo │ foodle │ no match (PM_ERR_INST) │
389 │foo │ foo │ match │
390 │foo bar │ foo │ match on short name (instance identifier) │
391 │foo bar │ foo bar │ match on full name (instance identifier) │
392 │foo │ foo bar │ bad match (-EDOM) │
393 │foo bar │ foo blah │ bad match (-EDOM) │
394 └────────┴─────────────────┴───────────────────────────────────────────┘
396 Cache persistence uses files with names constructed from the indom
397 within the $PCP_VAR_DIR/config/pmda directory.
398
400 BYTEORDER(3), PMAPI(3), PMDA(3), pmdaInit(3), pmdaInstance(3),
401 pmdaFetch(3), pmdaSetFetchCallback(3), pmErrStr(3) and pmGetInDom(3).
402Performance Co-Pilot PCP PMDACACHE(3)