1IBV_ADVISE_MR(3) Libibverbs Programmer’s Manual IBV_ADVISE_MR(3)
2
6 ibv_advise_mr - Gives advice or directions to the kernel about an ad‐
7 dress range belongs to a memory region (MR).
8
10 #include <infiniband/verbs.h>
11 int ibv_advise_mr(struct ibv_pd *pd,
13 enum ibv_advise_mr_advice advice,
14 uint32_t flags,
15 struct ibv_sge *sg_list,
16 uint32_t num_sge)
17
19 ibv_advise_mr() Give advice or directions to the kernel about an ad‐
20 dress range belonging to a memory region (MR). Applications that are
21 aware of future access patterns can use this verb in order to leverage
22 this knowledge to improve system or application performance.
23 Conventional advice values
25 IBV_ADVISE_MR_ADVICE_PREFETCH
27 Pre-fetch a range of an on-demand paging MR. Make pages present
28 with read-only permission before the actual IO is conducted.
29 This would provide a way to reduce latency by overlapping pag‐
30 ing-in and either compute time or IO to other ranges.
31 IBV_ADVISE_MR_ADVICE_PREFETCH_WRITE
33 Like IBV_ADVISE_MR_ADVICE_PREFETCH but with read-access and
34 write-access permission to the fetched memory.
35 IBV_ADVISE_MR_ADVICE_PREFETCH_NO_FAULT
37 Pre-fetch a range of an on-demand paging MR without faulting.
38 This allows presented pages in the CPU to become presented to
39 the device.
40
42 pd The protection domain (PD) associated with the MR.
43 advice The requested advise value (as listed above).
45 flags Describes the properties of the advise operation Conventional
47 advice values IBV_ADVISE_MR_FLAG_FLUSH : Request to be a syn‐
48 chronized operation. Return to the caller after the operation
49 is completed.
50 sg_list
52 Pointer to the s/g array When using IBV_ADVISE_OP_PREFETCH ad‐
53 vise value, all the lkeys of all the scatter gather elements
54 (SGEs) must be associated with ODP MRs (MRs that were registered
55 with IBV_ACCESS_ON_DEMAND).
56 num_sge
58 Number of elements in the s/g array
59
61 ibv_advise_mr() returns 0 when the call was successful, or the value of
62 errno on failure (which indicates the failure reason).
63 EOPNOTSUPP
65 libibverbs or provider driver doesn’t support the ibv_ad‐
66 vise_mr() verb (ENOSYS may sometimes be returned by old versions
67 of libibverbs).
68 ENOTSUP
70 The advise operation isn’t supported.
71 EFAULT In one of the following: o When the range requested is out of
73 the MR bounds, or when parts of it are not part of the process
74 address space. o One of the lkeys provided in the scatter gath‐
75 er list is invalid or with wrong write access.
76 EINVAL In one of the following: o The PD is invalid. o The flags are
78 invalid.
79
81 An application may pre-fetch any address range within an ODP MR when
82 using the IBV_ADVISE_MR_ADVICE_PREFETCH or IBV_ADVISE_MR_AD‐
83 VICE_PREFETCH_WRITE advice. Semantically, this operation is best-ef‐
84 fort. That means the kernel does not guarantee that underlying pages
85 are updated in the HCA or the pre-fetched pages would remain resident.
86 When using IBV_ADVISE_MR_ADVICE_PREFETCH or IBV_ADVISE_MR_AD‐
88 VICE_PREFETCH_WRITE advice, the operation will be done in the following
89 stages: o Page in the user pages to memory (pages aren’t pinned). o
90 Get the dma mapping of these user pages. o Post the underlying page
91 translations to the HCA.
92 If IBV_ADVISE_MR_FLAG_FLUSH is specified then the underlying pages are
94 guaranteed to be updated in the HCA before returning SUCCESS. Other‐
95 wise the driver can choose to postpone the posting of the new transla‐
96 tions to the HCA. When performing a local RDMA access operation it is
97 recommended to use IBV_ADVISE_MR_FLAG_FLUSH flag with one of the pre-
98 fetch advices to increase probability that the pages translations are
99 valid in the HCA and avoid future page faults.
100
102 ibv_reg_mr(3), ibv_rereg_mr(3), ibv_dereg_mr(3)
103
105 Aviad Yehezkel <aviadye@mellanox.com>
106libibverbs 2018-10-19 IBV_ADVISE_MR(3)