[ofa-general] Query on ibv_post_recv and IBV_SEND_INLINE

[Bharath Ramesh]

I have a question on using IBV_SEND_INLINE. From what I understand if I am
using IBV_SEND_INLINE I technically neednt register the buffer with
ibv_reg_mr as I dont need the L_Key. But still in the receiver end I
would have to have the buffer registered to receive the inline data that
has been posted. If I have an application where I am using a lot of
small messages, then memory registration would be expensive. Is there
any other option for receiving inline messages with out actually
registering the receive buffer.

Thanks,

Bharath

---
Bharath Ramesh <***@vt.edu> http://people.cs.vt.edu/~bramesh

[Sean Hefty]

Post by Bharath Ramesh
I have a question on using IBV_SEND_INLINE. From what I understand if I am
using IBV_SEND_INLINE I technically neednt register the buffer with
ibv_reg_mr as I dont need the L_Key. But still in the receiver end I
would have to have the buffer registered to receive the inline data that
has been posted. If I have an application where I am using a lot of
small messages, then memory registration would be expensive. Is there
any other option for receiving inline messages with out actually
registering the receive buffer.

You could register a larger buffer once and use it with multiple
receives. If you needed to, you could copy the data out of the
registered buffer into separate buffers for processing.

- Sean

[Dotan Barak]

Hi.

Post by Bharath Ramesh
I have a question on using IBV_SEND_INLINE. From what I understand if I am
using IBV_SEND_INLINE I technically neednt register the buffer with
ibv_reg_mr as I dont need the L_Key. But still in the receiver end I
would have to have the buffer registered to receive the inline data that
has been posted. If I have an application where I am using a lot of
small messages, then memory registration would be expensive. Is there
any other option for receiving inline messages with out actually
registering the receive buffer.

The flag IBV_SEND_INLINE describe to the low level driver how the data
will be read from the memory buffer
(the driver will read the data and give it to the HCA or the HCA will
fetch the data from the memory using
the gather list).

The data on the wire for inline messages looks the same like data which
was fetched by the HCA.

The receive side don't have any feature which can work with memory
buffer without any registration.


thanks
Dotan

[Tang, Changqing]

Hi, if anyone can answer,

I have a question about ibv_modify_qp() behavior. I use
ibv_modify_qp() to move QP state from INIT to RTR and then RTS.
when INIT-->RTR, I need to provide peer's qp number and port lid. If qp
number and port lid are not correct (for example, both
QPs are connected to different switch, and there is no bridge link
between the switches), my code is silently hanging.

Does ibv_modify_qp() check such error, or is there other way to
detect such error ? so my code can error out, not hanging ?

Thanks.

--CQ, HP MPI

[Michael S. Tsirkin]

Subject: does ibv_modify_qp() check error ?
Hi, if anyone can answer,
I have a question about ibv_modify_qp() behavior. I use
ibv_modify_qp() to move QP state from INIT to RTR and then RTS.
when INIT-->RTR, I need to provide peer's qp number and port lid. If qp
number and port lid are not correct (for example, both
QPs are connected to different switch, and there is no bridge link
between the switches), my code is silently hanging.
Does ibv_modify_qp() check such error, or is there other way to
detect such error ? so my code can error out, not hanging ?
Thanks.

Try posting a WR, e.g. a 0 size RDMA write.
You'll get a CQE with error if the QPN is wrong.