Use curl multi interface - #60
Conversation
|
See PDAL/PDAL#4985 |
|
|
||
| const auto code(curl_easy_perform(m_curl)); | ||
| curl_easy_getinfo(m_curl, CURLINFO_RESPONSE_CODE, &httpCode); | ||
| curl_easy_reset(m_curl); |
There was a problem hiding this comment.
I think we lost the curl_easy_reset - which would potentially allow settings from one request to persist through the next if not re-overridden.
There was a problem hiding this comment.
I'm not sure I understand. The settings are only made once when the Pool is made. Is this a problem?
There was a problem hiding this comment.
I'm saying that without the curl_easy_reset in perform, if we call two different verbs that reuse the same handle, their non-identical options would not be reset from the first. So, if we do a PUT, which does:
curl_easy_setopt(m_curl, CURLOPT_WRITEFUNCTION, Response::getCb);
And then we do a GET later, then the WRITEFUNCTION won't be reset. Or CURLOPT_NOBODY for a HEAD request.
only made once when the Pool is made
Maybe I'm misinterpreting the flow here, because I am talking about the context of perform executing on a per-call basis - not a one-time init thing.
There was a problem hiding this comment.
Ah. Yes, I think you're correct.
| { | ||
| throw std::runtime_error("Cannot acquire from empty pool"); | ||
| int msgCnt; | ||
| CURLMsg *m = curl_multi_info_read(m_multi, &msgCnt); |
There was a problem hiding this comment.
Docs for this function say that m->data.result holds the error code for the transfer which doesn't look checked here (although maybe this is handled outside this scope somehow) - is this the spot where we previously made up the error code 550 or whatever? What happens for a non-HTTP-status code response, e.g. timeout.
There was a problem hiding this comment.
I think the structure is generic. The result state for each transfer comes from the easy handle, not the multi handle and that's checked and set with curl_easy_getinfo a little farther down.
2 participants
This replaces arbiter's use of the curl "easy" interface with the curl "multi" interface. This was recommended by the author of curl in order to reduce file descriptor use. Each easy "handle" would use at least three file descriptors to facilitate an http transfer. The multi interface uses a single descriptor for each transfer in addition to a couple of descriptors to facilitate asynchronous I/O. This should allow more transfers to occur without running out of file descriptors on system where they are a limited resource.
This change also does modifies the interface internally so that fetched data is moved rather than copied where possible.