1OSSL_HTTP_TRANSFER(3ossl) OpenSSL OSSL_HTTP_TRANSFER(3ossl)
2
6 OSSL_HTTP_open, OSSL_HTTP_bio_cb_t, OSSL_HTTP_proxy_connect,
7 OSSL_HTTP_set1_request, OSSL_HTTP_exchange, OSSL_HTTP_get,
8 OSSL_HTTP_transfer, OSSL_HTTP_close - HTTP client high-level functions
9
11 #include <openssl/http.h>
12 typedef BIO *(*OSSL_HTTP_bio_cb_t)(BIO *bio, void *arg,
14 int connect, int detail);
15 OSSL_HTTP_REQ_CTX *OSSL_HTTP_open(const char *server, const char *port,
16 const char *proxy, const char *no_proxy,
17 int use_ssl, BIO *bio, BIO *rbio,
18 OSSL_HTTP_bio_cb_t bio_update_fn, void *arg,
19 int buf_size, int overall_timeout);
20 int OSSL_HTTP_proxy_connect(BIO *bio, const char *server, const char *port,
21 const char *proxyuser, const char *proxypass,
22 int timeout, BIO *bio_err, const char *prog);
23 int OSSL_HTTP_set1_request(OSSL_HTTP_REQ_CTX *rctx, const char *path,
24 const STACK_OF(CONF_VALUE) *headers,
25 const char *content_type, BIO *req,
26 const char *expected_content_type, int expect_asn1,
27 size_t max_resp_len, int timeout, int keep_alive);
28 BIO *OSSL_HTTP_exchange(OSSL_HTTP_REQ_CTX *rctx, char **redirection_url);
29 BIO *OSSL_HTTP_get(const char *url, const char *proxy, const char *no_proxy,
30 BIO *bio, BIO *rbio,
31 OSSL_HTTP_bio_cb_t bio_update_fn, void *arg,
32 int buf_size, const STACK_OF(CONF_VALUE) *headers,
33 const char *expected_content_type, int expect_asn1,
34 size_t max_resp_len, int timeout);
35 BIO *OSSL_HTTP_transfer(OSSL_HTTP_REQ_CTX **prctx,
36 const char *server, const char *port,
37 const char *path, int use_ssl,
38 const char *proxy, const char *no_proxy,
39 BIO *bio, BIO *rbio,
40 OSSL_HTTP_bio_cb_t bio_update_fn, void *arg,
41 int buf_size, const STACK_OF(CONF_VALUE) *headers,
42 const char *content_type, BIO *req,
43 const char *expected_content_type, int expect_asn1,
44 size_t max_resp_len, int timeout, int keep_alive);
45 int OSSL_HTTP_close(OSSL_HTTP_REQ_CTX *rctx, int ok);
46
48 OSSL_HTTP_open() initiates an HTTP session using the bio argument if
49 not NULL, else by connecting to a given server optionally via a proxy.
50 Typically the OpenSSL build supports sockets and the bio parameter is
52 NULL. In this case rbio must be NULL as well and the server must be
53 non-NULL. The function creates a network BIO internally using
54 BIO_new_connect(3) for connecting to the given server and the
55 optionally given port, defaulting to 80 for HTTP or 443 for HTTPS.
56 Then this internal BIO is used for setting up a connection and for
57 exchanging one or more request and response. If bio is given and rbio
58 is NULL then this bio is used instead. If both bio and rbio are given
59 (which may be memory BIOs for instance) then no explicit connection is
60 set up, but bio is used for writing requests and rbio for reading
61 responses. As soon as the client has flushed bio the server must be
62 ready to provide a response or indicate a waiting condition via rbio.
63 If bio is given, it is an error to provide proxy or no_proxy arguments,
65 while server and port arguments may be given to support diagnostic
66 output. If bio is NULL the optional proxy parameter can be used to set
67 an HTTP(S) proxy to use (unless overridden by "no_proxy" settings). If
68 TLS is not used this defaults to the environment variable "http_proxy"
69 if set, else "HTTP_PROXY". If use_ssl != 0 it defaults to
70 "https_proxy" if set, else "HTTPS_PROXY". An empty proxy string ""
71 forbids using a proxy. Else the format is
72 "[http[s]://][userinfo@]host[:port][/path][?query][#fragment]", where
73 any userinfo, path, query, and fragment given is ignored. The default
74 proxy port number is 80, or 443 in case "https:" is given. The HTTP
75 client functions connect via the given proxy unless the server is found
76 in the optional list no_proxy of proxy hostnames (if not NULL; default
77 is the environment variable "no_proxy" if set, else "NO_PROXY").
78 Proxying plain HTTP is supported directly, while using a proxy for
79 HTTPS connections requires a suitable callback function such as
80 OSSL_HTTP_proxy_connect(), described below.
81 If use_ssl is nonzero a TLS connection is requested and the
83 bio_update_fn parameter must be provided.
84 The parameter bio_update_fn, which is optional if use_ssl is 0, may be
86 used to modify the connection BIO used by the HTTP client, but cannot
87 be used when both bio and rbio are given. bio_update_fn is a BIO
88 connect/disconnect callback function with prototype
89 BIO *(*OSSL_HTTP_bio_cb_t)(BIO *bio, void *arg, int connect, int detail)
91 The callback function may modify the BIO provided in the bio argument,
93 whereby it may make use of a custom defined argument arg, which may for
94 instance point to an SSL_CTX structure. During connection
95 establishment, just after calling BIO_do_connect_retry(), the callback
96 function is invoked with the connect argument being 1 and detail being
97 1 if use_ssl is nonzero (i.e., HTTPS is requested), else 0. On
98 disconnect connect is 0 and detail is 1 if no error occurred, else 0.
99 For instance, on connect the callback may push an SSL BIO to implement
100 HTTPS; after disconnect it may do some diagnostic output and pop and
101 free the SSL BIO.
102 The callback function must return either the potentially modified BIO
104 bio. or NULL to indicate failure, in which case it should not modify
105 the BIO.
106 Here is a simple example that supports TLS connections (but not via a
108 proxy):
109 BIO *http_tls_cb(BIO *bio, void *arg, int connect, int detail)
111 {
112 if (connect && detail) { /* connecting with TLS */
113 SSL_CTX *ctx = (SSL_CTX *)arg;
114 BIO *sbio = BIO_new_ssl(ctx, 1);
115 bio = sbio != NULL ? BIO_push(sbio, bio) : NULL;
117 } else if (!connect) { /* disconnecting */
118 BIO *hbio;
119 if (!detail) { /* an error has occurred */
121 /* optionally add diagnostics here */
122 }
123 BIO_ssl_shutdown(bio);
124 hbio = BIO_pop(bio);
125 BIO_free(bio); /* SSL BIO */
126 bio = hbio;
127 }
128 return bio;
129 }
130 After disconnect the modified BIO will be deallocated using
132 BIO_free_all().
133 The buf_size parameter specifies the response header maximum line
135 length. A value <= 0 means that the OSSL_HTTP_DEFAULT_MAX_LINE_LEN
136 (4KiB) is used. buf_size is also used as the number of content bytes
137 that are read at a time.
138 If the overall_timeout parameter is > 0 this indicates the maximum
140 number of seconds the overall HTTP transfer (i.e., connection setup if
141 needed, sending requests, and receiving responses) is allowed to take
142 until completion. A value <= 0 enables waiting indefinitely, i.e., no
143 timeout.
144 OSSL_HTTP_proxy_connect() may be used by an above BIO connect callback
146 function to set up an SSL/TLS connection via an HTTPS proxy. It
147 promotes the given BIO bio representing a connection pre-established
148 with a TLS proxy using the HTTP CONNECT method, optionally using proxy
149 client credentials proxyuser and proxypass, to connect with TLS
150 protection ultimately to server and port. If the port argument is NULL
151 or the empty string it defaults to "443". If the timeout parameter is
152 > 0 this indicates the maximum number of seconds the connection setup
153 is allowed to take. A value <= 0 enables waiting indefinitely, i.e.,
154 no timeout. Since this function is typically called by applications
155 such as openssl-s_client(1) it uses the bio_err and prog parameters
156 (unless NULL) to print additional diagnostic information in a user-
157 oriented way.
158 OSSL_HTTP_set1_request() sets up in rctx the request header and content
160 data and expectations on the response using the following parameters.
161 If <rctx> indicates using a proxy for HTTP (but not HTTPS), the server
162 hostname (and optionally port) needs to be placed in the header and
163 thus must be present. If path is NULL it defaults to "/". If req is
164 NULL the HTTP GET method will be used to send the request else HTTP
165 POST with the contents of req and optional content_type, where the
166 length of the data in req does not need to be determined in advance:
167 the BIO will be read on-the-fly while sending the request, which
168 supports streaming. The optional list headers may contain additional
169 custom HTTP header lines. If the parameter expected_content_type is
170 not NULL then the client will check that the given content type string
171 is included in the HTTP header of the response and return an error if
172 not. If the expect_asn1 parameter is nonzero, a structure in ASN.1
173 encoding will be expected as response content. The max_resp_len
174 parameter specifies the maximum allowed response content length, where
175 the value 0 indicates no limit. If the timeout parameter is > 0 this
176 indicates the maximum number of seconds the subsequent HTTP transfer
177 (sending the request and receiving a response) is allowed to take. A
178 value of 0 enables waiting indefinitely, i.e., no timeout. A value < 0
179 indicates that the overall_timeout parameter value given when opening
180 the HTTP transfer will be used instead. If keep_alive is 0 the
181 connection is not kept open after receiving a response, which is the
182 default behavior for HTTP 1.0. If the value is 1 or 2 then a
183 persistent connection is requested. If the value is 2 then a
184 persistent connection is required, i.e., an error occurs in case the
185 server does not grant it.
186 OSSL_HTTP_exchange() exchanges any form of HTTP request and response as
188 specified by rctx, which must include both connection and request data,
189 typically set up using OSSL_HTTP_open() and OSSL_HTTP_set1_request().
190 It implements the core of the functions described below. If the HTTP
191 method is GET and redirection_url is not NULL the latter pointer is
192 used to provide any new location that the server may return with HTTP
193 code 301 (MOVED_PERMANENTLY) or 302 (FOUND). In this case the function
194 returns NULL and the caller is responsible for deallocating the URL
195 with OPENSSL_free(3). If the response header contains one or more
196 "Content-Length" header lines and/or an ASN.1-encoded response is
197 expected, which should include a total length, the length indications
198 received are checked for consistency and for not exceeding any given
199 maximum response length. If an ASN.1-encoded response is expected, the
200 function returns on success the contents buffered in a memory BIO,
201 which does not support streaming. Otherwise it returns directly the
202 read BIO that holds the response contents, which allows a response of
203 indefinite length and may support streaming. The caller is responsible
204 for freeing the BIO pointer obtained.
205 OSSL_HTTP_get() uses HTTP GET to obtain data from bio if non-NULL, else
207 from the server contained in the url, and returns it as a BIO. It
208 supports redirection via HTTP status code 301 or 302. It is meant for
209 transfers with a single round trip, so does not support persistent
210 connections. If bio is non-NULL, any host and port components in the
211 url are not used for connecting but the hostname is used, as usual, for
212 the "Host" header. Any userinfo and fragment components in the url are
213 ignored. Any query component is handled as part of the path component.
214 If the scheme component of the url is "https" a TLS connection is
215 requested and the bio_update_fn, as described for OSSL_HTTP_open(),
216 must be provided. Also the remaining parameters are interpreted as
217 described for OSSL_HTTP_open() and OSSL_HTTP_set1_request(),
218 respectively. The caller is responsible for freeing the BIO pointer
219 obtained.
220 OSSL_HTTP_transfer() exchanges an HTTP request and response over a
222 connection managed via prctx without supporting redirection. It
223 combines OSSL_HTTP_open(), OSSL_HTTP_set1_request(),
224 OSSL_HTTP_exchange(), and OSSL_HTTP_close(). If prctx is not NULL it
225 reuses any open connection represented by a non-NULL *prctx. It keeps
226 the connection open if a persistent connection is requested or required
227 and this was granted by the server, else it closes the connection and
228 assigns NULL to *prctx. The remaining parameters are interpreted as
229 described for OSSL_HTTP_open() and OSSL_HTTP_set1_request(),
230 respectively. The caller is responsible for freeing the BIO pointer
231 obtained.
232 OSSL_HTTP_close() closes the connection and releases rctx. The ok
234 parameter is passed to any BIO update function given during setup as
235 described above for OSSL_HTTP_open(). It must be 1 if no error
236 occurred during the HTTP transfer and 0 otherwise.
237
239 The names of the environment variables used by this implementation:
240 "http_proxy", "HTTP_PROXY", "https_proxy", "HTTPS_PROXY", "no_proxy",
241 and "NO_PROXY", have been chosen for maximal compatibility with other
242 HTTP client implementations such as wget, curl, and git.
243 When built with tracing enabled, OSSL_HTTP_transfer() and all functions
245 using it may be traced using OSSL_TRACE_CATEGORY_HTTP. See also
246 OSSL_trace_enabled(3) and "ENVIRONMENT" in openssl(1).
247
249 OSSL_HTTP_open() returns on success a OSSL_HTTP_REQ_CTX, else NULL.
250 OSSL_HTTP_proxy_connect() and OSSL_HTTP_set1_request() return 1 on
252 success, 0 on error.
253 On success, OSSL_HTTP_exchange(), OSSL_HTTP_get(), and
255 OSSL_HTTP_transfer() return a memory BIO that buffers all the data
256 received if an ASN.1-encoded response is expected, otherwise a BIO that
257 may support streaming. The BIO must be freed by the caller. On
258 failure, they return NULL. Failure conditions include
259 connection/transfer timeout, parse errors, etc. The caller is
260 responsible for freeing the BIO pointer obtained.
261 OSSL_HTTP_close() returns 0 if anything went wrong while disconnecting,
263 else 1.
264
266 OSSL_HTTP_parse_url(3), BIO_new_connect(3), ASN1_item_i2d_mem_bio(3),
267 ASN1_item_d2i_bio(3), OSSL_HTTP_is_alive(3), OSSL_trace_enabled(3)
268
270 All the functions described here were added in OpenSSL 3.0.
271
273 Copyright 2019-2022 The OpenSSL Project Authors. All Rights Reserved.
274 Licensed under the Apache License 2.0 (the "License"). You may not use
276 this file except in compliance with the License. You can obtain a copy
277 in the file LICENSE in the source distribution or at
278 <https://www.openssl.org/source/license.html>.
2793.1.1 2023-08-31 OSSL_HTTP_TRANSFER(3ossl)