1libcurl-tutorial(3)           libcurl programming          libcurl-tutorial(3)
2
3
4

NAME

6       libcurl-tutorial - libcurl programming tutorial
7

Objective

9       This  document  attempts  to  describe  the general principles and some
10       basic approaches to consider when programming with  libcurl.  The  text
11       will  focus  mainly  on  the C interface but might apply fairly well on
12       other interfaces as well as  they  usually  follow  the  C  one  pretty
13       closely.
14
15       This document will refer to 'the user' as the person writing the source
16       code that uses libcurl. That would probably be you or someone  in  your
17       position.   What will be generally referred to as 'the program' will be
18       the collected source code that you write  that  is  using  libcurl  for
19       transfers. The program is outside libcurl and libcurl is outside of the
20       program.
21
22       To get more details on all  options  and  functions  described  herein,
23       please refer to their respective man pages.
24
25

Building

27       There  are  many  different ways to build C programs. This chapter will
28       assume a Unix style build process. If you use a different build system,
29       you  can  still  read this to get general information that may apply to
30       your environment as well.
31
32       Compiling the Program
33              Your compiler needs  to  know  where  the  libcurl  headers  are
34              located.  Therefore you must set your compiler's include path to
35              point to the directory where you installed them. The  'curl-con‐
36              fig'[3] tool can be used to get this information:
37
38              $ curl-config --cflags
39
40
41       Linking the Program with libcurl
42              When  having  compiled the program, you need to link your object
43              files to create a single executable. For that  to  succeed,  you
44              need to link with libcurl and possibly also with other libraries
45              that libcurl itself depends on. Like the OpenSSL libraries,  but
46              even  some  standard  OS  libraries may be needed on the command
47              line. To figure out which flags to use, once  again  the  'curl-
48              config' tool comes to the rescue:
49
50              $ curl-config --libs
51
52
53       SSL or Not
54              libcurl  can  be  built  and customized in many ways. One of the
55              things that varies from different libraries and  builds  is  the
56              support  for SSL-based transfers, like HTTPS and FTPS. If a sup‐
57              ported SSL library was detected properly at build-time,  libcurl
58              will  be  built  with SSL support. To figure out if an installed
59              libcurl has been built with SSL support enabled, use  'curl-con‐
60              fig' like this:
61
62              $ curl-config --feature
63
64              And  if  SSL  is supported, the keyword 'SSL' will be written to
65              stdout, possibly together with a few other features  that  could
66              be either on or off on for different libcurls.
67
68              See also the "Features libcurl Provides" further down.
69
70       autoconf macro
71              When you write your configure script to detect libcurl and setup
72              variables accordingly, we offer a prewritten macro that probably
73              does     everything    you    need    in    this    area.    See
74              docs/libcurl/libcurl.m4 file - it includes docs on  how  to  use
75              it.
76
77

Portable Code in a Portable World

79       The  people  behind  libcurl  have  put  a  considerable effort to make
80       libcurl work on a large amount of different operating systems and envi‐
81       ronments.
82
83       You program libcurl the same way on all platforms that libcurl runs on.
84       There are only very few minor considerations that differ. If  you  just
85       make  sure to write your code portable enough, you may very well create
86       yourself a very portable program. libcurl shouldn't stop you from that.
87
88

Global Preparation

90       The program must initialize some of the libcurl functionality globally.
91       That means it should be done exactly once, no matter how many times you
92       intend to use the library. Once for your program's  entire  life  time.
93       This is done using
94
95        curl_global_init()
96
97       and  it  takes  one parameter which is a bit pattern that tells libcurl
98       what to initialize. Using CURL_GLOBAL_ALL will make it  initialize  all
99       known  internal  sub  modules,  and might be a good default option. The
100       current two bits that are specified are:
101
102              CURL_GLOBAL_WIN32
103                     which only does anything on Windows machines.  When  used
104                     on  a  Windows machine, it'll make libcurl initialize the
105                     win32 socket stuff. Without having that initialized prop‐
106                     erly,  your  program  cannot  use  sockets  properly. You
107                     should only do this once for each application, so if your
108                     program  already  does  this or of another library in use
109                     does it, you should not tell libcurl to do this as well.
110
111              CURL_GLOBAL_SSL
112                     which only does anything on libcurls compiled  and  built
113                     SSL-enabled.  On  these  systems,  this will make libcurl
114                     initialize the SSL library properly for this application.
115                     This  only  needs to be done once for each application so
116                     if your program or another  library  already  does  this,
117                     this bit should not be needed.
118
119       libcurl   has   a   default   protection   mechanism  that  detects  if
120       curl_global_init(3) hasn't been called by the time curl_easy_perform(3)
121       is  called  and  if  that is the case, libcurl runs the function itself
122       with a guessed bit pattern. Please note that depending solely  on  this
123       is not considered nice nor very good.
124
125       When   the   program   no   longer   uses   libcurl,   it  should  call
126       curl_global_cleanup(3), which is the opposite of the init call. It will
127       then   do   the  reversed  operations  to  cleanup  the  resources  the
128       curl_global_init(3) call initialized.
129
130       Repeated calls to curl_global_init(3) and curl_global_cleanup(3) should
131       be avoided. They should only be called once each.
132
133

Features libcurl Provides

135       It  is  considered  best-practice to determine libcurl features at run-
136       time rather than at build-time (if  possible  of  course).  By  calling
137       curl_version_info(3)  and  checking  out  the  details  of the returned
138       struct, your program can figure out exactly what the currently  running
139       libcurl supports.
140
141

Two Interfaces

143       libcurl  first  introduced the so called easy interface. All operations
144       in the easy interface are prefixed with 'curl_easy'. The easy interface
145       lets  you  do single transfers with a synchronous and blocking function
146       call.
147
148       libcurl also offers another interface that allows multiple simultaneous
149       transfers in a single thread, the so called multi interface. More about
150       that interface is detailed in a  separate  chapter  further  down.  You
151       still  need  to understand the easy interface first, so please continue
152       reading for better understanding.
153

Handle the Easy libcurl

155       To use the easy interface, you must first create yourself an easy  han‐
156       dle.  You  need  one  handle for each easy session you want to perform.
157       Basically, you should use one handle for every thread you plan  to  use
158       for  transferring.  You  must  never  share the same handle in multiple
159       threads.
160
161       Get an easy handle with
162
163        easyhandle = curl_easy_init();
164
165       It returns an easy handle. Using that you proceed  to  the  next  step:
166       setting  up your preferred actions. A handle is just a logic entity for
167       the upcoming transfer or series of transfers.
168
169       You   set   properties   and   options   for    this    handle    using
170       curl_easy_setopt(3). They control how the subsequent transfer or trans‐
171       fers will be made. Options remain set in the handle until set again  to
172       something  different. They are sticky. Multiple requests using the same
173       handle will use the same options.
174
175       If you at any point would like to blank all previously set options  for
176       a  single easy handle, you can call curl_easy_reset(3) and you can also
177       make a clone of an  easy  handle  (with  all  its  set  options)  using
178       curl_easy_duphandle(3).
179
180       Many  of the options you set in libcurl are "strings", pointers to data
181       terminated  with   a   zero   byte.   When   you   set   strings   with
182       curl_easy_setopt(3), libcurl makes its own copy so that they don't need
183       to be kept around in your application after being set[4].
184
185       One of the most basic properties to set in the handle is the  URL.  You
186       set your preferred URL to transfer with CURLOPT_URL(3) in a manner sim‐
187       ilar to:
188
189        curl_easy_setopt(handle, CURLOPT_URL, "http://domain.com/");
190
191       Let's assume for a while that you want to receive data as the URL iden‐
192       tifies  a  remote resource you want to get here. Since you write a sort
193       of application that needs this transfer, I assume that you  would  like
194       to  get  the  data  passed to you directly instead of simply getting it
195       passed to stdout. So, you write your own  function  that  matches  this
196       prototype:
197
198        size_t  write_data(void  *buffer,  size_t  size,  size_t  nmemb,  void
199       *userp);
200
201       You tell libcurl to pass all data to this function by issuing  a  func‐
202       tion similar to this:
203
204        curl_easy_setopt(easyhandle, CURLOPT_WRITEFUNCTION, write_data);
205
206       You  can  control  what  data your callback function gets in the fourth
207       argument by setting another property:
208
209        curl_easy_setopt(easyhandle, CURLOPT_WRITEDATA, &internal_struct);
210
211       Using that property, you can easily pass local data between your appli‐
212       cation  and  the  function that gets invoked by libcurl. libcurl itself
213       won't touch the data you pass with CURLOPT_WRITEDATA(3).
214
215       libcurl offers its own default internal callback that will take care of
216       the  data  if you don't set the callback with CURLOPT_WRITEFUNCTION(3).
217       It will then simply output the received data to stdout.  You  can  have
218       the default callback write the data to a different file handle by pass‐
219       ing a 'FILE *' to a file opened for  writing  with  the  CURLOPT_WRITE‐
220       DATA(3) option.
221
222       Now,  we need to take a step back and have a deep breath. Here's one of
223       those rare platform-dependent nitpicks. Did you spot it? On some  plat‐
224       forms[2],  libcurl won't be able to operate on files opened by the pro‐
225       gram. Thus, if you use the default callback and pass in  an  open  file
226       with  CURLOPT_WRITEDATA(3),  it  will crash. You should therefore avoid
227       this to make your program run fine virtually everywhere.
228
229       (CURLOPT_WRITEDATA(3) was formerly known as  CURLOPT_FILE.  Both  names
230       still work and do the same thing).
231
232       If you're using libcurl as a win32 DLL, you MUST use the CURLOPT_WRITE‐
233       FUNCTION(3) if you set CURLOPT_WRITEDATA(3) - or  you  will  experience
234       crashes.
235
236       There  are  of course many more options you can set, and we'll get back
237       to a few of them later. Let's instead continue to the actual transfer:
238
239        success = curl_easy_perform(easyhandle);
240
241       curl_easy_perform(3) will connect to the remote site, do the  necessary
242       commands  and receive the transfer. Whenever it receives data, it calls
243       the callback function we previously set. The function may get one  byte
244       at  a  time,  or it may get many kilobytes at once. libcurl delivers as
245       much as possible as often as possible. Your  callback  function  should
246       return  the number of bytes it "took care of". If that is not the exact
247       same amount of bytes that was passed to  it,  libcurl  will  abort  the
248       operation and return with an error code.
249
250       When  the transfer is complete, the function returns a return code that
251       informs you if it succeeded in its mission or not.  If  a  return  code
252       isn't  enough  for you, you can use the CURLOPT_ERRORBUFFER(3) to point
253       libcurl to a buffer of yours where it'll store a human  readable  error
254       message as well.
255
256       If  you  then  want to transfer another file, the handle is ready to be
257       used again. Mind you, it is even preferred that you re-use an  existing
258       handle  if  you  intend  to  make  another  transfer. libcurl will then
259       attempt to re-use the previous connection.
260
261       For some protocols,  downloading  a  file  can  involve  a  complicated
262       process  of logging in, setting the transfer mode, changing the current
263       directory and finally transferring the file data. libcurl takes care of
264       all  that complication for you. Given simply the URL to a file, libcurl
265       will take care of all the details needed to get the file moved from one
266       machine to another.
267
268

Multi-threading Issues

270       libcurl  is  thread  safe  but  there  are  a  few exceptions. Refer to
271       libcurl-thread(3) for more information.
272
273

When It Doesn't Work

275       There will always be times when the transfer fails for some reason. You
276       might  have  set  the  wrong  libcurl  option or misunderstood what the
277       libcurl option actually does, or the remote server  might  return  non-
278       standard replies that confuse the library which then confuses your pro‐
279       gram.
280
281       There's one golden rule when these things occur: set  the  CURLOPT_VER‐
282       BOSE(3)  option  to  1.  It'll cause the library to spew out the entire
283       protocol details it sends, some internal info and some received  proto‐
284       col  data  as  well  (especially when using FTP). If you're using HTTP,
285       adding the headers in the received output to study is also a clever way
286       to  get  a better understanding why the server behaves the way it does.
287       Include headers in the normal body output with CURLOPT_HEADER(3) set 1.
288
289       Of course, there are bugs left. We need to know about them to  be  able
290       to  fix them, so we're quite dependent on your bug reports! When you do
291       report suspected bugs in libcurl, please include as many details as you
292       possibly can: a protocol dump that CURLOPT_VERBOSE(3) produces, library
293       version, as much as possible of your code that uses libcurl,  operating
294       system name and version, compiler name and version etc.
295
296       If  CURLOPT_VERBOSE(3)  is  not enough, you increase the level of debug
297       data your application receive by using the CURLOPT_DEBUGFUNCTION(3).
298
299       Getting some in-depth knowledge about the protocols involved  is  never
300       wrong,  and  if  you're  trying to do funny things, you might very well
301       understand libcurl and how to use it better if you study the  appropri‐
302       ate RFC documents at least briefly.
303
304

Upload Data to a Remote Site

306       libcurl  tries  to  keep a protocol independent approach to most trans‐
307       fers, thus uploading to a remote FTP site is very similar to  uploading
308       data to an HTTP server with a PUT request.
309
310       Of  course,  first  you  either create an easy handle or you re-use one
311       existing one. Then you set the URL to operate on just like before. This
312       is the remote URL, that we now will upload.
313
314       Since  we  write an application, we most likely want libcurl to get the
315       upload data by asking us for it. To make it do that, we  set  the  read
316       callback and the custom pointer libcurl will pass to our read callback.
317       The read callback should have a prototype similar to:
318
319        size_t  function(char  *bufptr,  size_t  size,  size_t  nitems,   void
320       *userp);
321
322       Where  bufptr is the pointer to a buffer we fill in with data to upload
323       and size*nitems is the size of the buffer and therefore also the  maxi‐
324       mum  amount  of data we can return to libcurl in this call. The 'userp'
325       pointer is the custom pointer we set to point to a struct  of  ours  to
326       pass private data between the application and the callback.
327
328        curl_easy_setopt(easyhandle, CURLOPT_READFUNCTION, read_function);
329
330        curl_easy_setopt(easyhandle, CURLOPT_READDATA, &filedata);
331
332       Tell libcurl that we want to upload:
333
334        curl_easy_setopt(easyhandle, CURLOPT_UPLOAD, 1L);
335
336       A few protocols won't behave properly when uploads are done without any
337       prior knowledge of the expected file size. So, set the upload file size
338       using  the  CURLOPT_INFILESIZE_LARGE(3)  for  all known file sizes like
339       this[1]:
340
341        /* in this example, file_size must be an curl_off_t variable */
342        curl_easy_setopt(easyhandle, CURLOPT_INFILESIZE_LARGE, file_size);
343
344       When you call curl_easy_perform(3) this time,  it'll  perform  all  the
345       necessary operations and when it has invoked the upload it'll call your
346       supplied callback to get the data to upload. The program should  return
347       as much data as possible in every invoke, as that is likely to make the
348       upload perform as fast as possible. The callback should return the num‐
349       ber of bytes it wrote in the buffer. Returning 0 will signal the end of
350       the upload.
351
352

Passwords

354       Many protocols use or even require that user name and password are pro‐
355       vided to be able to download or upload the data of your choice. libcurl
356       offers several ways to specify them.
357
358       Most protocols support that you specify the name and  password  in  the
359       URL  itself. libcurl will detect this and use them accordingly. This is
360       written like this:
361
362        protocol://user:password@example.com/path/
363
364       If you need any odd letters in your user name or password,  you  should
365       enter them URL encoded, as %XX where XX is a two-digit hexadecimal num‐
366       ber.
367
368       libcurl also provides options to set various passwords. The  user  name
369       and  password as shown embedded in the URL can instead get set with the
370       CURLOPT_USERPWD(3) option. The argument passed to libcurl should  be  a
371       char  *  to  a  string  in the format "user:password". In a manner like
372       this:
373
374        curl_easy_setopt(easyhandle, CURLOPT_USERPWD, "myname:thesecret");
375
376       Another case where name and password might be needed at times,  is  for
377       those  users  who  need to authenticate themselves to a proxy they use.
378       libcurl offers another option for this, the CURLOPT_PROXYUSERPWD(3). It
379       is used quite similar to the CURLOPT_USERPWD(3) option like this:
380
381        curl_easy_setopt(easyhandle,    CURLOPT_PROXYUSERPWD,   "myname:these‐
382       cret");
383
384       There's a long time Unix "standard" way of storing FTP user  names  and
385       passwords,  namely  in  the  $HOME/.netrc file. The file should be made
386       private so that only the user may read it (see also the "Security  Con‐
387       siderations"  chapter), as it might contain the password in plain text.
388       libcurl has the ability to use this file to figure out what set of user
389       name  and password to use for a particular host. As an extension to the
390       normal functionality, libcurl also supports this file for non-FTP  pro‐
391       tocols  such  as  HTTP.  To  make  curl  use  this  file,  use the CUR‐
392       LOPT_NETRC(3) option:
393
394        curl_easy_setopt(easyhandle, CURLOPT_NETRC, 1L);
395
396       And a very basic example of how such a .netrc file may look like:
397
398        machine myhost.mydomain.com
399        login userlogin
400        password secretword
401
402       All these  examples  have  been  cases  where  the  password  has  been
403       optional,  or  at least you could leave it out and have libcurl attempt
404       to do its job without it. There  are  times  when  the  password  isn't
405       optional,  like  when you're using an SSL private key for secure trans‐
406       fers.
407
408       To pass the known private key password to libcurl:
409
410        curl_easy_setopt(easyhandle, CURLOPT_KEYPASSWD, "keypassword");
411
412

HTTP Authentication

414       The previous chapter showed how to set user name and password for  get‐
415       ting  URLs  that  require authentication. When using the HTTP protocol,
416       there are many different ways a client can provide those credentials to
417       the  server and you can control which way libcurl will (attempt to) use
418       them. The default HTTP authentication method is called  'Basic',  which
419       is  sending  the  name  and password in clear-text in the HTTP request,
420       base64-encoded. This is insecure.
421
422       At the time of this writing,  libcurl  can  be  built  to  use:  Basic,
423       Digest, NTLM, Negotiate (SPNEGO). You can tell libcurl which one to use
424       with CURLOPT_HTTPAUTH(3) as in:
425
426        curl_easy_setopt(easyhandle, CURLOPT_HTTPAUTH, CURLAUTH_DIGEST);
427
428       And when you send authentication to a proxy, you can also set authenti‐
429       cation type the same way but instead with CURLOPT_PROXYAUTH(3):
430
431        curl_easy_setopt(easyhandle, CURLOPT_PROXYAUTH, CURLAUTH_NTLM);
432
433       Both  these  options  allow  you  to  set multiple types (by ORing them
434       together), to make libcurl pick the most secure one out  of  the  types
435       the  server/proxy  claims  to  support.  This method does however add a
436       round-trip since libcurl must first ask the server what it supports:
437
438        curl_easy_setopt(easyhandle, CURLOPT_HTTPAUTH,
439        CURLAUTH_DIGEST|CURLAUTH_BASIC);
440
441       For convenience, you can use the 'CURLAUTH_ANY' define  (instead  of  a
442       list  with  specific types) which allows libcurl to use whatever method
443       it wants.
444
445       When asking for multiple types, libcurl will pick the available one  it
446       considers "best" in its own internal order of preference.
447
448

HTTP POSTing

450       We  get  many  questions regarding how to issue HTTP POSTs with libcurl
451       the proper way. This chapter will thus include examples using both dif‐
452       ferent versions of HTTP POST that libcurl supports.
453
454       The  first  version  is  the simple POST, the most common version, that
455       most HTML pages using the <form> tag uses. We provide a pointer to  the
456       data and tell libcurl to post it all to the remote site:
457
458           char *data="name=daniel&project=curl";
459           curl_easy_setopt(easyhandle, CURLOPT_POSTFIELDS, data);
460           curl_easy_setopt(easyhandle, CURLOPT_URL, "http://posthere.com/");
461
462           curl_easy_perform(easyhandle); /* post away! */
463
464       Simple  enough,  huh?  Since  you  set  the  POST options with the CUR‐
465       LOPT_POSTFIELDS(3), this automatically switches the handle to use  POST
466       in the upcoming request.
467
468       Ok,  so  what if you want to post binary data that also requires you to
469       set the Content-Type: header of the post? Well,  binary  posts  prevent
470       libcurl  from  being  able to do strlen() on the data to figure out the
471       size, so therefore we must tell libcurl the size of the post data. Set‐
472       ting headers in libcurl requests are done in a generic way, by building
473       a list of our own headers and then passing that list to libcurl.
474
475        struct curl_slist *headers=NULL;
476        headers = curl_slist_append(headers, "Content-Type: text/xml");
477
478        /* post binary data */
479        curl_easy_setopt(easyhandle, CURLOPT_POSTFIELDS, binaryptr);
480
481        /* set the size of the postfields data */
482        curl_easy_setopt(easyhandle, CURLOPT_POSTFIELDSIZE, 23L);
483
484        /* pass our list of custom made headers */
485        curl_easy_setopt(easyhandle, CURLOPT_HTTPHEADER, headers);
486
487        curl_easy_perform(easyhandle); /* post away! */
488
489        curl_slist_free_all(headers); /* free the header list */
490
491       While the simple examples above cover the majority of all  cases  where
492       HTTP  POST operations are required, they don't do multi-part formposts.
493       Multi-part formposts were introduced as a better way to post  (possibly
494       large) binary data and were first documented in the RFC1867 (updated in
495       RFC2388). They're called multi-part because they're built by a chain of
496       parts,  each  part  being  a single unit of data. Each part has its own
497       name and contents. You can in fact create and post a  multi-part  form‐
498       post  with  the  regular libcurl POST support described above, but that
499       would require that  you  build  a  formpost  yourself  and  provide  to
500       libcurl. To make that easier, libcurl provides a MIME API consisting in
501       several functions: using those, you can create and  fill  a  multi-part
502       form.   Function  curl_mime_init(3)  creates a multi-part body; you can
503       then append new parts to a multi-part body using  curl_mime_addpart(3).
504       There  are  three  possible  data  sources  for  a  part:  memory using
505       curl_mime_data(3), file using  curl_mime_filedata(3)  and  user-defined
506       data  read callback using curl_mime_data_cb(3).  curl_mime_name(3) sets
507       a part's (i.e.: form field) name, while curl_mime_filename(3) fills  in
508       the  remote  file  name.  With curl_mime_type(3), you can tell the MIME
509       type of a part, curl_mime_headers(3) allows defining the  part's  head‐
510       ers.  When  a  multi-part  body is no longer needed, you can destroy it
511       using curl_mime_free(3).
512
513       The following example sets two simple text  parts  with  plain  textual
514       contents,  and  then  a file with binary contents and uploads the whole
515       thing.
516
517        curl_mime *multipart = curl_mime_init(easyhandle);
518        curl_mimepart *part = curl_mime_addpart(mutipart);
519        curl_mime_name(part, "name");
520        curl_mime_data(part, "daniel", CURL_ZERO_TERMINATED);
521        part = curl_mime_addpart(mutipart);
522        curl_mime_name(part, "project");
523        curl_mime_data(part, "curl", CURL_ZERO_TERMINATED);
524        part = curl_mime_addpart(mutipart);
525        curl_mime_name(part, "logotype-image");
526        curl_mime_filedata(part, "curl.png");
527
528        /* Set the form info */
529        curl_easy_setopt(easyhandle, CURLOPT_MIMEPOST, multipart);
530
531        curl_easy_perform(easyhandle); /* post away! */
532
533        /* free the post data again */
534        curl_mime_free(multipart);
535
536       To post multiple files for a single form field, you  must  supply  each
537       file  in  a separate part, all with the same field name. Although func‐
538       tion curl_mime_subparts(3) implements nested multi-parts, this  way  of
539       multiple files posting is deprecated by RFC 7578, chapter 4.3.
540
541       To set the data source from an already opened FILE pointer, use:
542
543        curl_mime_data_cb(part, filesize, (curl_read_callback) fread,
544                          (curl_seek_callback) fseek, NULL, filepointer);
545
546       A  deprecated  curl_formadd(3)  function is still supported in libcurl.
547       It should however not be used anymore  for  new  designs  and  programs
548       using it ought to be converted to the MIME API. It is however described
549       here as an aid to conversion.
550
551       Using curl_formadd, you add parts to the form. When you're done  adding
552       parts, you post the whole form.
553
554       The MIME API example above is expressed as follows using this function:
555
556        struct curl_httppost *post=NULL;
557        struct curl_httppost *last=NULL;
558        curl_formadd(&post, &last,
559                     CURLFORM_COPYNAME, "name",
560                     CURLFORM_COPYCONTENTS, "daniel", CURLFORM_END);
561        curl_formadd(&post, &last,
562                     CURLFORM_COPYNAME, "project",
563                     CURLFORM_COPYCONTENTS, "curl", CURLFORM_END);
564        curl_formadd(&post, &last,
565                     CURLFORM_COPYNAME, "logotype-image",
566                     CURLFORM_FILECONTENT, "curl.png", CURLFORM_END);
567
568        /* Set the form info */
569        curl_easy_setopt(easyhandle, CURLOPT_HTTPPOST, post);
570
571        curl_easy_perform(easyhandle); /* post away! */
572
573        /* free the post data again */
574        curl_formfree(post);
575
576       Multipart formposts are chains of parts using MIME-style separators and
577       headers. It means that each one of these separate parts get a few head‐
578       ers  set that describe the individual content-type, size etc. To enable
579       your application to handicraft this formpost even more, libcurl  allows
580       you to supply your own set of custom headers to such an individual form
581       part. You can of course supply headers to as many parts  as  you  like,
582       but  this  little example will show how you set headers to one specific
583       part when you add that to the post handle:
584
585        struct curl_slist *headers=NULL;
586        headers = curl_slist_append(headers, "Content-Type: text/xml");
587
588        curl_formadd(&post, &last,
589                     CURLFORM_COPYNAME, "logotype-image",
590                     CURLFORM_FILECONTENT, "curl.xml",
591                     CURLFORM_CONTENTHEADER, headers,
592                     CURLFORM_END);
593
594        curl_easy_perform(easyhandle); /* post away! */
595
596        curl_formfree(post); /* free post */
597        curl_slist_free_all(headers); /* free custom header list */
598
599       Since all options on an easyhandle are "sticky", they remain  the  same
600       until changed even if you do call curl_easy_perform(3), you may need to
601       tell curl to go back to a plain GET request if you intend to do one  as
602       your  next  request. You force an easyhandle to go back to GET by using
603       the CURLOPT_HTTPGET(3) option:
604
605        curl_easy_setopt(easyhandle, CURLOPT_HTTPGET, 1L);
606
607       Just setting CURLOPT_POSTFIELDS(3)  to  ""  or  NULL  will  *not*  stop
608       libcurl  from  doing a POST. It will just make it POST without any data
609       to send!
610
611

Converting from deprecated form API to MIME API

613       Four rules have to be respected in building the multi-part:
614       - The easy handle must be created before building the multi-part.
615       - The multi-part is always created by a call to curl_mime_init(easyhan‐
616       dle).
617       - Each part is created by a call to curl_mime_addpart(multipart).
618       -  When complete, the multi-part must be bound to the easy handle using
619       CURLOPT_MIMEPOST(3) instead of CURLOPT_HTTPPOST(3).
620
621       Here are some example of curl_formadd calls to MIME API sequences:
622
623        curl_formadd(&post, &last,
624                     CURLFORM_COPYNAME, "id",
625                     CURLFORM_COPYCONTENTS, "daniel", CURLFORM_END);
626                     CURLFORM_CONTENTHEADER, headers,
627                     CURLFORM_END);
628       becomes:
629        part = curl_mime_addpart(multipart);
630        curl_mime_name(part, "id");
631        curl_mime_data(part, "daniel", CURL_ZERO_TERMINATED);
632        curl_mime_headers(part, headers, FALSE);
633
634       Setting the last curl_mime_headers argument to TRUE would  have  caused
635       the headers to be automatically released upon destroyed the multi-part,
636       thus saving a clean-up call to curl_slist_free_all(3).
637
638        curl_formadd(&post, &last,
639                     CURLFORM_PTRNAME, "logotype-image",
640                     CURLFORM_FILECONTENT, "-",
641                     CURLFORM_END);
642       becomes:
643        part = curl_mime_addpart(multipart);
644        curl_mime_name(part, "logotype-image");
645        curl_mime_data_cb(part, (curl_off_t) -1, fread, fseek, NULL, stdin);
646
647       curl_mime_name always copies the field name. The special file name  "-"
648       is  not  supported by curl_mime_file: to read an open file, use a call‐
649       back source using fread(). The transfer will be chunked since the  data
650       size is unknown.
651
652        curl_formadd(&post, &last,
653                     CURLFORM_COPYNAME, "datafile[]",
654                     CURLFORM_FILE, "file1",
655                     CURLFORM_FILE, "file2",
656                     CURLFORM_END);
657       becomes:
658        part = curl_mime_addpart(multipart);
659        curl_mime_name(part, "datafile[]");
660        curl_mime_filedata(part, "file1");
661        part = curl_mime_addpart(multipart);
662        curl_mime_name(part, "datafile[]");
663        curl_mime_filedata(part, "file2");
664
665       The  deprecated  multipart/mixed implementation of multiple files field
666       is translated to two distinct parts with the same name.
667
668        curl_easy_setopt(easyhandle, CURLOPT_READFUNCTION, myreadfunc);
669        curl_formadd(&post, &last,
670                     CURLFORM_COPYNAME, "stream",
671                     CURLFORM_STREAM, arg,
672                     CURLFORM_CONTENTLEN, (curl_off_t) datasize,
673                     CURLFORM_FILENAME, "archive.zip",
674                     CURLFORM_CONTENTTYPE, "application/zip",
675                     CURLFORM_END);
676       becomes:
677        part = curl_mime_addpart(multipart);
678        curl_mime_name(part, "stream");
679        curl_mime_data_cb(part, (curl_off_t) datasize,
680                          myreadfunc, NULL, NULL, arg);
681        curl_mime_filename(part, "archive.zip");
682        curl_mime_type(part, "application/zip");
683
684       CURLOPT_READFUNCTION callback is not used: it is  replace  by  directly
685       setting the part source data from the callback read function.
686
687        curl_formadd(&post, &last,
688                     CURLFORM_COPYNAME, "memfile",
689                     CURLFORM_BUFFER, "memfile.bin",
690                     CURLFORM_BUFFERPTR, databuffer,
691                     CURLFORM_BUFFERLENGTH, (long) sizeof databuffer,
692                     CURLFORM_END);
693       becomes:
694        part = curl_mime_addpart(multipart);
695        curl_mime_name(part, "memfile");
696        curl_mime_data(part, databuffer, (curl_off_t) sizeof databuffer);
697        curl_mime_filename(part, "memfile.bin");
698
699       curl_mime_data always copies the initial data: data buffer is thus free
700       for immediate reuse.
701
702        curl_formadd(&post, &last,
703                     CURLFORM_COPYNAME, "message",
704                     CURLFORM_FILECONTENT, "msg.txt",
705                     CURLFORM_END);
706       becomes:
707        part = curl_mime_addpart(multipart);
708        curl_mime_name(part, "message");
709        curl_mime_filedata(part, "msg.txt");
710        curl_mime_filename(part, NULL);
711
712       Use of curl_mime_filedata sets the remote file name as a  side  effect:
713       it  is  therefore necessary to clear it for CURLFORM_FILECONTENT emula‐
714       tion.
715
716

Showing Progress

718       For historical and traditional reasons, libcurl has a built-in progress
719       meter  that  can  be  switched  on and then makes it present a progress
720       meter in your terminal.
721
722       Switch  on  the  progress  meter  by,  oddly   enough,   setting   CUR‐
723       LOPT_NOPROGRESS(3) to zero. This option is set to 1 by default.
724
725       For  most  applications however, the built-in progress meter is useless
726       and what instead is interesting is the ability to  specify  a  progress
727       callback.  The function pointer you pass to libcurl will then be called
728       on irregular intervals with information about the current transfer.
729
730       Set the progress callback  by  using  CURLOPT_PROGRESSFUNCTION(3).  And
731       pass a pointer to a function that matches this prototype:
732
733        int progress_callback(void *clientp,
734                              double dltotal,
735                              double dlnow,
736                              double ultotal,
737                              double ulnow);
738
739       If any of the input arguments is unknown, a 0 will be passed. The first
740       argument, the 'clientp' is the pointer you pass to  libcurl  with  CUR‐
741       LOPT_PROGRESSDATA(3). libcurl won't touch it.
742
743

libcurl with C++

745       There's basically only one thing to keep in mind when using C++ instead
746       of C when interfacing libcurl:
747
748       The callbacks CANNOT be non-static class member functions
749
750       Example C++ code:
751
752       class AClass {
753           static size_t write_data(void *ptr, size_t size, size_t nmemb,
754                                    void *ourpointer)
755           {
756             /* do what you want with the data */
757           }
758        }
759
760

Proxies

762       What "proxy" means according to Merriam-Webster: "a  person  authorized
763       to  act  for  another"  but  also "the agency, function, or office of a
764       deputy who acts as a substitute for another".
765
766       Proxies are exceedingly common these days. Companies often  only  offer
767       Internet  access to employees through their proxies. Network clients or
768       user-agents ask the proxy for documents,  the  proxy  does  the  actual
769       request and then it returns them.
770
771       libcurl  supports  SOCKS  and HTTP proxies. When a given URL is wanted,
772       libcurl will ask the proxy for it instead of trying to connect  to  the
773       actual host identified in the URL.
774
775       If  you're using a SOCKS proxy, you may find that libcurl doesn't quite
776       support all operations through it.
777
778       For HTTP proxies: the fact that the proxy is an HTTP proxy puts certain
779       restrictions  on  what  can actually happen. A requested URL that might
780       not be a HTTP URL will be still be passed to the HTTP proxy to  deliver
781       back to libcurl. This happens transparently, and an application may not
782       need to know. I say "may", because at times it  is  very  important  to
783       understand  that  all operations over an HTTP proxy use the HTTP proto‐
784       col. For example, you can't invoke your own custom FTP commands or even
785       proper FTP directory listings.
786
787
788       Proxy Options
789
790              To tell libcurl to use a proxy at a given port number:
791
792               curl_easy_setopt(easyhandle,       CURLOPT_PROXY,       "proxy-
793              host.com:8080");
794
795              Some proxies  require  user  authentication  before  allowing  a
796              request, and you pass that information similar to this:
797
798               curl_easy_setopt(easyhandle,  CURLOPT_PROXYUSERPWD, "user:pass‐
799              word");
800
801              If you want to, you can specify the host name only in  the  CUR‐
802              LOPT_PROXY(3)  option,  and  set the port number separately with
803              CURLOPT_PROXYPORT(3).
804
805              Tell libcurl what kind of proxy it is with  CURLOPT_PROXYTYPE(3)
806              (if not, it will default to assume an HTTP proxy):
807
808               curl_easy_setopt(easyhandle,      CURLOPT_PROXYTYPE,      CURL‐
809              PROXY_SOCKS4);
810
811
812       Environment Variables
813
814              libcurl automatically checks and uses a set of environment vari‐
815              ables  to  know  what  proxies to use for certain protocols. The
816              names of the variables are following an ancient de  facto  stan‐
817              dard and are built up as "[protocol]_proxy" (note the lower cas‐
818              ing). Which makes the variable 'http_proxy' checked for  a  name
819              of a proxy to use when the input URL is HTTP. Following the same
820              rule, the variable named 'ftp_proxy' is checked  for  FTP  URLs.
821              Again,  the proxies are always HTTP proxies, the different names
822              of the variables simply allows  different  HTTP  proxies  to  be
823              used.
824
825              The  proxy environment variable contents should be in the format
826              "[protocol://][user:password@]machine[:port]". Where the  proto‐
827              col://  part  is  simply ignored if present (so http://proxy and
828              bluerk://proxy will do the same) and the  optional  port  number
829              specifies  on  which port the proxy operates on the host. If not
830              specified, the internal default port number  will  be  used  and
831              that is most likely *not* the one you would like it to be.
832
833              There are two special environment variables. 'all_proxy' is what
834              sets proxy for any URL in case the  protocol  specific  variable
835              wasn't  set,  and 'no_proxy' defines a list of hosts that should
836              not use a proxy even though a variable may say so. If 'no_proxy'
837              is a plain asterisk ("*") it matches all hosts.
838
839              To explicitly disable libcurl's checking for and using the proxy
840              environment variables, set the proxy  name  to  ""  -  an  empty
841              string - with CURLOPT_PROXY(3).
842
843       SSL and Proxies
844
845              SSL  is  for  secure  point-to-point  connections. This involves
846              strong encryption and similar things, which effectively makes it
847              impossible  for  a  proxy to operate as a "man in between" which
848              the proxy's task is, as previously discussed. Instead, the  only
849              way  to  have SSL work over an HTTP proxy is to ask the proxy to
850              tunnel trough everything without being able to check  or  fiddle
851              with the traffic.
852
853              Opening an SSL connection over an HTTP proxy is therefore a mat‐
854              ter of asking the proxy for a straight connection to the  target
855              host  on  a  specified  port. This is made with the HTTP request
856              CONNECT. ("please mr proxy, connect me to that remote host").
857
858              Because of the nature of this operation, where the proxy has  no
859              idea  what  kind  of data that is passed in and out through this
860              tunnel, this breaks some of the very few  advantages  that  come
861              from using a proxy, such as caching.  Many organizations prevent
862              this kind of tunneling to other destination  port  numbers  than
863              443 (which is the default HTTPS port number).
864
865
866       Tunneling Through Proxy
867              As  explained  above,  tunneling is required for SSL to work and
868              often even restricted to the operation intended for SSL; HTTPS.
869
870              This is however not the only time  proxy-tunneling  might  offer
871              benefits to you or your application.
872
873              As  tunneling opens a direct connection from your application to
874              the remote machine, it suddenly also re-introduces  the  ability
875              to  do  non-HTTP  operations over an HTTP proxy. You can in fact
876              use things such as FTP upload or FTP custom commands this way.
877
878              Again, this is often prevented by the administrators of  proxies
879              and is rarely allowed.
880
881              Tell libcurl to use proxy tunneling like this:
882
883               curl_easy_setopt(easyhandle, CURLOPT_HTTPPROXYTUNNEL, 1L);
884
885              In  fact,  there  might  even be times when you want to do plain
886              HTTP operations using a tunnel like this, as it then enables you
887              to  operate  on the remote server instead of asking the proxy to
888              do so. libcurl will not stand in the  way  for  such  innovative
889              actions either!
890
891
892       Proxy Auto-Config
893
894              Netscape  first  came  up  with this. It is basically a web page
895              (usually using a .pac extension) with  a  Javascript  that  when
896              executed by the browser with the requested URL as input, returns
897              information to the browser on how to connect  to  the  URL.  The
898              returned  information  might  be  "DIRECT" (which means no proxy
899              should be used), "PROXY host:port" (to tell  the  browser  where
900              the  proxy  for this particular URL is) or "SOCKS host:port" (to
901              direct the browser to a SOCKS proxy).
902
903              libcurl has no means to interpret  or  evaluate  Javascript  and
904              thus  it doesn't support this. If you get yourself in a position
905              where you face this nasty invention, the following  advice  have
906              been mentioned and used in the past:
907
908              - Depending on the Javascript complexity, write up a script that
909              translates it to another language and execute that.
910
911              - Read the Javascript code and rewrite the same logic in another
912              language.
913
914              -  Implement  a Javascript interpreter; people have successfully
915              used the Mozilla Javascript engine in the past.
916
917              - Ask your admins to stop this, for a static proxy setup or sim‐
918              ilar.
919
920

Persistence Is The Way to Happiness

922       Re-cycling  the  same  easy  handle  several  times when doing multiple
923       requests is the way to go.
924
925       After each single curl_easy_perform(3) operation, libcurl will keep the
926       connection  alive  and  open.  A subsequent request using the same easy
927       handle to the same host might just be able to use the already open con‐
928       nection! This reduces network impact a lot.
929
930       Even if the connection is dropped, all connections involving SSL to the
931       same host again, will benefit from  libcurl's  session  ID  cache  that
932       drastically reduces re-connection time.
933
934       FTP connections that are kept alive save a lot of time, as the command-
935       response round-trips are skipped,  and  also  you  don't  risk  getting
936       blocked without permission to login again like on many FTP servers only
937       allowing N persons to be logged in at the same time.
938
939       libcurl caches DNS name resolving results, to make lookups of a  previ‐
940       ously looked up name a lot faster.
941
942       Other  interesting  details  that  improve  performance  for subsequent
943       requests may also be added in the future.
944
945       Each easy handle will attempt to keep the last  few  connections  alive
946       for  a while in case they are to be used again. You can set the size of
947       this "cache" with the  CURLOPT_MAXCONNECTS(3)  option.  Default  is  5.
948       There is very seldom any point in changing this value, and if you think
949       of changing this it is often just a matter of thinking again.
950
951       To force your upcoming request to not use an already  existing  connec‐
952       tion  (it will even close one first if there happens to be one alive to
953       the same host you're about to operate on), you can do that  by  setting
954       CURLOPT_FRESH_CONNECT(3) to 1. In a similar spirit, you can also forbid
955       the upcoming request to be "lying"  around  and  possibly  get  re-used
956       after the request by setting CURLOPT_FORBID_REUSE(3) to 1.
957
958

HTTP Headers Used by libcurl

960       When  you use libcurl to do HTTP requests, it'll pass along a series of
961       headers automatically. It might be good for you to know and  understand
962       these.  You  can  replace  or  remove  them  by using the CURLOPT_HTTP‐
963       HEADER(3) option.
964
965
966       Host   This header is required by HTTP 1.1 and even  many  1.0  servers
967              and  should  be  the name of the server we want to talk to. This
968              includes the port number if anything but default.
969
970
971       Accept "*/*".
972
973
974       Expect When doing POST requests, libcurl sets this header to  "100-con‐
975              tinue"  to ask the server for an "OK" message before it proceeds
976              with sending the data part of  the  post.  If  the  POSTed  data
977              amount is deemed "small", libcurl will not use this header.
978
979

Customizing Operations

981       There is an ongoing development today where more and more protocols are
982       built upon HTTP for transport. This has obvious benefits as HTTP  is  a
983       tested  and reliable protocol that is widely deployed and has excellent
984       proxy-support.
985
986       When you use one of these protocols, and even when doing other kinds of
987       programming  you may need to change the traditional HTTP (or FTP or...)
988       manners. You may need to change words, headers or various data.
989
990       libcurl is your friend here too.
991
992
993       CUSTOMREQUEST
994              If just changing the actual HTTP request  keyword  is  what  you
995              want,  like  when  GET, HEAD or POST is not good enough for you,
996              CURLOPT_CUSTOMREQUEST(3) is there for you. It is very simple  to
997              use:
998
999               curl_easy_setopt(easyhandle,  CURLOPT_CUSTOMREQUEST,  "MYOWNRE‐
1000              QUEST");
1001
1002              When using the custom request, you change the request keyword of
1003              the actual request you are performing. Thus, by default you make
1004              a GET request but  you  can  also  make  a  POST  operation  (as
1005              described  before) and then replace the POST keyword if you want
1006              to. You're the boss.
1007
1008
1009       Modify Headers
1010              HTTP-like protocols pass a series of headers to the server  when
1011              doing  the  request, and you're free to pass any amount of extra
1012              headers that you think fit. Adding headers is this easy:
1013
1014               struct curl_slist *headers=NULL; /* init to NULL is important */
1015
1016               headers = curl_slist_append(headers, "Hey-server-hey: how are you?");
1017               headers = curl_slist_append(headers, "X-silly-content: yes");
1018
1019               /* pass our list of custom made headers */
1020               curl_easy_setopt(easyhandle, CURLOPT_HTTPHEADER, headers);
1021
1022               curl_easy_perform(easyhandle); /* transfer http */
1023
1024               curl_slist_free_all(headers); /* free the header list */
1025
1026              ... and if you think some of the internally  generated  headers,
1027              such as Accept: or Host: don't contain the data you want them to
1028              contain, you can replace them by simply setting them too:
1029
1030               headers = curl_slist_append(headers, "Accept: Agent-007");
1031               headers = curl_slist_append(headers, "Host: munged.host.line");
1032
1033
1034       Delete Headers
1035              If you replace an existing header with one with no contents, you
1036              will  prevent  the  header from being sent. For instance, if you
1037              want to completely prevent the "Accept:" header from being sent,
1038              you can disable it with code similar to this:
1039
1040               headers = curl_slist_append(headers, "Accept:");
1041
1042              Both  replacing  and  canceling  internal headers should be done
1043              with careful consideration and you should be aware that you  may
1044              violate the HTTP protocol when doing so.
1045
1046
1047       Enforcing chunked transfer-encoding
1048
1049              By making sure a request uses the custom header "Transfer-Encod‐
1050              ing: chunked" when doing a non-GET HTTP operation, libcurl  will
1051              switch  over  to  "chunked"  upload, even though the size of the
1052              data to upload might  be  known.  By  default,  libcurl  usually
1053              switches over to chunked upload automatically if the upload data
1054              size is unknown.
1055
1056
1057       HTTP Version
1058
1059              All HTTP requests includes the version number to tell the server
1060              which  version  we  support. libcurl speaks HTTP 1.1 by default.
1061              Some very old servers don't like getting 1.1-requests  and  when
1062              dealing with stubborn old things like that, you can tell libcurl
1063              to use 1.0 instead by doing something like this:
1064
1065               curl_easy_setopt(easyhandle,              CURLOPT_HTTP_VERSION,
1066              CURL_HTTP_VERSION_1_0);
1067
1068
1069       FTP Custom Commands
1070
1071              Not all protocols are HTTP-like, and thus the above may not help
1072              you when you want to make, for example, your  FTP  transfers  to
1073              behave differently.
1074
1075              Sending  custom commands to an FTP server means that you need to
1076              send the commands exactly as the FTP server expects them (RFC959
1077              is  a  good guide here), and you can only use commands that work
1078              on the control-connection alone.  All  kinds  of  commands  that
1079              require data interchange and thus need a data-connection must be
1080              left to libcurl's own judgement. Also be aware that libcurl will
1081              do  its  very  best  to change directory to the target directory
1082              before doing any transfer, so if you change directory (with  CWD
1083              or  similar)  you  might  confuse  libcurl and then it might not
1084              attempt to transfer the file in the correct remote directory.
1085
1086              A little example that deletes a given file before an operation:
1087
1088               headers = curl_slist_append(headers, "DELE file-to-remove");
1089
1090               /* pass the list of custom commands to the handle */
1091               curl_easy_setopt(easyhandle, CURLOPT_QUOTE, headers);
1092
1093               curl_easy_perform(easyhandle); /* transfer ftp data! */
1094
1095               curl_slist_free_all(headers); /* free the header list */
1096
1097              If you would instead want this operation  (or  chain  of  opera‐
1098              tions) to happen _after_ the data transfer took place the option
1099              to   curl_easy_setopt(3)   would   instead   be   called    CUR‐
1100              LOPT_POSTQUOTE(3) and used the exact same way.
1101
1102              The  custom FTP command will be issued to the server in the same
1103              order they are added to the list, and if a command gets an error
1104              code  returned  back  from  the server, no more commands will be
1105              issued  and  libcurl  will  bail  out   with   an   error   code
1106              (CURLE_QUOTE_ERROR).  Note  that  if you use CURLOPT_QUOTE(3) to
1107              send commands before a transfer, no transfer will actually  take
1108              place when a quote command has failed.
1109
1110              If  you set the CURLOPT_HEADER(3) to 1, you will tell libcurl to
1111              get information about the target file and output "headers" about
1112              it. The headers will be in "HTTP-style", looking like they do in
1113              HTTP.
1114
1115              The option to enable headers or to run custom FTP  commands  may
1116              be  useful  to combine with CURLOPT_NOBODY(3). If this option is
1117              set, no actual file content transfer will be performed.
1118
1119
1120       FTP Custom CUSTOMREQUEST
1121              If you do want to list the contents of an  FTP  directory  using
1122              your  own  defined FTP command, CURLOPT_CUSTOMREQUEST(3) will do
1123              just that. "NLST" is the default one for listing directories but
1124              you're free to pass in your idea of a good alternative.
1125
1126

Cookies Without Chocolate Chips

1128       In  the  HTTP  sense,  a  cookie  is a name with an associated value. A
1129       server sends the name and value to the client, and expects  it  to  get
1130       sent  back  on  every subsequent request to the server that matches the
1131       particular conditions set. The conditions include that the domain  name
1132       and path match and that the cookie hasn't become too old.
1133
1134       In  real-world cases, servers send new cookies to replace existing ones
1135       to update them. Server use cookies to "track" users and to  keep  "ses‐
1136       sions".
1137
1138       Cookies are sent from server to clients with the header Set-Cookie: and
1139       they're sent from clients to servers with the Cookie: header.
1140
1141       To just send whatever cookie you want to a server,  you  can  use  CUR‐
1142       LOPT_COOKIE(3) to set a cookie string like this:
1143
1144        curl_easy_setopt(easyhandle,        CURLOPT_COOKIE,       "name1=var1;
1145       name2=var2;");
1146
1147       In many cases, that is not enough. You might want to  dynamically  save
1148       whatever  cookies  the remote server passes to you, and make sure those
1149       cookies are then used accordingly on later requests.
1150
1151       One way to do this, is to save all headers you receive in a plain  file
1152       and  when  you  make  a  request, you tell libcurl to read the previous
1153       headers to figure out which cookies to use. Set the header file to read
1154       cookies from with CURLOPT_COOKIEFILE(3).
1155
1156       The  CURLOPT_COOKIEFILE(3) option also automatically enables the cookie
1157       parser in libcurl. Until the cookie parser is enabled, libcurl will not
1158       parse  or  understand  incoming  cookies and they will just be ignored.
1159       However, when the parser is enabled the cookies will be understood  and
1160       the  cookies  will  be  kept  in memory and used properly in subsequent
1161       requests when the same handle is used. Many times this is  enough,  and
1162       you may not have to save the cookies to disk at all. Note that the file
1163       you specify to CURLOPT_COOKIEFILE(3) doesn't have to  exist  to  enable
1164       the  parser, so a common way to just enable the parser and not read any
1165       cookies is to use the name of a file you know doesn't exist.
1166
1167       If you  would  rather  use  existing  cookies  that  you've  previously
1168       received  with  your Netscape or Mozilla browsers, you can make libcurl
1169       use that cookie file as input. The CURLOPT_COOKIEFILE(3)  is  used  for
1170       that  too,  as libcurl will automatically find out what kind of file it
1171       is and act accordingly.
1172
1173       Perhaps the most advanced cookie operation libcurl  offers,  is  saving
1174       the entire internal cookie state back into a Netscape/Mozilla formatted
1175       cookie file. We call that the cookie-jar. When you set a file name with
1176       CURLOPT_COOKIEJAR(3),  that  file name will be created and all received
1177       cookies will be stored in it when curl_easy_cleanup(3) is called.  This
1178       enables  cookies  to  get  passed  on properly between multiple handles
1179       without any information getting lost.
1180
1181

FTP Peculiarities We Need

1183       FTP transfers use a second TCP/IP connection  for  the  data  transfer.
1184       This is usually a fact you can forget and ignore but at times this fact
1185       will come back to haunt you. libcurl offers several different  ways  to
1186       customize how the second connection is being made.
1187
1188       libcurl  can  either  connect  to  the server a second time or tell the
1189       server to connect back to it. The first option is the default and it is
1190       also  what  works best for all the people behind firewalls, NATs or IP-
1191       masquerading setups.  libcurl then tells the server to open  up  a  new
1192       port  and  wait  for  a second connection. This is by default attempted
1193       with EPSV first, and if that doesn't work it tries PASV instead.  (EPSV
1194       is an extension to the original FTP spec and does not exist nor work on
1195       all FTP servers.)
1196
1197       You can prevent libcurl from first trying the EPSV command  by  setting
1198       CURLOPT_FTP_USE_EPSV(3) to zero.
1199
1200       In  some  cases, you will prefer to have the server connect back to you
1201       for the second connection. This might be when  the  server  is  perhaps
1202       behind  a firewall or something and only allows connections on a single
1203       port. libcurl then informs the remote server which IP address and  port
1204       number to connect to.  This is made with the CURLOPT_FTPPORT(3) option.
1205       If you set it to "-",  libcurl  will  use  your  system's  "default  IP
1206       address".  If  you want to use a particular IP, you can set the full IP
1207       address, a host name to resolve to an IP address or even a  local  net‐
1208       work interface name that libcurl will get the IP address from.
1209
1210       When  doing  the  "PORT" approach, libcurl will attempt to use the EPRT
1211       and the LPRT before trying PORT, as they work with more protocols.  You
1212       can disable this behavior by setting CURLOPT_FTP_USE_EPRT(3) to zero.
1213
1214

MIME API revisited for SMTP and IMAP

1216       In addition to support HTTP multi-part form fields, the MIME API can be
1217       used to build structured e-mail messages and  send  them  via  SMTP  or
1218       append such messages to IMAP directories.
1219
1220       A  structured  e-mail  message may contain several parts: some are dis‐
1221       played inline by the MUA, some  are  attachments.  Parts  can  also  be
1222       structured as multi-part, for example to include another e-mail message
1223       or to offer several text formats alternatives. This can  be  nested  to
1224       any level.
1225
1226       To  build such a message, you prepare the nth-level multi-part and then
1227       include it  as  a  source  to  the  parent  multi-part  using  function
1228       curl_mime_subparts(3). Once it has been bound to its parent multi-part,
1229       a nth-level multi-part belongs to it and should not  be  freed  explic‐
1230       itly.
1231
1232       E-mail messages data is not supposed to be non-ascii and line length is
1233       limited: fortunately, some transfer encodings are defined by the  stan‐
1234       dards  to  support the transmission of such incompatible data. Function
1235       curl_mime_encoder(3) tells a part that its source data must be  encoded
1236       before  being sent. It also generates the corresponding header for that
1237       part.  If the part data you want to send is already encoded in  such  a
1238       scheme,  do  not  use  this  function  (this would over-encode it), but
1239       explicitly set the corresponding part header.
1240
1241       Upon sending such a message, libcurl prepends it with the  header  list
1242       set with CURLOPT_HTTPHEADER(3), as 0th-level mime part headers.
1243
1244       Here is an example building an e-mail message with an inline plain/html
1245       text alternative and a file attachment encoded in base64:
1246
1247        curl_mime *message = curl_mime_init(easyhandle);
1248
1249        /* The inline part is an alternative proposing the html and the text
1250           versions of the e-mail. */
1251        curl_mime *alt = curl_mime_init(easyhandle);
1252
1253        /* HTML message. */
1254        curl_mimepart *part = curl_mime_addpart(alt);
1255        curl_mime_data(part, "<html><body><p>This is HTML</p></body></html>",
1256                             CURL_ZERO_TERMINATED);
1257        curl_mime_type(part, "text/html");
1258
1259        /* Text message. */
1260        part = curl_mime_addpart(alt);
1261        curl_mime_data(part, "This is plain text message",
1262                             CURL_ZERO_TERMINATED);
1263
1264        /* Create the inline part. */
1265        part = curl_mime_addpart(message);
1266        curl_mime_subparts(part, alt);
1267        curl_mime_type(part, "multipart/alternative");
1268        struct curl_slist *headers = curl_slist_append(NULL,
1269                          "Content-Disposition: inline");
1270        curl_mime_headers(part, headers, TRUE);
1271
1272        /* Add the attachment. */
1273        part = curl_mime_addpart(message);
1274        curl_mime_filedata(part, "manual.pdf");
1275        curl_mime_encoder(part, "base64");
1276
1277        /* Build the mail headers. */
1278        headers = curl_slist_append(NULL, "From: me@example.com");
1279        headers = curl_slist_append(headers, "To: you@example.com");
1280
1281        /* Set these into the easy handle. */
1282        curl_easy_setopt(easyhandle, CURLOPT_HTTPHEADER, headers);
1283        curl_easy_setopt(easyhandle, CURLOPT_MIMEPOST, mime);
1284
1285       It should be noted that  appending  a  message  to  an  IMAP  directory
1286       requires the message size to be known prior upload. It is therefore not
1287       possible to include parts with unknown data size in this context.
1288
1289

Headers Equal Fun

1291       Some protocols provide "headers", meta-data separated from  the  normal
1292       data.  These  headers  are  by  default not included in the normal data
1293       stream, but you can make them appear in the data stream by setting CUR‐
1294       LOPT_HEADER(3) to 1.
1295
1296       What  might  be  even more useful, is libcurl's ability to separate the
1297       headers from the data and thus make the callbacks differ. You  can  for
1298       example  set a different pointer to pass to the ordinary write callback
1299       by setting CURLOPT_HEADERDATA(3).
1300
1301       Or, you can set an entirely separate function to receive  the  headers,
1302       by using CURLOPT_HEADERFUNCTION(3).
1303
1304       The headers are passed to the callback function one by one, and you can
1305       depend on that fact. It makes it easier for you to  add  custom  header
1306       parsers etc.
1307
1308       "Headers"  for  FTP  transfers equal all the FTP server responses. They
1309       aren't actually true headers, but in this case we pretend they are! ;-)
1310
1311

Post Transfer Information

1313       See curl_easy_getinfo(3).
1314

The multi Interface

1316       The easy interface as described in detail in this document  is  a  syn‐
1317       chronous interface that transfers one file at a time and doesn't return
1318       until it is done.
1319
1320       The multi interface, on the other hand, allows your program to transfer
1321       multiple files in both directions at the same time, without forcing you
1322       to use multiple threads.  The name might make it seem  that  the  multi
1323       interface  is  for multi-threaded programs, but the truth is almost the
1324       reverse.  The multi interface allows a single-threaded  application  to
1325       perform  the same kinds of multiple, simultaneous transfers that multi-
1326       threaded programs can perform.  It  allows  many  of  the  benefits  of
1327       multi-threaded  transfers  without  the complexity of managing and syn‐
1328       chronizing many threads.
1329
1330       To complicate matters somewhat more, there are even two versions of the
1331       multi  interface. The event based one, also called multi_socket and the
1332       "normal one" designed for using with select(). See the  libcurl-multi.3
1333       man page for details on the multi_socket event based API, this descrip‐
1334       tion here is for the select() oriented one.
1335
1336       To use this interface, you are better off if you first  understand  the
1337       basics  of how to use the easy interface. The multi interface is simply
1338       a way to make multiple transfers at the same time by adding up multiple
1339       easy handles into a "multi stack".
1340
1341       You create the easy handles you want, one for each concurrent transfer,
1342       and you set all the options just like you learned above, and  then  you
1343       create  a  multi  handle with curl_multi_init(3) and add all those easy
1344       handles to that multi handle with curl_multi_add_handle(3).
1345
1346       When you've added the handles you have for the moment  (you  can  still
1347       add  new  ones  at  any  time),  you  start  the  transfers  by calling
1348       curl_multi_perform(3).
1349
1350       curl_multi_perform(3) is asynchronous. It will only perform what can be
1351       done  now  and then return back control to your program. It is designed
1352       to never block. You need to keep calling the function until all  trans‐
1353       fers are completed.
1354
1355       The  best usage of this interface is when you do a select() on all pos‐
1356       sible file descriptors or sockets to know when to call  libcurl  again.
1357       This  also makes it easy for you to wait and respond to actions on your
1358       own application's sockets/handles. You figure out what to select()  for
1359       by  using  curl_multi_fdset(3), that fills in a set of fd_set variables
1360       for you with the particular  file  descriptors  libcurl  uses  for  the
1361       moment.
1362
1363       When  you then call select(), it'll return when one of the file handles
1364       signal action and you then call curl_multi_perform(3) to allow  libcurl
1365       to  do  what  it  wants to do. Take note that libcurl does also feature
1366       some time-out code so we advise you to never use very long timeouts  on
1367       select()  before you call curl_multi_perform(3) again. curl_multi_time‐
1368       out(3) is provided to help you get a suitable timeout period.
1369
1370       Another precaution you  should  use:  always  call  curl_multi_fdset(3)
1371       immediately  before  the  select()  call  since the current set of file
1372       descriptors may change in any curl function invoke.
1373
1374       If you want to stop the transfer of one of  the  easy  handles  in  the
1375       stack,  you  can  use  curl_multi_remove_handle(3) to remove individual
1376       easy    handles.    Remember    that    easy    handles    should    be
1377       curl_easy_cleanup(3)ed.
1378
1379       When  a  transfer  within  the multi stack has finished, the counter of
1380       running  transfers  (as  filled  in  by   curl_multi_perform(3))   will
1381       decrease. When the number reaches zero, all transfers are done.
1382
1383       curl_multi_info_read(3)  can be used to get information about completed
1384       transfers. It then returns the CURLcode  for  each  easy  transfer,  to
1385       allow you to figure out success on each individual transfer.
1386
1387

SSL, Certificates and Other Tricks

1389        [ seeding, passwords, keys, certificates, ENGINE, ca certs ]
1390
1391

Sharing Data Between Easy Handles

1393       You can share some data between easy handles when the easy interface is
1394       used, and some data is share  automatically  when  you  use  the  multi
1395       interface.
1396
1397       When  you  add  easy handles to a multi handle, these easy handles will
1398       automatically share a lot of the data that otherwise would be kept on a
1399       per-easy handle basis when the easy interface is used.
1400
1401       The  DNS  cache is shared between handles within a multi handle, making
1402       subsequent name resolving faster, and the connection pool that is  kept
1403       to  better  allow  persistent connections and connection re-use is also
1404       shared. If you're using the easy interface, you can still  share  these
1405       between  specific  easy  handles  by  using  the  share  interface, see
1406       libcurl-share(3).
1407
1408       Some things are never shared automatically, not within  multi  handles,
1409       like  for  example  cookies  so  the only way to share that is with the
1410       share interface.
1411

Footnotes

1413       [1]    libcurl 7.10.3 and later have the  ability  to  switch  over  to
1414              chunked  Transfer-Encoding  in cases where HTTP uploads are done
1415              with data of an unknown size.
1416
1417       [2]    This happens on Windows machines when libcurl is built and  used
1418              as  a DLL. However, you can still do this on Windows if you link
1419              with a static library.
1420
1421       [3]    The curl-config tool is generated at  build-time  (on  Unix-like
1422              systems) and should be installed with the 'make install' or sim‐
1423              ilar instruction that installs the library,  header  files,  man
1424              pages etc.
1425
1426       [4]    This  behavior  was  different  in versions before 7.17.0, where
1427              strings  had   to   remain   valid   past   the   end   of   the
1428              curl_easy_setopt(3) call.
1429

SEE ALSO

1431       libcurl-errors(3), libcurl-multi(3), libcurl-easy(3)
1432
1433
1434
1435libcurl 7.61.1                  April 17, 2018             libcurl-tutorial(3)
Impressum