1explain_lca2010(1)          General Commands Manual         explain_lca2010(1)
2
3
4

NAME

6       explain_lca2010  -  No  medium  found: when it's time to stop trying to
7       read strerror(3)'s mind.
8

MOTIVATION

10       The idea for libexplain occurred to me back in the early 1980s.   When‐
11       ever a system call returns an error, the kernel knows exactly what went
12       wrong... and compresses this into less that  8  bits  of  errno.   User
13       space  has access to the same data as the kernel, it should be possible
14       for user space to figure out exactly what happened to provoke the error
15       return, and use this to write good error messages.
16
17       Could it be that simple?
18
19   Error messages as finesse
20       Good  error  messages  are  often  those  “one  percent” tasks that get
21       dropped when schedule pressure squeezes your project.  However, a  good
22       error message can make a huge, disproportionate improvement to the user
23       experience, when the user wanders into  scarey  unknown  territory  not
24       usually encountered.  This is no easy task.
25
26       As  a  larval  programmer, the author didn't see the problem with (com‐
27       pletely accurate) error messages like this one:
28              floating exception (core dumped)
29       until the alternative non‐programmer interpretation  was  pointed  out.
30       But  that  isn't  the  only  thing wrong with Unix error messages.  How
31       often do you see error messages like:
32              $ ./stupid
33              can't open file
34              $
35       There are two options for a developer at this point:
36
37       1.
38         you can run a debugger, such as gdb(1), or
39
40       2.
41         you can use strace(1) or truss(1) to look inside.
42
43       · Remember that your users may not even have access to these tools, let
44         alone  the  ability  to  use them.  (It's a very long time since Unix
45         beginner meant “has only written one device driver”.)
46
47       In this example, however, using strace(1) reveals
48              $ strace -e trace=open ./stupid
49              open("some/file", O_RDONLY) = -1 ENOENT (No such file or directory)
50              can't open file
51              $
52       This is considerably more information than the error message  provides.
53       Typically, the stupid source code looks like this
54              int fd = open("some/thing", O_RDONLY);
55              if (fd < 0)
56              {
57                  fprintf(stderr, "can't open file\n");
58                  exit(1);
59              }
60       The  user  isn't told which file, and also fails to tell the user which
61       error.  Was the file even there?  Was there a permissions problem?   It
62       does  tell  you  it was trying to open a file, but that was probably by
63       accident.
64
65       Grab your clue stick and go beat the larval programmer with  it.   Tell
66       him  about perror(3).  The next time you use the program you see a dif‐
67       ferent error message:
68              $ ./stupid
69              open: No such file or directory
70              $
71       Progress, but not what we expected.  How can the user fix  the  problem
72       if the error message doesn't tell him what the problem was?  Looking at
73       the source, we see
74              int fd = open("some/thing", O_RDONLY);
75              if (fd < 0)
76              {
77                  perror("open");
78                  exit(1);
79              }
80       Time for another run with the clue stick.  This time, the error message
81       takes one step forward and one step back:
82              $ ./stupid
83              some/thing: No such file or directory
84              $
85       Now  we know the file it was trying to open, but are no longer informed
86       that it was open(2) that failed.  In this case it is probably not  sig‐
87       nificant,  but  it can be significant for other system calls.  It could
88       have been creat(2) instead, an operation implying that  different  per‐
89       missions are necessary.
90              const char *filename = "some/thing";
91              int fd = open(filename, O_RDONLY);
92              if (fd < 0)
93              {
94                  perror(filename);
95                  exit(1);
96              }
97       The  above example code is unfortunately typical of non‐larval program‐
98       mers as well.  Time to tell our padawan learner about  the  strerror(3)
99       system call.
100              $ ./stupid
101              open some/thing: No such file or directory
102              $
103       This  maximizes the information that can be presented to the user.  The
104       code looks like this:
105              const char *filename = "some/thing";
106              int fd = open(filename, O_RDONLY);
107              if (fd < 0)
108              {
109                  fprintf(stderr, "open %s: %s\n", filename, strerror(errno));
110                  exit(1);
111              }
112       Now we have the system call, the filename, and the error string.   This
113       contains all the information that strace(1) printed.  That's as good as
114       it gets.
115
116       Or is it?
117
118   Limitations of perror and strerror
119       The problem the author saw, back in the 1980s, was that the error  mes‐
120       sage  is  incomplete.   Does  “no  such file or directory” refer to the
121some” directory, or to the “thing” file in the “some” directory?
122
123       A quick look at the man page for strerror(3) is telling:
124              strerror - return string describing error number
125       Note well: it is describing the error number, not the error.
126
127       On the other hand, the kernel knows what the error was.   There  was  a
128       specific  point  in  the  kernel  code, caused by a specific condition,
129       where the kernel code branched and said “no”.  Could a user‐space  pro‐
130       gram  figure  out  the specific condition and write a better error mes‐
131       sage?
132
133       However, the problem goes deeper.  What if the  problem  occurs  during
134       the  read(2)  system  call, rather than the open(2) call?  It is simple
135       for the error message associated with open(2) to include the file name,
136       it's  right  there.  But to be able to include a file name in the error
137       associated with the read(2) system call, you have to pass the file name
138       all the way down the call stack, as well as the file descriptor.
139
140       And  here  is  the  bit that grates: the kernel already knows what file
141       name the file descriptor is associated with.  Why should  a  programmer
142       have  to  pass  redundant  data all the way down the call stack just to
143       improve an error message that may never be issued?   In  reality,  many
144       programmers  don't  bother,  and  the  resulting error messages are the
145       worse for it.
146
147       But that was the 1980s, on a  PDP11,  with  limited  resources  and  no
148       shared  libraries.  Back then, no flavor of Unix included /proc even in
149       rudimentary form, and the lsof(1) program was over a decade  away.   So
150       the idea was shelved as impractical.
151
152   Level Infinity Support
153       Imagine that you are level infinity support.  Your job description says
154       that you never ever have to talk to users.  Why, then, is there still a
155       constant stream of people wanting you, the local Unix guru, to decipher
156       yet another error message?
157
158       Strangely, 25 years later, despite a simple permissions system,  imple‐
159       mented  with  complete  consistency, most Unix users still have no idea
160       how to decode “No such file or directory”, or any of the other  cryptic
161       error messages they see every day.  Or, at least, cryptic to them.
162
163       Wouldn't  it be nice if first level tech support didn't need error mes‐
164       sages deciphered?  Wouldn't it be nice  to  have  error  messages  that
165       users could understand without calling tech support?
166
167       These  days /proc on Linux is more than able to provide the information
168       necessary to decode the vast majority of error messages, and point  the
169       user  to  the proximate cause of their problem.  On systems with a lim‐
170       ited /proc implementation, the lsof(1) command can fill in many of  the
171       gaps.
172
173       In  2008, the stream of translation requests happened to the author way
174       too often.  It was time to re‐examine that 25 year old idea, and libex‐
175       plain is the result.
176

USING THE LIBRARY

178       The  interface  to  the library tries to be consistent, where possible.
179       Let's start with an example using strerror(3):
180              if (rename(old_path, new_path) < 0)
181              {
182                  fprintf(stderr, "rename %s %s: %s\n", old_path, new_path,
183                      strerror(errno));
184                  exit(1);
185              }
186       The idea behind libexplain is to provide a strerror(3)  equivalent  for
187       each system call, tailored specifically to that system call, so that it
188       can provide a more detailed  error  message,  containing  much  of  the
189       information  you  see under the “ERRORS” heading of section 2 and 3 man
190       pages, supplemented with information about  actual  conditions,  actual
191       argument values, and system limits.
192
193   The Simple Case
194       The strerror(3) replacement:
195              if (rename(old_path, new_path) < 0)
196              {
197                  fprintf(stderr, "%s\n", explain_rename(old_path, new_path));
198                  exit(1);
199              }
200
201   The Errno Case
202       It  is  also  possible  to pass an explicit errno(3) value, if you must
203       first do some processing that would disturb errno, such as error recov‐
204       ery:
205              if (rename(old_path, new_path < 0))
206              {
207                  int old_errno = errno;
208                  ...code that disturbs errno...
209                  fprintf(stderr, "%s\n", explain_errno_rename(old_errno,
210                      old_path, new_path));
211                  exit(1);
212              }
213
214   The Multi‐thread Cases
215       Some applications are multi‐threaded, and thus are unable to share lib‐
216       explain's internal buffer.  You can supply your own buffer using
217              if (unlink(pathname))
218              {
219                  char message[3000];
220                  explain_message_unlink(message, sizeof(message), pathname);
221                  error_dialog(message);
222                  return -1;
223              }
224       And for completeness, both errno(3) and thread‐safe:
225              ssize_t nbytes = read(fd, data, sizeof(data));
226              if (nbytes < 0)
227              {
228                  char message[3000];
229                  int old_errno = errno;
230                  ...error recovery...
231                  explain_message_errno_read(message, sizeof(message),
232                      old_errno, fd, data, sizeof(data));
233                  error_dialog(message);
234                  return -1;
235              }
236
237       These are replacements for strerror_r(3), on systems that have it.
238
239   Interface Sugar
240       A set of functions added as convenience functions, to  woo  programmers
241       to  use  the  libexplain library, turn out to be the author's most com‐
242       monly used libexplain functions in command line programs:
243              int fd = explain_creat_or_die(filename, 0666);
244       This function attempts to create a new file.  If it can't, it prints an
245       error  message  and  exits with EXIT_FAILURE.  If there is no error, it
246       returns the new file descriptor.
247
248       A related function:
249              int fd = explain_creat_on_error(filename, 0666);
250       will print the error message on failure, but also returns the  original
251       error result, and errno(3) is unmolested, as well.
252
253   All the other system calls
254       In general, every system call has its own include file
255              #include <libexplain/name.h>
256       that defines function prototypes for six functions:
257
258       · explain_name,
259
260       · explain_errno_name,
261
262       · explain_message_name,
263
264       · explain_message_errno_name,
265
266       · explain_name_or_die and
267
268       · explain_name_on_error.
269
270       Every function prototype has Doxygen documentation, and this documenta‐
271       tion is not stripped when the include files are installed.
272
273       The wait(2) system call (and friends) have  some  extra  variants  that
274       also  interpret  failure  to be an exit status that isn't EXIT_SUCCESS.
275       This applies to system(3) and pclose(3) as well.
276
277       Coverage includes 221 system calls and 547 ioctl requests.   There  are
278       many  more  system  calls  yet  to  implement.  System calls that never
279       return, such as exit(2), are not present in the library, and will never
280       be.  The exec family of system calls are supported, because they return
281       when there is an error.
282
283   Cat
284       This is what a hypothetical “cat” program could look  like,  with  full
285       error reporting, using libexplain.
286              #include <libexplain/libexplain.h>
287              #include <stdlib.h>
288              #include <unistd.h>
289       There  is one include for libexplain, plus the usual suspects.  (If you
290       wish to reduce the preprocessor load, you can use the specific  <libex‐
291       plain/name.h> includes.)
292              static void
293              process(FILE *fp)
294              {
295                  for (;;)
296                  {
297                      char buffer[4096];
298                      size_t n = explain_fread_or_die(buffer, 1, sizeof(buffer), fp);
299                      if (!n)
300                          break;
301                      explain_fwrite_or_die(buffer, 1, n, stdout);
302                  }
303              }
304       The  process  function  copies  a  file  stream to the standard output.
305       Should an error occur for either reading or  writing,  it  is  reported
306       (and  the pathname will be included in the error) and the command exits
307       with EXIT_FAILURE.  We don't even worry about tracking  the  pathnames,
308       or passing them down the call stack.
309              int
310              main(int argc, char **argv)
311              {
312                  for (;;)
313                  {
314                      int c = getopt(argc, argv, "o:");
315                      if (c == EOF)
316                          break;
317                      switch (c)
318                      {
319                      case 'o':
320                          explain_freopen_or_die(optarg, "w", stdout);
321                          break;
322       The  fun part of this code is that libexplain can report errors includ‐
323       ing the pathname even if you don't explicitly re‐open stdout as is done
324       here.  We don't even worry about tracking the file name.
325                      default:
326                          fprintf(stderr, "Usage: %ss [ -o <filename> ] <filename>...\n",
327                              argv[0]);
328                          return EXIT_FAILURE;
329                      }
330                  }
331                  if (optind == argc)
332                      process(stdin);
333                  else
334                  {
335                      while (optind < argc)
336                      {
337                          FILE *fp = explain_fopen_or_die(argv[optind]++, "r");
338                          process(fp);
339                          explain_fclose_or_die(fp);
340                      }
341                  }
342       The  standard  output  will  be  closed implicitly, but too late for an
343       error report to be issued, so  we  do  that  here,  just  in  case  the
344       buffered  I/O hasn't written anything yet, and there is an ENOSPC error
345       or something.
346                  explain_fflush_or_die(stdout);
347                  return EXIT_SUCCESS;
348              }
349       That's all.  Full error reporting, clear code.
350
351   Rusty's Scale of Interface Goodness
352       For those of you not familiar with it, Rusty Russel's “How  Do  I  Make
353       This Hard to Misuse?”  page is a must‐read for API designers.
354       http://ozlabs.org/~rusty/index.cgi/tech/2008‐03‐30.html
355
356       10. It's impossible to get wrong.
357
358       Goals  need  to be set high, ambitiously high, lest you accomplish them
359       and think you are finished when you are not.
360
361       The libexplain library detects bogus pointers and many other bogus sys‐
362       tem call parameters, and generally tries to avoid segfaults in even the
363       most trying circumstances.
364
365       The libexplain library is designed to be thread safe.  More  real‐world
366       use will likely reveal places this can be improved.
367
368       The  biggest  problem  is  with  the  actual function names themselves.
369       Because C does not have name‐spaces, the libexplain library always uses
370       an  explain_  name  prefix.   This is the traditional way of creating a
371       pseudo‐name‐space in order to  avoid  symbol  conflicts.   However,  it
372       results in some unnatural‐sounding names.
373
374       9. The compiler or linker won't let you get it wrong.
375
376       A  common  mistake is to use explain_open where explain_open_or_die was
377       intended.  Fortunately, the compiler will often issue a type  error  at
378       this point (e.g. can't assign const char * rvalue to an int lvalue).
379
380       8. The compiler will warn if you get it wrong.
381
382       If explain_rename is used when explain_rename_or_die was intended, this
383       can cause other problems.  GCC has a useful warn_unused_result function
384       attribute,   and   the  libexplain  library  attaches  it  to  all  the
385       explain_name function calls to produce a warning  when  you  make  this
386       mistake.   Combine  this  with  gcc  -Werror to promote this to level 9
387       goodness.
388
389       7. The obvious use is (probably) the correct one.
390
391       The function names have been chosen to convey their meaning,  but  this
392       is    not    always    successful.    While   explain_name_or_die   and
393       explain_name_on_error are fairly descriptive, the less‐used thread safe
394       variants  are  harder to decode.  The function prototypes help the com‐
395       piler towards understanding, and the Doxygen  comments  in  the  header
396       files help the user towards understanding.
397
398       6. The name tells you how to use it.
399
400       It  is  particularly  important to read explain_name_or_die as “explain
401       (name or die)”.  Using a consistent explain_ name‐space prefix has some
402       unfortunate side‐effects in the obviousness department, as well.
403
404       The  order  of  words in the names also indicate the order of the argu‐
405       ments.  The argument lists always end with the same arguments as passed
406       to  the  system call; all of them.  If _errno_ appears in the name, its
407       argument always precedes  the  system  call  arguments.   If  _message_
408       appears in the name, its two arguments always come first.
409
410       5. Do it right or it will break at runtime.
411
412       The libexplain library detects bogus pointers and many other bogus sys‐
413       tem call parameters, and generally tries to avoid segfaults in even the
414       most  trying circumstances.  It should never break at runtime, but more
415       real‐world use will no doubt improve this.
416
417       Some error messages are aimed at developers and maintainers rather than
418       end  users, as this can assist with bug resolution.  Not so much “break
419       at runtime” as “be informative  at  runtime”  (after  the  system  call
420       barfs).
421
422       4. Follow common convention and you'll get it right.
423
424       Because C does not have name‐spaces, the libexplain library always uses
425       an explain_ name prefix.  This is the traditional  way  of  creating  a
426       pseudo‐name‐space in order to avoid symbol conflicts.
427
428       The  trailing arguments of all the libexplain call are identical to the
429       system call they are describing.  This is intended to provide a consis‐
430       tent convention in common with the system calls themselves.
431
432       3. Read the documentation and you'll get it right.
433
434       The  libexplain library aims to have complete Doxygen documentation for
435       each and every public API call (and internally as well).
436

MESSAGE CONTENT

438       Working on libexplain is a bit like looking at the  underside  of  your
439       car  when  it  is up on the hoist at the mechanic's.  There's some ugly
440       stuff under there, plus mud and crud, and users rarely see it.  A  good
441       error  message  needs  to  be informative, even for a user who has been
442       fortunate enough not to have to look at the under‐side very often,  and
443       also  informative  for the mechanic listening to the user's description
444       over the phone.  This is no easy task.
445
446       Revisiting our first example, the code would like this if it uses  lib‐
447       explain:
448              int fd = explain_open_or_die("some/thing", O_RDONLY, 0);
449       will fail with an error message like this
450              open(pathname  =  "some/file", flags = O_RDONLY) failed, No such
451              file or directory (2, ENOENT) because there is no "some"  direc‐
452              tory in the current directory
453       This breaks down into three pieces
454              system‐call failed, system‐error because
455              explanation
456
457   Before Because
458       It  is  possible  to  see  the  part of the message before “because” as
459       overly technical to non‐technical users, mostly as a  result  of  accu‐
460       rately  printing  the  system call itself at the beginning of the error
461       message.  And it looks like strace(1) output, for bonus geek points.
462              open(pathname = "some/file", flags = O_RDONLY) failed,  No  such
463              file or directory (2, ENOENT)
464       This part of the error message is essential to the developer when he is
465       writing the code, and equally important to the maintainer  who  has  to
466       read  bug  reports  and  fix  bugs  in  the code.  It says exactly what
467       failed.
468
469       If this text is not presented to the user then the  user  cannot  copy‐
470       and‐paste  it  into a bug report, and if it isn't in the bug report the
471       maintainer can't know what actually went wrong.
472
473       Frequently tech staff will use strace(1) or truss(1) to get this  exact
474       information, but this avenue is not open when reading bug reports.  The
475       bug reporter's system is far far away, and, by now, in a far  different
476       state.   Thus,  this  information  needs to be in the bug report, which
477       means it must be in the error message.
478
479       The system call representation also gives context to the  rest  of  the
480       message.   If  need  arises,  the offending system call argument may be
481       referred to by name in the explanation after “because”.   In  addition,
482       all  strings  are  fully quoted and escaped C strings, so embedded new‐
483       lines and non‐printing characters will not cause the user's terminal to
484       go haywire.
485
486       The  system‐error is what comes out of strerror(2), plus the error sym‐
487       bol.  Impatient and expert sysadmins could stop reading at this  point,
488       but  the author's experience to date is that reading further is reward‐
489       ing.  (If it isn't rewarding, it's probably an area of libexplain  that
490       can be improved.  Code contributions are welcome, of course.)
491
492   After Because
493       This  is the portion of the error message aimed at non‐technical users.
494       It looks beyond the simple system call arguments, and looks  for  some‐
495       thing more specific.
496              there is no "some" directory in the current directory
497       This  portion  attempts  to  explain the proximal cause of the error in
498       plain language, and it is here that internationalization is essential.
499
500       In general, the policy is to include as much information  as  possible,
501       so  that  the user doesn't need to go looking for it (and doesn't leave
502       it out of the bug report).
503
504   Internationalization
505       Most of the error messages in the libexplain library have been interna‐
506       tionalized.   There  are  no  localizations  as yet, so if you want the
507       explanations in your native language, please contribute.
508
509       The “most of” qualifier, above, relates to the fact that the  proof‐of‐
510       concept  implementation  did  not include internationalization support.
511       The code base is being revised progressively, usually as  a  result  of
512       refactoring  messages  so that each error message string appears in the
513       code exactly once.
514
515       Provision has been made for languages that need to  assemble  the  por‐
516       tions of
517              system‐call failed, system‐error because explanation
518       in different orders for correct grammar in localized error messages.
519
520   Postmortem
521       There are times when a program has yet to use libexplain, and you can't
522       use strace(1) either.  There is an  explain(1)  command  included  with
523       libexplain that can be used to decipher error messages, if the state of
524       the underlying system hasn't changed too much.
525              $ explain rename foo /tmp/bar/baz -e ENOENT
526              rename(oldpath = "foo", newpath  =  "/tmp/bar/baz")  failed,  No
527              such  file  or  directory  (2, ENOENT) because there is no "bar"
528              directory in the newpath "/tmp" directory
529              $
530       Note how the path ambiguity is resolved by using the system call  argu‐
531       ment  name.   Of course, you have to know the error and the system call
532       for explain(1) to be useful.  As an aside, this is one of the ways used
533       by  the  libexplain  automatic  test suite to verify that libexplain is
534       working.
535
536   Philosophy
537       “Tell me everything, including stuff I didn't know to look for.”
538
539       The library is implemented in such a way that when  statically  linked,
540       only  the  code  you  actually use will be linked.  This is achieved by
541       having one function per source file, whenever feasible.
542
543       When it is possible to supply more information, libexplain will do  so.
544       The  less  the user has to track down for themselves, the better.  This
545       means that UIDs are accompanied by the user name, GIDs are  accompanied
546       by  the  group  name,  PIDs  are  accompanied by the process name, file
547       descriptors and streams are accompanied by the pathname, etc.
548
549       When resolving paths, if a path component does  not  exist,  libexplain
550       will look for similar names, in order to suggest alternatives for typo‐
551       graphical errors.
552
553       The libexplain library tries to use as little  heap  as  possible,  and
554       usually none.  This is to avoid perturbing the process state, as far as
555       possible, although sometimes it is unavoidable.
556
557       The libexplain library attempts to be thread safe, by  avoiding  global
558       variables,  keeping state on the stack as much as possible.  There is a
559       single common message buffer, and the functions that use it  are  docu‐
560       mented as not being thread safe.
561
562       The  libexplain  library  does not disturb a process's signal handlers.
563       This makes determining whether a pointer would  segfault  a  challenge,
564       but not impossible.
565
566       When  information  is  available via a system call as well as available
567       through a /proc entry, the system call is preferred.  This is to  avoid
568       disturbing  the  process's  state.   There  are also times when no file
569       descriptors are available.
570
571       The libexplain library is compiled with large file support.   There  is
572       no large/small schizophrenia.  Where this affects the argument types in
573       the API, and error will be issued if the necessary large  file  defines
574       are absent.
575
576       FIXME:  Work is needed to make sure that file system quotas are handled
577       in the code.  This applies to some getrlimit(2) boundaries, as well.
578
579       There are cases when relatives paths are uninformative.   For  example:
580       system  daemons,  servers  and  background  processes.  In these cases,
581       absolute paths are used in the error explanations.
582

PATH RESOLUTION

584       Short version: see path_resolution(7).
585
586       Long version: Most users have never heard  of  path_resolution(7),  and
587       many advanced users have never read it.  Here is an annotated version:
588
589   Step 1: Start of the resolution process
590       If  the  pathname  starts  with the slash (“/”) character, the starting
591       lookup directory is the root directory of the calling process.
592
593       If the pathname does not  start  with  the  slash(“/”)  character,  the
594       starting  lookup  directory  of  the  resolution process is the current
595       working directory of the process.
596
597   Step 2: Walk along the path
598       Set the current lookup directory  to  the  starting  lookup  directory.
599       Now, for each non‐final component of the pathname, where a component is
600       a substring delimited by slash  (“/”)  characters,  this  component  is
601       looked up in the current lookup directory.
602
603       If  the  process  does not have search permission on the current lookup
604       directory, an EACCES error is returned ("Permission denied").
605              open(pathname  =  "/home/archives/.ssh/private_key",   flags   =
606              O_RDONLY)  failed,  Permission  denied  (13, EACCES) because the
607              process  does  not  have  search  permission  to  the   pathname
608              "/home/archives/.ssh"  directory, the process effective GID 1000
609              "pmiller" does not match the directory owner 1001 "archives"  so
610              the  owner  permission mode "rwx" is ignored, the others permis‐
611              sion mode is "---", and the process is not privileged (does  not
612              have the DAC_READ_SEARCH capability)
613
614       If  the  component  is not found, an ENOENT error is returned ("No such
615       file or directory").
616              unlink(pathname =  "/home/microsoft/rubbish")  failed,  No  such
617              file  or  directory  (2, ENOENT) because there is no "microsoft"
618              directory in the pathname "/home" directory
619
620       There is also some support for users when they mis‐type pathnames, mak‐
621       ing suggestions when ENOENT is returned:
622              open(pathname   =  "/user/include/fcntl.h",  flags  =  O_RDONLY)
623              failed, No such file or directory (2, ENOENT) because  there  is
624              no  "user" directory in the pathname "/" directory, did you mean
625              the "usr" directory instead?
626
627       If  the  component  is found, but is neither a directory nor a symbolic
628       link, an ENOTDIR error is returned ("Not a directory").
629              open(pathname  =  "/home/pmiller/.netrc/lca",  flags = O_RDONLY)
630              failed, Not a directory (20, ENOTDIR) because the ".netrc" regu‐
631              lar file in the pathname "/home/pmiller" directory is being used
632              as a directory when it is not
633
634       If the component is found and is a directory, we set the current lookup
635       directory to that directory, and go to the next component.
636
637       If  the  component  is found and is a symbolic link (symlink), we first
638       resolve this symbolic link (with the current lookup directory as start‐
639       ing  lookup  directory).   Upon  error, that error is returned.  If the
640       result is not a directory, an ENOTDIR error is returned.
641              unlink(pathname = "/tmp/dangling/rubbish") failed, No such  file
642              or directory (2, ENOENT) because the "dangling" symbolic link in
643              the pathname "/tmp" directory refers to "nowhere" that does  not
644              exist
645       If the resolution of the symlink is successful and returns a directory,
646       we set the current lookup directory to that directory, and  go  to  the
647       next  component.  Note that the resolution process here involves recur‐
648       sion.  In order to protect the kernel against stack overflow, and  also
649       to  protect  against denial of service, there are limits on the maximum
650       recursion depth, and on the maximum number of symbolic links  followed.
651       An ELOOP error is returned when the maximum is exceeded ("Too many lev‐
652       els of symbolic links").
653              open(pathname = "/tmp/dangling", flags = O_RDONLY)  failed,  Too
654              many  levels  of  symbolic  links (40, ELOOP) because a symbolic
655              link loop was encountered in pathname,  starting  at  "/tmp/dan‐
656              gling"
657       It  is  also  possible to get an ELOOP or EMLINK error if there are too
658       many symlinks, but no loop was detected.
659              open(pathname = "/tmp/rabbit‐hole", flags  =  O_RDONLY)  failed,
660              Too  many  levels of symbolic links (40, ELOOP) because too many
661              symbolic links were encountered in pathname (8)
662       Notice how the actual limit is also printed.
663
664   Step 3: Find the final entry
665       The lookup of the final component of the pathname goes just  like  that
666       of  all  other  components, as described in the previous step, with two
667       differences:
668
669       (i) The final component need not be a directory (at least as far as the
670           path  resolution  process is concerned.  It may have to be a direc‐
671           tory, or a non‐directory, because of the requirements of  the  spe‐
672           cific system call).
673
674       (ii)
675           It is not necessarily an error if the final component is not found;
676           maybe we are just creating it.  The details on the treatment of the
677           final  entry are described in the manual pages of the specific sys‐
678           tem calls.
679
680       (iii)
681           It is also possible to have a problem with the last component if it
682           is  a  symbolic  link  and it should not be followed.  For example,
683           using the open(2) O_NOFOLLOW flag:
684           open(pathname = "a‐symlink", flags = O_RDONLY | O_NOFOLLOW) failed,
685           Too  many  levels  of symbolic links (ELOOP) because O_NOFOLLOW was
686           specified but pathname refers to a symbolic link
687
688       (iv)
689           It is common for users to make mistakes when typing pathnames.  The
690           libexplain  library  attempts  to  make  suggestions when ENOENT is
691           returned, for example:
692           open(pathname  =  "/usr/include/filecontrl.h",  flags  =  O_RDONLY)
693           failed,  No  such file or directory (2, ENOENT) because there is no
694           "filecontrl.h" regular file in the pathname  "/usr/include"  direc‐
695           tory, did you mean the "fcntl.h" regular file instead?
696
697       (v) It  is  also  possible  that  the final component is required to be
698           something other than a regular file:
699           readlink(pathname = "just‐a‐file", data = 0x7F930A50,  data_size  =
700           4097)  failed,  Invalid argument (22, EINVAL) because pathname is a
701           regular file, not a symbolic link
702
703       (vi)
704           FIXME: handling of the "t" bit.
705
706   Limits
707       There are a number of limits with regards to pathnames and filenames.
708
709       Pathname length limit
710               There is a maximum length for pathnames.  If the  pathname  (or
711               some  intermediate  pathname  obtained while resolving symbolic
712               links) is too long, an ENAMETOOLONG error  is  returned  ("File
713               name  too  long").   Notice how the system limit is included in
714               the error message.
715               open(pathname = "very...long", flags = O_RDONLY)  failed,  File
716               name  too  long (36, ENAMETOOLONG) because pathname exceeds the
717               system maximum path length (4096)
718
719       Filename length limit
720               Some Unix variants have a limit on the number of bytes in  each
721               path component.  Some of them deal with this silently, and some
722               give ENAMETOOLONG;  the  libexplain  library  uses  pathconf(3)
723               _PC_NO_TRUNC  to tell which.  If this error happens, the libex‐
724               plain library will state the limit in the  error  message,  the
725               limit  is  obtained  from pathconf(3) _PC_NAME_MAX.  Notice how
726               the system limit is included in the error message.
727               open(pathname  =  "system7/only-had-14-characters",   flags   =
728               O_RDONLY) failed, File name too long (36, ENAMETOOLONG) because
729               "only-had-14-characters" component is longer  than  the  system
730               limit (14)
731
732       Empty pathname
733               In  the  original Unix, the empty pathname referred to the cur‐
734               rent directory.  Nowadays POSIX decrees that an empty  pathname
735               must not be resolved successfully.
736               open(pathname  =  "", flags = O_RDONLY) failed, No such file or
737               directory (2, ENOENT) because POSIX decrees that an empty path‐
738               name must not be resolved successfully
739
740   Permissions
741       The  permission  bits  of a file consist of three groups of three bits.
742       The first group of three is used when the  effective  user  ID  of  the
743       calling  process  equals the owner ID of the file.  The second group of
744       three is used when the group ID of the file either equals the effective
745       group  ID  of the calling process, or is one of the supplementary group
746       IDs of the calling process.  When neither holds,  the  third  group  is
747       used.
748              open(pathname = "/etc/passwd", flags = O_WRONLY) failed, Permis‐
749              sion denied (13, EACCES) because the process does not have write
750              permission  to  the "passwd" regular file in the pathname "/etc"
751              directory, the process effective UID  1000  "pmiller"  does  not
752              match  the  regular  file owner 0 "root" so the owner permission
753              mode "rw-" is ignored, the others permission mode is "r--",  and
754              the  process  is  not privileged (does not have the DAC_OVERRIDE
755              capability)
756       Some considerable space is given to this explanation, as most users  do
757       not know that this is how the permissions system works.  In particular:
758       the owner, group and other permissions  are  exclusive,  they  are  not
759       “OR”ed together.
760

STRANGE AND INTERESTING SYSTEM CALLS

762       The  process  of  writing a specific error handler for each system call
763       often reveals interesting quirks and boundary  conditions,  or  obscure
764       errno(3) values.
765
766   ENOMEDIUM, No medium found
767       The act of copying a CD was the source of the title for this paper.
768              $ dd if=/dev/cdrom of=fubar.iso
769              dd: opening “/dev/cdrom”: No medium found
770              $
771       The  author  wondered why his computer was telling him there is no such
772       thing as a psychic medium.  Quite apart from the fact that huge numbers
773       of native English speakers are not even aware that “media” is a plural,
774       let alone that “medium” is its singular, the string  returned  by  str‐
775       error(3)  for  ENOMEDIUM is so terse as to be almost completely free of
776       content.
777
778       When open(2) returns ENOMEDIUM it  would  be  nice  if  the  libexplain
779       library  could  expand  a little on this, based on the type of drive it
780       is.  For example:
781         ... because there is no disk in the floppy drive
782         ... because there is no disc in the CD‐ROM drive
783         ... because there is no tape in the tape drive
784         ... because there is no memory stick in the card reader
785
786       And so it came to pass...
787              open(pathname =  "/dev/cdrom",  flags  =  O_RDONLY)  failed,  No
788              medium  found  (123, ENOMEDIUM) because there does not appear to
789              be a disc in the CD‐ROM drive
790       The trick, that the author was previously unaware of, was to  open  the
791       device  using the O_NONBLOCK flag, which will allow you to open a drive
792       with no medium in it.  You then issue device specific ioctl(2) requests
793       until  you figure out what the heck it is.  (Not sure if this is POSIX,
794       but it also seems to work that way in BSD and Solaris, according to the
795       wodim(1) sources.)
796
797       Note  also  the differing uses of “disk” and “disc” in context.  The CD
798       standard originated in France, but everything else has a “k”.
799
800   EFAULT, Bad address
801       Any system call that takes a pointer argument can return  EFAULT.   The
802       libexplain  library  can  figure out which argument is at fault, and it
803       does it without disturbing the process (or thread) signal handling.
804
805       When available, the mincore(2) system call is used, to ask if the  mem‐
806       ory  region  is  valid.  It can return three results: mapped but not in
807       physical memory, mapped and in physical memory, and not  mapped.   When
808       testing the validity of a pointer, the first two are “yes” and the last
809       one is “no”.
810
811       Checking C strings are more difficult, because instead of a pointer and
812       a size, we only have a pointer.  To determine the size we would have to
813       find the NUL, and that could segfault, catch‐22.
814
815       To work around this, the libexplain library  uses  the  lstat(2)  sysem
816       call  (with  a known good second argument) to test C strings for valid‐
817       ity.  A failure return && errno == EFAULT is a “no”, and  anythng  else
818       is a “yes”.  This, of course limits strings to PATH_MAX characters, but
819       that usually isn't a problem for the libexplain library,  because  that
820       is almost always the longest strings it cares about.
821
822   EMFILE, Too many open files
823       This error occurs when a process already has the maximum number of file
824       descriptors open.  If the actual limit is to be printed, and the libex‐
825       plain  library tries to, you can't open a file in /proc to read what it
826       is.
827              open_max = sysconf(_SC_OPEN_MAX);
828       This one wan't so difficult, there is a sysconf(3) way of obtaining the
829       limit.
830
831   ENFILE, Too many open files in system
832       This  error  occurs  when  the system limit on the total number of open
833       files has been reached.  In this case there is no handy sysconf(3)  way
834       of obtain the limit.
835
836       Digging  deeper,  one may discover that on Linux there is a /proc entry
837       we could read to obtain this value.   Catch‐22:  we  are  out  of  file
838       descriptors, so we can't open a file to read the limit.
839
840       On  Linux  there  is a system call to obtain it, but it has no [e]glibc
841       wrapper function, so you have to all it very carefully:
842              long
843              explain_maxfile(void)
844              {
845              #ifdef __linux__
846                  struct __sysctl_args args;
847                  int32_t maxfile;
848                  size_t maxfile_size = sizeof(maxfile);
849                  int name[] = { CTL_FS, FS_MAXFILE };
850                  memset(&args, 0, sizeof(struct __sysctl_args));
851                  args.name = name;
852                  args.nlen = 2;
853                  args.oldval = &maxfile;
854                  args.oldlenp = &maxfile_size;
855                  if (syscall(SYS__sysctl, &args) >= 0)
856                      return maxfile;
857              #endif
858                  return -1;
859              }
860       This permits the limit to be included in the error message, when avail‐
861       able.
862
863   EINVAL “Invalid argument” vs ENOSYS “Function not implemented”
864       Unsupported  actions  (such as symlink(2) on a FAT file system) are not
865       reported consistently from one system call to the next.  It is possible
866       to have either EINVAL or ENOSYS returned.
867
868       As  a  result,  attention must be paid to these error cases to get them
869       right, particularly as the EINVAL could also be referring  to  problems
870       with one or more system call arguments.
871
872   Note that errno(3) is not always set
873       There  are  times  when it is necessary to read the [e]glibc sources to
874       determine how and when errors are returned for some system calls.
875
876       feof(3), fileno(3)
877           It is often assumed that these functions cannot  return  an  error.
878           This is only true if the stream argument is valid, however they are
879           capable of detecting an invalid pointer.
880
881       fpathconf(3), pathconf(3)
882           The return value of fpathconf(2) and pathconf(2) could legitimately
883           be  -1,  so  it is necessary to see if errno(3) has been explicitly
884           set.
885
886       ioctl(2)
887           The return value of ioctl(2) could legitimately be  -1,  so  it  is
888           necessary to see if errno(3) has been explicitly set.
889
890       readdir(3)
891           The  return value of readdir(3) is NULL for both errors and end‐of‐
892           file.  It is necessary to see if errno(3) has been explicitly set.
893
894       setbuf(3), setbuffer(3), setlinebuf(3), setvbuf(3)
895           All but the last of these functions return void.  And setvbuf(3) is
896           only  documented as returning “non‐zero” on error.  It is necessary
897           to see if errno(3) has been explicitly set.
898
899       strtod(3), strtol(3), strtold(3), strtoll(3), strtoul(3), strtoull(3)
900           These functions return 0 on error, but that is  also  a  legitimate
901           return  value.  It is necessary to see if errno(3) has been explic‐
902           itly set.
903
904       ungetc(3)
905           While only a single character of backup is mandated by the  ANSI  C
906           standard,  it  turns  out  that  [e]glibc permits more...  but that
907           means it can fail with ENOMEM.  It can also fail with EBADF  if  fp
908           is  bogus.   Most difficult of all, if you pass EOF an error return
909           occurs, but errno is not set.
910
911       The libexplain library detects all of these errors correctly,  even  in
912       cases where the error values are poorly documented, if at all.
913
914   ENOSPC, No space left on device
915       When  this  error  refers  to  a  file on a file system, the libexplain
916       library prints the mount point of the file  system  with  the  problem.
917       This can make the source of the error much clearer.
918              write(fildes  =  1  "example", data = 0xbfff2340, data_size = 5)
919              failed, No space left on device (28, ENOSPC)  because  the  file
920              system containing fildes ("/home") has no more space for data
921       As more special device support is added, error messages are expected to
922       include the device name and actual size of the device.
923
924   EROFS, Read‐only file system
925       When this error refers to a file  on  a  file  system,  the  libexplain
926       library  prints  the  mount  point of the file system with the problem.
927       This can make the source of the error much clearer.
928
929       As more special device support is added, error messages are expected to
930       include the device name and type.
931              open(pathname = "/dev/fd0", O_RDWR, 0666) failed, Read‐only file
932              system (30, EROFS) because the floppy disk has the write protect
933              tab set
934
935       ...because a CD‐ROM is not writable
936       ...because the memory card has the write protect tab set
937       ...because the ½ inch magnetic tape does not have a write ring
938
939   rename
940       The  rename(2)  system call is used to change the location or name of a
941       file, moving it between directories if required.   If  the  destination
942       pathname  already  exists it will be atomically replaced, so that there
943       is no point at which another process attempting to access it will  find
944       it missing.
945
946       There  are limitations, however: you can only rename a directory on top
947       of another directory if the destination directory is not empty.
948              rename(oldpath = "foo", newpath = "bar") failed,  Directory  not
949              empty (39, ENOTEMPTY) because newpath is not an empty directory;
950              that is, it contains entries other than "." and ".."
951       You can't rename a directory on top of a non‐directory, either.
952              rename(oldpath = "foo", newpath = "bar") failed, Not a directory
953              (20,  ENOTDIR)  because oldpath is a directory, but newpath is a
954              regular file, not a directory
955       Nor is the reverse allowed
956              rename(oldpath = "foo", newpath = "bar") failed, Is a  directory
957              (21,  EISDIR)  because  newpath is a directory, but oldpath is a
958              regular file, not a directory
959
960       This, of course, makes the libexplain library's job  more  complicated,
961       because  the  unlink(2) or rmdir(2) system call is called implicitly by
962       rename(2), and so all of the  unlink(2)  or  rmdir(2)  errors  must  be
963       detected and handled, as well.
964
965   dup2
966       The dup2(2) system call is used to create a second file descriptor that
967       references the same object as the  first  file  descriptor.   Typically
968       this is used to implement shell input and output redirection.
969
970       The  fun  thing is that, just as rename(2) can atomically rename a file
971       on top of an existing file and remove the old file, dup2(2) can do this
972       onto an already‐open file descriptor.
973
974       Once  again,  this makes the libexplain library's job more complicated,
975       because the close(2) system call is called implicitly by  dup2(2),  and
976       so all of close(2)'s errors must be detected and handled, as well.
977

ADVENTURES IN IOCTL SUPPORT

979       The  ioctl(2)  system call provides device driver authors with a way to
980       communicate with user‐space that doesn't fit within the existing kernel
981       API.  See ioctl_list(2).
982
983   Decoding Request Numbers
984       From a cursory look at the ioctl(2) interface, there would appear to be
985       a large but finite number of possible ioctl(2) requests.  Each  differ‐
986       ent  ioctl(2)  request  is effectively another system call, but without
987       any type‐safety at all - the compiler can't help a programmer get these
988       right.  This was probably the motivation behind tcflush(3) and friends.
989
990       The initial impression is that you could decode ioctl(2) requests using
991       a huge switch statement.  This turns out to be infeasible  because  one
992       very rapidly discovers that it is impossible to include all of the nec‐
993       essary system headers defining the various ioctl(2)  requests,  because
994       they have a hard time playing nicely with each other.
995
996       A  deeper  look reveals that there is a range of “private” request num‐
997       bers, and device driver authors are encouraged to use them.  This means
998       that  there  is  a  far larger possible set of requests, with ambiguous
999       request numbers, than are immediately apparent.  Also, there  are  some
1000       historical ambiguities as well.
1001
1002       We  already  knew that the switch was impractical, but now we know that
1003       to select the appropriate request name and explanation we must consider
1004       not only the request number but also the file descriptor.
1005
1006       The implementation of ioctl(2) support within the libexplain library is
1007       to have a table of pointers to ioctl(2) request descriptors.   Each  of
1008       these  descriptors  includes  an  optional  pointer to a disambiguation
1009       function.
1010
1011       Each request is actually implemented in a separate source file, so that
1012       the  necessary  include  files  are  relieved of the obligation to play
1013       nicely with others.
1014
1015   Representation
1016       The philosophy behind the libexplain library  is  to  provide  as  much
1017       information  as  possible,  including an accurate representation of the
1018       system call.  In the case of ioctl(2) this means printing  the  correct
1019       request number (by name) and also a correct (or at least useful) repre‐
1020       sentation of the third argument.
1021
1022       The ioctl(2) prototype looks like this:
1023              int ioctl(int fildes, int request, ...);
1024       which should have your  type‐safety  alarms  going  off.   Internal  to
1025       [e]glibc, this is turned into a variety of forms:
1026              int __ioctl(int fildes, int request, long arg);
1027              int __ioctl(int fildes, int request, void *arg);
1028       and the Linux kernel syscall interface expects
1029              asmlinkage  long  sys_ioctl(unsigned  int  fildes,  unsigned int
1030              request, unsigned long arg);
1031       The extreme variability of the third argument is a challenge, when  the
1032       libexplain  library tries to print a representation of that third argu‐
1033       ment.  However, once the request number has  been  disambiguated,  each
1034       entry  in  the  the  libexplain  library's  ioctl  table  has  a custom
1035       print_data function (OO done manually).
1036
1037   Explanations
1038       There are fewer problems determining the explanation to be used.   Once
1039       the request number has been disambiguated, each entry in the libexplain
1040       library's ioctl table has a custom print_explanation  function  (again,
1041       OO done manually).
1042
1043       Unlike  section  2  and  section 3 system calls, most ioctl(2) requests
1044       have no errors documented.  This means, to  give  good  error  descrip‐
1045       tions, it is necessary to read kernel sources to discover
1046
1047       · what errno(3) values may be returned, and
1048
1049       · the cause of each error.
1050
1051       Because  of the OO nature of function call dispatching withing the ker‐
1052       nel, you need to read all sources implementing that  ioctl(2)  request,
1053       not just the generic implementation.  It is to be expected that differ‐
1054       ent kernels will have different  error  numbers  and  subtly  different
1055       error causes.
1056
1057   EINVAL vs ENOTTY
1058       The  situation  is  even  worse  for  ioctl(2) requests than for system
1059       calls, with EINVAL and ENOTTY both  being  used  to  indicate  that  an
1060       ioctl(2)  request  is  inappropriate  in that context, and occasionally
1061       ENOSYS, ENOTSUP and EOPNOTSUPP (meant to be used for sockets) as  well.
1062       There  are comments in the Linux kernel sources that seem to indicate a
1063       progressive cleanup is in progress.  For extra chaos, BSD adds ENOIOCTL
1064       to the confusion.
1065
1066       As  a  result,  attention must be paid to these error cases to get them
1067       right, particularly as the EINVAL could also be referring  to  problems
1068       with one or more system call arguments.
1069
1070   intptr_t
1071       The  C99 standard defines an integer type that is guaranteed to be able
1072       to hold any pointer without representation loss.
1073
1074       The above function syscall prototype would be better written
1075              long  sys_ioctl(unsigned  int  fildes,  unsigned  int   request,
1076              intptr_t arg);
1077       The  problem  is the cognitive dissonance induced by device‐specific or
1078       file‐system‐specific ioctl(2) implementations, such as:
1079              long vfs_ioctl(struct file *filp,  unsigned  int  cmd,  unsigned
1080              long arg);
1081       The majority of ioctl(2) requests actually have an int *arg third argu‐
1082       ment.  But having it declared long leads to code treating this as  long
1083       *arg.   This  is  harmless on 32‐bits (sizeof(long) == sizeof(int)) but
1084       nasty on 64‐bits  (sizeof(long)  !=  sizeof(int)).   Depending  on  the
1085       endian‐ness,  you  do or don't get the value you expect, but you always
1086       get a memory scribble or stack scribble as well.
1087
1088       Writing all of these as
1089              int ioctl(int fildes, int request, ...);
1090              int __ioctl(int fildes, int request, intptr_t arg);
1091              long  sys_ioctl(unsigned  int  fildes,  unsigned  int   request,
1092              intptr_t arg);
1093              long  vfs_ioctl(struct  file  *filp,  unsigned int cmd, intptr_t
1094              arg);
1095       emphasizes that the integer is only an integer to represent a  quantity
1096       that is almost always an unrelated pointer type.
1097

CONCLUSION

1099       Use libexplain, your users will like it.
1100
1102       libexplain version 1.4
1103       Copyright (C) 2008, 2009, 2010, 2011, 2012, 2013, 2014 Peter Miller
1104

AUTHOR

1106       Written by Peter Miller <pmiller@opensource.org.au>
1107
1108
1109
1110                                                            explain_lca2010(1)
Impressum