1explain_lca2010(1) General Commands Manual explain_lca2010(1)
2
6 explain_lca2010 - No medium found: when it's time to stop trying to
7 read strerror(3)'s mind.
8
10 The idea for libexplain occurred to me back in the early 1980s. When‐
11 ever a system call returns an error, the kernel knows exactly what went
12 wrong... and compresses this into less that 8 bits of errno. User
13 space has access to the same data as the kernel, it should be possible
14 for user space to figure out exactly what happened to provoke the error
15 return, and use this to write good error messages.
16 Could it be that simple?
18 Error messages as finesse
20 Good error messages are often those “one percent” tasks that get
21 dropped when schedule pressure squeezes your project. However, a good
22 error message can make a huge, disproportionate improvement to the user
23 experience, when the user wanders into scarey unknown territory not
24 usually encountered. This is no easy task.
25 As a larval programmer, the author didn't see the problem with (com‐
27 pletely accurate) error messages like this one:
28 floating exception (core dumped)
29 until the alternative non‐programmer interpretation was pointed out.
30 But that isn't the only thing wrong with Unix error messages. How
31 often do you see error messages like:
32 $ ./stupid
33 can't open file
34 $
35 There are two options for a developer at this point:
36 1.
38 you can run a debugger, such as gdb(1), or
39 2.
41 you can use strace(1) or truss(1) to look inside.
42 · Remember that your users may not even have access to these tools, let
44 alone the ability to use them. (It's a very long time since Unix
45 beginner meant “has only written one device driver”.)
46 In this example, however, using strace(1) reveals
48 $ strace -e trace=open ./stupid
49 open("some/file", O_RDONLY) = -1 ENOENT (No such file or directory)
50 can't open file
51 $
52 This is considerably more information than the error message provides.
53 Typically, the stupid source code looks like this
54 int fd = open("some/thing", O_RDONLY);
55 if (fd < 0)
56 {
57 fprintf(stderr, "can't open file\n");
58 exit(1);
59 }
60 The user isn't told which file, and also fails to tell the user which
61 error. Was the file even there? Was there a permissions problem? It
62 does tell you it was trying to open a file, but that was probably by
63 accident.
64 Grab your clue stick and go beat the larval programmer with it. Tell
66 him about perror(3). The next time you use the program you see a dif‐
67 ferent error message:
68 $ ./stupid
69 open: No such file or directory
70 $
71 Progress, but not what we expected. How can the user fix the problem
72 if the error message doesn't tell him what the problem was? Looking at
73 the source, we see
74 int fd = open("some/thing", O_RDONLY);
75 if (fd < 0)
76 {
77 perror("open");
78 exit(1);
79 }
80 Time for another run with the clue stick. This time, the error message
81 takes one step forward and one step back:
82 $ ./stupid
83 some/thing: No such file or directory
84 $
85 Now we know the file it was trying to open, but are no longer informed
86 that it was open(2) that failed. In this case it is probably not sig‐
87 nificant, but it can be significant for other system calls. It could
88 have been creat(2) instead, an operation implying that different per‐
89 missions are necessary.
90 const char *filename = "some/thing";
91 int fd = open(filename, O_RDONLY);
92 if (fd < 0)
93 {
94 perror(filename);
95 exit(1);
96 }
97 The above example code is unfortunately typical of non‐larval program‐
98 mers as well. Time to tell our padawan learner about the strerror(3)
99 system call.
100 $ ./stupid
101 open some/thing: No such file or directory
102 $
103 This maximizes the information that can be presented to the user. The
104 code looks like this:
105 const char *filename = "some/thing";
106 int fd = open(filename, O_RDONLY);
107 if (fd < 0)
108 {
109 fprintf(stderr, "open %s: %s\n", filename, strerror(errno));
110 exit(1);
111 }
112 Now we have the system call, the filename, and the error string. This
113 contains all the information that strace(1) printed. That's as good as
114 it gets.
115 Or is it?
117 Limitations of perror and strerror
119 The problem the author saw, back in the 1980s, was that the error mes‐
120 sage is incomplete. Does “no such file or directory” refer to the
121 “some” directory, or to the “thing” file in the “some” directory?
122 A quick look at the man page for strerror(3) is telling:
124 strerror - return string describing error number
125 Note well: it is describing the error number, not the error.
126 On the other hand, the kernel knows what the error was. There was a
128 specific point in the kernel code, caused by a specific condition,
129 where the kernel code branched and said “no”. Could a user‐space pro‐
130 gram figure out the specific condition and write a better error mes‐
131 sage?
132 However, the problem goes deeper. What if the problem occurs during
134 the read(2) system call, rather than the open(2) call? It is simple
135 for the error message associated with open(2) to include the file name,
136 it's right there. But to be able to include a file name in the error
137 associated with the read(2) system call, you have to pass the file name
138 all the way down the call stack, as well as the file descriptor.
139 And here is the bit that grates: the kernel already knows what file
141 name the file descriptor is associated with. Why should a programmer
142 have to pass redundant data all the way down the call stack just to
143 improve an error message that may never be issued? In reality, many
144 programmers don't bother, and the resulting error messages are the
145 worse for it.
146 But that was the 1980s, on a PDP11, with limited resources and no
148 shared libraries. Back then, no flavor of Unix included /proc even in
149 rudimentary form, and the lsof(1) program was over a decade away. So
150 the idea was shelved as impractical.
151 Level Infinity Support
153 Imagine that you are level infinity support. Your job description says
154 that you never ever have to talk to users. Why, then, is there still a
155 constant stream of people wanting you, the local Unix guru, to decipher
156 yet another error message?
157 Strangely, 25 years later, despite a simple permissions system, imple‐
159 mented with complete consistency, most Unix users still have no idea
160 how to decode “No such file or directory”, or any of the other cryptic
161 error messages they see every day. Or, at least, cryptic to them.
162 Wouldn't it be nice if first level tech support didn't need error mes‐
164 sages deciphered? Wouldn't it be nice to have error messages that
165 users could understand without calling tech support?
166 These days /proc on Linux is more than able to provide the information
168 necessary to decode the vast majority of error messages, and point the
169 user to the proximate cause of their problem. On systems with a lim‐
170 ited /proc implementation, the lsof(1) command can fill in many of the
171 gaps.
172 In 2008, the stream of translation requests happened to the author way
174 too often. It was time to re‐examine that 25 year old idea, and libex‐
175 plain is the result.
176
178 The interface to the library tries to be consistent, where possible.
179 Let's start with an example using strerror(3):
180 if (rename(old_path, new_path) < 0)
181 {
182 fprintf(stderr, "rename %s %s: %s\n", old_path, new_path,
183 strerror(errno));
184 exit(1);
185 }
186 The idea behind libexplain is to provide a strerror(3) equivalent for
187 each system call, tailored specifically to that system call, so that it
188 can provide a more detailed error message, containing much of the
189 information you see under the “ERRORS” heading of section 2 and 3 man
190 pages, supplemented with information about actual conditions, actual
191 argument values, and system limits.
192 The Simple Case
194 The strerror(3) replacement:
195 if (rename(old_path, new_path) < 0)
196 {
197 fprintf(stderr, "%s\n", explain_rename(old_path, new_path));
198 exit(1);
199 }
200 The Errno Case
202 It is also possible to pass an explicit errno(3) value, if you must
203 first do some processing that would disturb errno, such as error recov‐
204 ery:
205 if (rename(old_path, new_path < 0))
206 {
207 int old_errno = errno;
208 ...code that disturbs errno...
209 fprintf(stderr, "%s\n", explain_errno_rename(old_errno,
210 old_path, new_path));
211 exit(1);
212 }
213 The Multi‐thread Cases
215 Some applications are multi‐threaded, and thus are unable to share lib‐
216 explain's internal buffer. You can supply your own buffer using
217 if (unlink(pathname))
218 {
219 char message[3000];
220 explain_message_unlink(message, sizeof(message), pathname);
221 error_dialog(message);
222 return -1;
223 }
224 And for completeness, both errno(3) and thread‐safe:
225 ssize_t nbytes = read(fd, data, sizeof(data));
226 if (nbytes < 0)
227 {
228 char message[3000];
229 int old_errno = errno;
230 ...error recovery...
231 explain_message_errno_read(message, sizeof(message),
232 old_errno, fd, data, sizeof(data));
233 error_dialog(message);
234 return -1;
235 }
236 These are replacements for strerror_r(3), on systems that have it.
238 Interface Sugar
240 A set of functions added as convenience functions, to woo programmers
241 to use the libexplain library, turn out to be the author's most com‐
242 monly used libexplain functions in command line programs:
243 int fd = explain_creat_or_die(filename, 0666);
244 This function attempts to create a new file. If it can't, it prints an
245 error message and exits with EXIT_FAILURE. If there is no error, it
246 returns the new file descriptor.
247 A related function:
249 int fd = explain_creat_on_error(filename, 0666);
250 will print the error message on failure, but also returns the original
251 error result, and errno(3) is unmolested, as well.
252 All the other system calls
254 In general, every system call has its own include file
255 #include <libexplain/name.h>
256 that defines function prototypes for six functions:
257 · explain_name,
259 · explain_errno_name,
261 · explain_message_name,
263 · explain_message_errno_name,
265 · explain_name_or_die and
267 · explain_name_on_error.
269 Every function prototype has Doxygen documentation, and this documenta‐
271 tion is not stripped when the include files are installed.
272 The wait(2) system call (and friends) have some extra variants that
274 also interpret failure to be an exit status that isn't EXIT_SUCCESS.
275 This applies to system(3) and pclose(3) as well.
276 Coverage includes 221 system calls and 547 ioctl requests. There are
278 many more system calls yet to implement. System calls that never
279 return, such as exit(2), are not present in the library, and will never
280 be. The exec family of system calls are supported, because they return
281 when there is an error.
282 Cat
284 This is what a hypothetical “cat” program could look like, with full
285 error reporting, using libexplain.
286 #include <libexplain/libexplain.h>
287 #include <stdlib.h>
288 #include <unistd.h>
289 There is one include for libexplain, plus the usual suspects. (If you
290 wish to reduce the preprocessor load, you can use the specific <libex‐
291 plain/name.h> includes.)
292 static void
293 process(FILE *fp)
294 {
295 for (;;)
296 {
297 char buffer[4096];
298 size_t n = explain_fread_or_die(buffer, 1, sizeof(buffer), fp);
299 if (!n)
300 break;
301 explain_fwrite_or_die(buffer, 1, n, stdout);
302 }
303 }
304 The process function copies a file stream to the standard output.
305 Should an error occur for either reading or writing, it is reported
306 (and the pathname will be included in the error) and the command exits
307 with EXIT_FAILURE. We don't even worry about tracking the pathnames,
308 or passing them down the call stack.
309 int
310 main(int argc, char **argv)
311 {
312 for (;;)
313 {
314 int c = getopt(argc, argv, "o:");
315 if (c == EOF)
316 break;
317 switch (c)
318 {
319 case 'o':
320 explain_freopen_or_die(optarg, "w", stdout);
321 break;
322 The fun part of this code is that libexplain can report errors includ‐
323 ing the pathname even if you don't explicitly re‐open stdout as is done
324 here. We don't even worry about tracking the file name.
325 default:
326 fprintf(stderr, "Usage: %ss [ -o <filename> ] <filename>...\n",
327 argv[0]);
328 return EXIT_FAILURE;
329 }
330 }
331 if (optind == argc)
332 process(stdin);
333 else
334 {
335 while (optind < argc)
336 {
337 FILE *fp = explain_fopen_or_die(argv[optind]++, "r");
338 process(fp);
339 explain_fclose_or_die(fp);
340 }
341 }
342 The standard output will be closed implicitly, but too late for an
343 error report to be issued, so we do that here, just in case the
344 buffered I/O hasn't written anything yet, and there is an ENOSPC error
345 or something.
346 explain_fflush_or_die(stdout);
347 return EXIT_SUCCESS;
348 }
349 That's all. Full error reporting, clear code.
350 Rusty's Scale of Interface Goodness
352 For those of you not familiar with it, Rusty Russel's “How Do I Make
353 This Hard to Misuse?” page is a must‐read for API designers.
354 http://ozlabs.org/~rusty/index.cgi/tech/2008‐03‐30.html
355 10. It's impossible to get wrong.
357 Goals need to be set high, ambitiously high, lest you accomplish them
359 and think you are finished when you are not.
360 The libexplain library detects bogus pointers and many other bogus sys‐
362 tem call parameters, and generally tries to avoid segfaults in even the
363 most trying circumstances.
364 The libexplain library is designed to be thread safe. More real‐world
366 use will likely reveal places this can be improved.
367 The biggest problem is with the actual function names themselves.
369 Because C does not have name‐spaces, the libexplain library always uses
370 an explain_ name prefix. This is the traditional way of creating a
371 pseudo‐name‐space in order to avoid symbol conflicts. However, it
372 results in some unnatural‐sounding names.
373 9. The compiler or linker won't let you get it wrong.
375 A common mistake is to use explain_open where explain_open_or_die was
377 intended. Fortunately, the compiler will often issue a type error at
378 this point (e.g. can't assign const char * rvalue to an int lvalue).
379 8. The compiler will warn if you get it wrong.
381 If explain_rename is used when explain_rename_or_die was intended, this
383 can cause other problems. GCC has a useful warn_unused_result function
384 attribute, and the libexplain library attaches it to all the
385 explain_name function calls to produce a warning when you make this
386 mistake. Combine this with gcc -Werror to promote this to level 9
387 goodness.
388 7. The obvious use is (probably) the correct one.
390 The function names have been chosen to convey their meaning, but this
392 is not always successful. While explain_name_or_die and
393 explain_name_on_error are fairly descriptive, the less‐used thread safe
394 variants are harder to decode. The function prototypes help the com‐
395 piler towards understanding, and the Doxygen comments in the header
396 files help the user towards understanding.
397 6. The name tells you how to use it.
399 It is particularly important to read explain_name_or_die as “explain
401 (name or die)”. Using a consistent explain_ name‐space prefix has some
402 unfortunate side‐effects in the obviousness department, as well.
403 The order of words in the names also indicate the order of the argu‐
405 ments. The argument lists always end with the same arguments as passed
406 to the system call; all of them. If _errno_ appears in the name, its
407 argument always precedes the system call arguments. If _message_
408 appears in the name, its two arguments always come first.
409 5. Do it right or it will break at runtime.
411 The libexplain library detects bogus pointers and many other bogus sys‐
413 tem call parameters, and generally tries to avoid segfaults in even the
414 most trying circumstances. It should never break at runtime, but more
415 real‐world use will no doubt improve this.
416 Some error messages are aimed at developers and maintainers rather than
418 end users, as this can assist with bug resolution. Not so much “break
419 at runtime” as “be informative at runtime” (after the system call
420 barfs).
421 4. Follow common convention and you'll get it right.
423 Because C does not have name‐spaces, the libexplain library always uses
425 an explain_ name prefix. This is the traditional way of creating a
426 pseudo‐name‐space in order to avoid symbol conflicts.
427 The trailing arguments of all the libexplain call are identical to the
429 system call they are describing. This is intended to provide a consis‐
430 tent convention in common with the system calls themselves.
431 3. Read the documentation and you'll get it right.
433 The libexplain library aims to have complete Doxygen documentation for
435 each and every public API call (and internally as well).
436
438 Working on libexplain is a bit like looking at the underside of your
439 car when it is up on the hoist at the mechanic's. There's some ugly
440 stuff under there, plus mud and crud, and users rarely see it. A good
441 error message needs to be informative, even for a user who has been
442 fortunate enough not to have to look at the under‐side very often, and
443 also informative for the mechanic listening to the user's description
444 over the phone. This is no easy task.
445 Revisiting our first example, the code would like this if it uses lib‐
447 explain:
448 int fd = explain_open_or_die("some/thing", O_RDONLY, 0);
449 will fail with an error message like this
450 open(pathname = "some/file", flags = O_RDONLY) failed, No such
451 file or directory (2, ENOENT) because there is no "some" direc‐
452 tory in the current directory
453 This breaks down into three pieces
454 system‐call failed, system‐error because
455 explanation
456 Before Because
458 It is possible to see the part of the message before “because” as
459 overly technical to non‐technical users, mostly as a result of accu‐
460 rately printing the system call itself at the beginning of the error
461 message. And it looks like strace(1) output, for bonus geek points.
462 open(pathname = "some/file", flags = O_RDONLY) failed, No such
463 file or directory (2, ENOENT)
464 This part of the error message is essential to the developer when he is
465 writing the code, and equally important to the maintainer who has to
466 read bug reports and fix bugs in the code. It says exactly what
467 failed.
468 If this text is not presented to the user then the user cannot copy‐
470 and‐paste it into a bug report, and if it isn't in the bug report the
471 maintainer can't know what actually went wrong.
472 Frequently tech staff will use strace(1) or truss(1) to get this exact
474 information, but this avenue is not open when reading bug reports. The
475 bug reporter's system is far far away, and, by now, in a far different
476 state. Thus, this information needs to be in the bug report, which
477 means it must be in the error message.
478 The system call representation also gives context to the rest of the
480 message. If need arises, the offending system call argument may be
481 referred to by name in the explanation after “because”. In addition,
482 all strings are fully quoted and escaped C strings, so embedded new‐
483 lines and non‐printing characters will not cause the user's terminal to
484 go haywire.
485 The system‐error is what comes out of strerror(2), plus the error sym‐
487 bol. Impatient and expert sysadmins could stop reading at this point,
488 but the author's experience to date is that reading further is reward‐
489 ing. (If it isn't rewarding, it's probably an area of libexplain that
490 can be improved. Code contributions are welcome, of course.)
491 After Because
493 This is the portion of the error message aimed at non‐technical users.
494 It looks beyond the simple system call arguments, and looks for some‐
495 thing more specific.
496 there is no "some" directory in the current directory
497 This portion attempts to explain the proximal cause of the error in
498 plain language, and it is here that internationalization is essential.
499 In general, the policy is to include as much information as possible,
501 so that the user doesn't need to go looking for it (and doesn't leave
502 it out of the bug report).
503 Internationalization
505 Most of the error messages in the libexplain library have been interna‐
506 tionalized. There are no localizations as yet, so if you want the
507 explanations in your native language, please contribute.
508 The “most of” qualifier, above, relates to the fact that the proof‐of‐
510 concept implementation did not include internationalization support.
511 The code base is being revised progressively, usually as a result of
512 refactoring messages so that each error message string appears in the
513 code exactly once.
514 Provision has been made for languages that need to assemble the por‐
516 tions of
517 system‐call failed, system‐error because explanation
518 in different orders for correct grammar in localized error messages.
519 Postmortem
521 There are times when a program has yet to use libexplain, and you can't
522 use strace(1) either. There is an explain(1) command included with
523 libexplain that can be used to decipher error messages, if the state of
524 the underlying system hasn't changed too much.
525 $ explain rename foo /tmp/bar/baz -e ENOENT
526 rename(oldpath = "foo", newpath = "/tmp/bar/baz") failed, No
527 such file or directory (2, ENOENT) because there is no "bar"
528 directory in the newpath "/tmp" directory
529 $
530 Note how the path ambiguity is resolved by using the system call argu‐
531 ment name. Of course, you have to know the error and the system call
532 for explain(1) to be useful. As an aside, this is one of the ways used
533 by the libexplain automatic test suite to verify that libexplain is
534 working.
535 Philosophy
537 “Tell me everything, including stuff I didn't know to look for.”
538 The library is implemented in such a way that when statically linked,
540 only the code you actually use will be linked. This is achieved by
541 having one function per source file, whenever feasible.
542 When it is possible to supply more information, libexplain will do so.
544 The less the user has to track down for themselves, the better. This
545 means that UIDs are accompanied by the user name, GIDs are accompanied
546 by the group name, PIDs are accompanied by the process name, file
547 descriptors and streams are accompanied by the pathname, etc.
548 When resolving paths, if a path component does not exist, libexplain
550 will look for similar names, in order to suggest alternatives for typo‐
551 graphical errors.
552 The libexplain library tries to use as little heap as possible, and
554 usually none. This is to avoid perturbing the process state, as far as
555 possible, although sometimes it is unavoidable.
556 The libexplain library attempts to be thread safe, by avoiding global
558 variables, keeping state on the stack as much as possible. There is a
559 single common message buffer, and the functions that use it are docu‐
560 mented as not being thread safe.
561 The libexplain library does not disturb a process's signal handlers.
563 This makes determining whether a pointer would segfault a challenge,
564 but not impossible.
565 When information is available via a system call as well as available
567 through a /proc entry, the system call is preferred. This is to avoid
568 disturbing the process's state. There are also times when no file
569 descriptors are available.
570 The libexplain library is compiled with large file support. There is
572 no large/small schizophrenia. Where this affects the argument types in
573 the API, and error will be issued if the necessary large file defines
574 are absent.
575 FIXME: Work is needed to make sure that file system quotas are handled
577 in the code. This applies to some getrlimit(2) boundaries, as well.
578 There are cases when relatives paths are uninformative. For example:
580 system daemons, servers and background processes. In these cases,
581 absolute paths are used in the error explanations.
582
584 Short version: see path_resolution(7).
585 Long version: Most users have never heard of path_resolution(7), and
587 many advanced users have never read it. Here is an annotated version:
588 Step 1: Start of the resolution process
590 If the pathname starts with the slash (“/”) character, the starting
591 lookup directory is the root directory of the calling process.
592 If the pathname does not start with the slash(“/”) character, the
594 starting lookup directory of the resolution process is the current
595 working directory of the process.
596 Step 2: Walk along the path
598 Set the current lookup directory to the starting lookup directory.
599 Now, for each non‐final component of the pathname, where a component is
600 a substring delimited by slash (“/”) characters, this component is
601 looked up in the current lookup directory.
602 If the process does not have search permission on the current lookup
604 directory, an EACCES error is returned ("Permission denied").
605 open(pathname = "/home/archives/.ssh/private_key", flags =
606 O_RDONLY) failed, Permission denied (13, EACCES) because the
607 process does not have search permission to the pathname
608 "/home/archives/.ssh" directory, the process effective GID 1000
609 "pmiller" does not match the directory owner 1001 "archives" so
610 the owner permission mode "rwx" is ignored, the others permis‐
611 sion mode is "---", and the process is not privileged (does not
612 have the DAC_READ_SEARCH capability)
613 If the component is not found, an ENOENT error is returned ("No such
615 file or directory").
616 unlink(pathname = "/home/microsoft/rubbish") failed, No such
617 file or directory (2, ENOENT) because there is no "microsoft"
618 directory in the pathname "/home" directory
619 There is also some support for users when they mis‐type pathnames, mak‐
621 ing suggestions when ENOENT is returned:
622 open(pathname = "/user/include/fcntl.h", flags = O_RDONLY)
623 failed, No such file or directory (2, ENOENT) because there is
624 no "user" directory in the pathname "/" directory, did you mean
625 the "usr" directory instead?
626 If the component is found, but is neither a directory nor a symbolic
628 link, an ENOTDIR error is returned ("Not a directory").
629 open(pathname = "/home/pmiller/.netrc/lca", flags = O_RDONLY)
630 failed, Not a directory (20, ENOTDIR) because the ".netrc" regu‐
631 lar file in the pathname "/home/pmiller" directory is being used
632 as a directory when it is not
633 If the component is found and is a directory, we set the current lookup
635 directory to that directory, and go to the next component.
636 If the component is found and is a symbolic link (symlink), we first
638 resolve this symbolic link (with the current lookup directory as start‐
639 ing lookup directory). Upon error, that error is returned. If the
640 result is not a directory, an ENOTDIR error is returned.
641 unlink(pathname = "/tmp/dangling/rubbish") failed, No such file
642 or directory (2, ENOENT) because the "dangling" symbolic link in
643 the pathname "/tmp" directory refers to "nowhere" that does not
644 exist
645 If the resolution of the symlink is successful and returns a directory,
646 we set the current lookup directory to that directory, and go to the
647 next component. Note that the resolution process here involves recur‐
648 sion. In order to protect the kernel against stack overflow, and also
649 to protect against denial of service, there are limits on the maximum
650 recursion depth, and on the maximum number of symbolic links followed.
651 An ELOOP error is returned when the maximum is exceeded ("Too many lev‐
652 els of symbolic links").
653 open(pathname = "/tmp/dangling", flags = O_RDONLY) failed, Too
654 many levels of symbolic links (40, ELOOP) because a symbolic
655 link loop was encountered in pathname, starting at "/tmp/dan‐
656 gling"
657 It is also possible to get an ELOOP or EMLINK error if there are too
658 many symlinks, but no loop was detected.
659 open(pathname = "/tmp/rabbit‐hole", flags = O_RDONLY) failed,
660 Too many levels of symbolic links (40, ELOOP) because too many
661 symbolic links were encountered in pathname (8)
662 Notice how the actual limit is also printed.
663 Step 3: Find the final entry
665 The lookup of the final component of the pathname goes just like that
666 of all other components, as described in the previous step, with two
667 differences:
668 (i) The final component need not be a directory (at least as far as the
670 path resolution process is concerned. It may have to be a direc‐
671 tory, or a non‐directory, because of the requirements of the spe‐
672 cific system call).
673 (ii)
675 It is not necessarily an error if the final component is not found;
676 maybe we are just creating it. The details on the treatment of the
677 final entry are described in the manual pages of the specific sys‐
678 tem calls.
679 (iii)
681 It is also possible to have a problem with the last component if it
682 is a symbolic link and it should not be followed. For example,
683 using the open(2) O_NOFOLLOW flag:
684 open(pathname = "a‐symlink", flags = O_RDONLY | O_NOFOLLOW) failed,
685 Too many levels of symbolic links (ELOOP) because O_NOFOLLOW was
686 specified but pathname refers to a symbolic link
687 (iv)
689 It is common for users to make mistakes when typing pathnames. The
690 libexplain library attempts to make suggestions when ENOENT is
691 returned, for example:
692 open(pathname = "/usr/include/filecontrl.h", flags = O_RDONLY)
693 failed, No such file or directory (2, ENOENT) because there is no
694 "filecontrl.h" regular file in the pathname "/usr/include" direc‐
695 tory, did you mean the "fcntl.h" regular file instead?
696 (v) It is also possible that the final component is required to be
698 something other than a regular file:
699 readlink(pathname = "just‐a‐file", data = 0x7F930A50, data_size =
700 4097) failed, Invalid argument (22, EINVAL) because pathname is a
701 regular file, not a symbolic link
702 (vi)
704 FIXME: handling of the "t" bit.
705 Limits
707 There are a number of limits with regards to pathnames and filenames.
708 Pathname length limit
710 There is a maximum length for pathnames. If the pathname (or
711 some intermediate pathname obtained while resolving symbolic
712 links) is too long, an ENAMETOOLONG error is returned ("File
713 name too long"). Notice how the system limit is included in
714 the error message.
715 open(pathname = "very...long", flags = O_RDONLY) failed, File
716 name too long (36, ENAMETOOLONG) because pathname exceeds the
717 system maximum path length (4096)
718 Filename length limit
720 Some Unix variants have a limit on the number of bytes in each
721 path component. Some of them deal with this silently, and some
722 give ENAMETOOLONG; the libexplain library uses pathconf(3)
723 _PC_NO_TRUNC to tell which. If this error happens, the libex‐
724 plain library will state the limit in the error message, the
725 limit is obtained from pathconf(3) _PC_NAME_MAX. Notice how
726 the system limit is included in the error message.
727 open(pathname = "system7/only-had-14-characters", flags =
728 O_RDONLY) failed, File name too long (36, ENAMETOOLONG) because
729 "only-had-14-characters" component is longer than the system
730 limit (14)
731 Empty pathname
733 In the original Unix, the empty pathname referred to the cur‐
734 rent directory. Nowadays POSIX decrees that an empty pathname
735 must not be resolved successfully.
736 open(pathname = "", flags = O_RDONLY) failed, No such file or
737 directory (2, ENOENT) because POSIX decrees that an empty path‐
738 name must not be resolved successfully
739 Permissions
741 The permission bits of a file consist of three groups of three bits.
742 The first group of three is used when the effective user ID of the
743 calling process equals the owner ID of the file. The second group of
744 three is used when the group ID of the file either equals the effective
745 group ID of the calling process, or is one of the supplementary group
746 IDs of the calling process. When neither holds, the third group is
747 used.
748 open(pathname = "/etc/passwd", flags = O_WRONLY) failed, Permis‐
749 sion denied (13, EACCES) because the process does not have write
750 permission to the "passwd" regular file in the pathname "/etc"
751 directory, the process effective UID 1000 "pmiller" does not
752 match the regular file owner 0 "root" so the owner permission
753 mode "rw-" is ignored, the others permission mode is "r--", and
754 the process is not privileged (does not have the DAC_OVERRIDE
755 capability)
756 Some considerable space is given to this explanation, as most users do
757 not know that this is how the permissions system works. In particular:
758 the owner, group and other permissions are exclusive, they are not
759 “OR”ed together.
760
762 The process of writing a specific error handler for each system call
763 often reveals interesting quirks and boundary conditions, or obscure
764 errno(3) values.
765 ENOMEDIUM, No medium found
767 The act of copying a CD was the source of the title for this paper.
768 $ dd if=/dev/cdrom of=fubar.iso
769 dd: opening “/dev/cdrom”: No medium found
770 $
771 The author wondered why his computer was telling him there is no such
772 thing as a psychic medium. Quite apart from the fact that huge numbers
773 of native English speakers are not even aware that “media” is a plural,
774 let alone that “medium” is its singular, the string returned by str‐
775 error(3) for ENOMEDIUM is so terse as to be almost completely free of
776 content.
777 When open(2) returns ENOMEDIUM it would be nice if the libexplain
779 library could expand a little on this, based on the type of drive it
780 is. For example:
781 ... because there is no disk in the floppy drive
782 ... because there is no disc in the CD‐ROM drive
783 ... because there is no tape in the tape drive
784 ... because there is no memory stick in the card reader
785 And so it came to pass...
787 open(pathname = "/dev/cdrom", flags = O_RDONLY) failed, No
788 medium found (123, ENOMEDIUM) because there does not appear to
789 be a disc in the CD‐ROM drive
790 The trick, that the author was previously unaware of, was to open the
791 device using the O_NONBLOCK flag, which will allow you to open a drive
792 with no medium in it. You then issue device specific ioctl(2) requests
793 until you figure out what the heck it is. (Not sure if this is POSIX,
794 but it also seems to work that way in BSD and Solaris, according to the
795 wodim(1) sources.)
796 Note also the differing uses of “disk” and “disc” in context. The CD
798 standard originated in France, but everything else has a “k”.
799 EFAULT, Bad address
801 Any system call that takes a pointer argument can return EFAULT. The
802 libexplain library can figure out which argument is at fault, and it
803 does it without disturbing the process (or thread) signal handling.
804 When available, the mincore(2) system call is used, to ask if the mem‐
806 ory region is valid. It can return three results: mapped but not in
807 physical memory, mapped and in physical memory, and not mapped. When
808 testing the validity of a pointer, the first two are “yes” and the last
809 one is “no”.
810 Checking C strings are more difficult, because instead of a pointer and
812 a size, we only have a pointer. To determine the size we would have to
813 find the NUL, and that could segfault, catch‐22.
814 To work around this, the libexplain library uses the lstat(2) sysem
816 call (with a known good second argument) to test C strings for valid‐
817 ity. A failure return && errno == EFAULT is a “no”, and anythng else
818 is a “yes”. This, of course limits strings to PATH_MAX characters, but
819 that usually isn't a problem for the libexplain library, because that
820 is almost always the longest strings it cares about.
821 EMFILE, Too many open files
823 This error occurs when a process already has the maximum number of file
824 descriptors open. If the actual limit is to be printed, and the libex‐
825 plain library tries to, you can't open a file in /proc to read what it
826 is.
827 open_max = sysconf(_SC_OPEN_MAX);
828 This one wan't so difficult, there is a sysconf(3) way of obtaining the
829 limit.
830 ENFILE, Too many open files in system
832 This error occurs when the system limit on the total number of open
833 files has been reached. In this case there is no handy sysconf(3) way
834 of obtain the limit.
835 Digging deeper, one may discover that on Linux there is a /proc entry
837 we could read to obtain this value. Catch‐22: we are out of file
838 descriptors, so we can't open a file to read the limit.
839 On Linux there is a system call to obtain it, but it has no [e]glibc
841 wrapper function, so you have to all it very carefully:
842 long
843 explain_maxfile(void)
844 {
845 #ifdef __linux__
846 struct __sysctl_args args;
847 int32_t maxfile;
848 size_t maxfile_size = sizeof(maxfile);
849 int name[] = { CTL_FS, FS_MAXFILE };
850 memset(&args, 0, sizeof(struct __sysctl_args));
851 args.name = name;
852 args.nlen = 2;
853 args.oldval = &maxfile;
854 args.oldlenp = &maxfile_size;
855 if (syscall(SYS__sysctl, &args) >= 0)
856 return maxfile;
857 #endif
858 return -1;
859 }
860 This permits the limit to be included in the error message, when avail‐
861 able.
862 EINVAL “Invalid argument” vs ENOSYS “Function not implemented”
864 Unsupported actions (such as symlink(2) on a FAT file system) are not
865 reported consistently from one system call to the next. It is possible
866 to have either EINVAL or ENOSYS returned.
867 As a result, attention must be paid to these error cases to get them
869 right, particularly as the EINVAL could also be referring to problems
870 with one or more system call arguments.
871 Note that errno(3) is not always set
873 There are times when it is necessary to read the [e]glibc sources to
874 determine how and when errors are returned for some system calls.
875 feof(3), fileno(3)
877 It is often assumed that these functions cannot return an error.
878 This is only true if the stream argument is valid, however they are
879 capable of detecting an invalid pointer.
880 fpathconf(3), pathconf(3)
882 The return value of fpathconf(2) and pathconf(2) could legitimately
883 be -1, so it is necessary to see if errno(3) has been explicitly
884 set.
885 ioctl(2)
887 The return value of ioctl(2) could legitimately be -1, so it is
888 necessary to see if errno(3) has been explicitly set.
889 readdir(3)
891 The return value of readdir(3) is NULL for both errors and end‐of‐
892 file. It is necessary to see if errno(3) has been explicitly set.
893 setbuf(3), setbuffer(3), setlinebuf(3), setvbuf(3)
895 All but the last of these functions return void. And setvbuf(3) is
896 only documented as returning “non‐zero” on error. It is necessary
897 to see if errno(3) has been explicitly set.
898 strtod(3), strtol(3), strtold(3), strtoll(3), strtoul(3), strtoull(3)
900 These functions return 0 on error, but that is also a legitimate
901 return value. It is necessary to see if errno(3) has been explic‐
902 itly set.
903 ungetc(3)
905 While only a single character of backup is mandated by the ANSI C
906 standard, it turns out that [e]glibc permits more... but that
907 means it can fail with ENOMEM. It can also fail with EBADF if fp
908 is bogus. Most difficult of all, if you pass EOF an error return
909 occurs, but errno is not set.
910 The libexplain library detects all of these errors correctly, even in
912 cases where the error values are poorly documented, if at all.
913 ENOSPC, No space left on device
915 When this error refers to a file on a file system, the libexplain
916 library prints the mount point of the file system with the problem.
917 This can make the source of the error much clearer.
918 write(fildes = 1 "example", data = 0xbfff2340, data_size = 5)
919 failed, No space left on device (28, ENOSPC) because the file
920 system containing fildes ("/home") has no more space for data
921 As more special device support is added, error messages are expected to
922 include the device name and actual size of the device.
923 EROFS, Read‐only file system
925 When this error refers to a file on a file system, the libexplain
926 library prints the mount point of the file system with the problem.
927 This can make the source of the error much clearer.
928 As more special device support is added, error messages are expected to
930 include the device name and type.
931 open(pathname = "/dev/fd0", O_RDWR, 0666) failed, Read‐only file
932 system (30, EROFS) because the floppy disk has the write protect
933 tab set
934 ...because a CD‐ROM is not writable
936 ...because the memory card has the write protect tab set
937 ...because the ½ inch magnetic tape does not have a write ring
938 rename
940 The rename(2) system call is used to change the location or name of a
941 file, moving it between directories if required. If the destination
942 pathname already exists it will be atomically replaced, so that there
943 is no point at which another process attempting to access it will find
944 it missing.
945 There are limitations, however: you can only rename a directory on top
947 of another directory if the destination directory is not empty.
948 rename(oldpath = "foo", newpath = "bar") failed, Directory not
949 empty (39, ENOTEMPTY) because newpath is not an empty directory;
950 that is, it contains entries other than "." and ".."
951 You can't rename a directory on top of a non‐directory, either.
952 rename(oldpath = "foo", newpath = "bar") failed, Not a directory
953 (20, ENOTDIR) because oldpath is a directory, but newpath is a
954 regular file, not a directory
955 Nor is the reverse allowed
956 rename(oldpath = "foo", newpath = "bar") failed, Is a directory
957 (21, EISDIR) because newpath is a directory, but oldpath is a
958 regular file, not a directory
959 This, of course, makes the libexplain library's job more complicated,
961 because the unlink(2) or rmdir(2) system call is called implicitly by
962 rename(2), and so all of the unlink(2) or rmdir(2) errors must be
963 detected and handled, as well.
964 dup2
966 The dup2(2) system call is used to create a second file descriptor that
967 references the same object as the first file descriptor. Typically
968 this is used to implement shell input and output redirection.
969 The fun thing is that, just as rename(2) can atomically rename a file
971 on top of an existing file and remove the old file, dup2(2) can do this
972 onto an already‐open file descriptor.
973 Once again, this makes the libexplain library's job more complicated,
975 because the close(2) system call is called implicitly by dup2(2), and
976 so all of close(2)'s errors must be detected and handled, as well.
977
979 The ioctl(2) system call provides device driver authors with a way to
980 communicate with user‐space that doesn't fit within the existing kernel
981 API. See ioctl_list(2).
982 Decoding Request Numbers
984 From a cursory look at the ioctl(2) interface, there would appear to be
985 a large but finite number of possible ioctl(2) requests. Each differ‐
986 ent ioctl(2) request is effectively another system call, but without
987 any type‐safety at all - the compiler can't help a programmer get these
988 right. This was probably the motivation behind tcflush(3) and friends.
989 The initial impression is that you could decode ioctl(2) requests using
991 a huge switch statement. This turns out to be infeasible because one
992 very rapidly discovers that it is impossible to include all of the nec‐
993 essary system headers defining the various ioctl(2) requests, because
994 they have a hard time playing nicely with each other.
995 A deeper look reveals that there is a range of “private” request num‐
997 bers, and device driver authors are encouraged to use them. This means
998 that there is a far larger possible set of requests, with ambiguous
999 request numbers, than are immediately apparent. Also, there are some
1000 historical ambiguities as well.
1001 We already knew that the switch was impractical, but now we know that
1003 to select the appropriate request name and explanation we must consider
1004 not only the request number but also the file descriptor.
1005 The implementation of ioctl(2) support within the libexplain library is
1007 to have a table of pointers to ioctl(2) request descriptors. Each of
1008 these descriptors includes an optional pointer to a disambiguation
1009 function.
1010 Each request is actually implemented in a separate source file, so that
1012 the necessary include files are relieved of the obligation to play
1013 nicely with others.
1014 Representation
1016 The philosophy behind the libexplain library is to provide as much
1017 information as possible, including an accurate representation of the
1018 system call. In the case of ioctl(2) this means printing the correct
1019 request number (by name) and also a correct (or at least useful) repre‐
1020 sentation of the third argument.
1021 The ioctl(2) prototype looks like this:
1023 int ioctl(int fildes, int request, ...);
1024 which should have your type‐safety alarms going off. Internal to
1025 [e]glibc, this is turned into a variety of forms:
1026 int __ioctl(int fildes, int request, long arg);
1027 int __ioctl(int fildes, int request, void *arg);
1028 and the Linux kernel syscall interface expects
1029 asmlinkage long sys_ioctl(unsigned int fildes, unsigned int
1030 request, unsigned long arg);
1031 The extreme variability of the third argument is a challenge, when the
1032 libexplain library tries to print a representation of that third argu‐
1033 ment. However, once the request number has been disambiguated, each
1034 entry in the the libexplain library's ioctl table has a custom
1035 print_data function (OO done manually).
1036 Explanations
1038 There are fewer problems determining the explanation to be used. Once
1039 the request number has been disambiguated, each entry in the libexplain
1040 library's ioctl table has a custom print_explanation function (again,
1041 OO done manually).
1042 Unlike section 2 and section 3 system calls, most ioctl(2) requests
1044 have no errors documented. This means, to give good error descrip‐
1045 tions, it is necessary to read kernel sources to discover
1046 · what errno(3) values may be returned, and
1048 · the cause of each error.
1050 Because of the OO nature of function call dispatching withing the ker‐
1052 nel, you need to read all sources implementing that ioctl(2) request,
1053 not just the generic implementation. It is to be expected that differ‐
1054 ent kernels will have different error numbers and subtly different
1055 error causes.
1056 EINVAL vs ENOTTY
1058 The situation is even worse for ioctl(2) requests than for system
1059 calls, with EINVAL and ENOTTY both being used to indicate that an
1060 ioctl(2) request is inappropriate in that context, and occasionally
1061 ENOSYS, ENOTSUP and EOPNOTSUPP (meant to be used for sockets) as well.
1062 There are comments in the Linux kernel sources that seem to indicate a
1063 progressive cleanup is in progress. For extra chaos, BSD adds ENOIOCTL
1064 to the confusion.
1065 As a result, attention must be paid to these error cases to get them
1067 right, particularly as the EINVAL could also be referring to problems
1068 with one or more system call arguments.
1069 intptr_t
1071 The C99 standard defines an integer type that is guaranteed to be able
1072 to hold any pointer without representation loss.
1073 The above function syscall prototype would be better written
1075 long sys_ioctl(unsigned int fildes, unsigned int request,
1076 intptr_t arg);
1077 The problem is the cognitive dissonance induced by device‐specific or
1078 file‐system‐specific ioctl(2) implementations, such as:
1079 long vfs_ioctl(struct file *filp, unsigned int cmd, unsigned
1080 long arg);
1081 The majority of ioctl(2) requests actually have an int *arg third argu‐
1082 ment. But having it declared long leads to code treating this as long
1083 *arg. This is harmless on 32‐bits (sizeof(long) == sizeof(int)) but
1084 nasty on 64‐bits (sizeof(long) != sizeof(int)). Depending on the
1085 endian‐ness, you do or don't get the value you expect, but you always
1086 get a memory scribble or stack scribble as well.
1087 Writing all of these as
1089 int ioctl(int fildes, int request, ...);
1090 int __ioctl(int fildes, int request, intptr_t arg);
1091 long sys_ioctl(unsigned int fildes, unsigned int request,
1092 intptr_t arg);
1093 long vfs_ioctl(struct file *filp, unsigned int cmd, intptr_t
1094 arg);
1095 emphasizes that the integer is only an integer to represent a quantity
1096 that is almost always an unrelated pointer type.
1097
1099 Use libexplain, your users will like it.
1100
1102 libexplain version 1.4
1103 Copyright (C) 2008, 2009, 2010, 2011, 2012, 2013, 2014 Peter Miller
1104
1106 Written by Peter Miller <pmiller@opensource.org.au>
1107 explain_lca2010(1)