1ATOMICWRITES(1) atomicwrites ATOMICWRITES(1)
2
6 atomicwrites - atomicwrites Documentation Documentation Status
7 Atomic file writes.
9 from atomicwrites import atomic_write
11 with atomic_write('foo.txt', overwrite=True) as f:
13 f.write('Hello world.')
14 # "foo.txt" doesn't exist yet.
15 # Now it does.
17 See API documentation for more low-level interfaces.
19 Features that distinguish it from other similar libraries (see
21 Alternatives and Credit):
22 • Race-free assertion that the target file doesn't yet exist. This can
24 be controlled with the overwrite parameter.
25 • Windows support, although not well-tested. The MSDN resources are not
27 very explicit about which operations are atomic. I'm basing my as‐
28 sumptions off a comment by Doug Cook, who appears to be a Microsoft
29 employee:
30 Question: Is MoveFileEx atomic if the existing and new files are
31 both on the same drive?
32 The simple answer is "usually, but in some cases it will silently
34 fall-back to a non-atomic method, so don't count on it".
35 The implementation of MoveFileEx looks something like this: [...]
37 The problem is if the rename fails, you might end up with a Copy‐
39 File, which is definitely not atomic.
40 If you really need atomic-or-nothing, you can try calling NtSet‐
42 InformationFile, which is unsupported but is much more likely to
43 be atomic.
44 • Simple high-level API that wraps a very flexible class-based API.
46 • Consistent error handling across platforms.
48
50 It uses a temporary file in the same directory as the given path. This
51 ensures that the temporary file resides on the same filesystem.
52 The temporary file will then be atomically moved to the target loca‐
54 tion: On POSIX, it will use rename if files should be overwritten, oth‐
55 erwise a combination of link and unlink. On Windows, it uses MoveFileEx
56 through stdlib's ctypes with the appropriate flags.
57 Note that with link and unlink, there's a timewindow where the file
59 might be available under two entries in the filesystem: The name of the
60 temporary file, and the name of the target file.
61 Also note that the permissions of the target file may change this way.
63 In some situations a chmod can be issued without any concurrency prob‐
64 lems, but since that is not always the case, this library doesn't do it
65 by itself.
66 fsync
68 On POSIX, fsync is invoked on the temporary file after it is written
69 (to flush file content and metadata), and on the parent directory after
70 the file is moved (to flush filename).
71 fsync does not take care of disks' internal buffers, but there don't
73 seem to be any standard POSIX APIs for that. On OS X, fcntl is used
74 with F_FULLFSYNC instead of fsync for that reason.
75 On Windows, _commit is used, but there are no guarantees about disk in‐
77 ternal buffers.
78
80 Atomicwrites is directly inspired by the following libraries (and
81 shares a minimal amount of code):
82 • The Trac project's utility functions, also used in Werkzeug and
84 mitsuhiko/python-atomicfile. The idea to use ctypes instead of Py‐
85 Win32 originated there.
86 • abarnert/fatomic. Windows support (based on PyWin32) was originally
88 taken from there.
89 Other alternatives to atomicwrites include:
91 • sashka/atomicfile. Originally I considered using that, but at the
93 time it was lacking a lot of features I needed (Windows support,
94 overwrite-parameter, overriding behavior through subclassing).
95 • The Boltons library collection features a class for atomic file
97 writes, which seems to have a very similar overwrite parameter. It is
98 lacking Windows support though.
99
101 Licensed under the MIT, see LICENSE.
102
104 atomicwrites.atomic_write(path, writer_cls=<class 'atomicwrites.Atom‐
105 icWriter'>, **cls_kwargs)
106 Simple atomic writes. This wraps AtomicWriter:
107 with atomic_write(path) as f:
109 f.write(...)
110 Parameters
112 • path -- The target path to write to.
114 • writer_cls -- The writer class to use. This parameter
116 is useful if you subclassed AtomicWriter to change some
117 behavior and want to use that new subclass.
118 Additional keyword arguments are passed to the writer class. See
120 AtomicWriter.
121 Errorhandling
123 All filesystem errors are subclasses of OSError.
124 • On UNIX systems, errors from the Python stdlib calls are thrown.
126 • On Windows systems, errors from Python's ctypes are thrown.
128 In either case, the errno attribute on the thrown exception maps to an
130 errorcode in the errno module.
131 Low-level API
133 atomicwrites.replace_atomic(src, dst)
134 Move src to dst. If dst exists, it will be silently overwritten.
135 Both paths must reside on the same filesystem for the operation
137 to be atomic.
138 atomicwrites.move_atomic(src, dst)
140 Move src to dst. There might a timewindow where both filesystem
141 entries exist. If dst already exists, FileExistsError will be
142 raised.
143 Both paths must reside on the same filesystem for the operation
145 to be atomic.
146 class atomicwrites.AtomicWriter(path, mode='w', overwrite=False,
148 **open_kwargs)
149 A helper class for performing atomic writes. Usage:
150 with AtomicWriter(path).open() as f:
152 f.write(...)
153 Parameters
155 • path -- The destination filepath. May or may not exist.
157 • mode -- The filemode for the temporary file. This de‐
159 faults to wb in Python 2 and w in Python 3.
160 • overwrite -- If set to false, an error is raised if
162 path exists. Errors are only raised after the file has
163 been written to. Either way, the operation is atomic.
164 • open_kwargs -- Keyword-arguments to pass to the under‐
166 lying open() call. This can be used to set the encoding
167 when opening files in text-mode.
168 If you need further control over the exact behavior, you are en‐
170 couraged to subclass.
171 commit(f)
173 Move the temporary file to the target location.
174 get_fileobject(suffix='', prefix='tmp', dir=None, **kwargs)
176 Return the temporary file to use.
177 open() Open the temporary file.
179 rollback(f)
181 Clean up all temporary resources.
182 sync(f)
184 responsible for clearing as many file caches as possible
185 before commit
186
188 Copyright (c) 2015-2016 Markus Unterwaditzer
189 Permission is hereby granted, free of charge, to any person obtaining a
191 copy of this software and associated documentation files (the "Soft‐
192 ware"), to deal in the Software without restriction, including without
193 limitation the rights to use, copy, modify, merge, publish, distribute,
194 sublicense, and/or sell copies of the Software, and to permit persons
195 to whom the Software is furnished to do so, subject to the following
196 conditions:
197 The above copyright notice and this permission notice shall be included
199 in all copies or substantial portions of the Software.
200 THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
202 OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MER‐
203 CHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN
204 NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
205 CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
206 TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFT‐
207 WARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
208
210 Markus Unterwaditzer
211
213 2015-2023, Markus Unterwaditzer
2141.4 Jan 20, 2023 ATOMICWRITES(1)