1APACHECOUCHDB(1) Apache CouchDB® APACHECOUCHDB(1)
2
6 apachecouchdb - Apache CouchDB® 3.1.0
7
9 CouchDB is a database that completely embraces the web. Store your data
10 with JSON documents. Access your documents with your web browser, via
11 HTTP. Query, combine, and transform your documents with JavaScript.
12 CouchDB works well with modern web and mobile apps. You can distribute
13 your data, efficiently using CouchDB’s incremental replication. CouchDB
14 supports master-master setups with automatic conflict detection.
15 CouchDB comes with a suite of features, such as on-the-fly document
17 transformation and real-time change notifications, that make web devel‐
18 opment a breeze. It even comes with an easy to use web administration
19 console, served directly out of CouchDB! We care a lot about
20 distributed scaling. CouchDB is highly available and partition toler‐
21 ant, but is also eventually consistent. And we care a lot about your
22 data. CouchDB has a fault-tolerant storage engine that puts the safety
23 of your data first.
24 In this section you’ll learn about every basic bit of CouchDB, see upon
26 what conceptions and technologies it built and walk through short tuto‐
27 rial that teach how to use CouchDB.
28 Technical Overview
30 Document Storage
31 A CouchDB server hosts named databases, which store documents. Each
32 document is uniquely named in the database, and CouchDB provides a
33 RESTful HTTP API for reading and updating (add, edit, delete) database
34 documents.
35 Documents are the primary unit of data in CouchDB and consist of any
37 number of fields and attachments. Documents also include metadata
38 that’s maintained by the database system. Document fields are uniquely
39 named and contain values of varying types (text, number, boolean,
40 lists, etc), and there is no set limit to text size or element count.
41 The CouchDB document update model is lockless and optimistic. Document
43 edits are made by client applications loading documents, applying
44 changes, and saving them back to the database. If another client edit‐
45 ing the same document saves their changes first, the client gets an
46 edit conflict error on save. To resolve the update conflict, the latest
47 document version can be opened, the edits reapplied and the update
48 tried again.
49 Single document updates (add, edit, delete) are all or nothing, either
51 succeeding entirely or failing completely. The database never contains
52 partially saved or edited documents.
53 ACID Properties
55 The CouchDB file layout and commitment system features all Atomic Con‐
56 sistent Isolated Durable (ACID) properties. On-disk, CouchDB never
57 overwrites committed data or associated structures, ensuring the data‐
58 base file is always in a consistent state. This is a “crash-only”
59 design where the CouchDB server does not go through a shut down
60 process, it’s simply terminated.
61 Document updates (add, edit, delete) are serialized, except for binary
63 blobs which are written concurrently. Database readers are never locked
64 out and never have to wait on writers or other readers. Any number of
65 clients can be reading documents without being locked out or inter‐
66 rupted by concurrent updates, even on the same document. CouchDB read
67 operations use a Multi-Version Concurrency Control (MVCC) model where
68 each client sees a consistent snapshot of the database from the begin‐
69 ning to the end of the read operation. This means that CouchDB can
70 guarantee transactional semantics on a per-document basis.
71 Documents are indexed in B-trees by their name (DocID) and a Sequence
73 ID. Each update to a database instance generates a new sequential num‐
74 ber. Sequence IDs are used later for incrementally finding changes in
75 a database. These B-tree indexes are updated simultaneously when docu‐
76 ments are saved or deleted. The index updates always occur at the end
77 of the file (append-only updates).
78 Documents have the advantage of data being already conveniently pack‐
80 aged for storage rather than split out across numerous tables and rows
81 in most database systems. When documents are committed to disk, the
82 document fields and metadata are packed into buffers, sequentially one
83 document after another (helpful later for efficient building of views).
84 When CouchDB documents are updated, all data and associated indexes are
86 flushed to disk and the transactional commit always leaves the database
87 in a completely consistent state. Commits occur in two steps:
88 1. All document data and associated index updates are synchronously
90 flushed to disk.
91 2. The updated database header is written in two consecutive, identical
93 chunks to make up the first 4k of the file, and then synchronously
94 flushed to disk.
95 In the event of an OS crash or power failure during step 1, the par‐
97 tially flushed updates are simply forgotten on restart. If such a crash
98 happens during step 2 (committing the header), a surviving copy of the
99 previous identical headers will remain, ensuring coherency of all pre‐
100 viously committed data. Excepting the header area, consistency checks
101 or fix-ups after a crash or a power failure are never necessary.
102 Compaction
104 Wasted space is recovered by occasional compaction. On schedule, or
105 when the database file exceeds a certain amount of wasted space, the
106 compaction process clones all the active data to a new file and then
107 discards the old file. The database remains completely online the
108 entire time and all updates and reads are allowed to complete success‐
109 fully. The old database file is deleted only when all the data has been
110 copied and all users transitioned to the new file.
111 Views
113 ACID properties only deal with storage and updates, but we also need
114 the ability to show our data in interesting and useful ways. Unlike SQL
115 databases where data must be carefully decomposed into tables, data in
116 CouchDB is stored in semi-structured documents. CouchDB documents are
117 flexible and each has its own implicit structure, which alleviates the
118 most difficult problems and pitfalls of bi-directionally replicating
119 table schemas and their contained data.
120 But beyond acting as a fancy file server, a simple document model for
122 data storage and sharing is too simple to build real applications on –
123 it simply doesn’t do enough of the things we want and expect. We want
124 to slice and dice and see our data in many different ways. What is
125 needed is a way to filter, organize and report on data that hasn’t been
126 decomposed into tables.
127 SEE ALSO:
129 views
130 View Model
132 To address this problem of adding structure back to unstructured and
133 semi-structured data, CouchDB integrates a view model. Views are the
134 method of aggregating and reporting on the documents in a database, and
135 are built on-demand to aggregate, join and report on database docu‐
136 ments. Because views are built dynamically and don’t affect the under‐
137 lying document, you can have as many different view representations of
138 the same data as you like.
139 View definitions are strictly virtual and only display the documents
141 from the current database instance, making them separate from the data
142 they display and compatible with replication. CouchDB views are defined
143 inside special design documents and can replicate across database
144 instances like regular documents, so that not only data replicates in
145 CouchDB, but entire application designs replicate too.
146 JavaScript View Functions
148 Views are defined using JavaScript functions acting as the map part in
149 a map-reduce system. A view function takes a CouchDB document as an
150 argument and then does whatever computation it needs to do to determine
151 the data that is to be made available through the view, if any. It can
152 add multiple rows to the view based on a single document, or it can add
153 no rows at all.
154 SEE ALSO:
156 viewfun
157 View Indexes
159 Views are a dynamic representation of the actual document contents of a
160 database, and CouchDB makes it easy to create useful views of data.
161 But generating a view of a database with hundreds of thousands or mil‐
162 lions of documents is time and resource consuming, it’s not something
163 the system should do from scratch each time.
164 To keep view querying fast, the view engine maintains indexes of its
166 views, and incrementally updates them to reflect changes in the data‐
167 base. CouchDB’s core design is largely optimized around the need for
168 efficient, incremental creation of views and their indexes.
169 Views and their functions are defined inside special “design” docu‐
171 ments, and a design document may contain any number of uniquely named
172 view functions. When a user opens a view and its index is automati‐
173 cally updated, all the views in the same design document are indexed as
174 a single group.
175 The view builder uses the database sequence ID to determine if the view
177 group is fully up-to-date with the database. If not, the view engine
178 examines all database documents (in packed sequential order) changed
179 since the last refresh. Documents are read in the order they occur in
180 the disk file, reducing the frequency and cost of disk head seeks.
181 The views can be read and queried simultaneously while also being
183 refreshed. If a client is slowly streaming out the contents of a large
184 view, the same view can be concurrently opened and refreshed for
185 another client without blocking the first client. This is true for any
186 number of simultaneous client readers, who can read and query the view
187 while the index is concurrently being refreshed for other clients with‐
188 out causing problems for the readers.
189 As documents are processed by the view engine through your ‘map’ and
191 ‘reduce’ functions, their previous row values are removed from the view
192 indexes, if they exist. If the document is selected by a view function,
193 the function results are inserted into the view as a new row.
194 When view index changes are written to disk, the updates are always
196 appended at the end of the file, serving to both reduce disk head seek
197 times during disk commits and to ensure crashes and power failures can
198 not cause corruption of indexes. If a crash occurs while updating a
199 view index, the incomplete index updates are simply lost and rebuilt
200 incrementally from its previously committed state.
201 Security and Validation
203 To protect who can read and update documents, CouchDB has a simple
204 reader access and update validation model that can be extended to
205 implement custom security models.
206 SEE ALSO:
208 api/db/security
209 Administrator Access
211 CouchDB database instances have administrator accounts. Administrator
212 accounts can create other administrator accounts and update design doc‐
213 uments. Design documents are special documents containing view defini‐
214 tions and other special formulas, as well as regular fields and blobs.
215 Update Validation
217 As documents are written to disk, they can be validated dynamically by
218 JavaScript functions for both security and data validation. When the
219 document passes all the formula validation criteria, the update is
220 allowed to continue. If the validation fails, the update is aborted
221 and the user client gets an error response.
222 Both the user’s credentials and the updated document are given as
224 inputs to the validation formula, and can be used to implement custom
225 security models by validating a user’s permissions to update a docu‐
226 ment.
227 A basic “author only” update document model is trivial to implement,
229 where document updates are validated to check if the user is listed in
230 an “author” field in the existing document. More dynamic models are
231 also possible, like checking a separate user account profile for per‐
232 mission settings.
233 The update validations are enforced for both live usage and replicated
235 updates, ensuring security and data validation in a shared, distributed
236 system.
237 SEE ALSO:
239 vdufun
240 Distributed Updates and Replication
242 CouchDB is a peer-based distributed database system. It allows users
243 and servers to access and update the same shared data while discon‐
244 nected. Those changes can then be replicated bi-directionally later.
245 The CouchDB document storage, view and security models are designed to
247 work together to make true bi-directional replication efficient and
248 reliable. Both documents and designs can replicate, allowing full
249 database applications (including application design, logic and data) to
250 be replicated to laptops for offline use, or replicated to servers in
251 remote offices where slow or unreliable connections make sharing data
252 difficult.
253 The replication process is incremental. At the database level, replica‐
255 tion only examines documents updated since the last replication. If
256 replication fails at any step, due to network problems or crash for
257 example, the next replication restarts at the last checkpoint.
258 Partial replicas can be created and maintained. Replication can be fil‐
260 tered by a JavaScript function, so that only particular documents or
261 those meeting specific criteria are replicated. This can allow users to
262 take subsets of a large shared database application offline for their
263 own use, while maintaining normal interaction with the application and
264 that subset of data.
265 Conflicts
267 Conflict detection and management are key issues for any distributed
268 edit system. The CouchDB storage system treats edit conflicts as a com‐
269 mon state, not an exceptional one. The conflict handling model is sim‐
270 ple and “non-destructive” while preserving single document semantics
271 and allowing for decentralized conflict resolution.
272 CouchDB allows for any number of conflicting documents to exist simul‐
274 taneously in the database, with each database instance deterministi‐
275 cally deciding which document is the “winner” and which are conflicts.
276 Only the winning document can appear in views, while “losing” conflicts
277 are still accessible and remain in the database until deleted or purged
278 during database compaction. Because conflict documents are still regu‐
279 lar documents, they replicate just like regular documents and are sub‐
280 ject to the same security and validation rules.
281 When distributed edit conflicts occur, every database replica sees the
283 same winning revision and each has the opportunity to resolve the con‐
284 flict. Resolving conflicts can be done manually or, depending on the
285 nature of the data and the conflict, by automated agents. The system
286 makes decentralized conflict resolution possible while maintaining sin‐
287 gle document database semantics.
288 Conflict management continues to work even if multiple disconnected
290 users or agents attempt to resolve the same conflicts. If resolved con‐
291 flicts result in more conflicts, the system accommodates them in the
292 same manner, determining the same winner on each machine and maintain‐
293 ing single document semantics.
294 SEE ALSO:
296 replication/conflicts
297 Applications
299 Using just the basic replication model, many traditionally single
300 server database applications can be made distributed with almost no
301 extra work. CouchDB replication is designed to be immediately useful
302 for basic database applications, while also being extendable for more
303 elaborate and full-featured uses.
304 With very little database work, it is possible to build a distributed
306 document management application with granular security and full revi‐
307 sion histories. Updates to documents can be implemented to exploit
308 incremental field and blob replication, where replicated updates are
309 nearly as efficient and incremental as the actual edit differences
310 (“diffs”).
311 Implementation
313 CouchDB is built on the Erlang OTP platform, a functional, concurrent
314 programming language and development platform. Erlang was developed for
315 real-time telecom applications with an extreme emphasis on reliability
316 and availability.
317 Both in syntax and semantics, Erlang is very different from conven‐
319 tional programming languages like C or Java. Erlang uses lightweight
320 “processes” and message passing for concurrency, it has no shared state
321 threading and all data is immutable. The robust, concurrent nature of
322 Erlang is ideal for a database server.
323 CouchDB is designed for lock-free concurrency, in the conceptual model
325 and the actual Erlang implementation. Reducing bottlenecks and avoiding
326 locks keeps the entire system working predictably under heavy loads.
327 CouchDB can accommodate many clients replicating changes, opening and
328 updating documents, and querying views whose indexes are simultaneously
329 being refreshed for other clients, without needing locks.
330 For higher availability and more concurrent users, CouchDB is designed
332 for “shared nothing” clustering. In a “shared nothing” cluster, each
333 machine is independent and replicates data with its cluster mates,
334 allowing individual server failures with zero downtime. And because
335 consistency scans and fix-ups aren’t needed on restart, if the entire
336 cluster fails – due to a power outage in a datacenter, for example –
337 the entire CouchDB distributed system becomes immediately available
338 after a restart.
339 CouchDB is built from the start with a consistent vision of a distrib‐
341 uted document database system. Unlike cumbersome attempts to bolt dis‐
342 tributed features on top of the same legacy models and databases, it is
343 the result of careful ground-up design, engineering and integration.
344 The document, view, security and replication models, the special pur‐
345 pose query language, the efficient and robust disk layout and the con‐
346 current and reliable nature of the Erlang platform are all carefully
347 integrated for a reliable and efficient system.
348 Why CouchDB?
350 Apache CouchDB is one of a new breed of database management systems.
351 This topic explains why there’s a need for new systems as well as the
352 motivations behind building CouchDB.
353 As CouchDB developers, we’re naturally very excited to be using
355 CouchDB. In this topic we’ll share with you the reasons for our enthu‐
356 siasm. We’ll show you how CouchDB’s schema-free document model is a
357 better fit for common applications, how the built-in query engine is a
358 powerful way to use and process your data, and how CouchDB’s design
359 lends itself to modularization and scalability.
360 Relax
362 If there’s one word to describe CouchDB, it is relax. It is the byline
363 to CouchDB’s official logo and when you start CouchDB, you see:
364 Apache CouchDB has started. Time to relax.
366 Why is relaxation important? Developer productivity roughly doubled in
368 the last five years. The chief reason for the boost is more powerful
369 tools that are easier to use. Take Ruby on Rails as an example. It is
370 an infinitely complex framework, but it’s easy to get started with.
371 Rails is a success story because of the core design focus on ease of
372 use. This is one reason why CouchDB is relaxing: learning CouchDB and
373 understanding its core concepts should feel natural to most everybody
374 who has been doing any work on the Web. And it is still pretty easy to
375 explain to non-technical people.
376 Getting out of the way when creative people try to build specialized
378 solutions is in itself a core feature and one thing that CouchDB aims
379 to get right. We found existing tools too cumbersome to work with dur‐
380 ing development or in production, and decided to focus on making
381 CouchDB easy, even a pleasure, to use.
382 Another area of relaxation for CouchDB users is the production setting.
384 If you have a live running application, CouchDB again goes out of its
385 way to avoid troubling you. Its internal architecture is fault-toler‐
386 ant, and failures occur in a controlled environment and are dealt with
387 gracefully. Single problems do not cascade through an entire server
388 system but stay isolated in single requests.
389 CouchDB’s core concepts are simple (yet powerful) and well understood.
391 Operations teams (if you have a team; otherwise, that’s you) do not
392 have to fear random behavior and untraceable errors. If anything should
393 go wrong, you can easily find out what the problem is, but these situa‐
394 tions are rare.
395 CouchDB is also designed to handle varying traffic gracefully. For
397 instance, if a website is experiencing a sudden spike in traffic,
398 CouchDB will generally absorb a lot of concurrent requests without
399 falling over. It may take a little more time for each request, but they
400 all get answered. When the spike is over, CouchDB will work with regu‐
401 lar speed again.
402 The third area of relaxation is growing and shrinking the underlying
404 hardware of your application. This is commonly referred to as scaling.
405 CouchDB enforces a set of limits on the programmer. On first look,
406 CouchDB might seem inflexible, but some features are left out by design
407 for the simple reason that if CouchDB supported them, it would allow a
408 programmer to create applications that couldn’t deal with scaling up or
409 down.
410 NOTE:
412 CouchDB doesn’t let you do things that would get you in trouble
413 later on. This sometimes means you’ll have to unlearn best prac‐
414 tices you might have picked up in your current or past work.
415 A Different Way to Model Your Data
417 We believe that CouchDB will drastically change the way you build docu‐
418 ment-based applications. CouchDB combines an intuitive document storage
419 model with a powerful query engine in a way that’s so simple you’ll
420 probably be tempted to ask, “Why has no one built something like this
421 before?”
422 Django may be built for the Web, but CouchDB is built of the Web.
423 I’ve never seen software that so completely embraces the philoso‐
424 phies behind HTTP. CouchDB makes Django look old-school in the same
425 way that Django makes ASP look outdated. — Jacob Kaplan-Moss,
426 Django developer
427 CouchDB’s design borrows heavily from web architecture and the concepts
429 of resources, methods, and representations. It augments this with pow‐
430 erful ways to query, map, combine, and filter your data. Add fault tol‐
431 erance, extreme scalability, and incremental replication, and CouchDB
432 defines a sweet spot for document databases.
433 A Better Fit for Common Applications
435 We write software to improve our lives and the lives of others. Usually
436 this involves taking some mundane information such as contacts,
437 invoices, or receipts and manipulating it using a computer application.
438 CouchDB is a great fit for common applications like this because it
439 embraces the natural idea of evolving, self-contained documents as the
440 very core of its data model.
441 Self-Contained Data
443 An invoice contains all the pertinent information about a single trans‐
444 action the seller, the buyer, the date, and a list of the items or ser‐
445 vices sold. As shown in Figure 1. Self-contained documents, there’s no
446 abstract reference on this piece of paper that points to some other
447 piece of paper with the seller’s name and address. Accountants appreci‐
448 ate the simplicity of having everything in one place. And given the
449 choice, programmers appreciate that, too.
450 [image: Self-contained documents] [image] Figure 1. Self-contained
451 documents.UNINDENT
452 Yet using references is exactly how we model our data in a relational
454 database! Each invoice is stored in a table as a row that refers to
455 other rows in other tables one row for seller information, one for
456 the buyer, one row for each item billed, and more rows still to
457 describe the item details, manufacturer details, and so on and so
458 forth.
459 This isn’t meant as a detraction of the relational model, which is
461 widely applicable and extremely useful for a number of reasons. Hope‐
462 fully, though, it illustrates the point that sometimes your model may
463 not “fit” your data in the way it occurs in the real world.
464 Let’s take a look at the humble contact database to illustrate a dif‐
466 ferent way of modeling data, one that more closely “fits” its
467 real-world counterpart – a pile of business cards. Much like our
468 invoice example, a business card contains all the important informa‐
469 tion, right there on the cardstock. We call this “self-contained”
470 data, and it’s an important concept in understanding document data‐
471 bases like CouchDB.
472 Syntax and Semantics
474 Most business cards contain roughly the same information – someone’s
475 identity, an affiliation, and some contact information. While the exact
476 form of this information can vary between business cards, the general
477 information being conveyed remains the same, and we’re easily able to
478 recognize it as a business card. In this sense, we can describe a busi‐
479 ness card as a real-world document.
480 Jan’s business card might contain a phone number but no fax number,
482 whereas J. Chris’s business card contains both a phone and a fax num‐
483 ber. Jan does not have to make his lack of a fax machine explicit by
484 writing something as ridiculous as “Fax: None” on the business card.
485 Instead, simply omitting a fax number implies that he doesn’t have one.
486 We can see that real-world documents of the same type, such as business
488 cards, tend to be very similar in semantics – the sort of information
489 they carry, but can vary hugely in syntax, or how that information is
490 structured. As human beings, we’re naturally comfortable dealing with
491 this kind of variation.
492 While a traditional relational database requires you to model your data
494 up front, CouchDB’s schema-free design unburdens you with a powerful
495 way to aggregate your data after the fact, just like we do with
496 real-world documents. We’ll look in depth at how to design applications
497 with this underlying storage paradigm.
498 Building Blocks for Larger Systems
500 CouchDB is a storage system useful on its own. You can build many
501 applications with the tools CouchDB gives you. But CouchDB is designed
502 with a bigger picture in mind. Its components can be used as building
503 blocks that solve storage problems in slightly different ways for
504 larger and more complex systems.
505 Whether you need a system that’s crazy fast but isn’t too concerned
507 with reliability (think logging), or one that guarantees storage in two
508 or more physically separated locations for reliability, but you’re
509 willing to take a performance hit, CouchDB lets you build these sys‐
510 tems.
511 There are a multitude of knobs you could turn to make a system work
513 better in one area, but you’ll affect another area when doing so. One
514 example would be the CAP theorem discussed in intro/consistency. To
515 give you an idea of other things that affect storage systems, see
516 Figure 2 and Figure 3.
517 By reducing latency for a given system (and that is true not only for
519 storage systems), you affect concurrency and throughput capabilities.
520 [image: Throughput, latency, or concurrency] [image] Figure 2.
521 Throughput, latency, or concurrency.UNINDENT
522 [image: Scaling: read requests, write requests, or data] [image] Fig‐
523 ure 3. Scaling: read requests, write requests, or data.UNINDENT
524 When you want to scale out, there are three distinct issues to deal
526 with: scaling read requests, write requests, and data. Orthogonal to
527 all three and to the items shown in Figure 2 and Figure 3 are many
528 more attributes like reliability or simplicity. You can draw many of
529 these graphs that show how different features or attributes pull into
530 different directions and thus shape the system they describe.
531 CouchDB is very flexible and gives you enough building blocks to cre‐
533 ate a system shaped to suit your exact problem. That’s not saying
534 that CouchDB can be bent to solve any problem – CouchDB is no silver
535 bullet – but in the area of data storage, it can get you a long way.
536 CouchDB Replication
538 CouchDB replication is one of these building blocks. Its fundamental
539 function is to synchronize two or more CouchDB databases. This may
540 sound simple, but the simplicity is key to allowing replication to
541 solve a number of problems: reliably synchronize databases between mul‐
542 tiple machines for redundant data storage; distribute data to a cluster
543 of CouchDB instances that share a subset of the total number of
544 requests that hit the cluster (load balancing); and distribute data
545 between physically distant locations, such as one office in New York
546 and another in Tokyo.
547 CouchDB replication uses the same REST API all clients use. HTTP is
549 ubiquitous and well understood. Replication works incrementally; that
550 is, if during replication anything goes wrong, like dropping your net‐
551 work connection, it will pick up where it left off the next time it
552 runs. It also only transfers data that is needed to synchronize data‐
553 bases.
554 A core assumption CouchDB makes is that things can go wrong, like net‐
556 work connection troubles, and it is designed for graceful error recov‐
557 ery instead of assuming all will be well. The replication system’s
558 incremental design shows that best. The ideas behind “things that can
559 go wrong” are embodied in the Fallacies of Distributed Computing:
560 · The network is reliable.
562 · Latency is zero.
564 · Bandwidth is infinite.
566 · The network is secure.
568 · Topology doesn’t change.
570 · There is one administrator.
572 · Transport cost is zero.
574 · The network is homogeneous.
576 Existing tools often try to hide the fact that there is a network and
578 that any or all of the previous conditions don’t exist for a particular
579 system. This usually results in fatal error scenarios when something
580 finally goes wrong. In contrast, CouchDB doesn’t try to hide the net‐
581 work; it just handles errors gracefully and lets you know when actions
582 on your end are required.
583 Local Data Is King
585 CouchDB takes quite a few lessons learned from the Web, but there is
586 one thing that could be improved about the Web: latency. Whenever you
587 have to wait for an application to respond or a website to render, you
588 almost always wait for a network connection that isn’t as fast as you
589 want it at that point. Waiting a few seconds instead of milliseconds
590 greatly affects user experience and thus user satisfaction.
591 What do you do when you are offline? This happens all the time – your
593 DSL or cable provider has issues, or your iPhone, G1, or Blackberry has
594 no bars, and no connectivity means no way to get to your data.
595 CouchDB can solve this scenario as well, and this is where scaling is
597 important again. This time it is scaling down. Imagine CouchDB
598 installed on phones and other mobile devices that can synchronize data
599 with centrally hosted CouchDBs when they are on a network. The synchro‐
600 nization is not bound by user interface constraints like sub-second
601 response times. It is easier to tune for high bandwidth and higher
602 latency than for low bandwidth and very low latency. Mobile applica‐
603 tions can then use the local CouchDB to fetch data, and since no remote
604 networking is required for that, latency is low by default.
605 Can you really use CouchDB on a phone? Erlang, CouchDB’s implementation
607 language has been designed to run on embedded devices magnitudes
608 smaller and less powerful than today’s phones.
609 Wrapping Up
611 The next document intro/consistency further explores the distributed
612 nature of CouchDB. We should have given you enough bites to whet your
613 interest. Let’s go!
614 Eventual Consistency
616 In the previous document intro/why, we saw that CouchDB’s flexibility
617 allows us to evolve our data as our applications grow and change. In
618 this topic, we’ll explore how working “with the grain” of CouchDB pro‐
619 motes simplicity in our applications and helps us naturally build scal‐
620 able, distributed systems.
621 Working with the Grain
623 A distributed system is a system that operates robustly over a wide
624 network. A particular feature of network computing is that network
625 links can potentially disappear, and there are plenty of strategies for
626 managing this type of network segmentation. CouchDB differs from others
627 by accepting eventual consistency, as opposed to putting absolute con‐
628 sistency ahead of raw availability, like RDBMS or Paxos. What these
629 systems have in common is an awareness that data acts differently when
630 many people are accessing it simultaneously. Their approaches differ
631 when it comes to which aspects of consistency, availability, or parti‐
632 tion tolerance they prioritize.
633 Engineering distributed systems is tricky. Many of the caveats and
635 “gotchas” you will face over time aren’t immediately obvious. We don’t
636 have all the solutions, and CouchDB isn’t a panacea, but when you work
637 with CouchDB’s grain rather than against it, the path of least resis‐
638 tance leads you to naturally scalable applications.
639 Of course, building a distributed system is only the beginning. A web‐
641 site with a database that is available only half the time is next to
642 worthless. Unfortunately, the traditional relational database approach
643 to consistency makes it very easy for application programmers to rely
644 on global state, global clocks, and other high availability no-nos,
645 without even realizing that they’re doing so. Before examining how
646 CouchDB promotes scalability, we’ll look at the constraints faced by a
647 distributed system. After we’ve seen the problems that arise when parts
648 of your application can’t rely on being in constant contact with each
649 other, we’ll see that CouchDB provides an intuitive and useful way for
650 modeling applications around high availability.
651 The CAP Theorem
653 The CAP theorem describes a few different strategies for distributing
654 application logic across networks. CouchDB’s solution uses replication
655 to propagate application changes across participating nodes. This is a
656 fundamentally different approach from consensus algorithms and rela‐
657 tional databases, which operate at different intersections of consis‐
658 tency, availability, and partition tolerance.
659 The CAP theorem, shown in Figure 1. The CAP theorem, identifies three
661 distinct concerns:
662 · Consistency: All database clients see the same data, even with con‐
664 current updates.
665 · Availability: All database clients are able to access some version of
667 the data.
668 · Partition tolerance: The database can be split over multiple servers.
670 Pick two.
672 [image: The CAP theorem] [image] Figure 1. The CAP theorem.UNINDENT
673 When a system grows large enough that a single database node is
675 unable to handle the load placed on it, a sensible solution is to add
676 more servers. When we add nodes, we have to start thinking about how
677 to partition data between them. Do we have a few databases that share
678 exactly the same data? Do we put different sets of data on different
679 database servers? Do we let only certain database servers write data
680 and let others handle the reads?
681 Regardless of which approach we take, the one problem we’ll keep
683 bumping into is that of keeping all these database servers in sync.
684 If you write some information to one node, how are you going to make
685 sure that a read request to another database server reflects this
686 newest information? These events might be milliseconds apart. Even
687 with a modest collection of database servers, this problem can become
688 extremely complex.
689 When it’s absolutely critical that all clients see a consistent view
691 of the database, the users of one node will have to wait for any
692 other nodes to come into agreement before being able to read or write
693 to the database. In this instance, we see that availability takes a
694 backseat to consistency. However, there are situations where avail‐
695 ability trumps consistency:
696 Each node in a system should be able to make decisions purely based
697 on local state. If you need to do something under high load with
698 failures occurring and you need to reach agreement, you’re lost. If
699 you’re concerned about scalability, any algorithm that forces you to
700 run agreement will eventually become your bottleneck. Take that as a
701 given. — Werner Vogels, Amazon CTO and Vice President
702 If availability is a priority, we can let clients write data to one
704 node of the database without waiting for other nodes to come into
705 agreement. If the database knows how to take care of reconciling these
706 operations between nodes, we achieve a sort of “eventual consistency”
707 in exchange for high availability. This is a surprisingly applicable
708 trade-off for many applications.
709 Unlike traditional relational databases, where each action performed is
711 necessarily subject to database-wide consistency checks, CouchDB makes
712 it really simple to build applications that sacrifice immediate consis‐
713 tency for the huge performance improvements that come with simple dis‐
714 tribution.
715 Local Consistency
717 Before we attempt to understand how CouchDB operates in a cluster, it’s
718 important that we understand the inner workings of a single CouchDB
719 node. The CouchDB API is designed to provide a convenient but thin
720 wrapper around the database core. By taking a closer look at the struc‐
721 ture of the database core, we’ll have a better understanding of the API
722 that surrounds it.
723 The Key to Your Data
725 At the heart of CouchDB is a powerful B-tree storage engine. A B-tree
726 is a sorted data structure that allows for searches, insertions, and
727 deletions in logarithmic time. As Figure 2. Anatomy of a view request
728 illustrates, CouchDB uses this B-tree storage engine for all internal
729 data, documents, and views. If we understand one, we will understand
730 them all.
731 [image: Anatomy of a view request] [image] Figure 2. Anatomy of a
732 view request.UNINDENT
733 CouchDB uses MapReduce to compute the results of a view. MapReduce
735 makes use of two functions, “map” and “reduce”, which are applied to
736 each document in isolation. Being able to isolate these operations
737 means that view computation lends itself to parallel and incremental
738 computation. More important, because these functions produce
739 key/value pairs, CouchDB is able to insert them into the B-tree stor‐
740 age engine, sorted by key. Lookups by key, or key range, are
741 extremely efficient operations with a B-tree, described in big O
742 notation as O(log N) and O(log N + K), respectively.
743 In CouchDB, we access documents and view results by key or key range.
745 This is a direct mapping to the underlying operations performed on
746 CouchDB’s B-tree storage engine. Along with document inserts and
747 updates, this direct mapping is the reason we describe CouchDB’s API
748 as being a thin wrapper around the database core.
749 Being able to access results by key alone is a very important
751 restriction because it allows us to make huge performance gains. As
752 well as the massive speed improvements, we can partition our data
753 over multiple nodes, without affecting our ability to query each node
754 in isolation. BigTable, Hadoop, SimpleDB, and memcached restrict
755 object lookups by key for exactly these reasons.
756 No Locking
758 A table in a relational database is a single data structure. If you
759 want to modify a table – say, update a row – the database system must
760 ensure that nobody else is trying to update that row and that nobody
761 can read from that row while it is being updated. The common way to
762 handle this uses what’s known as a lock. If multiple clients want to
763 access a table, the first client gets the lock, making everybody else
764 wait. When the first client’s request is processed, the next client is
765 given access while everybody else waits, and so on. This serial execu‐
766 tion of requests, even when they arrived in parallel, wastes a signifi‐
767 cant amount of your server’s processing power. Under high load, a
768 relational database can spend more time figuring out who is allowed to
769 do what, and in which order, than it does doing any actual work.
770 NOTE:
772 Modern relational databases avoid locks by implementing MVCC under
773 the hood, but hide it from the end user, requiring them to coordi‐
774 nate concurrent changes of single rows or fields.
775 Instead of locks, CouchDB uses Multi-Version Concurrency Control (MVCC)
777 to manage concurrent access to the database. Figure 3. MVCC means no
778 locking illustrates the differences between MVCC and traditional lock‐
779 ing mechanisms. MVCC means that CouchDB can run at full speed, all the
780 time, even under high load. Requests are run in parallel, making excel‐
781 lent use of every last drop of processing power your server has to
782 offer.
783 [image: MVCC means no locking] [image] Figure 3. MVCC means no lock‐
784 ing.UNINDENT
785 Documents in CouchDB are versioned, much like they would be in a reg‐
787 ular version control system such as Subversion. If you want to change
788 a value in a document, you create an entire new version of that docu‐
789 ment and save it over the old one. After doing this, you end up with
790 two versions of the same document, one old and one new.
791 How does this offer an improvement over locks? Consider a set of
793 requests wanting to access a document. The first request reads the
794 document. While this is being processed, a second request changes
795 the document. Since the second request includes a completely new
796 version of the document, CouchDB can simply append it to the database
797 without having to wait for the read request to finish.
798 When a third request wants to read the same document, CouchDB will
800 point it to the new version that has just been written. During this
801 whole process, the first request could still be reading the original
802 version.
803 A read request will always see the most recent snapshot of your data‐
805 base at the time of the beginning of the request.
806 Validation
808 As application developers, we have to think about what sort of input we
809 should accept and what we should reject. The expressive power to do
810 this type of validation over complex data within a traditional rela‐
811 tional database leaves a lot to be desired. Fortunately, CouchDB pro‐
812 vides a powerful way to perform per-document validation from within the
813 database.
814 CouchDB can validate documents using JavaScript functions similar to
816 those used for MapReduce. Each time you try to modify a document,
817 CouchDB will pass the validation function a copy of the existing docu‐
818 ment, a copy of the new document, and a collection of additional infor‐
819 mation, such as user authentication details. The validation function
820 now has the opportunity to approve or deny the update.
821 By working with the grain and letting CouchDB do this for us, we save
823 ourselves a tremendous amount of CPU cycles that would otherwise have
824 been spent serializing object graphs from SQL, converting them into
825 domain objects, and using those objects to do application-level valida‐
826 tion.
827 Distributed Consistency
829 Maintaining consistency within a single database node is relatively
830 easy for most databases. The real problems start to surface when you
831 try to maintain consistency between multiple database servers. If a
832 client makes a write operation on server A, how do we make sure that
833 this is consistent with server B, or C, or D? For relational databases,
834 this is a very complex problem with entire books devoted to its solu‐
835 tion. You could use multi-master, single-master, partitioning, shard‐
836 ing, write-through caches, and all sorts of other complex techniques.
837 Incremental Replication
839 CouchDB’s operations take place within the context of a single docu‐
840 ment. As CouchDB achieves eventual consistency between multiple data‐
841 bases by using incremental replication you no longer have to worry
842 about your database servers being able to stay in constant communica‐
843 tion. Incremental replication is a process where document changes are
844 periodically copied between servers. We are able to build what’s known
845 as a shared nothing cluster of databases where each node is independent
846 and self-sufficient, leaving no single point of contention across the
847 system.
848 Need to scale out your CouchDB database cluster? Just throw in another
850 server.
851 As illustrated in Figure 4. Incremental replication between CouchDB
853 nodes, with CouchDB’s incremental replication, you can synchronize your
854 data between any two databases however you like and whenever you like.
855 After replication, each database is able to work independently.
856 You could use this feature to synchronize database servers within a
858 cluster or between data centers using a job scheduler such as cron, or
859 you could use it to synchronize data with your laptop for offline work
860 as you travel. Each database can be used in the usual fashion, and
861 changes between databases can be synchronized later in both directions.
862 [image: Incremental replication between CouchDB nodes] [image] Figure
863 4. Incremental replication between CouchDB nodes.UNINDENT
864 What happens when you change the same document in two different data‐
866 bases and want to synchronize these with each other? CouchDB’s repli‐
867 cation system comes with automatic conflict detection and resolution.
868 When CouchDB detects that a document has been changed in both data‐
869 bases, it flags this document as being in conflict, much like they
870 would be in a regular version control system.
871 This isn’t as troublesome as it might first sound. When two versions
873 of a document conflict during replication, the winning version is
874 saved as the most recent version in the document’s history. Instead
875 of throwing the losing version away, as you might expect, CouchDB
876 saves this as a previous version in the document’s history, so that
877 you can access it if you need to. This happens automatically and con‐
878 sistently, so both databases will make exactly the same choice.
879 It is up to you to handle conflicts in a way that makes sense for
881 your application. You can leave the chosen document versions in
882 place, revert to the older version, or try to merge the two versions
883 and save the result.
884 Case Study
886 Greg Borenstein, a friend and coworker, built a small library for con‐
887 verting Songbird playlists to JSON objects and decided to store these
888 in CouchDB as part of a backup application. The completed software uses
889 CouchDB’s MVCC and document revisions to ensure that Songbird playlists
890 are backed up robustly between nodes.
891 NOTE:
893 Songbird is a free software media player with an integrated web
894 browser, based on the Mozilla XULRunner platform. Songbird is avail‐
895 able for Microsoft Windows, Apple Mac OS X, Solaris, and Linux.
896 Let’s examine the workflow of the Songbird backup application, first as
898 a user backing up from a single computer, and then using Songbird to
899 synchronize playlists between multiple computers. We’ll see how docu‐
900 ment revisions turn what could have been a hairy problem into something
901 that just works.
902 The first time we use this backup application, we feed our playlists to
904 the application and initiate a backup. Each playlist is converted to a
905 JSON object and handed to a CouchDB database. As illustrated in Figure
906 5. Backing up to a single database, CouchDB hands back the document ID
907 and revision of each playlist as it’s saved to the database.
908 [image: Backing up to a single database] [image] Figure 5. Backing up
909 to a single database.UNINDENT
910 After a few days, we find that our playlists have been updated and we
912 want to back up our changes. After we have fed our playlists to the
913 backup application, it fetches the latest versions from CouchDB,
914 along with the corresponding document revisions. When the application
915 hands back the new playlist document, CouchDB requires that the docu‐
916 ment revision is included in the request.
917 CouchDB then makes sure that the document revision handed to it in
919 the request matches the current revision held in the database.
920 Because CouchDB updates the revision with every modification, if
921 these two are out of sync it suggests that someone else has made
922 changes to the document between the time we requested it from the
923 database and the time we sent our updates. Making changes to a docu‐
924 ment after someone else has modified it without first inspecting
925 those changes is usually a bad idea.
926 Forcing clients to hand back the correct document revision is the
928 heart of CouchDB’s optimistic concurrency.
929 We have a laptop we want to keep synchronized with our desktop com‐
931 puter. With all our playlists on our desktop, the first step is to
932 “restore from backup” onto our laptop. This is the first time we’ve
933 done this, so afterward our laptop should hold an exact replica of
934 our desktop playlist collection.
935 After editing our Argentine Tango playlist on our laptop to add a few
937 new songs we’ve purchased, we want to save our changes. The backup
938 application replaces the playlist document in our laptop CouchDB
939 database and a new document revision is generated. A few days later,
940 we remember our new songs and want to copy the playlist across to our
941 desktop computer. As illustrated in Figure 6. Synchronizing between
942 two databases, the backup application copies the new document and the
943 new revision to the desktop CouchDB database. Both CouchDB databases
944 now have the same document revision.
945 [image: Synchronizing between two databases] [image] Figure 6. Syn‐
946 chronizing between two databases.UNINDENT
947 Because CouchDB tracks document revisions, it ensures that updates
949 like these will work only if they are based on current information.
950 If we had made modifications to the playlist backups between synchro‐
951 nization, things wouldn’t go as smoothly.
952 We back up some changes on our laptop and forget to synchronize. A
954 few days later, we’re editing playlists on our desktop computer, make
955 a backup, and want to synchronize this to our laptop. As illustrated
956 in Figure 7. Synchronization conflicts between two databases, when
957 our backup application tries to replicate between the two databases,
958 CouchDB sees that the changes being sent from our desktop computer
959 are modifications of out-of-date documents and helpfully informs us
960 that there has been a conflict.
961 Recovering from this error is easy to accomplish from an application
963 perspective. Just download CouchDB’s version of the playlist and pro‐
964 vide an opportunity to merge the changes or save local modifications
965 into a new playlist.
966 [image: Synchronization conflicts between two databases] [image] Fig‐
967 ure 7. Synchronization conflicts between two databases.UNINDENT
968 Wrapping Up
970 CouchDB’s design borrows heavily from web architecture and the lessons
971 learned deploying massively distributed systems on that architecture.
972 By understanding why this architecture works the way it does, and by
973 learning to spot which parts of your application can be easily distrib‐
974 uted and which parts cannot, you’ll enhance your ability to design dis‐
975 tributed and scalable applications, with CouchDB or without it.
976 We’ve covered the main issues surrounding CouchDB’s consistency model
978 and hinted at some of the benefits to be had when you work with CouchDB
979 and not against it. But enough theory – let’s get up and running and
980 see what all the fuss is about!
981 cURL: Your Command Line Friend
983 The curl utility is a command line tool available on Unix, Linux, Mac
984 OS X, Windows, and many other platforms. curl provides easy access to
985 the HTTP protocol (among others) directly from the command line and is
986 therefore an ideal way of interacting with CouchDB over the HTTP REST
987 API.
988 For simple GET requests you can supply the URL of the request. For
990 example, to get the database information:
991 shell> curl http://admin:password@127.0.0.1:5984
993 This returns the database information (formatted in the output below
995 for clarity):
996 {
998 "couchdb": "Welcome",
999 "version": "3.0.0",
1000 "git_sha": "83bdcf693",
1001 "uuid": "56f16e7c93ff4a2dc20eb6acc7000b71",
1002 "features": [
1003 "access-ready",
1004 "partitioned",
1005 "pluggable-storage-engines",
1006 "reshard",
1007 "scheduler"
1008 ],
1009 "vendor": {
1010 "name": "The Apache Software Foundation"
1011 }
1012 }
1013 NOTE:
1015 For some URLs, especially those that include special characters such
1016 as ampersand, exclamation mark, or question mark, you should quote
1017 the URL you are specifying on the command line. For example:
1018 shell> curl 'http://couchdb:5984/_uuids?count=5'
1020 NOTE:
1022 On Microsoft Windows, use double-quotes anywhere you see sin‐
1023 gle-quotes in the following examples. Use doubled double-quotes (“”)
1024 anywhere you see single quotes. For example, if you see:
1025 shell> curl -X PUT 'http:/127.0.0.1:5984/demo/doc' -d '{"motto": "I love gnomes"}'
1027 you should replace it with:
1029 shell> curl -X PUT "http://127.0.0.1:5984/demo/doc" -d "{""motto"": ""I love gnomes""}"
1031 If you prefer, ^" and \" may be used to escape the double-quote
1033 character in quoted strings instead.
1034 You can explicitly set the HTTP command using the -X command line
1036 option. For example, when creating a database, you set the name of the
1037 database in the URL you send using a PUT request:
1038 shell> curl -X PUT http://user:pass@127.0.0.1:5984/demo
1040 {"ok":true}
1041 But to obtain the database information you use a GET request (with the
1043 return information formatted for clarity):
1044 shell> curl -X GET http://user:pass@127.0.0.1:5984/demo
1046 {
1047 "compact_running" : false,
1048 "doc_count" : 0,
1049 "db_name" : "demo",
1050 "purge_seq" : 0,
1051 "committed_update_seq" : 0,
1052 "doc_del_count" : 0,
1053 "disk_format_version" : 5,
1054 "update_seq" : 0,
1055 "instance_start_time" : "0",
1056 "disk_size" : 79
1057 }
1058 For certain operations, you must specify the content type of request,
1060 which you do by specifying the Content-Type header using the -H com‐
1061 mand-line option:
1062 shell> curl -H 'Content-Type: application/json' http://127.0.0.1:5984/_uuids
1064 You can also submit ‘payload’ data, that is, data in the body of the
1066 HTTP request using the -d option. This is useful if you need to submit
1067 JSON structures, for example document data, as part of the request. For
1068 example, to submit a simple document to the demo database:
1069 shell> curl -H 'Content-Type: application/json' \
1071 -X POST http://user:pass@127.0.0.1:5984/demo \
1072 -d '{"company": "Example, Inc."}'
1073 {"ok":true,"id":"8843faaf0b831d364278331bc3001bd8",
1074 "rev":"1-33b9fbce46930280dab37d672bbc8bb9"}
1075 In the above example, the argument after the -d option is the JSON of
1077 the document we want to submit.
1078 The document can be accessed by using the automatically generated docu‐
1080 ment ID that was returned:
1081 shell> curl -X GET http://user:pass@127.0.0.1:5984/demo/8843faaf0b831d364278331bc3001bd8
1083 {"_id":"8843faaf0b831d364278331bc3001bd8",
1084 "_rev":"1-33b9fbce46930280dab37d672bbc8bb9",
1085 "company":"Example, Inc."}
1086 The API samples in the api/basics show the HTTP command, URL and any
1088 payload information that needs to be submitted (and the expected return
1089 value). All of these examples can be reproduced using curl with the
1090 command-line examples shown above.
1091 Security
1093 In this document, we’ll look at the basic security mechanisms in
1094 CouchDB: Basic Authentication and Cookie Authentication. This is how
1095 CouchDB handles users and protects their credentials.
1096 Authentication
1098 CouchDB has the idea of an admin user (e.g. an administrator, a super
1099 user, or root) that is allowed to do anything to a CouchDB installa‐
1100 tion. By default, one admin user must be created for CouchDB to start
1101 up successfully.
1102 CouchDB also defines a set of requests that only admin users are
1104 allowed to do. If you have defined one or more specific admin users,
1105 CouchDB will ask for identification for certain requests:
1106 · Creating a database (PUT /database)
1108 · Deleting a database (DELETE /database)
1110 · Setup a database security (PUT /database/_security)
1112 · Creating a design document (PUT /database/_design/app)
1114 · Updating a design document (PUT /database/_design/app?rev=1-4E2)
1116 · Deleting a design document (DELETE /database/_design/app?rev=2-6A7)
1118 · Triggering compaction (POST /database/_compact)
1120 · Reading the task status list (GET /_active_tasks)
1122 · Restarting the server on a given node (POST
1124 /_node/{node-name}/_restart </_restart>)
1125 · Reading the active configuration (GET /_node/{node-name}/_config
1127 </_config>)
1128 · Updating the active configuration (PUT /_node/{node-name}/_con‐
1130 fig/section/key </_config/{section}/{key}>)
1131 Creating a New Admin User
1133 If your installation process did not set up an admin user, you will
1134 have to add one to the configuration file by hand and restart CouchDB
1135 first. For the purposes of this example, we’ll create a default admin
1136 user with the password password.
1137 WARNING:
1139 Don’t just type in the following without thinking! Pick a good name
1140 for your administrator user that isn’t easily guessable, and pick a
1141 secure password.
1142 To the end of your etc/local.ini file, after the [admins] line, add the
1144 text admin = password, so it looks like this:
1145 [admins]
1147 admin = password
1148 (Don’t worry about the password being in plain text; we’ll come back to
1150 this.)
1151 Now, restart CouchDB using the method appropriate for your operating
1153 system. You should now be able to access CouchDB using your new admin‐
1154 istrator account:
1155 > curl http://admin:password@127.0.0.1:5984/_up
1157 {"status":"ok","seeds":{}}
1158 Great!
1160 Let’s create an admin user through the HTTP API. We’ll call her anna,
1162 and her password is secret. Note the double quotes in the following
1163 code; they are needed to denote a string value for the configuration
1164 API:
1165 > HOST="http://admin:password@127.0.0.1:5984"
1167 > NODENAME="_local"
1168 > curl -X PUT $HOST/_node/$NODENAME/_config/admins/anna -d '"secret"'
1169 ""
1170 As per the _config API’s behavior, we’re getting the previous value for
1172 the config item we just wrote. Since our admin user didn’t exist, we
1173 get an empty string.
1174 Please note that _local serves as an alias for the local node name, so
1176 for all configuration URLs, NODENAME may be set to _local, to interact
1177 with the local node’s configuration.
1178 SEE ALSO:
1180 Node Management
1181 Hashing Passwords
1183 Seeing the plain-text password is scary, isn’t it? No worries, CouchDB
1184 doesn’t show the plain-text password anywhere. It gets hashed right
1185 away. Go ahead and look at your local.ini file now. You’ll see that
1186 CouchDB has rewritten the plain text passwords so they are hashed:
1187 [admins]
1189 admin = -pbkdf2-71c01cb429088ac1a1e95f3482202622dc1e53fe,226701bece4ae0fc9a373a5e02bf5d07,10
1190 anna = -pbkdf2-2d86831c82b440b8887169bd2eebb356821d621b,5e11b9a9228414ab92541beeeacbf125,10
1191 The hash is that big, ugly, long string that starts out with -pbkdf2-.
1193 To compare a plain-text password during authentication with the stored
1195 hash, the hashing algorithm is run and the resulting hash is compared
1196 to the stored hash. The probability of two identical hashes for differ‐
1197 ent passwords is too insignificant to mention (c.f. Bruce Schneier).
1198 Should the stored hash fall into the hands of an attacker, it is, by
1199 current standards, way too inconvenient (i.e., it’d take a lot of money
1200 and time) to find the plain-text password from the hash.
1201 When CouchDB starts up, it reads a set of .ini files with config set‐
1203 tings. It loads these settings into an internal data store (not a data‐
1204 base). The config API lets you read the current configuration as well
1205 as change it and create new entries. CouchDB writes any changes back to
1206 the .ini files.
1207 The .ini files can also be edited by hand when CouchDB is not running.
1209 Instead of creating the admin user as we showed previously, you could
1210 have stopped CouchDB, opened your local.ini, added anna = secret to the
1211 admins, and restarted CouchDB. Upon reading the new line from
1212 local.ini, CouchDB would run the hashing algorithm and write back the
1213 hash to local.ini, replacing the plain-text password — just as it did
1214 for our original admin user. To make sure CouchDB only hashes
1215 plain-text passwords and not an existing hash a second time, it pre‐
1216 fixes the hash with -pbkdf2-, to distinguish between plain-text pass‐
1217 words and PBKDF2 hashed passwords. This means your plain-text password
1218 can’t start with the characters -pbkdf2-, but that’s pretty unlikely to
1219 begin with.
1220 Basic Authentication
1222 CouchDB will not allow us to create new databases unless we give the
1223 correct admin user credentials. Let’s verify:
1224 > HOST="http://127.0.0.1:5984"
1226 > curl -X PUT $HOST/somedatabase
1227 {"error":"unauthorized","reason":"You are not a server admin."}
1228 That looks about right. Now we try again with the correct credentials:
1230 > HOST="http://anna:secret@127.0.0.1:5984"
1232 > curl -X PUT $HOST/somedatabase
1233 {"ok":true}
1234 If you have ever accessed a website or FTP server that was pass‐
1236 word-protected, the username:password@ URL variant should look famil‐
1237 iar.
1238 If you are security conscious, the missing s in http:// will make you
1240 nervous. We’re sending our password to CouchDB in plain text. This is a
1241 bad thing, right? Yes, but consider our scenario: CouchDB listens on
1242 127.0.0.1 on a development box that we’re the sole user of. Who could
1243 possibly sniff our password?
1244 If you are in a production environment, however, you need to recon‐
1246 sider. Will your CouchDB instance communicate over a public network?
1247 Even a LAN shared with other collocation customers is public. There are
1248 multiple ways to secure communication between you or your application
1249 and CouchDB that exceed the scope of this documentation. CouchDB as of
1250 version 1.1.0 comes with SSL built in.
1251 SEE ALSO:
1253 Basic Authentication API Reference
1254 Cookie Authentication
1256 Basic authentication that uses plain-text passwords is nice and conve‐
1257 nient, but not very secure if no extra measures are taken. It is also a
1258 very poor user experience. If you use basic authentication to identify
1259 admins, your application’s users need to deal with an ugly, unstylable
1260 browser modal dialog that says non-professional at work more than any‐
1261 thing else.
1262 To remedy some of these concerns, CouchDB supports cookie authentica‐
1264 tion. With cookie authentication your application doesn’t have to
1265 include the ugly login dialog that the users’ browsers come with. You
1266 can use a regular HTML form to submit logins to CouchDB. Upon receipt,
1267 CouchDB will generate a one-time token that the client can use in its
1268 next request to CouchDB. When CouchDB sees the token in a subsequent
1269 request, it will authenticate the user based on the token without the
1270 need to see the password again. By default, a token is valid for 10
1271 minutes.
1272 To obtain the first token and thus authenticate a user for the first
1274 time, the username and password must be sent to the _session API. The
1275 API is smart enough to decode HTML form submissions, so you don’t have
1276 to resort to any smarts in your application.
1277 If you are not using HTML forms to log in, you need to send an HTTP
1279 request that looks as if an HTML form generated it. Luckily, this is
1280 super simple:
1281 > HOST="http://127.0.0.1:5984"
1283 > curl -vX POST $HOST/_session \
1284 -H 'Content-Type:application/x-www-form-urlencoded' \
1285 -d 'name=anna&password=secret'
1286 CouchDB replies, and we’ll give you some more detail:
1288 < HTTP/1.1 200 OK
1290 < Set-Cookie: AuthSession=YW5uYTo0QUIzOTdFQjrC4ipN-D-53hw1sJepVzcVxnriEw;
1291 < Version=1; Path=/; HttpOnly
1292 > ...
1293 <
1294 {"ok":true}
1295 A 200 OK response code tells us all is well, a Set-Cookie header
1297 includes the token we can use for the next request, and the standard
1298 JSON response tells us again that the request was successful.
1299 Now we can use this token to make another request as the same user
1301 without sending the username and password again:
1302 > curl -vX PUT $HOST/mydatabase \
1304 --cookie AuthSession=YW5uYTo0QUIzOTdFQjrC4ipN-D-53hw1sJepVzcVxnriEw \
1305 -H "X-CouchDB-WWW-Authenticate: Cookie" \
1306 -H "Content-Type:application/x-www-form-urlencoded"
1307 {"ok":true}
1308 You can keep using this token for 10 minutes by default. After 10 min‐
1310 utes you need to authenticate your user again. The token lifetime can
1311 be configured with the timeout (in seconds) setting in the
1312 couch_httpd_auth configuration section.
1313 SEE ALSO:
1315 Cookie Authentication API Reference
1316 Authentication Database
1318 You may already note that CouchDB administrators are defined within the
1319 config file and are wondering if regular users are also stored there.
1320 No, they are not. CouchDB has a special authentication database, named
1321 _users by default, that stores all registered users as JSON documents.
1322 This special database is a system database. This means that while it
1324 shares the common database API, there are some special security-related
1325 constraints applied. Below is a list of how the authentication database
1326 is different from the other databases.
1327 · Only administrators may browse list of all documents (GET
1329 /_users/_all_docs)
1330 · Only administrators may listen to changes feed (GET /_users/_changes)
1332 · Only administrators may execute design functions like views.
1334 · There is a special design document _auth that cannot be modified
1336 · Every document except the design documents represent registered
1338 CouchDB users and belong to them
1339 · Users may only access (GET /_users/org.couchdb.user:Jan) or modify
1341 (PUT /_users/org.couchdb.user:Jan) documents that they own
1342 These draconian rules are necessary since CouchDB cares about its
1344 users’ personal information and will not disclose it to just anyone.
1345 Often, user documents contain system information like login, password
1346 hash and roles, apart from sensitive personal information like real
1347 name, email, phone, special internal identifications and more. This is
1348 not information that you want to share with the World.
1349 Users Documents
1351 Each CouchDB user is stored in document format. These documents contain
1352 several mandatory fields, that CouchDB needs for authentication:
1353 · _id (string): Document ID. Contains user’s login with special prefix
1355 Why the org.couchdb.user: prefix?
1356 · derived_key (string): PBKDF2 key
1358 · name (string): User’s name aka login. Immutable e.g. you cannot
1360 rename an existing user - you have to create new one
1361 · roles (array of string): List of user roles. CouchDB doesn’t provide
1363 any built-in roles, so you’re free to define your own depending on
1364 your needs. However, you cannot set system roles like _admin there.
1365 Also, only administrators may assign roles to users - by default all
1366 users have no roles
1367 · password_sha (string): Hashed password with salt. Used for simple
1369 password_scheme
1370 · password_scheme (string): Password hashing scheme. May be simple or
1372 pbkdf2
1373 · salt (string): Hash salt. Used for simple password_scheme
1375 · type (string): Document type. Constantly has the value user
1377 Additionally, you may specify any custom fields that relate to the tar‐
1379 get user.
1380 Why the org.couchdb.user: prefix?
1382 The reason there is a special prefix before a user’s login name is to
1383 have namespaces that users belong to. This prefix is designed to pre‐
1384 vent replication conflicts when you try merging two or more _user data‐
1385 bases.
1386 For current CouchDB releases, all users belong to the same
1388 org.couchdb.user namespace and this cannot be changed. This may be
1389 changed in future releases.
1390 Creating a New User
1392 Creating a new user is a very trivial operation. You just need to do a
1393 PUT request with the user’s data to CouchDB. Let’s create a user with
1394 login jan and password apple:
1395 curl -X PUT http://localhost:5984/_users/org.couchdb.user:jan \
1397 -H "Accept: application/json" \
1398 -H "Content-Type: application/json" \
1399 -d '{"name": "jan", "password": "apple", "roles": [], "type": "user"}'
1400 This curl command will produce the following HTTP request:
1402 PUT /_users/org.couchdb.user:jan HTTP/1.1
1404 Accept: application/json
1405 Content-Length: 62
1406 Content-Type: application/json
1407 Host: localhost:5984
1408 User-Agent: curl/7.31.0
1409 And CouchDB responds with:
1411 HTTP/1.1 201 Created
1413 Cache-Control: must-revalidate
1414 Content-Length: 83
1415 Content-Type: application/json
1416 Date: Fri, 27 Sep 2013 07:33:28 GMT
1417 ETag: "1-e0ebfb84005b920488fc7a8cc5470cc0"
1418 Location: http://localhost:5984/_users/org.couchdb.user:jan
1419 Server: CouchDB (Erlang OTP)
1420 {"ok":true,"id":"org.couchdb.user:jan","rev":"1-e0ebfb84005b920488fc7a8cc5470cc0"}
1422 The document was successfully created! The user jan should now exist in
1424 our database. Let’s check if this is true:
1425 curl -X POST http://localhost:5984/_session -d 'name=jan&password=apple'
1427 CouchDB should respond with:
1429 {"ok":true,"name":"jan","roles":[]}
1431 This means that the username was recognized and the password’s hash
1433 matches with the stored one. If we specify an incorrect login and/or
1434 password, CouchDB will notify us with the following error message:
1435 {"error":"unauthorized","reason":"Name or password is incorrect."}
1437 Password Changing
1439 Let’s define what is password changing from the point of view of
1440 CouchDB and the authentication database. Since “users” are “documents”,
1441 this operation is just updating the document with a special field pass‐
1442 word which contains the plain text password. Scared? No need to be. The
1443 authentication database has a special internal hook on document update
1444 which looks for this field and replaces it with the secured hash
1445 depending on the chosen password_scheme.
1446 Summarizing the above process - we need to get the document’s content,
1448 add the password field with the new password in plain text and then
1449 store the JSON result to the authentication database.
1450 curl -X GET http://localhost:5984/_users/org.couchdb.user:jan
1452 {
1454 "_id": "org.couchdb.user:jan",
1455 "_rev": "1-e0ebfb84005b920488fc7a8cc5470cc0",
1456 "derived_key": "e579375db0e0c6a6fc79cd9e36a36859f71575c3",
1457 "iterations": 10,
1458 "name": "jan",
1459 "password_scheme": "pbkdf2",
1460 "roles": [],
1461 "salt": "1112283cf988a34f124200a050d308a1",
1462 "type": "user"
1463 }
1464 Here is our user’s document. We may strip hashes from the stored docu‐
1466 ment to reduce the amount of posted data:
1467 curl -X PUT http://localhost:5984/_users/org.couchdb.user:jan \
1469 -H "Accept: application/json" \
1470 -H "Content-Type: application/json" \
1471 -H "If-Match: 1-e0ebfb84005b920488fc7a8cc5470cc0" \
1472 -d '{"name":"jan", "roles":[], "type":"user", "password":"orange"}'
1473 {"ok":true,"id":"org.couchdb.user:jan","rev":"2-ed293d3a0ae09f0c624f10538ef33c6f"}
1475 Updated! Now let’s check that the password was really changed:
1477 curl -X POST http://localhost:5984/_session -d 'name=jan&password=apple'
1479 CouchDB should respond with:
1481 {"error":"unauthorized","reason":"Name or password is incorrect."}
1483 Looks like the password apple is wrong, what about orange?
1485 curl -X POST http://localhost:5984/_session -d 'name=jan&password=orange'
1487 CouchDB should respond with:
1489 {"ok":true,"name":"jan","roles":[]}
1491 Hooray! You may wonder why this was so complex - we need to retrieve
1493 user’s document, add a special field to it, and post it back.
1494 NOTE:
1496 There is no password confirmation for API request: you should imple‐
1497 ment it in your application layer.
1498 Users Public Information
1500 New in version 1.4.
1501 Sometimes users want to share some information with the world. For
1504 instance, their contact email to let other users get in touch with
1505 them. To solve this problem, but still keep sensitive and private
1506 information secured, there is a special configuration option pub‐
1507 lic_fields. In this option you may define a comma-separated list of
1508 users document fields that will be publicly available.
1509 Normally, if you request a user document and you’re not an administra‐
1511 tor or the document’s owner, CouchDB will respond with 404 Not Found:
1512 curl http://localhost:5984/_users/org.couchdb.user:robert
1514 {"error":"not_found","reason":"missing"}
1516 This response is constant for both cases when user exists or doesn’t
1518 exist for security reasons.
1519 Now let’s share the field name. First, set up the public_fields config‐
1521 uration option. Remember, that this action requires administrator priv‐
1522 ileges. The next command will prompt you for user admin’s password:
1523 curl -X PUT http://localhost:5984/_node/nonode@nohost/_config/couch_httpd_auth/public_fields \
1525 -H "Content-Type: application/json" \
1526 -d '"name"' \
1527 -u admin
1528 What has changed? Let’s check Robert’s document once again:
1530 curl http://localhost:5984/_users/org.couchdb.user:robert
1532 {"_id":"org.couchdb.user:robert","_rev":"6-869e2d3cbd8b081f9419f190438ecbe7","name":"robert"}
1534 Good news! Now we may read the field name in every user document with‐
1536 out needing to be an administrator. Keep in mind, though, not to pub‐
1537 lish sensitive information, especially without user’s consent!
1538 Authorization
1540 Now that you have a few users who can log in, you probably want to set
1541 up some restrictions on what actions they can perform based on their
1542 identity and their roles. Each database on a CouchDB server can con‐
1543 tain its own set of authorization rules that specify which users are
1544 allowed to read and write documents, create design documents, and
1545 change certain database configuration parameters. The authorization
1546 rules are set up by a server admin and can be modified at any time.
1547 Database authorization rules assign a user into one of two classes:
1549 · members, who are allowed to read all documents and create and modify
1551 any document except for design documents.
1552 · admins, who can read and write all types of documents, modify which
1554 users are members or admins, and set certain per-database configura‐
1555 tion options.
1556 Note that a database admin is not the same as a server admin – the
1558 actions of a database admin are restricted to a specific database.
1559 When a database is first created, there are no members or admins. HTTP
1561 requests that have no authentication credentials or have credentials
1562 for a normal user are treated as members, and those with server admin
1563 credentials are treated as database admins. To change the default per‐
1564 missions, you must create a _security document in the database:
1565 > curl -X PUT http://localhost:5984/mydatabase/_security \
1567 -u anna:secret \
1568 -H "Content-Type: application/json" \
1569 -d '{"admins": { "names": [], "roles": [] }, "members": { "names": ["jan"], "roles": [] } }'
1570 The HTTP request to create the _security document must contain the cre‐
1572 dentials of a server admin. CouchDB will respond with:
1573 {"ok":true}
1575 The database is now secured against anonymous reads and writes:
1577 > curl http://localhost:5984/mydatabase/
1579 {"error":"unauthorized","reason":"You are not authorized to access this db."}
1581 You declared user “jan” as a member in this database, so he is able to
1583 read and write normal documents:
1584 > curl -u jan:apple http://localhost:5984/mydatabase/
1586 {"db_name":"mydatabase","doc_count":1,"doc_del_count":0,"update_seq":3,"purge_seq":0,
1588 "compact_running":false,"sizes":{"active":272,"disk":12376,"external":350},
1589 "instance_start_time":"0","disk_format_version":6,"committed_update_seq":3}
1590 If Jan attempted to create a design doc, however, CouchDB would return
1592 a 401 Unauthorized error because the username “jan” is not in the list
1593 of admin names and the /_users/org.couchdb.user:jan document doesn’t
1594 contain a role that matches any of the declared admin roles. If you
1595 want to promote Jan to an admin, you can update the security document
1596 to add “jan” to the names array under admin. Keeping track of individ‐
1597 ual database admin usernames is tedious, though, so you would likely
1598 prefer to create a database admin role and assign that role to the
1599 org.couchdb.user:jan user document:
1600 > curl -X PUT http://localhost:5984/mydatabase/_security \
1602 -u anna:secret \
1603 -H "Content-Type: application/json" \
1604 -d '{"admins": { "names": [], "roles": ["mydatabase_admin"] }, "members": { "names": [], "roles": [] } }'
1605 See the _security document reference page for additional details about
1607 specifying database members and admins.
1608 Getting Started
1610 In this document, we’ll take a quick tour of CouchDB’s features. We’ll
1611 create our first document and experiment with CouchDB views.
1612 All Systems Are Go!
1614 We’ll have a very quick look at CouchDB’s bare-bones Application Pro‐
1615 gramming Interface (API) by using the command-line utility curl. Please
1616 note that this is not the only way of talking to CouchDB. We will show
1617 you plenty more throughout the rest of the documents. What’s interest‐
1618 ing about curl is that it gives you control over raw HTTP requests, and
1619 you can see exactly what is going on “underneath the hood” of your
1620 database.
1621 Make sure CouchDB is still running, and then do:
1623 curl http://127.0.0.1:5984/
1625 This issues a GET request to your newly installed CouchDB instance.
1627 The reply should look something like:
1629 {
1631 "couchdb": "Welcome",
1632 "version": "3.0.0",
1633 "git_sha": "83bdcf693",
1634 "uuid": "56f16e7c93ff4a2dc20eb6acc7000b71",
1635 "features": [
1636 "access-ready",
1637 "partitioned",
1638 "pluggable-storage-engines",
1639 "reshard",
1640 "scheduler"
1641 ],
1642 "vendor": {
1643 "name": "The Apache Software Foundation"
1644 }
1645 }
1646 Not all that spectacular. CouchDB is saying “hello” with the running
1648 version number.
1649 Next, we can get a list of databases:
1651 curl -X GET http://admin:password@127.0.0.1:5984/_all_dbs
1653 All we added to the previous request is the _all_dbs string, and our
1655 admin user name and password (set when installing CouchDB).
1656 The response should look like:
1658 ["_replicator","_users"]
1660 NOTE:
1662 In case this returns an empty Array for you, it means you haven’t
1663 finished installation correctly. Please refer to setup for further
1664 information on this.
1665 For the purposes of this example, we’ll not be showing the system
1667 databases past this point. In your installation, any time you GET
1668 /_all_dbs, you should see the system databases in the list, too.
1669 Oh, that’s right, we didn’t create any user databases yet!
1671 NOTE:
1673 The curl command issues GET requests by default. You can issue POST
1674 requests using curl -X POST. To make it easy to work with our termi‐
1675 nal history, we usually use the -X option even when issuing GET
1676 requests. If we want to send a POST next time, all we have to
1677 change is the method.
1678 HTTP does a bit more under the hood than you can see in the examples
1680 here. If you’re interested in every last detail that goes over the
1681 wire, pass in the -v option (e.g., curl -vX GET), which will show
1682 you the server curl tries to connect to, the request headers it
1683 sends, and response headers it receives back. Great for debugging!
1684 Let’s create a database:
1686 curl -X PUT http://admin:password@127.0.0.1:5984/baseball
1688 CouchDB will reply with:
1690 {"ok":true}
1692 Retrieving the list of databases again shows some useful results this
1694 time:
1695 curl -X GET http://admin:password@127.0.0.1:5984/_all_dbs
1697 ["baseball"]
1699 NOTE:
1701 We should mention JavaScript Object Notation (JSON) here, the data
1702 format CouchDB speaks. JSON is a lightweight data interchange format
1703 based on JavaScript syntax. Because JSON is natively compatible with
1704 JavaScript, your web browser is an ideal client for CouchDB.
1705 Brackets ([]) represent ordered lists, and curly braces ({}) repre‐
1707 sent key/value dictionaries. Keys must be strings, delimited by
1708 quotes ("), and values can be strings, numbers, booleans, lists, or
1709 key/value dictionaries. For a more detailed description of JSON, see
1710 Appendix E, JSON Primer.
1711 Let’s create another database:
1713 curl -X PUT http://admin:password@127.0.0.1:5984/baseball
1715 CouchDB will reply with:
1717 {"error":"file_exists","reason":"The database could not be created,
1719 the file already exists."}
1720 We already have a database with that name, so CouchDB will respond with
1722 an error. Let’s try again with a different database name:
1723 curl -X PUT http://admin:password@127.0.0.1:5984/plankton
1725 CouchDB will reply with:
1727 {"ok":true}
1729 Retrieving the list of databases yet again shows some useful results:
1731 curl -X GET http://admin:password@127.0.0.1:5984/_all_dbs
1733 CouchDB will respond with:
1735 ["baseball", "plankton"]
1737 To round things off, let’s delete the second database:
1739 curl -X DELETE http://admin:password@127.0.0.1:5984/plankton
1741 CouchDB will reply with:
1743 {"ok":true}
1745 The list of databases is now the same as it was before:
1747 curl -X GET http://admin:password@127.0.0.1:5984/_all_dbs
1749 CouchDB will respond with:
1751 ["baseball"]
1753 For brevity, we’ll skip working with documents, as the next section
1755 covers a different and potentially easier way of working with CouchDB
1756 that should provide experience with this. As we work through the exam‐
1757 ple, keep in mind that “under the hood” everything is being done by the
1758 application exactly as you have been doing here manually. Everything
1759 is done using GET, PUT, POST, and DELETE with a URI.
1760 Welcome to Fauxton
1762 After having seen CouchDB’s raw API, let’s get our feet wet by playing
1763 with Fauxton, the built-in administration interface. Fauxton provides
1764 full access to all of CouchDB’s features and makes it easy to work with
1765 some of the more complex ideas involved. With Fauxton we can create and
1766 destroy databases; view and edit documents; compose and run MapReduce
1767 views; and trigger replication between databases.
1768 To load Fauxton in your browser, visit:
1770 http://127.0.0.1:5984/_utils/
1772 and log in when prompted with your admin password.
1774 In later documents, we’ll focus on using CouchDB from server-side lan‐
1776 guages such as Ruby and Python. As such, this document is a great
1777 opportunity to showcase an example of natively serving up a dynamic web
1778 application using nothing more than CouchDB’s integrated web server,
1779 something you may wish to do with your own applications.
1780 The first thing we should do with a fresh installation of CouchDB is
1782 run the test suite to verify that everything is working properly. This
1783 assures us that any problems we may run into aren’t due to bothersome
1784 issues with our setup. By the same token, failures in the Fauxton test
1785 suite are a red flag, telling us to double-check our installation
1786 before attempting to use a potentially broken database server, saving
1787 us the confusion when nothing seems to be working quite like we expect!
1788 To validate your installation, click on the Verify link on the
1790 left-hand side, then press the green Verify Installation button. All
1791 tests should pass with a check mark. If any fail, re-check your instal‐
1792 lation steps.
1793 Your First Database and Document
1795 Creating a database in Fauxton is simple. From the overview page, click
1796 “Create Database.” When asked for a name, enter hello-world and click
1797 the Create button.
1798 After your database has been created, Fauxton will display a list of
1800 all its documents. This list will start out empty, so let’s create our
1801 first document. Click the plus sign next to “All Documents” and select
1802 the “New Doc” link. CouchDB will generate a UUID for you.
1803 For demoing purposes, having CouchDB assign a UUID is fine. When you
1805 write your first programs, we recommend assigning your own UUIDs. If
1806 you rely on the server to generate the UUID and you end up making two
1807 POST requests because the first POST request bombed out, you might gen‐
1808 erate two docs and never find out about the first one because only the
1809 second one will be reported back. Generating your own UUIDs makes sure
1810 that you’ll never end up with duplicate documents.
1811 Fauxton will display the newly created document, with its _id field. To
1813 create a new field, simply use the editor to write valid JSON. Add a
1814 new field by appending a comma to the _id value, then adding the text:
1815 "hello": "my new value"
1817 Click the green Create Document button to finalize creating the docu‐
1819 ment.
1820 You can experiment with other JSON values; e.g., [1, 2, "c"] or {"foo":
1822 "bar"}.
1823 You’ll notice that the document’s _rev has been added. We’ll go into
1825 more detail about this in later documents, but for now, the important
1826 thing to note is that _rev acts like a safety feature when saving a
1827 document. As long as you and CouchDB agree on the most recent _rev of a
1828 document, you can successfully save your changes.
1829 For clarity, you may want to display the contents of the document in
1831 the all document view. To enable this, from the upper-right corner of
1832 the window, select Options, then check the Include Docs option.
1833 Finally, press the Run Query button. The full document should be dis‐
1834 played along with the _id and _rev values.
1835 Running a Mango Query
1837 Now that we have stored documents successfully, we want to be able to
1838 query them. The easiest way to do this in CouchDB is running a Mango
1839 Query. There are always two parts to a Mango Query: the index and the
1840 selector.
1841 The index specifies which fields we want to be able to query on, and
1843 the selector includes the actual query parameters that define what we
1844 are looking for exactly.
1845 Indexes are stored as rows that are kept sorted by the fields you spec‐
1847 ify. This makes retrieving data from a range of keys efficient even
1848 when there are thousands or millions of rows.
1849 Before we can run an example query, we’ll need some data to run it on.
1851 We’ll create documents with information about movies. Let’s create doc‐
1852 uments for three movies. (Allow CouchDB to generate the _id and _rev
1853 fields.) Use Fauxton to create documents that have a final JSON struc‐
1854 ture that look like this:
1855 {
1857 "_id": "00a271787f89c0ef2e10e88a0c0001f4",
1858 "type": "movie",
1859 "title": "My Neighbour Totoro",
1860 "year": 1988,
1861 "director": "miyazaki",
1862 "rating": 8.2
1863 }
1864 {
1866 "_id": "00a271787f89c0ef2e10e88a0c0003f0",
1867 "type": "movie",
1868 "title": "Kikis Delivery Service",
1869 "year": 1989,
1870 "director": "miyazaki",
1871 "rating": 7.8
1872 }
1873 {
1875 "_id": "00a271787f89c0ef2e10e88a0c00048b",
1876 "type": "movie",
1877 "title": "Princess Mononoke",
1878 "year": 1997,
1879 "director": "miyazaki",
1880 "rating": 8.4
1881 }
1882 Now we want to be able to find a movie by its release year, we need to
1884 create a Mango Index. To do this, go to “Run A Query with Mango” in the
1885 Database overview. Then click on “manage indexes”, and change the index
1886 field on the left to look like this:
1887 {
1889 "index": {
1890 "fields": [
1891 "year"
1892 ]
1893 },
1894 "name": "year-json-index",
1895 "type": "json"
1896 }
1897 This defines an index on the field year and allows us to send queries
1899 for documents from a specific year.
1900 Next, click on “edit query” and change the Mango Query to look like
1902 this:
1903 {
1905 "selector": {
1906 "year": {
1907 "$eq": 1988
1908 }
1909 }
1910 }
1911 Then click on ”Run Query”.
1913 The result should be a single result, the movie “My Neighbour Totoro”
1915 which has the year value of 1988. $eq here stands for “equal”.
1916 NOTE:
1918 Note that if you skip adding the index, the query will still return
1919 the correct results, although you will see a warning about not using
1920 a pre-existing index. Not using an index will work fine on small
1921 databases and is acceptable for testing out queries in development
1922 or training, but we very strongly discourage doing this in any other
1923 case, since an index is absolutely vital to good query performance.
1924 You can also query for all movies during the 1980s, with this selector:
1926 {
1928 "selector": {
1929 "year": {
1930 "$lt": 1990,
1931 "$gte": 1980
1932 }
1933 }
1934 }
1935 The result are the two movies from 1988 and 1989. $lt here means “lower
1937 than”, and $gte means “greater than or equal to”. The latter currently
1938 doesn’t have any effect, given that all of our movies are more recent
1939 than 1980, but this makes the query future-proof and allows us to add
1940 older movies later.
1941 Triggering Replication
1943 Fauxton can trigger replication between two local databases, between a
1944 local and remote database, or even between two remote databases. We’ll
1945 show you how to replicate data from one local database to another,
1946 which is a simple way of making backups of your databases as we’re
1947 working through the examples.
1948 First we’ll need to create an empty database to be the target of repli‐
1950 cation. Return to the Databases overview and create a database called
1951 hello-replication. Now click “Replication” in the sidebar and choose
1952 hello-world as the source and hello-replication as the target. Click
1953 “Replicate” to replicate your database.
1954 To view the result of your replication, click on the Databases tab
1956 again. You should see the hello-replication database has the same num‐
1957 ber of documents as the hello-world database, and it should take up
1958 roughly the same size as well.
1959 NOTE:
1961 For larger databases, replication can take much longer. It is impor‐
1962 tant to leave the browser window open while replication is taking
1963 place. As an alternative, you can trigger replication via curl or
1964 some other HTTP client that can handle long-running connections. If
1965 your client closes the connection before replication finishes,
1966 you’ll have to retrigger it. Luckily, CouchDB’s replication can
1967 take over from where it left off instead of starting from scratch.
1968 Wrapping Up
1970 Now that you’ve seen most of Fauxton’s features, you’ll be prepared to
1971 dive in and inspect your data as we build our example application in
1972 the next few documents. Fauxton’s pure JavaScript approach to managing
1973 CouchDB shows how it’s possible to build a fully featured web applica‐
1974 tion using only CouchDB’s HTTP API and integrated web server.
1975 But before we get there, we’ll have another look at CouchDB’s HTTP API
1977 – now with a magnifying glass. Let’s curl up on the couch and relax.
1978 The Core API
1980 This document explores the CouchDB in minute detail. It shows all the
1981 nitty-gritty and clever bits. We show you best practices and guide you
1982 around common pitfalls.
1983 We start out by revisiting the basic operations we ran in the previous
1985 document intro/tour, looking behind the scenes. We also show what Faux‐
1986 ton needs to do behind its user interface to give us the nice features
1987 we saw earlier.
1988 This document is both an introduction to the core CouchDB API as well
1990 as a reference. If you can’t remember how to run a particular request
1991 or why some parameters are needed, you can always come back here and
1992 look things up (we are probably the heaviest users of this document).
1993 While explaining the API bits and pieces, we sometimes need to take a
1995 larger detour to explain the reasoning for a particular request. This
1996 is a good opportunity for us to tell you why CouchDB works the way it
1997 does.
1998 The API can be subdivided into the following sections. We’ll explore
2000 them individually:
2001 · Server
2003 · Databases
2005 · Documents
2007 · Replication
2009 · Wrapping Up
2011 Server
2013 This one is basic and simple. It can serve as a sanity check to see if
2014 CouchDB is running at all. It can also act as a safety guard for
2015 libraries that require a certain version of CouchDB. We’re using the
2016 curl utility again:
2017 curl http://127.0.0.1:5984/
2019 CouchDB replies, all excited to get going:
2021 {
2023 "couchdb": "Welcome",
2024 "version": "3.0.0",
2025 "git_sha": "83bdcf693",
2026 "uuid": "56f16e7c93ff4a2dc20eb6acc7000b71",
2027 "features": [
2028 "access-ready",
2029 "partitioned",
2030 "pluggable-storage-engines",
2031 "reshard",
2032 "scheduler"
2033 ],
2034 "vendor": {
2035 "name": "The Apache Software Foundation"
2036 }
2037 }
2038 You get back a JSON string, that, if parsed into a native object or
2040 data structure of your programming language, gives you access to the
2041 welcome string and version information.
2042 This is not terribly useful, but it illustrates nicely the way CouchDB
2044 behaves. You send an HTTP request and you receive a JSON string in the
2045 HTTP response as a result.
2046 Databases
2048 Now let’s do something a little more useful: create databases. For the
2049 strict, CouchDB is a database management system (DMS). That means it
2050 can hold multiple databases. A database is a bucket that holds “related
2051 data”. We’ll explore later what that means exactly. In practice, the
2052 terminology is overlapping – often people refer to a DMS as “a data‐
2053 base” and also a database within the DMS as “a database.” We might fol‐
2054 low that slight oddity, so don’t get confused by it. In general, it
2055 should be clear from the context if we are talking about the whole of
2056 CouchDB or a single database within CouchDB.
2057 Now let’s make one! We want to store our favorite music albums, and we
2059 creatively give our database the name albums. Note that we’re now using
2060 the -X option again to tell curl to send a PUT request instead of the
2061 default GET request:
2062 curl -X PUT http://admin:password@127.0.0.1:5984/albums
2064 CouchDB replies:
2066 {"ok":true}
2068 That’s it. You created a database and CouchDB told you that all went
2070 well. What happens if you try to create a database that already
2071 exists? Let’s try to create that database again:
2072 curl -X PUT http://admin:password@127.0.0.1:5984/albums
2074 CouchDB replies:
2076 {"error":"file_exists","reason":"The database could not be created, the file already exists."}
2078 We get back an error. This is pretty convenient. We also learn a little
2080 bit about how CouchDB works. CouchDB stores each database in a single
2081 file. Very simple.
2082 Let’s create another database, this time with curl’s -v (for “verbose”)
2084 option. The verbose option tells curl to show us not only the essen‐
2085 tials – the HTTP response body – but all the underlying request and
2086 response details:
2087 curl -vX PUT http://admin:password@127.0.0.1:5984/albums-backup
2089 curl elaborates:
2091 * About to connect() to 127.0.0.1 port 5984 (#0)
2093 * Trying 127.0.0.1... connected
2094 * Connected to 127.0.0.1 (127.0.0.1) port 5984 (#0)
2095 > PUT /albums-backup HTTP/1.1
2096 > User-Agent: curl/7.16.3 (powerpc-apple-darwin9.0) libcurl/7.16.3 OpenSSL/0.9.7l zlib/1.2.3
2097 > Host: 127.0.0.1:5984
2098 > Accept: */*
2099 >
2100 < HTTP/1.1 201 Created
2101 < Server: CouchDB (Erlang/OTP)
2102 < Date: Sun, 05 Jul 2009 22:48:28 GMT
2103 < Content-Type: text/plain;charset=utf-8
2104 < Content-Length: 12
2105 < Cache-Control: must-revalidate
2106 <
2107 {"ok":true}
2108 * Connection #0 to host 127.0.0.1 left intact
2109 * Closing connection #0
2110 What a mouthful. Let’s step through this line by line to understand
2112 what’s going on and find out what’s important. Once you’ve seen this
2113 output a few times, you’ll be able to spot the important bits more eas‐
2114 ily.
2115 * About to connect() to 127.0.0.1 port 5984 (#0)
2117 This is curl telling us that it is going to establish a TCP connection
2119 to the CouchDB server we specified in our request URI. Not at all
2120 important, except when debugging networking issues.
2121 * Trying 127.0.0.1... connected
2123 * Connected to 127.0.0.1 (127.0.0.1) port 5984 (#0)
2124 curl tells us it successfully connected to CouchDB. Again, not impor‐
2126 tant if you aren’t trying to find problems with your network.
2127 The following lines are prefixed with > and < characters. The > means
2129 the line was sent to CouchDB verbatim (without the actual >). The <
2130 means the line was sent back to curl by CouchDB.
2131 > PUT /albums-backup HTTP/1.1
2133 This initiates an HTTP request. Its method is PUT, the URI is
2135 /albums-backup, and the HTTP version is HTTP/1.1. There is also
2136 HTTP/1.0, which is simpler in some cases, but for all practical reasons
2137 you should be using HTTP/1.1.
2138 Next, we see a number of request headers. These are used to provide
2140 additional details about the request to CouchDB.
2141 > User-Agent: curl/7.16.3 (powerpc-apple-darwin9.0) libcurl/7.16.3 OpenSSL/0.9.7l zlib/1.2.3
2143 The User-Agent header tells CouchDB which piece of client software is
2145 doing the HTTP request. We don’t learn anything new: it’s curl. This
2146 header is often useful in web development when there are known errors
2147 in client implementations that a server might want to prepare the
2148 response for. It also helps to determine which platform a user is on.
2149 This information can be used for technical and statistical reasons. For
2150 CouchDB, the User-Agent header is irrelevant.
2151 > Host: 127.0.0.1:5984
2153 The Host header is required by HTTP 1.1. It tells the server the host‐
2155 name that came with the request.
2156 > Accept: */*
2158 The Accept header tells CouchDB that curl accepts any media type.
2160 We’ll look into why this is useful a little later.
2161 >
2163 An empty line denotes that the request headers are now finished and the
2165 rest of the request contains data we’re sending to the server. In this
2166 case, we’re not sending any data, so the rest of the curl output is
2167 dedicated to the HTTP response.
2168 < HTTP/1.1 201 Created
2170 The first line of CouchDB’s HTTP response includes the HTTP version
2172 information (again, to acknowledge that the requested version could be
2173 processed), an HTTP status code, and a status code message. Different
2174 requests trigger different response codes. There’s a whole range of
2175 them telling the client (curl in our case) what effect the request had
2176 on the server. Or, if an error occurred, what kind of error. RFC 2616
2177 (the HTTP 1.1 specification) defines clear behavior for response codes.
2178 CouchDB fully follows the RFC.
2179 The 201 Created status code tells the client that the resource the
2181 request was made against was successfully created. No surprise here,
2182 but if you remember that we got an error message when we tried to cre‐
2183 ate this database twice, you now know that this response could include
2184 a different response code. Acting upon responses based on response
2185 codes is a common practice. For example, all response codes of 400 Bad
2186 Request or larger tell you that some error occurred. If you want to
2187 shortcut your logic and immediately deal with the error, you could just
2188 check a >= 400 response code.
2189 < Server: CouchDB (Erlang/OTP)
2191 The Server header is good for diagnostics. It tells us which CouchDB
2193 version and which underlying Erlang version we are talking to. In gen‐
2194 eral, you can ignore this header, but it is good to know it’s there if
2195 you need it.
2196 < Date: Sun, 05 Jul 2009 22:48:28 GMT
2198 The Date header tells you the time of the server. Since client and
2200 server time are not necessarily synchronized, this header is purely
2201 informational. You shouldn’t build any critical application logic on
2202 top of this!
2203 < Content-Type: text/plain;charset=utf-8
2205 The Content-Type header tells you which MIME type the HTTP response
2207 body is and its encoding. We already know CouchDB returns JSON strings.
2208 The appropriate Content-Type header is application/json. Why do we see
2209 text/plain? This is where pragmatism wins over purity. Sending an
2210 application/json Content-Type header will make a browser offer you the
2211 returned JSON for download instead of just displaying it. Since it is
2212 extremely useful to be able to test CouchDB from a browser, CouchDB
2213 sends a text/plain content type, so all browsers will display the JSON
2214 as text.
2215 NOTE:
2217 There are some extensions that make your browser JSON-aware, but
2218 they are not installed by default. For more information, look at the
2219 popular JSONView extension, available for both Firefox and Chrome.
2220 Do you remember the Accept request header and how it is set to */* to
2222 express interest in any MIME type? If you send Accept: application/json
2223 in your request, CouchDB knows that you can deal with a pure JSON
2224 response with the proper Content-Type header and will use it instead of
2225 text/plain.
2226 < Content-Length: 12
2228 The Content-Length header simply tells us how many bytes the response
2230 body has.
2231 < Cache-Control: must-revalidate
2233 This Cache-Control header tells you, or any proxy server between
2235 CouchDB and you, not to cache this response.
2236 <
2238 This empty line tells us we’re done with the response headers and what
2240 follows now is the response body.
2241 {"ok":true}
2243 We’ve seen this before.
2245 * Connection #0 to host 127.0.0.1 left intact
2247 * Closing connection #0
2248 The last two lines are curl telling us that it kept the TCP connection
2250 it opened in the beginning open for a moment, but then closed it after
2251 it received the entire response.
2252 Throughout the documents, we’ll show more requests with the -v option,
2254 but we’ll omit some of the headers we’ve seen here and include only
2255 those that are important for the particular request.
2256 Creating databases is all fine, but how do we get rid of one? Easy –
2258 just change the HTTP method:
2259 > curl -vX DELETE http://admin:password@127.0.0.1:5984/albums-backup
2261 This deletes a CouchDB database. The request will remove the file that
2263 the database contents are stored in. There is no “Are you sure?” safety
2264 net or any “Empty the trash” magic you’ve got to do to delete a data‐
2265 base. Use this command with care. Your data will be deleted without a
2266 chance to bring it back easily if you don’t have a backup copy.
2267 This section went knee-deep into HTTP and set the stage for discussing
2269 the rest of the core CouchDB API. Next stop: documents.
2270 Documents
2272 Documents are CouchDB’s central data structure. The idea behind a docu‐
2273 ment is, unsurprisingly, that of a real-world document – a sheet of
2274 paper such as an invoice, a recipe, or a business card. We already
2275 learned that CouchDB uses the JSON format to store documents. Let’s see
2276 how this storing works at the lowest level.
2277 Each document in CouchDB has an ID. This ID is unique per database. You
2279 are free to choose any string to be the ID, but for best results we
2280 recommend a UUID (or GUID), i.e., a Universally (or Globally) Unique
2281 IDentifier. UUIDs are random numbers that have such a low collision
2282 probability that everybody can make thousands of UUIDs a minute for
2283 millions of years without ever creating a duplicate. This is a great
2284 way to ensure two independent people cannot create two different docu‐
2285 ments with the same ID. Why should you care what somebody else is
2286 doing? For one, that somebody else could be you at a later time or on a
2287 different computer; secondly, CouchDB replication lets you share docu‐
2288 ments with others and using UUIDs ensures that it all works. But more
2289 on that later; let’s make some documents:
2290 curl -X PUT http://admin:password@127.0.0.1:5984/albums/6e1295ed6c29495e54cc05947f18c8af -d '{"title":"There is Nothing Left to Lose","artist":"Foo Fighters"}'
2292 CouchDB replies:
2294 {"ok":true,"id":"6e1295ed6c29495e54cc05947f18c8af","rev":"1-2902191555"}
2296 The curl command appears complex, but let’s break it down. First, -X
2298 PUT tells curl to make a PUT request. It is followed by the URL that
2299 specifies your CouchDB IP address and port. The resource part of the
2300 URL /albums/6e1295ed6c29495e54cc05947f18c8af specifies the location of
2301 a document inside our albums database. The wild collection of numbers
2302 and characters is a UUID. This UUID is your document’s ID. Finally, the
2303 -d flag tells curl to use the following string as the body for the PUT
2304 request. The string is a simple JSON structure including title and
2305 artist attributes with their respective values.
2306 NOTE:
2308 If you don’t have a UUID handy, you can ask CouchDB to give you one
2309 (in fact, that is what we did just now without showing you). Simply
2310 send a GET /_uuids request:
2311 curl -X GET http://127.0.0.1:5984/_uuids
2313 CouchDB replies:
2315 {"uuids":["6e1295ed6c29495e54cc05947f18c8af"]}
2317 Voilà, a UUID. If you need more than one, you can pass in the
2319 ?count=10 HTTP parameter to request 10 UUIDs, or really, any number
2320 you need.
2321 To double-check that CouchDB isn’t lying about having saved your docu‐
2323 ment (it usually doesn’t), try to retrieve it by sending a GET request:
2324 curl -X GET http://admin:password@127.0.0.1:5984/albums/6e1295ed6c29495e54cc05947f18c8af
2326 We hope you see a pattern here. Everything in CouchDB has an address, a
2328 URI, and you use the different HTTP methods to operate on these URIs.
2329 CouchDB replies:
2331 {"_id":"6e1295ed6c29495e54cc05947f18c8af","_rev":"1-2902191555","title":"There is Nothing Left to Lose","artist":"Foo Fighters"}
2333 This looks a lot like the document you asked CouchDB to save, which is
2335 good. But you should notice that CouchDB added two fields to your JSON
2336 structure. The first is _id, which holds the UUID we asked CouchDB to
2337 save our document under. We always know the ID of a document if it is
2338 included, which is very convenient.
2339 The second field is _rev. It stands for revision.
2341 Revisions
2343 If you want to change a document in CouchDB, you don’t tell it to go
2344 and find a field in a specific document and insert a new value.
2345 Instead, you load the full document out of CouchDB, make your changes
2346 in the JSON structure (or object, when you are doing actual program‐
2347 ming), and save the entire new revision (or version) of that document
2348 back into CouchDB. Each revision is identified by a new _rev value.
2349 If you want to update or delete a document, CouchDB expects you to
2351 include the _rev field of the revision you wish to change. When CouchDB
2352 accepts the change, it will generate a new revision number. This mecha‐
2353 nism ensures that, in case somebody else made a change without you
2354 knowing before you got to request the document update, CouchDB will not
2355 accept your update because you are likely to overwrite data you didn’t
2356 know existed. Or simplified: whoever saves a change to a document
2357 first, wins. Let’s see what happens if we don’t provide a _rev field
2358 (which is equivalent to providing a outdated value):
2359 curl -X PUT http://admin:password@127.0.0.1:5984/albums/6e1295ed6c29495e54cc05947f18c8af \
2361 -d '{"title":"There is Nothing Left to Lose","artist":"Foo Fighters","year":"1997"}'
2362 CouchDB replies:
2364 {"error":"conflict","reason":"Document update conflict."}
2366 If you see this, add the latest revision number of your document to the
2368 JSON structure:
2369 curl -X PUT http://admin:password@127.0.0.1:5984/albums/6e1295ed6c29495e54cc05947f18c8af \
2371 -d '{"_rev":"1-2902191555","title":"There is Nothing Left to Lose","artist":"Foo Fighters","year":"1997"}'
2372 Now you see why it was handy that CouchDB returned that _rev when we
2374 made the initial request. CouchDB replies:
2375 {"ok":true,"id":"6e1295ed6c29495e54cc05947f18c8af","rev":"2-8aff9ee9d06671fa89c99d20a4b3ae"}
2377 CouchDB accepted your write and also generated a new revision number.
2379 The revision number is the MD5 hash of the transport representation of
2380 a document with an N- prefix denoting the number of times a document
2381 got updated. This is useful for replication. See replication/conflicts
2382 for more information.
2383 There are multiple reasons why CouchDB uses this revision system, which
2385 is also called Multi-Version Concurrency Control (MVCC). They all work
2386 hand-in-hand, and this is a good opportunity to explain some of them.
2387 One of the aspects of the HTTP protocol that CouchDB uses is that it is
2389 stateless. What does that mean? When talking to CouchDB you need to
2390 make requests. Making a request includes opening a network connection
2391 to CouchDB, exchanging bytes, and closing the connection. This is done
2392 every time you make a request. Other protocols allow you to open a con‐
2393 nection, exchange bytes, keep the connection open, exchange more bytes
2394 later – maybe depending on the bytes you exchanged at the beginning –
2395 and eventually close the connection. Holding a connection open for
2396 later use requires the server to do extra work. One common pattern is
2397 that for the lifetime of a connection, the client has a consistent and
2398 static view of the data on the server. Managing huge amounts of paral‐
2399 lel connections is a significant amount of work. HTTP connections are
2400 usually short-lived, and making the same guarantees is a lot easier.
2401 As a result, CouchDB can handle many more concurrent connections.
2402 Another reason CouchDB uses MVCC is that this model is simpler concep‐
2404 tually and, as a consequence, easier to program. CouchDB uses less code
2405 to make this work, and less code is always good because the ratio of
2406 defects per lines of code is static.
2407 The revision system also has positive effects on replication and stor‐
2409 age mechanisms, but we’ll explore these later in the documents.
2410 WARNING:
2412 The terms version and revision might sound familiar (if you are pro‐
2413 gramming without version control, stop reading this guide right now
2414 and start learning one of the popular systems). Using new versions
2415 for document changes works a lot like version control, but there’s
2416 an important difference: CouchDB does not guarantee that older ver‐
2417 sions are kept around. Don’t use the ``_rev`` token in CouchDB as a
2418 revision control system for your documents.
2419 Documents in Detail
2421 Now let’s have a closer look at our document creation requests with the
2422 curl -v flag that was helpful when we explored the database API ear‐
2423 lier. This is also a good opportunity to create more documents that we
2424 can use in later examples.
2425 We’ll add some more of our favorite music albums. Get a fresh UUID from
2427 the /_uuids resource. If you don’t remember how that works, you can
2428 look it up a few pages back.
2429 curl -vX PUT http://admin:password@127.0.0.1:5984/albums/70b50bfa0a4b3aed1f8aff9e92dc16a0 \
2431 -d '{"title":"Blackened Sky","artist":"Biffy Clyro","year":2002}'
2432 NOTE:
2434 By the way, if you happen to know more information about your
2435 favorite albums, don’t hesitate to add more properties. And don’t
2436 worry about not knowing all the information for all the albums.
2437 CouchDB’s schema-less documents can contain whatever you know. After
2438 all, you should relax and not worry about data.
2439 Now with the -v option, CouchDB’s reply (with only the important bits
2441 shown) looks like this:
2442 > PUT /albums/70b50bfa0a4b3aed1f8aff9e92dc16a0 HTTP/1.1
2444 >
2445 < HTTP/1.1 201 Created
2446 < Location: http://127.0.0.1:5984/albums/70b50bfa0a4b3aed1f8aff9e92dc16a0
2447 < ETag: "1-e89c99d29d06671fa0a4b3ae8aff9e"
2448 <
2449 {"ok":true,"id":"70b50bfa0a4b3aed1f8aff9e92dc16a0","rev":"1-e89c99d29d06671fa0a4b3ae8aff9e"}
2450 We’re getting back the 201 Created HTTP status code in the response
2452 headers, as we saw earlier when we created a database. The Location
2453 header gives us a full URL to our newly created document. And there’s a
2454 new header. An ETag in HTTP-speak identifies a specific version of a
2455 resource. In this case, it identifies a specific version (the first
2456 one) of our new document. Sound familiar? Yes, conceptually, an ETag is
2457 the same as a CouchDB document revision number, and it shouldn’t come
2458 as a surprise that CouchDB uses revision numbers for ETags. ETags are
2459 useful for caching infrastructures.
2460 Attachments
2462 CouchDB documents can have attachments just like an email message can
2463 have attachments. An attachment is identified by a name and includes
2464 its MIME type (or Content-Type) and the number of bytes the attachment
2465 contains. Attachments can be any data. It is easiest to think about
2466 attachments as files attached to a document. These files can be text,
2467 images, Word documents, music, or movie files. Let’s make one.
2468 Attachments get their own URL where you can upload data. Say we want to
2470 add the album artwork to the 6e1295ed6c29495e54cc05947f18c8af document
2471 (“There is Nothing Left to Lose”), and let’s also say the artwork is in
2472 a file artwork.jpg in the current directory:
2473 curl -vX PUT http://admin:password@127.0.0.1:5984/albums/6e1295ed6c29495e54cc05947f18c8af/artwork.jpg?rev=2-2739352689 \
2475 --data-binary @artwork.jpg -H "Content-Type:image/jpg"
2476 NOTE:
2478 The --data-binary @ option tells curl to read a file’s contents into
2479 the HTTP request body. We’re using the -H option to tell CouchDB
2480 that we’re uploading a JPEG file. CouchDB will keep this information
2481 around and will send the appropriate header when requesting this
2482 attachment; in case of an image like this, a browser will render the
2483 image instead of offering you the data for download. This will come
2484 in handy later. Note that you need to provide the current revision
2485 number of the document you’re attaching the artwork to, just as if
2486 you would update the document. Because, after all, attaching some
2487 data is changing the document.
2488 You should now see your artwork image if you point your browser to
2490 http://127.0.0.1:5984/albums/6e1295ed6c29495e54cc05947f18c8af/artwork.jpg
2491 If you request the document again, you’ll see a new member:
2493 curl http://admin:password@127.0.0.1:5984/albums/6e1295ed6c29495e54cc05947f18c8af
2495 CouchDB replies:
2497 {
2499 "_id": "6e1295ed6c29495e54cc05947f18c8af",
2500 "_rev": "3-131533518",
2501 "title": "There is Nothing Left to Lose",
2502 "artist": "Foo Fighters",
2503 "year": "1997",
2504 "_attachments": {
2505 "artwork.jpg": {
2506 "stub": true,
2507 "content_type": "image/jpg",
2508 "length": 52450
2509 }
2510 }
2511 }
2512 _attachments is a list of keys and values where the values are JSON
2514 objects containing the attachment metadata. stub=true tells us that
2515 this entry is just the metadata. If we use the ?attachments=true HTTP
2516 option when requesting this document, we’d get a Base64 encoded string
2517 containing the attachment data.
2518 We’ll have a look at more document request options later as we explore
2520 more features of CouchDB, such as replication, which is the next topic.
2521 Replication
2523 CouchDB replication is a mechanism to synchronize databases. Much like
2524 rsync synchronizes two directories locally or over a network, replica‐
2525 tion synchronizes two databases locally or remotely.
2526 In a simple POST request, you tell CouchDB the source and the target of
2528 a replication and CouchDB will figure out which documents and new docu‐
2529 ment revisions are on source that are not yet on target, and will pro‐
2530 ceed to move the missing documents and revisions over.
2531 We’ll take an in-depth look at replication in the document replica‐
2533 tion/intro; in this document, we’ll just show you how to use it.
2534 First, we’ll create a target database. Note that CouchDB won’t automat‐
2536 ically create a target database for you, and will return a replication
2537 failure if the target doesn’t exist (likewise for the source, but that
2538 mistake isn’t as easy to make):
2539 curl -X PUT http://admin:password@127.0.0.1:5984/albums-replica
2541 Now we can use the database albums-replica as a replication target:
2543 curl -vX POST http://admin:password@127.0.0.1:5984/_replicate \
2545 -d '{"source":"http://127.0.0.1:5984/albums","target":"http://127.0.0.1:5984/albums-replica"}' \
2546 -H "Content-Type: application/json"
2547 NOTE:
2549 As of CouchDB 2.0.0, fully qualified URLs are required for both the
2550 replication source and target parameters.
2551 NOTE:
2553 CouchDB supports the option "create_target":true placed in the JSON
2554 POSTed to the _replicate URL. It implicitly creates the target data‐
2555 base if it doesn’t exist.
2556 CouchDB replies (this time we formatted the output so you can read it
2558 more easily):
2559 {
2561 "history": [
2562 {
2563 "start_last_seq": 0,
2564 "missing_found": 2,
2565 "docs_read": 2,
2566 "end_last_seq": 5,
2567 "missing_checked": 2,
2568 "docs_written": 2,
2569 "doc_write_failures": 0,
2570 "end_time": "Sat, 11 Jul 2009 17:36:21 GMT",
2571 "start_time": "Sat, 11 Jul 2009 17:36:20 GMT"
2572 }
2573 ],
2574 "source_last_seq": 5,
2575 "session_id": "924e75e914392343de89c99d29d06671",
2576 "ok": true
2577 }
2578 CouchDB maintains a session history of replications. The response for a
2580 replication request contains the history entry for this replication
2581 session. It is also worth noting that the request for replication will
2582 stay open until replication closes. If you have a lot of documents,
2583 it’ll take a while until they are all replicated and you won’t get back
2584 the replication response until all documents are replicated. It is
2585 important to note that replication replicates the database only as it
2586 was at the point in time when replication was started. So, any addi‐
2587 tions, modifications, or deletions subsequent to the start of replica‐
2588 tion will not be replicated.
2589 We’ll punt on the details again – the "ok": true at the end tells us
2591 all went well. If you now have a look at the albums-replica database,
2592 you should see all the documents that you created in the albums data‐
2593 base. Neat, eh?
2594 What you just did is called local replication in CouchDB terms. You
2596 created a local copy of a database. This is useful for backups or to
2597 keep snapshots of a specific state of your data around for later. You
2598 might want to do this if you are developing your applications but want
2599 to be able to roll back to a stable version of your code and data.
2600 There are more types of replication useful in other situations. The
2602 source and target members of our replication request are actually links
2603 (like in HTML) and so far we’ve seen links relative to the server we’re
2604 working on (hence local). You can also specify a remote database as the
2605 target:
2606 curl -vX POST http://admin:password@127.0.0.1:5984/_replicate \
2608 -d '{"source":"http://127.0.0.1:5984/albums","target":"http://example.org:5984/albums-replica"}' \
2609 -H "Content-Type:application/json"
2610 Using a local source and a remote target database is called push repli‐
2612 cation. We’re pushing changes to a remote server.
2613 NOTE:
2615 Since we don’t have a second CouchDB server around just yet, we’ll
2616 just use the absolute address of our single server, but you should
2617 be able to infer from this that you can put any remote server in
2618 there.
2619 This is great for sharing local changes with remote servers or buddies
2621 next door.
2622 You can also use a remote source and a local target to do a pull repli‐
2624 cation. This is great for getting the latest changes from a server that
2625 is used by others:
2626 curl -vX POST http://admin:password@127.0.0.1:5984/_replicate \
2628 -d '{"source":"http://example.org:5984/albums-replica","target":"http://127.0.0.1:5984/albums"}' \
2629 -H "Content-Type:application/json"
2630 Finally, you can run remote replication, which is mostly useful for
2632 management operations:
2633 curl -vX POST http://admin:password@127.0.0.1:5984/_replicate \
2635 -d '{"source":"http://example.org:5984/albums","target":"http://example.org:5984/albums-replica"}' \
2636 -H"Content-Type: application/json"
2637 NOTE:
2639 CouchDB and REST
2640 CouchDB prides itself on having a RESTful API, but these replication
2642 requests don’t look very RESTy to the trained eye. What’s up with
2643 that? While CouchDB’s core database, document, and attachment API
2644 are RESTful, not all of CouchDB’s API is. The replication API is one
2645 example. There are more, as we’ll see later in the documents.
2646 Why are there RESTful and non-RESTful APIs mixed up here? Have the
2648 developers been too lazy to go REST all the way? Remember, REST is
2649 an architectural style that lends itself to certain architectures
2650 (such as the CouchDB document API). But it is not a
2651 one-size-fits-all. Triggering an event like replication does not
2652 make a whole lot of sense in the REST world. It is more like a tra‐
2653 ditional remote procedure call. And there is nothing wrong with
2654 this.
2655 We very much believe in the “use the right tool for the job” philos‐
2657 ophy, and REST does not fit every job. For support, we refer to
2658 Leonard Richardson and Sam Ruby who wrote RESTful Web Services
2659 (O’Reilly), as they share our view.
2660 Wrapping Up
2662 This is still not the full CouchDB API, but we discussed the essentials
2663 in great detail. We’re going to fill in the blanks as we go. For now,
2664 we believe you’re ready to start building CouchDB applications.
2665 SEE ALSO:
2667 Complete HTTP API Reference:
2668 · Server API Reference
2670 · Database API Reference
2672 · Document API Reference
2674 · Replication API
2676
2678 Replication is an incremental one way process involving two databases
2679 (a source and a destination).
2680 The aim of replication is that at the end of the process, all active
2682 documents in the source database are also in the destination database
2683 and all documents that were deleted in the source database are also
2684 deleted in the destination database (if they even existed).
2685 The replication process only copies the last revision of a document, so
2687 all previous revisions that were only in the source database are not
2688 copied to the destination database.
2689 Introduction to Replication
2691 One of CouchDB’s strengths is the ability to synchronize two copies of
2692 the same database. This enables users to distribute data across several
2693 nodes or data centers, but also to move data more closely to clients.
2694 Replication involves a source and a destination database, which can be
2696 on the same or on different CouchDB instances. The aim of replication
2697 is that at the end of the process, all active documents in the source
2698 database are also in the destination database and all documents that
2699 were deleted in the source database are also deleted in the destination
2700 database (if they even existed).
2701 Transient and Persistent Replication
2703 There are two different ways to set up a replication. The first one
2704 that was introduced into CouchDB leads to a replication that could be
2705 called transient. Transient means that there are no documents backing
2706 up the replication. So after a restart of the CouchDB server the repli‐
2707 cation will disappear. Later, the _replicator database was introduced,
2708 which keeps documents containing your replication parameters. Such a
2709 replication can be called persistent. Transient replications were kept
2710 for backward compatibility. Both replications can have different repli‐
2711 cation states.
2712 Triggering, Stopping and Monitoring Replications
2714 A persistent replication is controlled through a document in the
2715 _replicator database, where each document describes one replication
2716 process (see replication-settings). For setting up a transient replica‐
2717 tion the api endpoint /_replicate can be used. A replication is trig‐
2718 gered by sending a JSON object either to the _replicate endpoint or
2719 storing it as a document into the _replicator database.
2720 If a replication is currently running its status can be inspected
2722 through the active tasks API (see api/server/active_tasks, replica‐
2723 tion-status and api/server/_scheduler/jobs).
2724 For document based-replications, api/server/_scheduler/docs can be used
2726 to get a complete state summary. This API is preferred as it will show
2727 the state of the replication document before it becomes a replication
2728 job.
2729 For transient replications there is no way to query their state when
2731 the job is finished.
2732 A replication can be stopped by deleting the document, or by updating
2734 it with its cancel property set to true.
2735 Replication Procedure
2737 During replication, CouchDB will compare the source and the destination
2738 database to determine which documents differ between the source and the
2739 destination database. It does so by following the changes on the source
2740 and comparing the documents to the destination. Changes are submitted
2741 to the destination in batches where they can introduce conflicts. Docu‐
2742 ments that already exist on the destination in the same revision are
2743 not transferred. As the deletion of documents is represented by a new
2744 revision, a document deleted on the source will also be deleted on the
2745 target.
2746 A replication task will finish once it reaches the end of the changes
2748 feed. If its continuous property is set to true, it will wait for new
2749 changes to appear until the task is canceled. Replication tasks also
2750 create checkpoint documents on the destination to ensure that a
2751 restarted task can continue from where it stopped, for example after it
2752 has crashed.
2753 When a replication task is initiated on the sending node, it is called
2755 push replication, if it is initiated by the receiving node, it is
2756 called pull replication.
2757 Master - Master replication
2759 One replication task will only transfer changes in one direction. To
2760 achieve master-master replication, it is possible to set up two repli‐
2761 cation tasks in opposite direction. When a change is replicated from
2762 database A to B by the first task, the second task from B to A will
2763 discover that the new change on B already exists in A and will wait for
2764 further changes.
2765 Controlling which Documents to Replicate
2767 There are three options for controlling which documents are replicated,
2768 and which are skipped:
2769 1. Defining documents as being local.
2771 2. Using selectorobj.
2773 3. Using filterfun.
2775 Local documents are never replicated (see api/local).
2777 selectorobj can be included in a replication document (see replica‐
2779 tion-settings). A selector object contains a query expression that is
2780 used to test whether a document should be replicated.
2781 filterfun can be used in a replication (see replication-settings). The
2783 replication task evaluates the filter function for each document in the
2784 changes feed. The document is only replicated if the filter returns
2785 true.
2786 NOTE:
2788 Using a selector provides performance benefits when compared with
2789 using a filterfun. You should use selectorobj where possible.
2790 NOTE:
2792 When using replication filters that depend on the document’s con‐
2793 tent, deleted documents may pose a problem, since the document
2794 passed to the filter will not contain any of the document’s content.
2795 This can be resolved by adding a _deleted:true field to the document
2796 instead of using the DELETE HTTP method, paired with the use of a
2797 validate document update handler to ensure the fields required for
2798 replication filters are always present. Take note, though, that the
2799 deleted document will still contain all of its data (including
2800 attachments)!
2801 Migrating Data to Clients
2803 Replication can be especially useful for bringing data closer to
2804 clients. PouchDB implements the replication algorithm of CouchDB in
2805 JavaScript, making it possible to make data from a CouchDB database
2806 available in an offline browser application, and synchronize changes
2807 back to CouchDB.
2808 Replicator Database
2810 Changed in version 2.1.0: Scheduling replicator was introduced. Repli‐
2811 cation states, by default are not written back to documents anymore.
2812 There are new replication job states and new API endpoints _sched‐
2813 uler/jobs and _scheduler/docs.
2814 The _replicator database works like any other in CouchDB, but documents
2817 added to it will trigger replications. Create (PUT or POST) a document
2818 to start replication. DELETE a replication document to cancel an ongo‐
2819 ing replication.
2820 These documents have exactly the same content as the JSON objects we
2822 used to POST to _replicate (fields source, target, create_target, con‐
2823 tinuous, doc_ids, filter, query_params, use_checkpoints, check‐
2824 point_interval).
2825 Replication documents can have a user defined _id (handy for finding a
2827 specific replication request later). Design Documents (and _local docu‐
2828 ments) added to the replicator database are ignored.
2829 The default replicator database is _replicator. Additional replicator
2831 databases can be created. To be recognized as such by the system, their
2832 database names should end with /_replicator.
2833 Basics
2835 Let’s say you POST the following document into _replicator:
2836 {
2838 "_id": "my_rep",
2839 "source": "http://myserver.com/foo",
2840 "target": "http://user:pass@localhost:5984/bar",
2841 "create_target": true,
2842 "continuous": true
2843 }
2844 In the couch log you’ll see 2 entries like these:
2846 [notice] 2017-04-05T17:16:19.646716Z node1@127.0.0.1 <0.29432.0> -------- Replication `"a81a78e822837e66df423d54279c15fe+continuous+create_target"` is using:
2848 4 worker processes
2849 a worker batch size of 500
2850 20 HTTP connections
2851 a connection timeout of 30000 milliseconds
2852 10 retries per request
2853 socket options are: [{keepalive,true},{nodelay,false}]
2854 [notice] 2017-04-05T17:16:19.646759Z node1@127.0.0.1 <0.29432.0> -------- Document `my_rep` triggered replication `a81a78e822837e66df423d54279c15fe+continuous+create_target`
2855 Replication state of this document can then be queried from
2857 http://adm:pass@localhost:5984/_scheduler/docs/_replicator/my_rep
2858 {
2860 "database": "_replicator",
2861 "doc_id": "my_rep",
2862 "error_count": 0,
2863 "id": "a81a78e822837e66df423d54279c15fe+continuous+create_target",
2864 "info": {
2865 "revisions_checked": 113,
2866 "missing_revisions_found": 113,
2867 "docs_read": 113,
2868 "docs_written": 113,
2869 "changes_pending": 0,
2870 "doc_write_failures": 0,
2871 "checkpointed_source_seq": "113-g1AAAACTeJzLYWBgYMpgTmHgz8tPSTV0MDQy1zMAQsMckEQiQ1L9____szKYE01ygQLsZsYGqcamiZjKcRqRxwIkGRqA1H-oSbZgk1KMLCzTDE0wdWUBAF6HJIQ",
2872 "source_seq": "113-g1AAAACTeJzLYWBgYMpgTmHgz8tPSTV0MDQy1zMAQsMckEQiQ1L9____szKYE01ygQLsZsYGqcamiZjKcRqRxwIkGRqA1H-oSbZgk1KMLCzTDE0wdWUBAF6HJIQ",
2873 "through_seq": "113-g1AAAACTeJzLYWBgYMpgTmHgz8tPSTV0MDQy1zMAQsMckEQiQ1L9____szKYE01ygQLsZsYGqcamiZjKcRqRxwIkGRqA1H-oSbZgk1KMLCzTDE0wdWUBAF6HJIQ"
2874 },
2875 "last_updated": "2017-04-05T19:18:15Z",
2876 "node": "node1@127.0.0.1",
2877 "source_proxy": null,
2878 "target_proxy": null,
2879 "source": "http://myserver.com/foo/",
2880 "start_time": "2017-04-05T19:18:15Z",
2881 "state": "running",
2882 "target": "http://adm:*****@localhost:5984/bar/"
2883 }
2884 The state is running. That means replicator has scheduled this replica‐
2886 tion job to run. Replication document contents stay the same. Previ‐
2887 ously, before version 2.1, it was updated with the triggered state.
2888 The replication job will also appear in
2890 http://adm:pass@localhost:5984/_scheduler/jobs
2892 {
2894 "jobs": [
2895 {
2896 "database": "_replicator",
2897 "doc_id": "my_rep",
2898 "history": [
2899 {
2900 "timestamp": "2017-04-05T19:18:15Z",
2901 "type": "started"
2902 },
2903 {
2904 "timestamp": "2017-04-05T19:18:15Z",
2905 "type": "added"
2906 }
2907 ],
2908 "id": "a81a78e822837e66df423d54279c15fe+continuous+create_target",
2909 "info": {
2910 "changes_pending": 0,
2911 "checkpointed_source_seq": "113-g1AAAACTeJzLYWBgYMpgTmHgz8tPSTV0MDQy1zMAQsMckEQiQ1L9____szKYE01ygQLsZsYGqcamiZjKcRqRxwIkGRqA1H-oSbZgk1KMLCzTDE0wdWUBAF6HJIQ",
2912 "doc_write_failures": 0,
2913 "docs_read": 113,
2914 "docs_written": 113,
2915 "missing_revisions_found": 113,
2916 "revisions_checked": 113,
2917 "source_seq": "113-g1AAAACTeJzLYWBgYMpgTmHgz8tPSTV0MDQy1zMAQsMckEQiQ1L9____szKYE01ygQLsZsYGqcamiZjKcRqRxwIkGRqA1H-oSbZgk1KMLCzTDE0wdWUBAF6HJIQ",
2918 "through_seq": "113-g1AAAACTeJzLYWBgYMpgTmHgz8tPSTV0MDQy1zMAQsMckEQiQ1L9____szKYE01ygQLsZsYGqcamiZjKcRqRxwIkGRqA1H-oSbZgk1KMLCzTDE0wdWUBAF6HJIQ"
2919 },
2920 "node": "node1@127.0.0.1",
2921 "pid": "<0.1174.0>",
2922 "source": "http://myserver.com/foo/",
2923 "start_time": "2017-04-05T19:18:15Z",
2924 "target": "http://adm:*****@localhost:5984/bar/",
2925 "user": null
2926 }
2927 ],
2928 "offset": 0,
2929 "total_rows": 1
2930 }
2931 _scheduler/jobs shows more information, such as a detailed history of
2933 state changes. If a persistent replication has not yet started, has
2934 failed, or is completed, information about its state can only be found
2935 in _scheduler/docs. Keep in mind that some replication documents could
2936 be invalid and could not become a replication job. Others might be
2937 delayed because they are fetching data from a slow source database.
2938 If there is an error, for example if the source database is missing,
2940 the replication job will crash and retry after a wait period. Each suc‐
2941 cessive crash will result in a longer waiting period.
2942 For example, POST-ing this document
2944 {
2946 "_id": "my_rep_crashing",
2947 "source": "http://myserver.com/missing",
2948 "target": "http://user:pass@localhost:5984/bar",
2949 "create_target": true,
2950 "continuous": true
2951 }
2952 when source database is missing, will result in periodic starts and
2954 crashes with an increasingly larger interval. The history list from
2955 _scheduler/jobs for this replication would look something like this:
2956 [
2958 {
2959 "reason": "db_not_found: could not open http://adm:*****@localhost:5984/missing/",
2960 "timestamp": "2017-04-05T20:55:10Z",
2961 "type": "crashed"
2962 },
2963 {
2964 "timestamp": "2017-04-05T20:55:10Z",
2965 "type": "started"
2966 },
2967 {
2968 "reason": "db_not_found: could not open http://adm:*****@localhost:5984/missing/",
2969 "timestamp": "2017-04-05T20:47:10Z",
2970 "type": "crashed"
2971 },
2972 {
2973 "timestamp": "2017-04-05T20:47:10Z",
2974 "type": "started"
2975 }
2976 ]
2977 _scheduler/docs shows a shorter summary:
2979 {
2981 "database": "_replicator",
2982 "doc_id": "my_rep_crashing",
2983 "error_count": 6,
2984 "id": "cb78391640ed34e9578e638d9bb00e44+create_target",
2985 "info": {
2986 "error": "db_not_found: could not open http://adm:*****@localhost:5984/missing/"
2987 },
2988 "last_updated": "2017-04-05T20:55:10Z",
2989 "node": "node1@127.0.0.1",
2990 "source_proxy": null,
2991 "target_proxy": null,
2992 "source": "http://adm:*****@localhost:5984/missing/",
2993 "start_time": "2017-04-05T20:38:34Z",
2994 "state": "crashing",
2995 "target": "http://adm:*****@localhost:5984/bar/"
2996 }
2997 Repeated crashes are described as a crashing state. -ing suffix implies
2999 this is a temporary state. User at any moment could create the missing
3000 database and then replication job could return back to the normal.
3001 Documents describing the same replication
3003 Lets suppose 2 documents are added to the _replicator database in the
3004 following order:
3005 {
3007 "_id": "my_rep",
3008 "source": "http://myserver.com/foo",
3009 "target": "http://user:pass@localhost:5984/bar",
3010 "create_target": true,
3011 "continuous": true
3012 }
3013 and
3015 {
3017 "_id": "my_rep_dup",
3018 "source": "http://myserver.com/foo",
3019 "target": "http://user:pass@localhost:5984/bar",
3020 "create_target": true,
3021 "continuous": true
3022 }
3023 Both describe exactly the same replication (only their _ids differ).
3025 In this case document my_rep triggers the replication, while
3026 my_rep_dup` will fail. Inspecting _scheduler/docs explains exactly why
3027 it failed:
3028 {
3030 "database": "_replicator",
3031 "doc_id": "my_rep_dup",
3032 "error_count": 1,
3033 "id": null,
3034 "info": {
3035 "error": "Replication `a81a78e822837e66df423d54279c15fe+continuous+create_target` specified by document `my_rep_dup` already started, triggered by document `my_rep` from db `_replicator`"
3036 },
3037 "last_updated": "2017-04-05T21:41:51Z",
3038 "source": "http://myserver.com/foo/",
3039 "start_time": "2017-04-05T21:41:51Z",
3040 "state": "failed",
3041 "target": "http://adm:*****@localhost:5984/bar/"
3042 }
3043 Notice the state for this replication is failed. Unlike crashing,
3045 failed state is terminal. As long as both documents are present the
3046 replicator will not retry to run my_rep_dup replication. Another reason
3047 could be malformed documents. For example if worker process count is
3048 specified as a string ("worker_processes": "a few") instead of an inte‐
3049 ger, failure will occur.
3050 Replication Scheduler
3052 Once replication jobs are created they are managed by the scheduler.
3053 The scheduler is the replication component which periodically stops
3054 some jobs and starts others. This behavior makes it possible to have a
3055 larger number of jobs than the cluster could run simultaneously.
3056 Replication jobs which keep failing will be penalized and forced to
3057 wait. The wait time increases exponentially with each consecutive fail‐
3058 ure.
3059 When deciding which jobs to stop and which to start, the scheduler uses
3061 a round-robin algorithm to ensure fairness. Jobs which have been run‐
3062 ning the longest time will be stopped, and jobs which have been waiting
3063 the longest time will be started.
3064 NOTE:
3066 Non-continuous (normal) replication are treated differently once
3067 they start running. See Normal vs Continuous Replications section
3068 for more information.
3069 The behavior of the scheduler can configured via max_jobs, interval and
3071 max_churn options. See Replicator configuration section for additional
3072 information.
3073 Replication states
3075 Replication jobs during their life-cycle pass through various states.
3076 This is a diagram of all the states and transitions between them:
3077 [image: Replication state diagram] [image] Replication state dia‐
3078 gram.UNINDENT
3079 Blue and yellow shapes represent replication job states.
3081 Trapezoidal shapes represent external APIs, that’s how users interact
3083 with the replicator. Writing documents to _replicator is the pre‐
3084 ferred way of creating replications, but posting to the _replicate
3085 HTTP endpoint is also supported.
3086 Six-sided shapes are internal API boundaries. They are optional for
3088 this diagram and are only shown as additional information to help
3089 clarify how the replicator works. There are two processing stages:
3090 the first is where replication documents are parsed and become repli‐
3091 cation jobs, and the second is the scheduler itself. The scheduler
3092 runs replication jobs, periodically stopping and starting some. Jobs
3093 posted via the _replicate endpoint bypass the first component and go
3094 straight to the scheduler.
3095 States descriptions
3097 Before explaining the details of each state, it is worth noticing that
3098 color and shape of each state in the diagram:
3099 Blue vs yellow partitions states into “healthy” and “unhealthy”,
3101 respectively. Unhealthy states indicate something has gone wrong and it
3102 might need user’s attention.
3103 Rectangle vs oval separates “terminal” states from “non-terminal” ones.
3105 Terminal states are those which will not transition to other states any
3106 more. Informally, jobs in a terminal state will not be retried and
3107 don’t consume memory or CPU resources.
3108 · Initializing: Indicates replicator has noticed the change from the
3110 replication document. Jobs should transition quickly through this
3111 state. Being stuck here for a while could mean there is an inter‐
3112 nal error.
3113 · Failed: Replication document could not be processed and turned
3115 into a valid replication job for the scheduler. This state is ter‐
3116 minal and requires user intervention to fix the problem. A typical
3117 reason for ending up in this state is a malformed document. For
3118 example, specifying an integer for a parameter which accepts a
3119 boolean. Another reason for failure could be specifying a dupli‐
3120 cate replication. A duplicate replication is a replication with
3121 identical parameters but a different document ID.
3122 · Error: Replication document update could not be turned into a
3124 replication job. Unlike the Failed state, this one is temporary,
3125 and replicator will keep retrying periodically. There is an expo‐
3126 nential backoff applied in case of consecutive failures. The main
3127 reason this state exists is to handle filtered replications with
3128 custom user functions. Filter function content is needed in order
3129 to calculate the replication ID. A replication job could not be
3130 created until the function code is retrieved. Because retrieval
3131 happens over the network, temporary failures have to be handled.
3132 · Running: Replication job is running normally. This means, there
3134 might be a change feed open, and if changes are noticed, they
3135 would be processed and posted to the target. Job is still consid‐
3136 ered Running even if its workers are currently not streaming
3137 changes from source to target and are just waiting on the change
3138 feed. Continuous replications will most likely end up in this
3139 state.
3140 · Pending: Replication job is not running and is waiting its turn.
3142 This state is reached when the number of replication jobs added to
3143 the scheduler exceeds replicator.max_jobs. In that case scheduler
3144 will periodically stop and start subsets of jobs trying to give
3145 each one a fair chance at making progress.
3146 · Crashing: Replication job has been successfully added to the
3148 replication scheduler. However an error was encountered during the
3149 last run. Error could be a network failure, a missing source data‐
3150 base, a permissions error, etc. Repeated consecutive crashes
3151 result in an exponential backoff. This state is considered tempo‐
3152 rary (non-terminal) and replication jobs will be periodically
3153 retried. Maximum backoff interval is around a day or so.
3154 · Completed: This is a terminal, successful state for non-continuous
3156 replications. Once in this state the replication is “forgotten” by
3157 the scheduler and it doesn’t consume any more CPU or memory
3158 resources. Continuous replication jobs will never reach this
3159 state.
3160 Normal vs Continuous Replications
3162 Normal (non-continuous) replications once started will be allowed to
3163 run to completion. That behavior is to preserve their semantics of
3164 replicating a snapshot of the source database to the target. For exam‐
3165 ple if new documents are added to the source after the replication are
3166 started, those updates should not show up on the target database.
3167 Stopping and restring a normal replication would violate that con‐
3168 straint.
3169 WARNING:
3171 When there is a mix of continuous and normal replications, once nor‐
3172 mal replication are scheduled to run, they might temporarily starve
3173 continuous replication jobs.
3174 However, normal replications will still be stopped and rescheduled if
3176 an operator reduces the value for the maximum number of replications.
3177 This is so that if an operator decides replications are overwhelming a
3178 node that it has the ability to recover. Any stopped replications will
3179 be resubmitted to the queue to be rescheduled.
3180 Compatibility Mode
3182 Previous version of CouchDB replicator wrote state updates back to
3183 replication documents. In cases where user code programmatically read
3184 those states, there is compatibility mode enabled via a configuration
3185 setting:
3186 [replicator]
3188 update_docs = true
3189 In this mode replicator will continue to write state updates to the
3191 documents.
3192 To effectively disable the scheduling behavior, which periodically stop
3194 and starts jobs, set max_jobs configuration setting to a large number.
3195 For example:
3196 [replicator]
3198 max_jobs = 9999999
3199 See Replicator configuration section for other replicator configuration
3201 options.
3202 Canceling replications
3204 To cancel a replication simply DELETE the document which triggered the
3205 replication. To update a replication, for example, change the number of
3206 worker or the source, simply update the document with new data. If
3207 there is extra application-specific data in the replication documents,
3208 that data is ignored by the replicator.
3209 Server restart
3211 When CouchDB is restarted, it checks its _replicator databases and
3212 restarts replications described by documents if they are not already in
3213 in a completed or failed state. If they are, they are ignored.
3214 Clustering
3216 In a cluster, replication jobs are balanced evenly among all the nodes
3217 nodes such that a replication job runs on only one node at a time.
3218 Every time there is a cluster membership change, that is when nodes are
3220 added or removed, as it happens in a rolling reboot, replicator appli‐
3221 cation will notice the change, rescan all the document and running
3222 replication, and re-evaluate their cluster placement in light of the
3223 new set of live nodes. This mechanism also provides replication
3224 fail-over in case a node fails. Replication jobs started from replica‐
3225 tion documents (but not those started from _replicate HTTP endpoint)
3226 will automatically migrate one of the live nodes.
3227 Additional Replicator Databases
3229 Imagine replicator database (_replicator) has these two documents which
3230 represent pull replications from servers A and B:
3231 {
3233 "_id": "rep_from_A",
3234 "source": "http://aserver.com:5984/foo",
3235 "target": "http://user:pass@localhost:5984/foo_a",
3236 "continuous": true
3237 }
3238 {
3240 "_id": "rep_from_B",
3241 "source": "http://bserver.com:5984/foo",
3242 "target": "http://user:pass@localhost:5984/foo_b",
3243 "continuous": true
3244 }
3245 Now without stopping and restarting CouchDB, add another replicator
3247 database. For example another/_replicator:
3248 $ curl -X PUT http://user:pass@localhost:5984/another%2F_replicator/
3250 {"ok":true}
3251 NOTE:
3253 A / character in a database name, when used in a URL, should be
3254 escaped.
3255 Then add a replication document to the new replicator database:
3257 {
3259 "_id": "rep_from_X",
3260 "source": "http://xserver.com:5984/foo",
3261 "target": "http://user:pass@localhost:5984/foo_x",
3262 "continuous": true
3263 }
3264 From now on, there are three replications active in the system: two
3266 replications from A and B, and a new one from X.
3267 Then remove the additional replicator database:
3269 $ curl -X DELETE http://user:pass@localhost:5984/another%2F_replicator/
3271 {"ok":true}
3272 After this operation, replication pulling from server X will be stopped
3274 and the replications in the _replicator database (pulling from servers
3275 A and B) will continue.
3276 Replicating the replicator database
3278 Imagine you have in server C a replicator database with the two follow‐
3279 ing pull replication documents in it:
3280 {
3282 "_id": "rep_from_A",
3283 "source": "http://aserver.com:5984/foo",
3284 "target": "http://user:pass@localhost:5984/foo_a",
3285 "continuous": true
3286 }
3287 {
3289 "_id": "rep_from_B",
3290 "source": "http://bserver.com:5984/foo",
3291 "target": "http://user:pass@localhost:5984/foo_b",
3292 "continuous": true
3293 }
3294 Now you would like to have the same pull replications going on in
3296 server D, that is, you would like to have server D pull replicating
3297 from servers A and B. You have two options:
3298 · Explicitly add two documents to server’s D replicator database
3300 · Replicate server’s C replicator database into server’s D replicator
3302 database
3303 Both alternatives accomplish exactly the same goal.
3305 Delegations
3307 Replication documents can have a custom user_ctx property. This prop‐
3308 erty defines the user context under which a replication runs. For the
3309 old way of triggering a replication (POSTing to /_replicate/), this
3310 property is not needed. That’s because information about the authenti‐
3311 cated user is readily available during the replication, which is not
3312 persistent in that case. Now, with the replicator database, the problem
3313 is that information about which user is starting a particular replica‐
3314 tion is only present when the replication document is written. The
3315 information in the replication document and the replication itself are
3316 persistent, however. This implementation detail implies that in the
3317 case of a non-admin user, a user_ctx property containing the user’s
3318 name and a subset of their roles must be defined in the replication
3319 document. This is enforced by the document update validation function
3320 present in the default design document of the replicator database. The
3321 validation function also ensures that non-admin users are unable to set
3322 the value of the user context’s name property to anything other than
3323 their own user name. The same principle applies for roles.
3324 For admins, the user_ctx property is optional, and if it’s missing it
3326 defaults to a user context with name null and an empty list of roles,
3327 which means design documents won’t be written to local targets. If
3328 writing design documents to local targets is desired, the role _admin
3329 must be present in the user context’s list of roles.
3330 Also, for admins the user_ctx property can be used to trigger a repli‐
3332 cation on behalf of another user. This is the user context that will be
3333 passed to local target database document validation functions.
3334 NOTE:
3336 The user_ctx property only has effect for local endpoints.
3337 Example delegated replication document:
3339 {
3341 "_id": "my_rep",
3342 "source": "http://bserver.com:5984/foo",
3343 "target": "http://user:pass@localhost:5984/bar",
3344 "continuous": true,
3345 "user_ctx": {
3346 "name": "joe",
3347 "roles": ["erlanger", "researcher"]
3348 }
3349 }
3350 As stated before, the user_ctx property is optional for admins, while
3352 being mandatory for regular (non-admin) users. When the roles property
3353 of user_ctx is missing, it defaults to the empty list [].
3354 Selector Objects
3356 Including a Selector Object in the replication document enables you to
3357 use a query expression to determine if a document should be included in
3358 the replication.
3359 The selector specifies fields in the document, and provides an expres‐
3361 sion to evaluate with the field content or other data. If the expres‐
3362 sion resolves to true, the document is replicated.
3363 The selector object must:
3365 · Be structured as valid JSON.
3367 · Contain a valid query expression.
3369 The syntax for a selector is the same as the selectorsyntax used for
3371 _find.
3372 Using a selector is significantly more efficient than using a
3374 JavaScript filter function, and is the recommended option if filtering
3375 on document attributes only.
3376 Replication and conflict model
3378 Let’s take the following example to illustrate replication and conflict
3379 handling.
3380 · Alice has a document containing Bob’s business card;
3382 · She synchronizes it between her desktop PC and her laptop;
3384 · On the desktop PC, she updates Bob’s E-mail address; Without syncing
3386 again, she updates Bob’s mobile number on the laptop;
3387 · Then she replicates the two to each other again.
3389 So on the desktop the document has Bob’s new E-mail address and his old
3391 mobile number, and on the laptop it has his old E-mail address and his
3392 new mobile number.
3393 The question is, what happens to these conflicting updated documents?
3395 CouchDB replication
3397 CouchDB works with JSON documents inside databases. Replication of
3398 databases takes place over HTTP, and can be either a “pull” or a
3399 “push”, but is unidirectional. So the easiest way to perform a full
3400 sync is to do a “push” followed by a “pull” (or vice versa).
3401 So, Alice creates v1 and sync it. She updates to v2a on one side and
3403 v2b on the other, and then replicates. What happens?
3404 The answer is simple: both versions exist on both sides!
3406 DESKTOP LAPTOP
3408 +---------+
3409 | /db/bob | INITIAL
3410 | v1 | CREATION
3411 +---------+
3412 +---------+ +---------+
3414 | /db/bob | -----------------> | /db/bob | PUSH
3415 | v1 | | v1 |
3416 +---------+ +---------+
3417 +---------+ +---------+ INDEPENDENT
3419 | /db/bob | | /db/bob | LOCAL
3420 | v2a | | v2b | EDITS
3421 +---------+ +---------+
3422 +---------+ +---------+
3424 | /db/bob | -----------------> | /db/bob | PUSH
3425 | v2a | | v2a |
3426 +---------+ | v2b |
3427 +---------+
3428 +---------+ +---------+
3430 | /db/bob | <----------------- | /db/bob | PULL
3431 | v2a | | v2a |
3432 | v2b | | v2b |
3433 +---------+ +---------+
3434 After all, this is not a file system, so there’s no restriction that
3436 only one document can exist with the name /db/bob. These are just “con‐
3437 flicting” revisions under the same name.
3438 Because the changes are always replicated, the data is safe. Both
3440 machines have identical copies of both documents, so failure of a hard
3441 drive on either side won’t lose any of the changes.
3442 Another thing to notice is that peers do not have to be configured or
3444 tracked. You can do regular replications to peers, or you can do
3445 one-off, ad-hoc pushes or pulls. After the replication has taken place,
3446 there is no record kept of which peer any particular document or revi‐
3447 sion came from.
3448 So the question now is: what happens when you try to read /db/bob? By
3450 default, CouchDB picks one arbitrary revision as the “winner”, using a
3451 deterministic algorithm so that the same choice will be made on all
3452 peers. The same happens with views: the deterministically-chosen winner
3453 is the only revision fed into your map function.
3454 Let’s say that the winner is v2a. On the desktop, if Alice reads the
3456 document she’ll see v2a, which is what she saved there. But on the lap‐
3457 top, after replication, she’ll also see only v2a. It could look as if
3458 the changes she made there have been lost - but of course they have
3459 not, they have just been hidden away as a conflicting revision. But
3460 eventually she’ll need these changes merged into Bob’s business card,
3461 otherwise they will effectively have been lost.
3462 Any sensible business-card application will, at minimum, have to
3464 present the conflicting versions to Alice and allow her to create a new
3465 version incorporating information from them all. Ideally it would merge
3466 the updates itself.
3467 Conflict avoidance
3469 When working on a single node, CouchDB will avoid creating conflicting
3470 revisions by returning a 409 Conflict error. This is because, when you
3471 PUT a new version of a document, you must give the _rev of the previous
3472 version. If that _rev has already been superseded, the update is
3473 rejected with a 409 Conflict response.
3474 So imagine two users on the same node are fetching Bob’s business card,
3476 updating it concurrently, and writing it back:
3477 USER1 -----------> GET /db/bob
3479 <----------- {"_rev":"1-aaa", ...}
3480 USER2 -----------> GET /db/bob
3482 <----------- {"_rev":"1-aaa", ...}
3483 USER1 -----------> PUT /db/bob?rev=1-aaa
3485 <----------- {"_rev":"2-bbb", ...}
3486 USER2 -----------> PUT /db/bob?rev=1-aaa
3488 <----------- 409 Conflict (not saved)
3489 User2’s changes are rejected, so it’s up to the app to fetch /db/bob
3491 again, and either:
3492 1. apply the same changes as were applied to the earlier revision, and
3494 submit a new PUT
3495 2. redisplay the document so the user has to edit it again
3497 3. just overwrite it with the document being saved before (which is not
3499 advisable, as user1’s changes will be silently lost)
3500 So when working in this mode, your application still has to be able to
3502 handle these conflicts and have a suitable retry strategy, but these
3503 conflicts never end up inside the database itself.
3504 Revision tree
3506 When you update a document in CouchDB, it keeps a list of the previous
3507 revisions. In the case where conflicting updates are introduced, this
3508 history branches into a tree, where the current conflicting revisions
3509 for this document form the tips (leaf nodes) of this tree:
3510 ,--> r2a
3512 r1 --> r2b
3513 `--> r2c
3514 Each branch can then extend its history - for example if you read revi‐
3516 sion r2b and then PUT with ?rev=r2b then you will make a new revision
3517 along that particular branch.
3518 ,--> r2a -> r3a -> r4a
3520 r1 --> r2b -> r3b
3521 `--> r2c -> r3c
3522 Here, (r4a, r3b, r3c) are the set of conflicting revisions. The way you
3524 resolve a conflict is to delete the leaf nodes along the other
3525 branches. So when you combine (r4a+r3b+r3c) into a single merged docu‐
3526 ment, you would replace r4a and delete r3b and r3c.
3527 ,--> r2a -> r3a -> r4a -> r5a
3529 r1 --> r2b -> r3b -> (r4b deleted)
3530 `--> r2c -> r3c -> (r4c deleted)
3531 Note that r4b and r4c still exist as leaf nodes in the history tree,
3533 but as deleted docs. You can retrieve them but they will be marked
3534 "_deleted":true.
3535 When you compact a database, the bodies of all the non-leaf documents
3537 are discarded. However, the list of historical _revs is retained, for
3538 the benefit of later conflict resolution in case you meet any old
3539 replicas of the database at some time in future. There is “revision
3540 pruning” to stop this getting arbitrarily large.
3541 Working with conflicting documents
3543 The basic /{doc}/{docid} operation will not show you any information
3544 about conflicts. You see only the deterministically-chosen winner, and
3545 get no indication as to whether other conflicting revisions exist or
3546 not:
3547 {
3549 "_id":"test",
3550 "_rev":"2-b91bb807b4685080c6a651115ff558f5",
3551 "hello":"bar"
3552 }
3553 If you do GET /db/test?conflicts=true, and the document is in a con‐
3555 flict state, then you will get the winner plus a _conflicts member con‐
3556 taining an array of the revs of the other, conflicting revision(s). You
3557 can then fetch them individually using subsequent GET /db/test?rev=xxxx
3558 operations:
3559 {
3561 "_id":"test",
3562 "_rev":"2-b91bb807b4685080c6a651115ff558f5",
3563 "hello":"bar",
3564 "_conflicts":[
3565 "2-65db2a11b5172bf928e3bcf59f728970",
3566 "2-5bc3c6319edf62d4c624277fdd0ae191"
3567 ]
3568 }
3569 If you do GET /db/test?open_revs=all then you will get all the leaf
3571 nodes of the revision tree. This will give you all the current con‐
3572 flicts, but will also give you leaf nodes which have been deleted (i.e.
3573 parts of the conflict history which have since been resolved). You can
3574 remove these by filtering out documents with "_deleted":true:
3575 [
3577 {"ok":{"_id":"test","_rev":"2-5bc3c6319edf62d4c624277fdd0ae191","hello":"foo"}},
3578 {"ok":{"_id":"test","_rev":"2-65db2a11b5172bf928e3bcf59f728970","hello":"baz"}},
3579 {"ok":{"_id":"test","_rev":"2-b91bb807b4685080c6a651115ff558f5","hello":"bar"}}
3580 ]
3581 The "ok" tag is an artifact of open_revs, which also lets you list
3583 explicit revisions as a JSON array, e.g. open_revs=[rev1,rev2,rev3]. In
3584 this form, it would be possible to request a revision which is now
3585 missing, because the database has been compacted.
3586 NOTE:
3588 The order of revisions returned by open_revs=all is NOT related to
3589 the deterministic “winning” algorithm. In the above example, the
3590 winning revision is 2-b91b… and happens to be returned last, but in
3591 other cases it can be returned in a different position.
3592 Once you have retrieved all the conflicting revisions, your application
3594 can then choose to display them all to the user. Or it could attempt to
3595 merge them, write back the merged version, and delete the conflicting
3596 versions - that is, to resolve the conflict permanently.
3597 As described above, you need to update one revision and delete all the
3599 conflicting revisions explicitly. This can be done using a single POST
3600 to _bulk_docs, setting "_deleted":true on those revisions you wish to
3601 delete.
3602 Multiple document API
3604 Finding conflicted documents with Mango
3605 New in version 2.2.0.
3606 CouchDB’s Mango system allows easy querying of documents with con‐
3609 flicts, returning the full body of each document as well.
3610 Here’s how to use it to find all conflicts in a database:
3612 $ curl -X POST http://127.0.0.1/dbname/_find \
3614 -d '{"selector": {"_conflicts": { "$exists": true}}, "conflicts": true}' \
3615 -Hcontent-type:application/json
3616 {"docs": [
3618 {"_id":"doc","_rev":"1-3975759ccff3842adf690a5c10caee42","a":2,"_conflicts":["1-23202479633c2b380f79507a776743d5"]}
3619 ],
3620 "bookmark": "g1AAAABheJzLYWBgYMpgSmHgKy5JLCrJTq2MT8lPzkzJBYozA1kgKQ6YVA5QkBFMgKSVDHWNjI0MjEzMLc2MjZONkowtDNLMLU0NzBPNzc3MTYxTTLOysgCY2ReV"}
3621 The bookmark value can be used to navigate through additional pages of
3623 results if necessary. Mango by default only returns 25 results per
3624 request.
3625 If you expect to run this query often, be sure to create a Mango sec‐
3627 ondary index to speed the query:
3628 $ curl -X POST http://127.0.0.1/dbname/_index \
3630 -d '{"index":{"fields": ["_conflicts"]}}' \
3631 -Hcontent-type:application/json
3632 Of course, the selector can be enhanced to filter documents on addi‐
3634 tional keys in the document. Be sure to add those keys to your sec‐
3635 ondary index as well, or a full database scan will be triggered.
3636 Finding conflicted documents using the _all_docs index
3638 You can fetch multiple documents at once using include_docs=true on a
3639 view. However, a conflicts=true request is ignored; the “doc” part of
3640 the value never includes a _conflicts member. Hence you would need to
3641 do another query to determine for each document whether it is in a con‐
3642 flicting state:
3643 $ curl 'http://127.0.0.1:5984/conflict_test/_all_docs?include_docs=true&conflicts=true'
3645 {
3647 "total_rows":1,
3648 "offset":0,
3649 "rows":[
3650 {
3651 "id":"test",
3652 "key":"test",
3653 "value":{"rev":"2-b91bb807b4685080c6a651115ff558f5"},
3654 "doc":{
3655 "_id":"test",
3656 "_rev":"2-b91bb807b4685080c6a651115ff558f5",
3657 "hello":"bar"
3658 }
3659 }
3660 ]
3661 }
3662 $ curl 'http://127.0.0.1:5984/conflict_test/test?conflicts=true'
3664 {
3666 "_id":"test",
3667 "_rev":"2-b91bb807b4685080c6a651115ff558f5",
3668 "hello":"bar",
3669 "_conflicts":[
3670 "2-65db2a11b5172bf928e3bcf59f728970",
3671 "2-5bc3c6319edf62d4c624277fdd0ae191"
3672 ]
3673 }
3674 View map functions
3676 Views only get the winning revision of a document. However they do also
3677 get a _conflicts member if there are any conflicting revisions. This
3678 means you can write a view whose job is specifically to locate docu‐
3679 ments with conflicts. Here is a simple map function which achieves
3680 this:
3681 function(doc) {
3683 if (doc._conflicts) {
3684 emit(null, [doc._rev].concat(doc._conflicts));
3685 }
3686 }
3687 which gives the following output:
3689 {
3691 "total_rows":1,
3692 "offset":0,
3693 "rows":[
3694 {
3695 "id":"test",
3696 "key":null,
3697 "value":[
3698 "2-b91bb807b4685080c6a651115ff558f5",
3699 "2-65db2a11b5172bf928e3bcf59f728970",
3700 "2-5bc3c6319edf62d4c624277fdd0ae191"
3701 ]
3702 }
3703 ]
3704 }
3705 If you do this, you can have a separate “sweep” process which periodi‐
3707 cally scans your database, looks for documents which have conflicts,
3708 fetches the conflicting revisions, and resolves them.
3709 Whilst this keeps the main application simple, the problem with this
3711 approach is that there will be a window between a conflict being intro‐
3712 duced and it being resolved. From a user’s viewpoint, this may appear
3713 that the document they just saved successfully may suddenly lose their
3714 changes, only to be resurrected some time later. This may or may not be
3715 acceptable.
3716 Also, it’s easy to forget to start the sweeper, or not to implement it
3718 properly, and this will introduce odd behaviour which will be hard to
3719 track down.
3720 CouchDB’s “winning” revision algorithm may mean that information drops
3722 out of a view until a conflict has been resolved. Consider Bob’s busi‐
3723 ness card again; suppose Alice has a view which emits mobile numbers,
3724 so that her telephony application can display the caller’s name based
3725 on caller ID. If there are conflicting documents with Bob’s old and new
3726 mobile numbers, and they happen to be resolved in favour of Bob’s old
3727 number, then the view won’t be able to recognise his new one. In this
3728 particular case, the application might have preferred to put informa‐
3729 tion from both the conflicting documents into the view, but this cur‐
3730 rently isn’t possible.
3731 Suggested algorithm to fetch a document with conflict resolution:
3733 1. Get document via GET docid?conflicts=true request
3735 2. For each member in the _conflicts array call GET docid?rev=xxx. If
3737 any errors occur at this stage, restart from step 1. (There could
3738 be a race where someone else has already resolved this conflict and
3739 deleted that rev)
3740 3. Perform application-specific merging
3742 4. Write _bulk_docs with an update to the first rev and deletes of the
3744 other revs.
3745 This could either be done on every read (in which case you could
3747 replace all calls to GET in your application with calls to a library
3748 which does the above), or as part of your sweeper code.
3749 And here is an example of this in Ruby using the low-level RestClient:
3751 require 'rubygems'
3753 require 'rest_client'
3754 require 'json'
3755 DB="http://127.0.0.1:5984/conflict_test"
3756 # Write multiple documents
3758 def writem(docs)
3759 JSON.parse(RestClient.post("#{DB}/_bulk_docs", {
3760 "docs" => docs,
3761 }.to_json))
3762 end
3763 # Write one document, return the rev
3765 def write1(doc, id=nil, rev=nil)
3766 doc['_id'] = id if id
3767 doc['_rev'] = rev if rev
3768 writem([doc]).first['rev']
3769 end
3770 # Read a document, return *all* revs
3772 def read1(id)
3773 retries = 0
3774 loop do
3775 # FIXME: escape id
3776 res = [JSON.parse(RestClient.get("#{DB}/#{id}?conflicts=true"))]
3777 if revs = res.first.delete('_conflicts')
3778 begin
3779 revs.each do |rev|
3780 res << JSON.parse(RestClient.get("#{DB}/#{id}?rev=#{rev}"))
3781 end
3782 rescue
3783 retries += 1
3784 raise if retries >= 5
3785 next
3786 end
3787 end
3788 return res
3789 end
3790 end
3791 # Create DB
3793 RestClient.delete DB rescue nil
3794 RestClient.put DB, {}.to_json
3795 # Write a document
3797 rev1 = write1({"hello"=>"xxx"},"test")
3798 p read1("test")
3799 # Make three conflicting versions
3801 write1({"hello"=>"foo"},"test",rev1)
3802 write1({"hello"=>"bar"},"test",rev1)
3803 write1({"hello"=>"baz"},"test",rev1)
3804 res = read1("test")
3806 p res
3807 # Now let's replace these three with one
3809 res.first['hello'] = "foo+bar+baz"
3810 res.each_with_index do |r,i|
3811 unless i == 0
3812 r.replace({'_id'=>r['_id'], '_rev'=>r['_rev'], '_deleted'=>true})
3813 end
3814 end
3815 writem(res)
3816 p read1("test")
3818 An application written this way never has to deal with a PUT 409, and
3820 is automatically multi-master capable.
3821 You can see that it’s straightforward enough when you know what you’re
3823 doing. It’s just that CouchDB doesn’t currently provide a convenient
3824 HTTP API for “fetch all conflicting revisions”, nor “PUT to supersede
3825 these N revisions”, so you need to wrap these yourself. At the time of
3826 writing, there are no known client-side libraries which provide support
3827 for this.
3828 Merging and revision history
3830 Actually performing the merge is an application-specific function. It
3831 depends on the structure of your data. Sometimes it will be easy: e.g.
3832 if a document contains a list which is only ever appended to, then you
3833 can perform a union of the two list versions.
3834 Some merge strategies look at the changes made to an object, compared
3836 to its previous version. This is how Git’s merge function works.
3837 For example, to merge Bob’s business card versions v2a and v2b, you
3839 could look at the differences between v1 and v2b, and then apply these
3840 changes to v2a as well.
3841 With CouchDB, you can sometimes get hold of old revisions of a docu‐
3843 ment. For example, if you fetch /db/bob?rev=v2b&revs_info=true you’ll
3844 get a list of the previous revision ids which ended up with revision
3845 v2b. Doing the same for v2a you can find their common ancestor revi‐
3846 sion. However if the database has been compacted, the content of that
3847 document revision will have been lost. revs_info will still show that
3848 v1 was an ancestor, but report it as “missing”:
3849 BEFORE COMPACTION AFTER COMPACTION
3851 ,-> v2a v2a
3853 v1
3854 `-> v2b v2b
3855 So if you want to work with diffs, the recommended way is to store
3857 those diffs within the new revision itself. That is: when you replace
3858 v1 with v2a, include an extra field or attachment in v2a which says
3859 which fields were changed from v1 to v2a. This unfortunately does mean
3860 additional book-keeping for your application.
3861 Comparison with other replicating data stores
3863 The same issues arise with other replicating systems, so it can be
3864 instructive to look at these and see how they compare with CouchDB.
3865 Please feel free to add other examples.
3866 Unison
3868 Unison is a bi-directional file synchronisation tool. In this case, the
3869 business card would be a file, say bob.vcf.
3870 When you run unison, changes propagate both ways. If a file has changed
3872 on one side but not the other, the new replaces the old. Unison main‐
3873 tains a local state file so that it knows whether a file has changed
3874 since the last successful replication.
3875 In our example it has changed on both sides. Only one file called
3877 bob.vcf can exist within the file system. Unison solves the problem by
3878 simply ducking out: the user can choose to replace the remote version
3879 with the local version, or vice versa (both of which would lose data),
3880 but the default action is to leave both sides unchanged.
3881 From Alice’s point of view, at least this is a simple solution. When‐
3883 ever she’s on the desktop she’ll see the version she last edited on the
3884 desktop, and whenever she’s on the laptop she’ll see the version she
3885 last edited there.
3886 But because no replication has actually taken place, the data is not
3888 protected. If her laptop hard drive dies, she’ll lose all her changes
3889 made on the laptop; ditto if her desktop hard drive dies.
3890 It’s up to her to copy across one of the versions manually (under a
3892 different filename), merge the two, and then finally push the merged
3893 version to the other side.
3894 Note also that the original file (version v1) has been lost at this
3896 point. So it’s not going to be known from inspection alone whether v2a
3897 or v2b has the most up-to-date E-mail address for Bob, or which version
3898 has the most up-to-date mobile number. Alice has to remember which one
3899 she entered last.
3900 Git
3902 Git is a well-known distributed source control system. Like Unison, Git
3903 deals with files. However, Git considers the state of a whole set of
3904 files as a single object, the “tree”. Whenever you save an update, you
3905 create a “commit” which points to both the updated tree and the previ‐
3906 ous commit(s), which in turn point to the previous tree(s). You there‐
3907 fore have a full history of all the states of the files. This history
3908 forms a branch, and a pointer is kept to the tip of the branch, from
3909 which you can work backwards to any previous state. The “pointer” is an
3910 SHA1 hash of the tip commit.
3911 If you are replicating with one or more peers, a separate branch is
3913 made for each of those peers. For example, you might have:
3914 master -- my local branch
3916 remotes/foo/master -- branch on peer 'foo'
3917 remotes/bar/master -- branch on peer 'bar'
3918 In the regular workflow, replication is a “pull”, importing changes
3920 from a remote peer into the local repository. A “pull” does two things:
3921 first “fetch” the state of the peer into the remote tracking branch for
3922 that peer; and then attempt to “merge” those changes into the local
3923 branch.
3924 Now let’s consider the business card. Alice has created a Git repo con‐
3926 taining bob.vcf, and cloned it across to the other machine. The
3927 branches look like this, where AAAAAAAA is the SHA1 of the commit:
3928 ---------- desktop ---------- ---------- laptop ----------
3930 master: AAAAAAAA master: AAAAAAAA
3931 remotes/laptop/master: AAAAAAAA remotes/desktop/master: AAAAAAAA
3932 Now she makes a change on the desktop, and commits it into the desktop
3934 repo; then she makes a different change on the laptop, and commits it
3935 into the laptop repo:
3936 ---------- desktop ---------- ---------- laptop ----------
3938 master: BBBBBBBB master: CCCCCCCC
3939 remotes/laptop/master: AAAAAAAA remotes/desktop/master: AAAAAAAA
3940 Now on the desktop she does git pull laptop. First, the remote objects
3942 are copied across into the local repo and the remote tracking branch is
3943 updated:
3944 ---------- desktop ---------- ---------- laptop ----------
3946 master: BBBBBBBB master: CCCCCCCC
3947 remotes/laptop/master: CCCCCCCC remotes/desktop/master: AAAAAAAA
3948 NOTE:
3950 The repo still contains AAAAAAAA because commits BBBBBBBB and CCCCC‐
3951 CCC point to it.
3952 Then Git will attempt to merge the changes in. Knowing that the parent
3954 commit to CCCCCCCC is AAAAAAAA, it takes a diff between AAAAAAAA and
3955 CCCCCCCC and tries to apply it to BBBBBBBB.
3956 If this is successful, then you’ll get a new version with a merge com‐
3958 mit:
3959 ---------- desktop ---------- ---------- laptop ----------
3961 master: DDDDDDDD master: CCCCCCCC
3962 remotes/laptop/master: CCCCCCCC remotes/desktop/master: AAAAAAAA
3963 Then Alice has to logon to the laptop and run git pull desktop. A simi‐
3965 lar process occurs. The remote tracking branch is updated:
3966 ---------- desktop ---------- ---------- laptop ----------
3968 master: DDDDDDDD master: CCCCCCCC
3969 remotes/laptop/master: CCCCCCCC remotes/desktop/master: DDDDDDDD
3970 Then a merge takes place. This is a special case: CCCCCCCC is one of
3972 the parent commits of DDDDDDDD, so the laptop can fast forward update
3973 from CCCCCCCC to DDDDDDDD directly without having to do any complex
3974 merging. This leaves the final state as:
3975 ---------- desktop ---------- ---------- laptop ----------
3977 master: DDDDDDDD master: DDDDDDDD
3978 remotes/laptop/master: CCCCCCCC remotes/desktop/master: DDDDDDDD
3979 Now this is all and good, but you may wonder how this is relevant when
3981 thinking about CouchDB.
3982 First, note what happens in the case when the merge algorithm fails.
3984 The changes are still propagated from the remote repo into the local
3985 one, and are available in the remote tracking branch. So, unlike Uni‐
3986 son, you know the data is protected. It’s just that the local working
3987 copy may fail to update, or may diverge from the remote version. It’s
3988 up to you to create and commit the combined version yourself, but you
3989 are guaranteed to have all the history you might need to do this.
3990 Note that while it is possible to build new merge algorithms into Git,
3992 the standard ones are focused on line-based changes to source code.
3993 They don’t work well for XML or JSON if it’s presented without any line
3994 breaks.
3995 The other interesting consideration is multiple peers. In this case you
3997 have multiple remote tracking branches, some of which may match your
3998 local branch, some of which may be behind you, and some of which may be
3999 ahead of you (i.e. contain changes that you haven’t yet merged):
4000 master: AAAAAAAA
4002 remotes/foo/master: BBBBBBBB
4003 remotes/bar/master: CCCCCCCC
4004 remotes/baz/master: AAAAAAAA
4005 Note that each peer is explicitly tracked, and therefore has to be
4007 explicitly created. If a peer becomes stale or is no longer needed,
4008 it’s up to you to remove it from your configuration and delete the
4009 remote tracking branch. This is different from CouchDB, which doesn’t
4010 keep any peer state in the database.
4011 Another difference between CouchDB and Git is that it maintains all
4013 history back to time zero - Git compaction keeps diffs between all
4014 those versions in order to reduce size, but CouchDB discards them. If
4015 you are constantly updating a document, the size of a Git repo would
4016 grow forever. It is possible (with some effort) to use “history rewrit‐
4017 ing” to make Git forget commits earlier than a particular one.
4018 What is the CouchDB replication protocol? Is it like Git?
4020 Author Jason Smith
4021 Date 2011-01-29
4023 Source StackOverflow
4025 Key points
4027 If you know Git, then you know how Couch replication works. Replicating
4029 is very similar to pushing or pulling with distributed source managers
4030 like Git.
4031 CouchDB replication does not have its own protocol. A replicator simply
4033 connects to two DBs as a client, then reads from one and writes to the
4034 other. Push replication is reading the local data and updating the
4035 remote DB; pull replication is vice versa.
4036 · Fun fact 1: The replicator is actually an independent Erlang applica‐
4038 tion, in its own process. It connects to both couches, then reads
4039 records from one and writes them to the other.
4040 · Fun fact 2: CouchDB has no way of knowing who is a normal client and
4042 who is a replicator (let alone whether the replication is push or
4043 pull). It all looks like client connections. Some of them read
4044 records. Some of them write records.
4045 Everything flows from the data model
4047 The replication algorithm is trivial, uninteresting. A trained monkey
4049 could design it. It’s simple because the cleverness is the data model,
4050 which has these useful characteristics:
4051 1. Every record in CouchDB is completely independent of all others.
4053 That sucks if you want to do a JOIN or a transaction, but it’s awe‐
4054 some if you want to write a replicator. Just figure out how to
4055 replicate one record, and then repeat that for each record.
4056 2. Like Git, records have a linked-list revision history. A record’s
4058 revision ID is the checksum of its own data. Subsequent revision IDs
4059 are checksums of: the new data, plus the revision ID of the previ‐
4060 ous.
4061 3. In addition to application data ({"name": "Jason", "awesome":
4063 true}), every record stores the evolutionary time line of all previ‐
4064 ous revision IDs leading up to itself.
4065 · Exercise: Take a moment of quiet reflection. Consider any two dif‐
4067 ferent records, A and B. If A’s revision ID appears in B’s time
4068 line, then B definitely evolved from A. Now consider Git’s
4069 fast-forward merges. Do you hear that? That is the sound of your
4070 mind being blown.
4071 4. Git isn’t really a linear list. It has forks, when one parent has
4073 multiple children. CouchDB has that too.
4074 · Exercise: Compare two different records, A and B. A’s revision ID
4076 does not appear in B’s time line; however, one revision ID, C, is
4077 in both A’s and B’s time line. Thus A didn’t evolve from B. B
4078 didn’t evolve from A. But rather, A and B have a common ancestor
4079 C. In Git, that is a “fork.” In CouchDB, it’s a “conflict.”
4080 · In Git, if both children go on to develop their time lines inde‐
4082 pendently, that’s cool. Forks totally support that.
4083 · In CouchDB, if both children go on to develop their time lines
4085 independently, that cool too. Conflicts totally support that.
4086 · Fun fact 3: CouchDB “conflicts” do not correspond to Git “con‐
4088 flicts.” A Couch conflict is a divergent revision history, what
4089 Git calls a “fork.” For this reason the CouchDB community pro‐
4090 nounces “conflict” with a silent n: “co-flicked.”
4091 5. Git also has merges, when one child has multiple parents. CouchDB
4093 sort of has that too.
4094 · In the data model, there is no merge. The client simply marks one
4096 time line as deleted and continues to work with the only extant
4097 time line.
4098 · In the application, it feels like a merge. Typically, the client
4100 merges the data from each time line in an application-specific
4101 way. Then it writes the new data to the time line. In Git, this
4102 is like copying and pasting the changes from branch A into branch
4103 B, then committing to branch B and deleting branch A. The data was
4104 merged, but there was no git merge.
4105 · These behaviors are different because, in Git, the time line
4107 itself is important; but in CouchDB, the data is important and the
4108 time line is incidental—it’s just there to support replication.
4109 That is one reason why CouchDB’s built-in revisioning is inappro‐
4110 priate for storing revision data like a wiki page.
4111 Final notes
4113 At least one sentence in this writeup (possibly this one) is complete
4115 BS.
4116 CouchDB Replication Protocol
4118 Version
4119 3
4120 The CouchDB Replication Protocol is a protocol for synchronising JSON
4122 documents between 2 peers over HTTP/1.1 by using the public CouchDB
4123 REST API and is based on the Apache CouchDB MVCC Data model.
4124 Preface
4126 Language
4127 The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”,
4128 “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in this
4129 document are to be interpreted as described in RFC 2119.
4130 Goals
4132 The primary goal of this specification is to describe the CouchDB
4133 Replication Protocol under the hood.
4134 The secondary goal is to provide enough detailed information about the
4136 protocol to make it easy to build tools on any language and platform
4137 that can synchronize data with CouchDB.
4138 Definitions
4140 JSON: JSON is a text format for the serialization of structured data.
4141 It is described in ECMA-262 and RFC 4627.
4142 URI: A URI is defined by RFC 3986. It can be a URL as defined in RFC
4144 1738.
4145 ID: An identifier (could be a UUID) as described in RFC 4122.
4147 Revision:
4149 A MVCC token value of following pattern: N-sig where N is ALWAYS
4150 a positive integer and sig is the Document signature (custom).
4151 Don’t mix it up with the revision in version control systems!
4152 Leaf Revision:
4154 The last Document Revision in a series of changes. Documents may
4155 have multiple Leaf Revisions (aka Conflict Revisions) due to
4156 concurrent updates.
4157 Document:
4159 A document is a JSON object with an ID and Revision defined in
4160 _id and _rev fields respectively. A Document’s ID MUST be unique
4161 within the Database where it is stored.
4162 Database:
4164 A collection of Documents with a unique URI.
4165 Changes Feed:
4167 A stream of Document-changing events (create, update, delete)
4168 for the specified Database.
4169 Sequence ID:
4171 An ID provided by the Changes Feed. It MUST be incremental, but
4172 MAY NOT always be an integer.
4173 Source:
4175 Database from where the Documents are replicated.
4176 Target:
4178 Database where the Documents are replicated to.
4179 Replication:
4181 The one-way directed synchronization process of Source and Tar‐
4182 get endpoints.
4183 Checkpoint:
4185 Intermediate Recorded Sequence ID used for Replication recovery.
4186 Replicator:
4188 A service or an application which initiates and runs Replica‐
4189 tion.
4190 Filter Function:
4192 A special function of any programming language that is used to
4193 filter Documents during Replication (see filterfun)
4194 Filter Function Name:
4196 An ID of a Filter Function that may be used as a symbolic refer‐
4197 ence (aka callback function) to apply the related Filter Func‐
4198 tion to Replication.
4199 Filtered Replication:
4201 Replication of Documents from Source to Target using a Filter
4202 Function.
4203 Full Replication:
4205 Replication of all Documents from Source to Target.
4206 Push Replication:
4208 Replication process where Source is a local endpoint and Target
4209 is remote.
4210 Pull Replication:
4212 Replication process where Source is a remote endpoint and Target
4213 is local.
4214 Continuous Replication:
4216 Replication that “never stops”: after processing all events from
4217 the Changes Feed, the Replicator doesn’t close the connection,
4218 but awaits new change events from the Source. The connection is
4219 kept alive by periodic heartbeats.
4220 Replication Log:
4222 A special Document that holds Replication history (recorded
4223 Checkpoints and a few more statistics) between Source and Tar‐
4224 get.
4225 Replication ID:
4227 A unique value that unambiguously identifies the Replication
4228 Log.
4229 Replication Protocol Algorithm
4231 The CouchDB Replication Protocol is not magical, but an agreement on
4232 usage of the public CouchDB HTTP REST API to enable Documents to be
4233 replicated from Source to Target.
4234 The reference implementation, written in Erlang, is provided by the
4236 couch_replicator module in Apache CouchDB.
4237 It is RECOMMENDED that one follow this algorithm specification, use the
4239 same HTTP endpoints, and run requests with the same parameters to pro‐
4240 vide a completely compatible implementation. Custom Replicator imple‐
4241 mentations MAY use different HTTP API endpoints and request parameters
4242 depending on their local specifics and they MAY implement only part of
4243 the Replication Protocol to run only Push or Pull Replication. However,
4244 while such solutions could also run the Replication process, they loose
4245 compatibility with the CouchDB Replicator.
4246 Verify Peers
4248 + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - +
4249 ' Verify Peers: '
4250 ' '
4251 ' 404 Not Found +--------------------------------+ '
4252 ' +----------------------- | Check Source Existence | '
4253 ' | +--------------------------------+ '
4254 ' | | HEAD /source | '
4255 ' | +--------------------------------+ '
4256 ' | | '
4257 ' | | 200 OK '
4258 ' | v '
4259 ' | +--------------------------------+ '
4260 ' | | Check Target Existence | ----+ '
4261 ' | +--------------------------------+ | '
4262 ' | | HEAD /target | | '
4263 ' | +--------------------------------+ | '
4264 ' | | | '
4265 ' | | 404 Not Found | '
4266 ' v v | '
4267 ' +-------+ No +--------------------------------+ | '
4268 ' | Abort | <----------------- | Create Target? | | '
4269 ' +-------+ +--------------------------------+ | '
4270 ' ^ | | '
4271 ' | | Yes | '
4272 ' | v | '
4273 ' | Failure +--------------------------------+ | '
4274 ' +----------------------- | Create Target | | '
4275 ' +--------------------------------+ | '
4276 ' | PUT /target | | '
4277 ' +--------------------------------+ | '
4278 ' | | '
4279 ' | 201 Created 200 OK | '
4280 ' | | '
4281 + - - - - - - - - - - - - - - - - | - - - - - - - - - - - - - - - - - | - +
4282 | |
4283 + - - - - - - - - - - - - - - - - | - - - - - - - - - - - - - - - - - | - +
4284 ' Get Peers Information: | | '
4285 ' +------------------------------------+ '
4286 ' | '
4287 ' v '
4288 ' +--------------------------------+ '
4289 ' | Get Source Information | '
4290 ' +--------------------------------+ '
4291 ' '
4292 + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - +
4293 The Replicator MUST ensure that both Source and Target exist by using
4295 HEAD /{db} requests.
4296 Check Source Existence
4298 Request:
4299 HEAD /source HTTP/1.1
4301 Host: localhost:5984
4302 User-Agent: CouchDB
4303 Response:
4305 HTTP/1.1 200 OK
4307 Cache-Control: must-revalidate
4308 Content-Type: application/json
4309 Date: Sat, 05 Oct 2013 08:50:39 GMT
4310 Server: CouchDB (Erlang/OTP)
4311 Check Target Existence
4313 Request:
4314 HEAD /target HTTP/1.1
4316 Host: localhost:5984
4317 User-Agent: CouchDB
4318 Response:
4320 HTTP/1.1 200 OK
4322 Cache-Control: must-revalidate
4323 Content-Type: application/json
4324 Date: Sat, 05 Oct 2013 08:51:11 GMT
4325 Server: CouchDB (Erlang/OTP)
4326 Create Target?
4328 In case of a non-existent Target, the Replicator MAY make a PUT /{db}
4329 request to create the Target:
4330 Request:
4331 PUT /target HTTP/1.1
4333 Accept: application/json
4334 Host: localhost:5984
4335 User-Agent: CouchDB
4336 Response:
4338 HTTP/1.1 201 Created
4340 Content-Length: 12
4341 Content-Type: application/json
4342 Date: Sat, 05 Oct 2013 08:58:41 GMT
4343 Server: CouchDB (Erlang/OTP)
4344 {
4346 "ok": true
4347 }
4348 However, the Replicator’s PUT request MAY NOT succeeded due to insuffi‐
4350 cient privileges (which are granted by the provided credential) and so
4351 receive a 401 Unauthorized or a 403 Forbidden error. Such errors SHOULD
4352 be expected and well handled:
4353 HTTP/1.1 500 Internal Server Error
4355 Cache-Control: must-revalidate
4356 Content-Length: 108
4357 Content-Type: application/json
4358 Date: Fri, 09 May 2014 13:50:32 GMT
4359 Server: CouchDB (Erlang OTP)
4360 {
4362 "error": "unauthorized",
4363 "reason": "unauthorized to access or create database http://localhost:5984/target"
4364 }
4365 Abort
4367 In case of a non-existent Source or Target, Replication SHOULD be
4368 aborted with an HTTP error response:
4369 HTTP/1.1 500 Internal Server Error
4371 Cache-Control: must-revalidate
4372 Content-Length: 56
4373 Content-Type: application/json
4374 Date: Sat, 05 Oct 2013 08:55:29 GMT
4375 Server: CouchDB (Erlang OTP)
4376 {
4378 "error": "db_not_found",
4379 "reason": "could not open source"
4380 }
4381 Get Peers Information
4383 + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -+
4384 ' Verify Peers: '
4385 ' +------------------------+ '
4386 ' | Check Target Existence | '
4387 ' +------------------------+ '
4388 ' | '
4389 ' | 200 OK '
4390 ' | '
4391 + - - - - - - - - - - - - - - - - - - | - - - - - - - - - - - - - -+
4392 |
4393 + - - - - - - - - - - - - - - - - - - | - - - - - - - - - - - - - -+
4394 ' Get Peers Information: | '
4395 ' v '
4396 ' +------------------------+ '
4397 ' | Get Source Information | '
4398 ' +------------------------+ '
4399 ' | GET /source | '
4400 ' +------------------------+ '
4401 ' | '
4402 ' | 200 OK '
4403 ' v '
4404 ' +------------------------+ '
4405 ' | Get Target Information | '
4406 ' +------------------------+ '
4407 ' | GET /target | '
4408 ' +------------------------+ '
4409 ' | '
4410 ' | 200 OK '
4411 ' | '
4412 + - - - - - - - - - - - - - - - - - - | - - - - - - - - - - - - - -+
4413 |
4414 + - - - - - - - - - - - - - - - - - - | - - - - - - - - - - - - - -+
4415 ' Find Common Ancestry: | '
4416 ' | '
4417 ' v '
4418 ' +-------------------------+ '
4419 ' | Generate Replication ID | '
4420 ' +-------------------------+ '
4421 ' '
4422 + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -+
4423 The Replicator retrieves basic information both from Source and Target
4425 using GET /{db} requests. The GET response MUST contain JSON objects
4426 with the following mandatory fields:
4427 · instance_start_time (string): Always "0". (Returned for legacy rea‐
4429 sons.)
4430 · update_seq (number / string): The current database Sequence ID.
4432 Any other fields are optional. The information that the Replicator
4434 needs is the update_seq field: this value will be used to define a tem‐
4435 porary (because Database data is subject to change) upper bound for
4436 changes feed listening and statistic calculating to show proper Repli‐
4437 cation progress.
4438 Get Source Information
4440 Request:
4441 GET /source HTTP/1.1
4443 Accept: application/json
4444 Host: localhost:5984
4445 User-Agent: CouchDB
4446 Response:
4448 HTTP/1.1 200 OK
4450 Cache-Control: must-revalidate
4451 Content-Length: 256
4452 Content-Type: application/json
4453 Date: Tue, 08 Oct 2013 07:53:08 GMT
4454 Server: CouchDB (Erlang OTP)
4455 {
4457 "committed_update_seq": 61772,
4458 "compact_running": false,
4459 "db_name": "source",
4460 "disk_format_version": 6,
4461 "doc_count": 41961,
4462 "doc_del_count": 3807,
4463 "instance_start_time": "0",
4464 "purge_seq": 0,
4465 "sizes": {
4466 "active": 70781613961,
4467 "disk": 79132913799,
4468 "external": 72345632950
4469 },
4470 "update_seq": 61772
4471 }
4472 Get Target Information
4474 Request:
4475 GET /target/ HTTP/1.1
4477 Accept: application/json
4478 Host: localhost:5984
4479 User-Agent: CouchDB
4480 Response:
4482 HTTP/1.1 200 OK
4484 Content-Length: 363
4485 Content-Type: application/json
4486 Date: Tue, 08 Oct 2013 12:37:01 GMT
4487 Server: CouchDB (Erlang/OTP)
4488 {
4490 "compact_running": false,
4491 "db_name": "target",
4492 "disk_format_version": 5,
4493 "doc_count": 1832,
4494 "doc_del_count": 1,
4495 "instance_start_time": "0",
4496 "purge_seq": 0,
4497 "sizes": {
4498 "active": 50829452,
4499 "disk": 77001455,
4500 "external": 60326450
4501 },
4502 "update_seq": "1841-g1AAAADveJzLYWBgYMlgTmGQT0lKzi9KdUhJMtbLSs1LLUst0k"
4503 }
4504 Find Common Ancestry
4506 + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - +
4507 ' Get Peers Information: '
4508 ' '
4509 ' +-------------------------------------------+ '
4510 ' | Get Target Information | '
4511 ' +-------------------------------------------+ '
4512 ' | '
4513 + - - - - - - - - - - - - - - - | - - - - - - - - - - - - - - - - - - - - - +
4514 |
4515 + - - - - - - - - - - - - - - - | - - - - - - - - - - - - - - - - - - - - - +
4516 ' Find Common Ancestry: v '
4517 ' +-------------------------------------------+ '
4518 ' | Generate Replication ID | '
4519 ' +-------------------------------------------+ '
4520 ' | '
4521 ' | '
4522 ' v '
4523 ' +-------------------------------------------+ '
4524 ' | Get Replication Log from Source | '
4525 ' +-------------------------------------------+ '
4526 ' | GET /source/_local/replication-id | '
4527 ' +-------------------------------------------+ '
4528 ' | '
4529 ' | 200 OK '
4530 ' | 404 Not Found '
4531 ' v '
4532 ' +-------------------------------------------+ '
4533 ' | Get Replication Log from Target | '
4534 ' +-------------------------------------------+ '
4535 ' | GET /target/_local/replication-id | '
4536 ' +-------------------------------------------+ '
4537 ' | '
4538 ' | 200 OK '
4539 ' | 404 Not Found '
4540 ' v '
4541 ' +-------------------------------------------+ '
4542 ' | Compare Replication Logs | '
4543 ' +-------------------------------------------+ '
4544 ' | '
4545 ' | Use latest common sequence as start point '
4546 ' | '
4547 + - - - - - - - - - - - - - - - | - - - - - - - - - - - - - - - - - - - - - +
4548 |
4549 |
4550 + - - - - - - - - - - - - - - - | - - - - - - - - - - - - - - - - - - - - - +
4551 ' Locate Changed Documents: | '
4552 ' | '
4553 ' v '
4554 ' +-------------------------------------------+ '
4555 ' | Listen Source Changes Feed | '
4556 ' +-------------------------------------------+ '
4557 ' '
4558 + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - +
4559 Generate Replication ID
4561 Before Replication is started, the Replicator MUST generate a Replica‐
4562 tion ID. This value is used to track Replication History, resume and
4563 continue previously interrupted Replication process.
4564 The Replication ID generation algorithm is implementation specific.
4566 Whatever algorithm is used it MUST uniquely identify the Replication
4567 process. CouchDB’s Replicator, for example, uses the following factors
4568 in generating a Replication ID:
4569 · Persistent Peer UUID value. For CouchDB, the local Server UUID is
4571 used
4572 · Source and Target URI and if Source or Target are local or remote
4574 Databases
4575 · If Target needed to be created
4577 · If Replication is Continuous
4579 · Any custom headers
4581 · Filter function code if used
4583 · Changes Feed query parameters, if any
4585 NOTE:
4587 See couch_replicator_ids.erl for an example of a Replication ID gen‐
4588 eration implementation.
4589 Retrieve Replication Logs from Source and Target
4591 Once the Replication ID has been generated, the Replicator SHOULD
4592 retrieve the Replication Log from both Source and Target using GET
4593 /{db}/_local/{docid}:
4594 Request:
4595 GET /source/_local/b3e44b920ee2951cb2e123b63044427a HTTP/1.1
4597 Accept: application/json
4598 Host: localhost:5984
4599 User-Agent: CouchDB
4600 Response:
4602 HTTP/1.1 200 OK
4604 Cache-Control: must-revalidate
4605 Content-Length: 1019
4606 Content-Type: application/json
4607 Date: Thu, 10 Oct 2013 06:18:56 GMT
4608 ETag: "0-8"
4609 Server: CouchDB (Erlang OTP)
4610 {
4612 "_id": "_local/b3e44b920ee2951cb2e123b63044427a",
4613 "_rev": "0-8",
4614 "history": [
4615 {
4616 "doc_write_failures": 0,
4617 "docs_read": 2,
4618 "docs_written": 2,
4619 "end_last_seq": 5,
4620 "end_time": "Thu, 10 Oct 2013 05:56:38 GMT",
4621 "missing_checked": 2,
4622 "missing_found": 2,
4623 "recorded_seq": 5,
4624 "session_id": "d5a34cbbdafa70e0db5cb57d02a6b955",
4625 "start_last_seq": 3,
4626 "start_time": "Thu, 10 Oct 2013 05:56:38 GMT"
4627 },
4628 {
4629 "doc_write_failures": 0,
4630 "docs_read": 1,
4631 "docs_written": 1,
4632 "end_last_seq": 3,
4633 "end_time": "Thu, 10 Oct 2013 05:56:12 GMT",
4634 "missing_checked": 1,
4635 "missing_found": 1,
4636 "recorded_seq": 3,
4637 "session_id": "11a79cdae1719c362e9857cd1ddff09d",
4638 "start_last_seq": 2,
4639 "start_time": "Thu, 10 Oct 2013 05:56:12 GMT"
4640 },
4641 {
4642 "doc_write_failures": 0,
4643 "docs_read": 2,
4644 "docs_written": 2,
4645 "end_last_seq": 2,
4646 "end_time": "Thu, 10 Oct 2013 05:56:04 GMT",
4647 "missing_checked": 2,
4648 "missing_found": 2,
4649 "recorded_seq": 2,
4650 "session_id": "77cdf93cde05f15fcb710f320c37c155",
4651 "start_last_seq": 0,
4652 "start_time": "Thu, 10 Oct 2013 05:56:04 GMT"
4653 }
4654 ],
4655 "replication_id_version": 3,
4656 "session_id": "d5a34cbbdafa70e0db5cb57d02a6b955",
4657 "source_last_seq": 5
4658 }
4659 The Replication Log SHOULD contain the following fields:
4661 · history (array of object): Replication history. Required
4663 · doc_write_failures (number): Number of failed writes
4665 · docs_read (number): Number of read documents
4667 · docs_written (number): Number of written documents
4669 · end_last_seq (number): Last processed Update Sequence ID
4671 · end_time (string): Replication completion timestamp in RFC 5322
4673 format
4674 · missing_checked (number): Number of checked revisions on Source
4676 · missing_found (number): Number of missing revisions found on Target
4678 · recorded_seq (number): Recorded intermediate Checkpoint. Required
4680 · session_id (string): Unique session ID. Commonly, a random UUID
4682 value is used. Required
4683 · start_last_seq (number): Start update Sequence ID
4685 · start_time (string): Replication start timestamp in RFC 5322 format
4687 · replication_id_version (number): Replication protocol version.
4689 Defines Replication ID calculation algorithm, HTTP API calls and the
4690 others routines. Required
4691 · session_id (string): Unique ID of the last session. Shortcut to the
4693 session_id field of the latest history object. Required
4694 · source_last_seq (number): Last processed Checkpoint. Shortcut to the
4696 recorded_seq field of the latest history object. Required
4697 This request MAY fall with a 404 Not Found response:
4699 Request:
4700 GET /source/_local/b6cef528f67aa1a8a014dd1144b10e09 HTTP/1.1
4702 Accept: application/json
4703 Host: localhost:5984
4704 User-Agent: CouchDB
4705 Response:
4707 HTTP/1.1 404 Object Not Found
4709 Cache-Control: must-revalidate
4710 Content-Length: 41
4711 Content-Type: application/json
4712 Date: Tue, 08 Oct 2013 13:31:10 GMT
4713 Server: CouchDB (Erlang OTP)
4714 {
4716 "error": "not_found",
4717 "reason": "missing"
4718 }
4719 That’s OK. This means that there is no information about the current
4721 Replication so it must not have been run previously and as such the
4722 Replicator MUST run a Full Replication.
4723 Compare Replication Logs
4725 If the Replication Logs are successfully retrieved from both Source and
4726 Target then the Replicator MUST determine their common ancestry by fol‐
4727 lowing the next algorithm:
4728 · Compare session_id values for the chronological last session - if
4730 they match both Source and Target have a common Replication history
4731 and it seems to be valid. Use source_last_seq value for the startup
4732 Checkpoint
4733 · In case of mismatch, iterate over the history collection to search
4735 for the latest (chronologically) common session_id for Source and
4736 Target. Use value of recorded_seq field as startup Checkpoint
4737 If Source and Target has no common ancestry, the Replicator MUST run
4739 Full Replication.
4740 Locate Changed Documents
4742 + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - +
4743 ' Find Common Ancestry: '
4744 ' '
4745 ' +------------------------------+ '
4746 ' | Compare Replication Logs | '
4747 ' +------------------------------+ '
4748 ' | '
4749 ' | '
4750 + - - - - - - - - - - - - - - - - - - - - | - - - - - - - - - - - - - - - +
4751 |
4752 + - - - - - - - - - - - - - - - - - - - - | - - - - - - - - - - - - - - - +
4753 ' Locate Changed Documents: | '
4754 ' | '
4755 ' | '
4756 ' v '
4757 ' +-------------------------------+ '
4758 ' +------> | Listen to Changes Feed | -----+ '
4759 ' | +-------------------------------+ | '
4760 ' | | GET /source/_changes | | '
4761 ' | | POST /source/_changes | | '
4762 ' | +-------------------------------+ | '
4763 ' | | | '
4764 ' | | | '
4765 ' | There are new changes | | No more changes '
4766 ' | | | '
4767 ' | v v '
4768 ' | +-------------------------------+ +-----------------------+ '
4769 ' | | Read Batch of Changes | | Replication Completed | '
4770 ' | +-------------------------------+ +-----------------------+ '
4771 ' | | '
4772 ' | No | '
4773 ' | v '
4774 ' | +-------------------------------+ '
4775 ' | | Compare Documents Revisions | '
4776 ' | +-------------------------------+ '
4777 ' | | POST /target/_revs_diff | '
4778 ' | +-------------------------------+ '
4779 ' | | '
4780 ' | 200 OK | '
4781 ' | v '
4782 ' | +-------------------------------+ '
4783 ' +------- | Any Differences Found? | '
4784 ' +-------------------------------+ '
4785 ' | '
4786 ' Yes | '
4787 ' | '
4788 + - - - - - - - - - - - - - - - - - - - - | - - - - - - - - - - - - - - - +
4789 |
4790 + - - - - - - - - - - - - - - - - - - - - | - - - - - - - - - - - - - - - +
4791 ' Replicate Changes: | '
4792 ' v '
4793 ' +-------------------------------+ '
4794 ' | Fetch Next Changed Document | '
4795 ' +-------------------------------+ '
4796 ' '
4797 + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - +
4798 Listen to Changes Feed
4800 When the start up Checkpoint has been defined, the Replicator SHOULD
4801 read the Source’s Changes Feed by using a GET /{db}/_changes request.
4802 This request MUST be made with the following query parameters:
4803 · feed parameter defines the Changes Feed response style: for Continu‐
4805 ous Replication the continuous value SHOULD be used, otherwise - nor‐
4806 mal.
4807 · style=all_docs query parameter tells the Source that it MUST include
4809 all Revision leaves for each document’s event in output.
4810 · For Continuous Replication the heartbeat parameter defines the heart‐
4812 beat period in milliseconds. The RECOMMENDED value by default is
4813 10000 (10 seconds).
4814 · If a startup Checkpoint was found during the Replication Logs compar‐
4816 ison, the since query parameter MUST be passed with this value. In
4817 case of Full Replication it MAY be 0 (number zero) or be omitted.
4818 Additionally, the filter query parameter MAY be specified to enable a
4820 filter function on Source side. Other custom parameters MAY also be
4821 provided.
4822 Read Batch of Changes
4824 Reading the whole feed in a single shot may not be an optimal use of
4825 resources. It is RECOMMENDED to process the feed in small chunks. How‐
4826 ever, there is no specific recommendation on chunk size since it is
4827 heavily dependent on available resources: large chunks requires more
4828 memory while they reduce I/O operations and vice versa.
4829 Note, that Changes Feed output format is different for a request with
4831 feed=normal and with feed=continuous query parameter.
4832 Normal Feed:
4834 Request:
4835 GET /source/_changes?feed=normal&style=all_docs&heartbeat=10000 HTTP/1.1
4837 Accept: application/json
4838 Host: localhost:5984
4839 User-Agent: CouchDB
4840 Response:
4842 HTTP/1.1 200 OK
4844 Cache-Control: must-revalidate
4845 Content-Type: application/json
4846 Date: Fri, 09 May 2014 16:20:41 GMT
4847 Server: CouchDB (Erlang OTP)
4848 Transfer-Encoding: chunked
4849 {"results":[
4851 {"seq":14,"id":"f957f41e","changes":[{"rev":"3-46a3"}],"deleted":true}
4852 {"seq":29,"id":"ddf339dd","changes":[{"rev":"10-304b"}]}
4853 {"seq":37,"id":"d3cc62f5","changes":[{"rev":"2-eec2"}],"deleted":true}
4854 {"seq":39,"id":"f13bd08b","changes":[{"rev":"1-b35d"}]}
4855 {"seq":41,"id":"e0a99867","changes":[{"rev":"2-c1c6"}]}
4856 {"seq":42,"id":"a75bdfc5","changes":[{"rev":"1-967a"}]}
4857 {"seq":43,"id":"a5f467a0","changes":[{"rev":"1-5575"}]}
4858 {"seq":45,"id":"470c3004","changes":[{"rev":"11-c292"}]}
4859 {"seq":46,"id":"b1cb8508","changes":[{"rev":"10-ABC"}]}
4860 {"seq":47,"id":"49ec0489","changes":[{"rev":"157-b01f"},{"rev":"123-6f7c"}]}
4861 {"seq":49,"id":"dad10379","changes":[{"rev":"1-9346"},{"rev":"6-5b8a"}]}
4862 {"seq":50,"id":"73464877","changes":[{"rev":"1-9f08"}]}
4863 {"seq":51,"id":"7ae19302","changes":[{"rev":"1-57bf"}]}
4864 {"seq":63,"id":"6a7a6c86","changes":[{"rev":"5-acf6"}],"deleted":true}
4865 {"seq":64,"id":"dfb9850a","changes":[{"rev":"1-102f"}]}
4866 {"seq":65,"id":"c532afa7","changes":[{"rev":"1-6491"}]}
4867 {"seq":66,"id":"af8a9508","changes":[{"rev":"1-3db2"}]}
4868 {"seq":67,"id":"caa3dded","changes":[{"rev":"1-6491"}]}
4869 {"seq":68,"id":"79f3b4e9","changes":[{"rev":"1-102f"}]}
4870 {"seq":69,"id":"1d89d16f","changes":[{"rev":"1-3db2"}]}
4871 {"seq":71,"id":"abae7348","changes":[{"rev":"2-7051"}]}
4872 {"seq":77,"id":"6c25534f","changes":[{"rev":"9-CDE"},{"rev":"3-00e7"},{"rev":"1-ABC"}]}
4873 {"seq":78,"id":"SpaghettiWithMeatballs","changes":[{"rev":"22-5f95"}]}
4874 ],
4875 "last_seq":78}
4876 Continuous Feed:
4878 Request:
4879 GET /source/_changes?feed=continuous&style=all_docs&heartbeat=10000 HTTP/1.1
4881 Accept: application/json
4882 Host: localhost:5984
4883 User-Agent: CouchDB
4884 Response:
4886 HTTP/1.1 200 OK
4888 Cache-Control: must-revalidate
4889 Content-Type: application/json
4890 Date: Fri, 09 May 2014 16:22:22 GMT
4891 Server: CouchDB (Erlang OTP)
4892 Transfer-Encoding: chunked
4893 {"seq":14,"id":"f957f41e","changes":[{"rev":"3-46a3"}],"deleted":true}
4895 {"seq":29,"id":"ddf339dd","changes":[{"rev":"10-304b"}]}
4896 {"seq":37,"id":"d3cc62f5","changes":[{"rev":"2-eec2"}],"deleted":true}
4897 {"seq":39,"id":"f13bd08b","changes":[{"rev":"1-b35d"}]}
4898 {"seq":41,"id":"e0a99867","changes":[{"rev":"2-c1c6"}]}
4899 {"seq":42,"id":"a75bdfc5","changes":[{"rev":"1-967a"}]}
4900 {"seq":43,"id":"a5f467a0","changes":[{"rev":"1-5575"}]}
4901 {"seq":45,"id":"470c3004","changes":[{"rev":"11-c292"}]}
4902 {"seq":46,"id":"b1cb8508","changes":[{"rev":"10-ABC"}]}
4903 {"seq":47,"id":"49ec0489","changes":[{"rev":"157-b01f"},{"rev":"123-6f7c"}]}
4904 {"seq":49,"id":"dad10379","changes":[{"rev":"1-9346"},{"rev":"6-5b8a"}]}
4905 {"seq":50,"id":"73464877","changes":[{"rev":"1-9f08"}]}
4906 {"seq":51,"id":"7ae19302","changes":[{"rev":"1-57bf"}]}
4907 {"seq":63,"id":"6a7a6c86","changes":[{"rev":"5-acf6"}],"deleted":true}
4908 {"seq":64,"id":"dfb9850a","changes":[{"rev":"1-102f"}]}
4909 {"seq":65,"id":"c532afa7","changes":[{"rev":"1-6491"}]}
4910 {"seq":66,"id":"af8a9508","changes":[{"rev":"1-3db2"}]}
4911 {"seq":67,"id":"caa3dded","changes":[{"rev":"1-6491"}]}
4912 {"seq":68,"id":"79f3b4e9","changes":[{"rev":"1-102f"}]}
4913 {"seq":69,"id":"1d89d16f","changes":[{"rev":"1-3db2"}]}
4914 {"seq":71,"id":"abae7348","changes":[{"rev":"2-7051"}]}
4915 {"seq":75,"id":"SpaghettiWithMeatballs","changes":[{"rev":"21-5949"}]}
4916 {"seq":77,"id":"6c255","changes":[{"rev":"9-CDE"},{"rev":"3-00e7"},{"rev":"1-ABC"}]}
4917 {"seq":78,"id":"SpaghettiWithMeatballs","changes":[{"rev":"22-5f95"}]}
4918 For both Changes Feed formats record-per-line style is preserved to
4920 simplify iterative fetching and decoding JSON objects with less memory
4921 footprint.
4922 Calculate Revision Difference
4924 After reading the batch of changes from the Changes Feed, the Replica‐
4925 tor forms a JSON mapping object for Document ID and related leaf Revi‐
4926 sions and sends the result to Target via a POST /{db}/_revs_diff
4927 request:
4928 Request:
4929 POST /target/_revs_diff HTTP/1.1
4931 Accept: application/json
4932 Content-Length: 287
4933 Content-Type: application/json
4934 Host: localhost:5984
4935 User-Agent: CouchDB
4936 {
4938 "baz": [
4939 "2-7051cbe5c8faecd085a3fa619e6e6337"
4940 ],
4941 "foo": [
4942 "3-6a540f3d701ac518d3b9733d673c5484"
4943 ],
4944 "bar": [
4945 "1-d4e501ab47de6b2000fc8a02f84a0c77",
4946 "1-967a00dff5e02add41819138abb3284d"
4947 ]
4948 }
4949 Response:
4951 HTTP/1.1 200 OK
4953 Cache-Control: must-revalidate
4954 Content-Length: 88
4955 Content-Type: application/json
4956 Date: Fri, 25 Oct 2013 14:44:41 GMT
4957 Server: CouchDB (Erlang/OTP)
4958 {
4960 "baz": {
4961 "missing": [
4962 "2-7051cbe5c8faecd085a3fa619e6e6337"
4963 ]
4964 },
4965 "bar": {
4966 "missing": [
4967 "1-d4e501ab47de6b2000fc8a02f84a0c77"
4968 ]
4969 }
4970 }
4971 In the response the Replicator receives a Document ID – Revisions map‐
4973 ping, but only for Revisions that do not exist in Target and are
4974 REQUIRED to be transferred from Source.
4975 If all Revisions in the request match the current state of the Docu‐
4977 ments then the response will contain an empty JSON object:
4978 Request
4979 POST /target/_revs_diff HTTP/1.1
4981 Accept: application/json
4982 Content-Length: 160
4983 Content-Type: application/json
4984 Host: localhost:5984
4985 User-Agent: CouchDB
4986 {
4988 "foo": [
4989 "3-6a540f3d701ac518d3b9733d673c5484"
4990 ],
4991 "bar": [
4992 "1-967a00dff5e02add41819138abb3284d"
4993 ]
4994 }
4995 Response:
4997 HTTP/1.1 200 OK
4999 Cache-Control: must-revalidate
5000 Content-Length: 2
5001 Content-Type: application/json
5002 Date: Fri, 25 Oct 2013 14:45:00 GMT
5003 Server: CouchDB (Erlang/OTP)
5004 {}
5006 Replication Completed
5008 When there are no more changes left to process and no more Documents
5009 left to replicate, the Replicator finishes the Replication process. If
5010 Replication wasn’t Continuous, the Replicator MAY return a response to
5011 client with statistics about the process.
5012 HTTP/1.1 200 OK
5014 Cache-Control: must-revalidate
5015 Content-Length: 414
5016 Content-Type: application/json
5017 Date: Fri, 09 May 2014 15:14:19 GMT
5018 Server: CouchDB (Erlang OTP)
5019 {
5021 "history": [
5022 {
5023 "doc_write_failures": 2,
5024 "docs_read": 2,
5025 "docs_written": 0,
5026 "end_last_seq": 2939,
5027 "end_time": "Fri, 09 May 2014 15:14:19 GMT",
5028 "missing_checked": 1835,
5029 "missing_found": 2,
5030 "recorded_seq": 2939,
5031 "session_id": "05918159f64842f1fe73e9e2157b2112",
5032 "start_last_seq": 0,
5033 "start_time": "Fri, 09 May 2014 15:14:18 GMT"
5034 }
5035 ],
5036 "ok": true,
5037 "replication_id_version": 3,
5038 "session_id": "05918159f64842f1fe73e9e2157b2112",
5039 "source_last_seq": 2939
5040 }
5041 Replicate Changes
5043 + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - +
5044 ' Locate Changed Documents: '
5045 ' '
5046 ' +-------------------------------------+ '
5047 ' | Any Differences Found? | '
5048 ' +-------------------------------------+ '
5049 ' | '
5050 ' | '
5051 ' | '
5052 + - - - - - - - - - - - - - - - - - - - - - - - - - | - - - - - - - - - - - - - - +
5053 |
5054 + - - - - - - - - - - - - - - - - - - - - - - - - - | - - - - - - - - - - - - - - +
5055 ' Replicate Changes: | '
5056 ' v '
5057 ' +-------------------------------------+ '
5058 ' +---------> | Fetch Next Changed Document | <---------------------+ '
5059 ' | +-------------------------------------+ | '
5060 ' | | GET /source/docid | | '
5061 ' | +-------------------------------------+ | '
5062 ' | | | '
5063 ' | | | '
5064 ' | | 201 Created | '
5065 ' | | 200 OK 401 Unauthorized | '
5066 ' | | 403 Forbidden | '
5067 ' | | | '
5068 ' | v | '
5069 ' | +-------------------------------------+ | '
5070 ' | +------ | Document Has Changed Attachments? | | '
5071 ' | | +-------------------------------------+ | '
5072 ' | | | | '
5073 ' | | | | '
5074 ' | | | Yes | '
5075 ' | | | | '
5076 ' | | v | '
5077 ' | | +------------------------+ Yes +---------------------------+ '
5078 ' | | No | Are They Big Enough? | -------> | Update Document on Target | '
5079 ' | | +------------------------+ +---------------------------+ '
5080 ' | | | | PUT /target/docid | '
5081 ' | | | +---------------------------+ '
5082 ' | | | '
5083 ' | | | No '
5084 ' | | | '
5085 ' | | v '
5086 ' | | +-------------------------------------+ '
5087 ' | +-----> | Put Document Into the Stack | '
5088 ' | +-------------------------------------+ '
5089 ' | | '
5090 ' | | '
5091 ' | v '
5092 ' | No +-------------------------------------+ '
5093 ' +---------- | Stack is Full? | '
5094 ' | +-------------------------------------+ '
5095 ' | | '
5096 ' | | Yes '
5097 ' | | '
5098 ' | v '
5099 ' | +-------------------------------------+ '
5100 ' | | Upload Stack of Documents to Target | '
5101 ' | +-------------------------------------+ '
5102 ' | | POST /target/_bulk_docs | '
5103 ' | +-------------------------------------+ '
5104 ' | | '
5105 ' | | 201 Created '
5106 ' | v '
5107 ' | +-------------------------------------+ '
5108 ' | | Ensure in Commit | '
5109 ' | +-------------------------------------+ '
5110 ' | | POST /target/_ensure_full_commit | '
5111 ' | +-------------------------------------+ '
5112 ' | | '
5113 ' | | 201 Created '
5114 ' | v '
5115 ' | +-------------------------------------+ '
5116 ' | | Record Replication Checkpoint | '
5117 ' | +-------------------------------------+ '
5118 ' | | PUT /source/_local/replication-id | '
5119 ' | | PUT /target/_local/replication-id | '
5120 ' | +-------------------------------------+ '
5121 ' | | '
5122 ' | | 201 Created '
5123 ' | v '
5124 ' | No +-------------------------------------+ '
5125 ' +---------- | All Documents from Batch Processed? | '
5126 ' +-------------------------------------+ '
5127 ' | '
5128 ' Yes | '
5129 ' | '
5130 + - - - - - - - - - - - - - - - - - - - - - - - - - | - - - - - - - - - - - - - - +
5131 |
5132 + - - - - - - - - - - - - - - - - - - - - - - - - - | - - - - - - - - - - - - - - +
5133 ' Locate Changed Documents: | '
5134 ' v '
5135 ' +-------------------------------------+ '
5136 ' | Listen to Changes Feed | '
5137 ' +-------------------------------------+ '
5138 ' '
5139 + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - +
5140 Fetch Changed Documents
5142 At this step the Replicator MUST fetch all Document Leaf Revisions from
5143 Source that are missed at Target. This operation is effective if Repli‐
5144 cation WILL use previously calculated Revision differences since they
5145 define missing Documents and their Revisions.
5146 To fetch the Document the Replicator will make a GET /{db}/{docid}
5148 request with the following query parameters:
5149 · revs=true: Instructs the Source to include the list of all known
5151 revisions into the Document in the _revisions field. This information
5152 is needed to synchronize the Document’s ancestors history between
5153 Source and Target
5154 · The open_revs query parameter contains a JSON array with a list of
5156 Leaf Revisions that are needed to be fetched. If the specified Revi‐
5157 sion exists then the Document MUST be returned for this Revision.
5158 Otherwise, Source MUST return an object with the single field missing
5159 with the missed Revision as the value. In case the Document contains
5160 attachments, Source MUST return information only for those ones that
5161 had been changed (added or updated) since the specified Revision val‐
5162 ues. If an attachment was deleted, the Document MUST NOT have stub
5163 information for it
5164 · latest=true: Ensures, that Source will return the latest Document
5166 Revision regardless of which one was specified in the open_revs query
5167 parameter. This parameter solves a race condition problem where the
5168 requested Document may be changed in between this step and handling
5169 related events on the Changes Feed
5170 In the response Source SHOULD return multipart/mixed or respond instead
5172 with application/json unless the Accept header specifies a different
5173 mime type. The multipart/mixed content type allows handling the
5174 response data as a stream, since there could be multiple documents (one
5175 per each Leaf Revision) plus several attachments. These attachments are
5176 mostly binary and JSON has no way to handle such data except as base64
5177 encoded strings which are very ineffective for transfer and processing
5178 operations.
5179 With a multipart/mixed response the Replicator handles multiple Docu‐
5181 ment Leaf Revisions and their attachments one by one as raw data with‐
5182 out any additional encoding applied. There is also one agreement to
5183 make data processing more effective: the Document ALWAYS goes before
5184 its attachments, so the Replicator has no need to process all the data
5185 to map related Documents-Attachments and may handle it as stream with
5186 lesser memory footprint.
5187 Request:
5188 GET /source/SpaghettiWithMeatballs?revs=true&open_revs=[%225-00ecbbc%22,%221-917fa23%22,%223-6bcedf1%22]&latest=true HTTP/1.1
5190 Accept: multipart/mixed
5191 Host: localhost:5984
5192 User-Agent: CouchDB
5193 Response:
5195 HTTP/1.1 200 OK
5197 Content-Type: multipart/mixed; boundary="7b1596fc4940bc1be725ad67f11ec1c4"
5198 Date: Thu, 07 Nov 2013 15:10:16 GMT
5199 Server: CouchDB (Erlang OTP)
5200 Transfer-Encoding: chunked
5201 --7b1596fc4940bc1be725ad67f11ec1c4
5203 Content-Type: application/json
5204 {
5206 "_id": "SpaghettiWithMeatballs",
5207 "_rev": "1-917fa23",
5208 "_revisions": {
5209 "ids": [
5210 "917fa23"
5211 ],
5212 "start": 1
5213 },
5214 "description": "An Italian-American delicious dish",
5215 "ingredients": [
5216 "spaghetti",
5217 "tomato sauce",
5218 "meatballs"
5219 ],
5220 "name": "Spaghetti with meatballs"
5221 }
5222 --7b1596fc4940bc1be725ad67f11ec1c4
5223 Content-Type: multipart/related; boundary="a81a77b0ca68389dda3243a43ca946f2"
5224 --a81a77b0ca68389dda3243a43ca946f2
5226 Content-Type: application/json
5227 {
5229 "_attachments": {
5230 "recipe.txt": {
5231 "content_type": "text/plain",
5232 "digest": "md5-R5CrCb6fX10Y46AqtNn0oQ==",
5233 "follows": true,
5234 "length": 87,
5235 "revpos": 7
5236 }
5237 },
5238 "_id": "SpaghettiWithMeatballs",
5239 "_rev": "7-474f12e",
5240 "_revisions": {
5241 "ids": [
5242 "474f12e",
5243 "5949cfc",
5244 "00ecbbc",
5245 "fc997b6",
5246 "3552c87",
5247 "404838b",
5248 "5defd9d",
5249 "dc1e4be"
5250 ],
5251 "start": 7
5252 },
5253 "description": "An Italian-American delicious dish",
5254 "ingredients": [
5255 "spaghetti",
5256 "tomato sauce",
5257 "meatballs",
5258 "love"
5259 ],
5260 "name": "Spaghetti with meatballs"
5261 }
5262 --a81a77b0ca68389dda3243a43ca946f2
5263 Content-Disposition: attachment; filename="recipe.txt"
5264 Content-Type: text/plain
5265 Content-Length: 87
5266 1. Cook spaghetti
5268 2. Cook meetballs
5269 3. Mix them
5270 4. Add tomato sauce
5271 5. ...
5272 6. PROFIT!
5273 --a81a77b0ca68389dda3243a43ca946f2--
5275 --7b1596fc4940bc1be725ad67f11ec1c4
5276 Content-Type: application/json; error="true"
5277 {"missing":"3-6bcedf1"}
5279 --7b1596fc4940bc1be725ad67f11ec1c4--
5280 After receiving the response, the Replicator puts all the received data
5282 into a local stack for further bulk upload to utilize network bandwidth
5283 effectively. The local stack size could be limited by number of Docu‐
5284 ments or bytes of handled JSON data. When the stack is full the Repli‐
5285 cator uploads all the handled Document in bulk mode to the Target.
5286 While bulk operations are highly RECOMMENDED to be used, in certain
5287 cases the Replicator MAY upload Documents to Target one by one.
5288 NOTE:
5290 Alternative Replicator implementations MAY use alternative ways to
5291 retrieve Documents from Source. For instance, PouchDB doesn’t use
5292 the Multipart API and fetches only the latest Document Revision with
5293 inline attachments as a single JSON object. While this is still
5294 valid CouchDB HTTP API usage, such solutions MAY require a different
5295 API implementation for non-CouchDB Peers.
5296 Upload Batch of Changed Documents
5298 To upload multiple Documents in a single shot the Replicator sends a
5299 POST /{db}/_bulk_docs request to Target with payload containing a JSON
5300 object with the following mandatory fields:
5301 · docs (array of objects): List of Document objects to update on Tar‐
5303 get. These Documents MUST contain the _revisions field that holds a
5304 list of the full Revision history to let Target create Leaf Revisions
5305 that correctly preserve ancestry
5306 · new_edits (boolean): Special flag that instructs Target to store Doc‐
5308 uments with the specified Revision (field _rev) value as-is without
5309 generating a new revision. Always false
5310 The request also MAY contain X-Couch-Full-Commit that used to control
5312 CouchDB <3.0 behavior when delayed commits were enabled. Other Peers
5313 MAY ignore this header or use it to control similar local feature.
5314 Request:
5315 POST /target/_bulk_docs HTTP/1.1
5317 Accept: application/json
5318 Content-Length: 826
5319 Content-Type:application/json
5320 Host: localhost:5984
5321 User-Agent: CouchDB
5322 X-Couch-Full-Commit: false
5323 {
5325 "docs": [
5326 {
5327 "_id": "SpaghettiWithMeatballs",
5328 "_rev": "1-917fa2381192822767f010b95b45325b",
5329 "_revisions": {
5330 "ids": [
5331 "917fa2381192822767f010b95b45325b"
5332 ],
5333 "start": 1
5334 },
5335 "description": "An Italian-American delicious dish",
5336 "ingredients": [
5337 "spaghetti",
5338 "tomato sauce",
5339 "meatballs"
5340 ],
5341 "name": "Spaghetti with meatballs"
5342 },
5343 {
5344 "_id": "LambStew",
5345 "_rev": "1-34c318924a8f327223eed702ddfdc66d",
5346 "_revisions": {
5347 "ids": [
5348 "34c318924a8f327223eed702ddfdc66d"
5349 ],
5350 "start": 1
5351 },
5352 "servings": 6,
5353 "subtitle": "Delicious with scone topping",
5354 "title": "Lamb Stew"
5355 },
5356 {
5357 "_id": "FishStew",
5358 "_rev": "1-9c65296036141e575d32ba9c034dd3ee",
5359 "_revisions": {
5360 "ids": [
5361 "9c65296036141e575d32ba9c034dd3ee"
5362 ],
5363 "start": 1
5364 },
5365 "servings": 4,
5366 "subtitle": "Delicious with fresh bread",
5367 "title": "Fish Stew"
5368 }
5369 ],
5370 "new_edits": false
5371 }
5372 In its response Target MUST return a JSON array with a list of Document
5374 update statuses. If the Document has been stored successfully, the list
5375 item MUST contain the field ok with true value. Otherwise it MUST con‐
5376 tain error and reason fields with error type and a human-friendly rea‐
5377 son description.
5378 Document updating failure isn’t fatal as Target MAY reject the update
5380 for its own reasons. It’s RECOMMENDED to use error type forbidden for
5381 rejections, but other error types can also be used (like invalid field
5382 name etc.). The Replicator SHOULD NOT retry uploading rejected docu‐
5383 ments unless there are good reasons for doing so (e.g. there is special
5384 error type for that).
5385 Note that while a update may fail for one Document in the response,
5387 Target can still return a 201 Created response. Same will be true if
5388 all updates fail for all uploaded Documents.
5389 Response:
5390 HTTP/1.1 201 Created
5392 Cache-Control: must-revalidate
5393 Content-Length: 246
5394 Content-Type: application/json
5395 Date: Sun, 10 Nov 2013 19:02:26 GMT
5396 Server: CouchDB (Erlang/OTP)
5397 [
5399 {
5400 "ok": true,
5401 "id": "SpaghettiWithMeatballs",
5402 "rev":" 1-917fa2381192822767f010b95b45325b"
5403 },
5404 {
5405 "ok": true,
5406 "id": "FishStew",
5407 "rev": "1-9c65296036141e575d32ba9c034dd3ee"
5408 },
5409 {
5410 "error": "forbidden",
5411 "id": "LambStew",
5412 "reason": "sorry",
5413 "rev": "1-34c318924a8f327223eed702ddfdc66d"
5414 }
5415 ]
5416 Upload Document with Attachments
5418 There is a special optimization case when then Replicator WILL NOT use
5419 bulk upload of changed Documents. This case is applied when Documents
5420 contain a lot of attached files or the files are too big to be effi‐
5421 ciently encoded with Base64.
5422 For this case the Replicator issues a /{db}/{docid}?new_edits=false
5424 request with multipart/related content type. Such a request allows one
5425 to easily stream the Document and all its attachments one by one with‐
5426 out any serialization overhead.
5427 Request:
5428 PUT /target/SpaghettiWithMeatballs?new_edits=false HTTP/1.1
5430 Accept: application/json
5431 Content-Length: 1030
5432 Content-Type: multipart/related; boundary="864d690aeb91f25d469dec6851fb57f2"
5433 Host: localhost:5984
5434 User-Agent: CouchDB
5435 --2fa48cba80d0cdba7829931fe8acce9d
5437 Content-Type: application/json
5438 {
5440 "_attachments": {
5441 "recipe.txt": {
5442 "content_type": "text/plain",
5443 "digest": "md5-R5CrCb6fX10Y46AqtNn0oQ==",
5444 "follows": true,
5445 "length": 87,
5446 "revpos": 7
5447 }
5448 },
5449 "_id": "SpaghettiWithMeatballs",
5450 "_rev": "7-474f12eb068c717243487a9505f6123b",
5451 "_revisions": {
5452 "ids": [
5453 "474f12eb068c717243487a9505f6123b",
5454 "5949cfcd437e3ee22d2d98a26d1a83bf",
5455 "00ecbbc54e2a171156ec345b77dfdf59",
5456 "fc997b62794a6268f2636a4a176efcd6",
5457 "3552c87351aadc1e4bea2461a1e8113a",
5458 "404838bc2862ce76c6ebed046f9eb542",
5459 "5defd9d813628cea6e98196eb0ee8594"
5460 ],
5461 "start": 7
5462 },
5463 "description": "An Italian-American delicious dish",
5464 "ingredients": [
5465 "spaghetti",
5466 "tomato sauce",
5467 "meatballs",
5468 "love"
5469 ],
5470 "name": "Spaghetti with meatballs"
5471 }
5472 --2fa48cba80d0cdba7829931fe8acce9d
5473 Content-Disposition: attachment; filename="recipe.txt"
5474 Content-Type: text/plain
5475 Content-Length: 87
5476 1. Cook spaghetti
5478 2. Cook meetballs
5479 3. Mix them
5480 4. Add tomato sauce
5481 5. ...
5482 6. PROFIT!
5483 --2fa48cba80d0cdba7829931fe8acce9d--
5485 Response:
5487 HTTP/1.1 201 Created
5489 Cache-Control: must-revalidate
5490 Content-Length: 105
5491 Content-Type: application/json
5492 Date: Fri, 08 Nov 2013 16:35:27 GMT
5493 Server: CouchDB (Erlang/OTP)
5494 {
5496 "ok": true,
5497 "id": "SpaghettiWithMeatballs",
5498 "rev": "7-474f12eb068c717243487a9505f6123b"
5499 }
5500 Unlike bulk updating via POST /{db}/_bulk_docs endpoint, the response
5502 MAY come with a different status code. For instance, in the case when
5503 the Document is rejected, Target SHOULD respond with a 403 Forbidden:
5504 Response:
5505 HTTP/1.1 403 Forbidden
5507 Cache-Control: must-revalidate
5508 Content-Length: 39
5509 Content-Type: application/json
5510 Date: Fri, 08 Nov 2013 16:35:27 GMT
5511 Server: CouchDB (Erlang/OTP)
5512 {
5514 "error": "forbidden",
5515 "reason": "sorry"
5516 }
5517 Replicator SHOULD NOT retry requests in case of a 401 Unauthorized, 403
5519 Forbidden, 409 Conflict or 412 Precondition Failed since repeating the
5520 request couldn’t solve the issue with user credentials or uploaded
5521 data.
5522 Ensure In Commit
5524 Once a batch of changes has been successfully uploaded to Target, the
5525 Replicator issues a POST /{db}/_ensure_full_commit request to ensure
5526 that every transferred bit is laid down on disk or other persistent
5527 storage place. Target MUST return 201 Created response with a JSON
5528 object containing the following mandatory fields:
5529 · instance_start_time (string): Timestamp of when the database was
5531 opened, expressed in microseconds since the epoch
5532 · ok (boolean): Operation status. Constantly true
5534 Request:
5536 POST /target/_ensure_full_commit HTTP/1.1
5538 Accept: application/json
5539 Content-Type: application/json
5540 Host: localhost:5984
5541 Response:
5543 HTTP/1.1 201 Created
5545 Cache-Control: must-revalidate
5546 Content-Length: 53
5547 Content-Type: application/json
5548 Date: Web, 06 Nov 2013 18:20:43 GMT
5549 Server: CouchDB (Erlang/OTP)
5550 {
5552 "instance_start_time": "0",
5553 "ok": true
5554 }
5555 Record Replication Checkpoint
5557 Since batches of changes were uploaded and committed successfully, the
5558 Replicator updates the Replication Log both on Source and Target
5559 recording the current Replication state. This operation is REQUIRED so
5560 that in the case of Replication failure the replication can resume from
5561 last point of success, not from the very beginning.
5562 Replicator updates Replication Log on Source:
5564 Request:
5565 PUT /source/_local/afa899a9e59589c3d4ce5668e3218aef HTTP/1.1
5567 Accept: application/json
5568 Content-Length: 591
5569 Content-Type: application/json
5570 Host: localhost:5984
5571 User-Agent: CouchDB
5572 {
5574 "_id": "_local/afa899a9e59589c3d4ce5668e3218aef",
5575 "_rev": "0-1",
5576 "_revisions": {
5577 "ids": [
5578 "31f36e40158e717fbe9842e227b389df"
5579 ],
5580 "start": 1
5581 },
5582 "history": [
5583 {
5584 "doc_write_failures": 0,
5585 "docs_read": 6,
5586 "docs_written": 6,
5587 "end_last_seq": 26,
5588 "end_time": "Thu, 07 Nov 2013 09:42:17 GMT",
5589 "missing_checked": 6,
5590 "missing_found": 6,
5591 "recorded_seq": 26,
5592 "session_id": "04bf15bf1d9fa8ac1abc67d0c3e04f07",
5593 "start_last_seq": 0,
5594 "start_time": "Thu, 07 Nov 2013 09:41:43 GMT"
5595 }
5596 ],
5597 "replication_id_version": 3,
5598 "session_id": "04bf15bf1d9fa8ac1abc67d0c3e04f07",
5599 "source_last_seq": 26
5600 }
5601 Response:
5603 HTTP/1.1 201 Created
5605 Cache-Control: must-revalidate
5606 Content-Length: 75
5607 Content-Type: application/json
5608 Date: Thu, 07 Nov 2013 09:42:17 GMT
5609 Server: CouchDB (Erlang/OTP)
5610 {
5612 "id": "_local/afa899a9e59589c3d4ce5668e3218aef",
5613 "ok": true,
5614 "rev": "0-2"
5615 }
5616 …and on Target too:
5618 Request:
5619 PUT /target/_local/afa899a9e59589c3d4ce5668e3218aef HTTP/1.1
5621 Accept: application/json
5622 Content-Length: 591
5623 Content-Type: application/json
5624 Host: localhost:5984
5625 User-Agent: CouchDB
5626 {
5628 "_id": "_local/afa899a9e59589c3d4ce5668e3218aef",
5629 "_rev": "1-31f36e40158e717fbe9842e227b389df",
5630 "_revisions": {
5631 "ids": [
5632 "31f36e40158e717fbe9842e227b389df"
5633 ],
5634 "start": 1
5635 },
5636 "history": [
5637 {
5638 "doc_write_failures": 0,
5639 "docs_read": 6,
5640 "docs_written": 6,
5641 "end_last_seq": 26,
5642 "end_time": "Thu, 07 Nov 2013 09:42:17 GMT",
5643 "missing_checked": 6,
5644 "missing_found": 6,
5645 "recorded_seq": 26,
5646 "session_id": "04bf15bf1d9fa8ac1abc67d0c3e04f07",
5647 "start_last_seq": 0,
5648 "start_time": "Thu, 07 Nov 2013 09:41:43 GMT"
5649 }
5650 ],
5651 "replication_id_version": 3,
5652 "session_id": "04bf15bf1d9fa8ac1abc67d0c3e04f07",
5653 "source_last_seq": 26
5654 }
5655 Response:
5657 HTTP/1.1 201 Created
5659 Cache-Control: must-revalidate
5660 Content-Length: 106
5661 Content-Type: application/json
5662 Date: Thu, 07 Nov 2013 09:42:17 GMT
5663 Server: CouchDB (Erlang/OTP)
5664 {
5666 "id": "_local/afa899a9e59589c3d4ce5668e3218aef",
5667 "ok": true,
5668 "rev": "2-9b5d1e36bed6ae08611466e30af1259a"
5669 }
5670 Continue Reading Changes
5672 Once a batch of changes had been processed and transferred to Target
5673 successfully, the Replicator can continue to listen to the Changes Feed
5674 for new changes. If there are no new changes to process the Replication
5675 is considered to be done.
5676 For Continuous Replication, the Replicator MUST continue to wait for
5678 new changes from Source.
5679 Protocol Robustness
5681 Since the CouchDB Replication Protocol works on top of HTTP, which is
5682 based on TCP/IP, the Replicator SHOULD expect to be working within an
5683 unstable environment with delays, losses and other bad surprises that
5684 might eventually occur. The Replicator SHOULD NOT count every HTTP
5685 request failure as a fatal error. It SHOULD be smart enough to detect
5686 timeouts, repeat failed requests, be ready to process incomplete or
5687 malformed data and so on. Data must flow - that’s the rule.
5688 Error Responses
5690 In case something goes wrong the Peer MUST respond with a JSON object
5691 with the following REQUIRED fields:
5692 · error (string): Error type for programs and developers
5694 · reason (string): Error description for humans
5696 Bad Request
5698 If a request contains malformed data (like invalid JSON) the Peer MUST
5699 respond with a HTTP 400 Bad Request and bad_request as error type:
5700 {
5702 "error": "bad_request",
5703 "reason": "invalid json"
5704 }
5705 Unauthorized
5707 If a Peer REQUIRES credentials be included with the request and the
5708 request does not contain acceptable credentials then the Peer MUST
5709 respond with the HTTP 401 Unauthorized and unauthorized as error type:
5710 {
5712 "error": "unauthorized",
5713 "reason": "Name or password is incorrect"
5714 }
5715 Forbidden
5717 If a Peer receives valid user credentials, but the requester does not
5718 have sufficient permissions to perform the operation then the Peer MUST
5719 respond with a HTTP 403 Forbidden and forbidden as error type:
5720 {
5722 "error": "forbidden",
5723 "reason": "You may only update your own user document."
5724 }
5725 Resource Not Found
5727 If the requested resource, Database or Document wasn’t found on a Peer,
5728 the Peer MUST respond with a HTTP 404 Not Found and not_found as error
5729 type:
5730 {
5732 "error": "not_found",
5733 "reason": "database \"target\" does not exists"
5734 }
5735 Method Not Allowed
5737 If an unsupported method was used then the Peer MUST respond with a
5738 HTTP 405 Method Not Allowed and method_not_allowed as error type:
5739 {
5741 "error": "method_not_allowed",
5742 "reason": "Only GET, PUT, DELETE allowed"
5743 }
5744 Resource Conflict
5746 A resource conflict error occurs when there are concurrent updates of
5747 the same resource by multiple clients. In this case the Peer MUST
5748 respond with a HTTP 409 Conflict and conflict as error type:
5749 {
5751 "error": "conflict",
5752 "reason": "document update conflict"
5753 }
5754 Precondition Failed
5756 The HTTP 412 Precondition Failed response may be sent in case of an
5757 attempt to create a Database (error type db_exists) that already exists
5758 or some attachment information is missing (error type missing_stub).
5759 There is no explicit error type restrictions, but it is RECOMMEND to
5760 use error types that are previously mentioned:
5761 {
5763 "error": "db_exists",
5764 "reason": "database \"target\" exists"
5765 }
5766 Server Error
5768 Raised in case an error is fatal and the Replicator cannot do anything
5769 to continue Replication. In this case the Replicator MUST return a HTTP
5770 500 Internal Server Error response with an error description (no
5771 restrictions on error type applied):
5772 {
5774 "error": "worker_died",
5775 "reason": "kaboom!"
5776 }
5777 Optimisations
5779 There are RECOMMENDED approaches to optimize the Replication process:
5780 · Keep the number of HTTP requests at a reasonable minimum
5782 · Try to work with a connection pool and make parallel/multiple
5784 requests whenever possible
5785 · Don’t close sockets after each request: respect the keep-alive option
5787 · Use continuous sessions (cookies, etc.) to reduce authentication
5789 overhead
5790 · Try to use bulk requests for every operations with Documents
5792 · Find out optimal batch size for Changes feed processing
5794 · Preserve Replication Logs and resume Replication from the last Check‐
5796 point whenever possible
5797 · Optimize filter functions: let them run as fast as possible
5799 · Get ready for surprises: networks are very unstable environments
5801 API Reference
5803 Common Methods
5804 · HEAD /{db} – Check Database existence
5805 · GET /{db} – Retrieve Database information
5807 · GET /{db}/_local/{docid} – Read the last Checkpoint
5809 · PUT /{db}/_local/{docid} – Save a new Checkpoint
5811 For Target
5813 · PUT /{db} – Create Target if it not exists and the option was pro‐
5814 vided
5815 · POST /{db}/_revs_diff – Locate Revisions that are not known to Target
5817 · POST /{db}/_bulk_docs – Upload Revisions to Target
5819 · PUT /{db}/{docid} – Upload a single Document with attachments to Tar‐
5821 get
5822 · POST /{db}/_ensure_full_commit – Ensure that all changes are stored
5824 on disk
5825 For Source
5827 · GET /{db}/_changes – Fetch changes since the last pull of Source
5828 · POST /{db}/_changes – Fetch changes for specified Document IDs since
5830 the last pull of Source
5831 · GET /{db}/{docid} – Retrieve a single Document from Source with
5833 attachments
5834 Reference
5836 · Refuge RCouch wiki
5837 · CouchBase Lite IOS wiki
5839
5841 CouchDB supports special documents within databases known as “design
5842 documents”. These documents, mostly driven by JavaScript you write, are
5843 used to build indexes, validate document updates, format query results,
5844 and filter replications.
5845 Design Documents
5847 In this section we’ll show how to write design documents, using the
5848 built-in JavaScript Query Server.
5849 But before we start to write our first document, let’s take a look at
5851 the list of common objects that will be used during our code journey -
5852 we’ll be using them extensively within each function:
5853 · Database information object
5855 · Request object
5857 · Response object
5859 · UserCtx object
5861 · Database Security object
5863 · Guide to JavaScript Query Server
5865 Creation
5867 Design documents are denoted by an id field with the format
5868 _design/{name}.
5869 Example:
5871 {
5873 "_id": "_design/example",
5874 }
5875 View Functions
5877 Views are the primary tool used for querying and reporting on CouchDB
5878 databases.
5879 Map Functions
5881 mapfun(doc)
5882 Arguments
5884 · doc – The document that is being processed
5886 Map functions accept a single document as the argument and (optionally)
5888 emit() key/value pairs that are stored in a view.
5889 function (doc) {
5891 if (doc.type === 'post' && doc.tags && Array.isArray(doc.tags)) {
5892 doc.tags.forEach(function (tag) {
5893 emit(tag.toLowerCase(), 1);
5894 });
5895 }
5896 }
5897 In this example a key/value pair is emitted for each value in the tags
5899 array of a document with a type of “post”. Note that emit() may be
5900 called many times for a single document, so the same document may be
5901 available by several different keys.
5902 Also keep in mind that each document is sealed to prevent the situation
5904 where one map function changes document state and another receives a
5905 modified version.
5906 For efficiency reasons, documents are passed to a group of map func‐
5908 tions - each document is processed by a group of map functions from all
5909 views of the related design document. This means that if you trigger an
5910 index update for one view in the design document, all others will get
5911 updated too.
5912 Since version 1.1.0, map supports CommonJS modules and the require()
5914 function.
5915 Reduce and Rereduce Functions
5917 redfun(keys, values[, rereduce])
5918 Arguments
5920 · keys – Array of pairs of key-docid for related map
5922 function results. Always null if rereduce is running
5923 (has true value).
5924 · values – Array of map function result values.
5926 · rereduce – Boolean flag to indicate a rereduce run.
5928 Returns
5930 Reduces values
5931 Reduce functions take two required arguments of keys and values lists -
5933 the result of the related map function - and an optional third value
5934 which indicates if rereduce mode is active or not. Rereduce is used for
5935 additional reduce values list, so when it is true there is no informa‐
5936 tion about related keys (first argument is null).
5937 Note that if the result of a reduce function is longer than the initial
5939 values list then a Query Server error will be raised. However, this
5940 behavior can be disabled by setting reduce_limit config option to
5941 false:
5942 [query_server_config]
5944 reduce_limit = false
5945 While disabling reduce_limit might be useful for debug proposes, remem‐
5947 ber that the main task of reduce functions is to reduce the mapped
5948 result, not to make it bigger. Generally, your reduce function should
5949 converge rapidly to a single value - which could be an array or similar
5950 object.
5951 Built-in Reduce Functions
5953 Additionally, CouchDB has a set of built-in reduce functions. These are
5954 implemented in Erlang and run inside CouchDB, so they are much faster
5955 than the equivalent JavaScript functions.
5956 _approx_count_distinct
5958 New in version 2.2.
5960 Aproximates the number of distinct keys in a view index using a variant
5963 of the HyperLogLog algorithm. This algorithm enables an efficient, par‐
5964 allelizable computation of cardinality using fixed memory resources.
5965 CouchDB has configured the underlying data structure to have a relative
5966 error of ~2%.
5967 As this reducer ignores the emitted values entirely, an invocation with
5969 group=true will simply return a value of 1 for every distinct key in
5970 the view. In the case of array keys, querying the view with a
5971 group_level specified will return the number of distinct keys that
5972 share the common group prefix in each row. The algorithm is also cog‐
5973 nizant of the startkey and endkey boundaries and will return the number
5974 of distinct keys within the specified key range.
5975 A final note regarding Unicode collation: this reduce function uses the
5977 binary representation of each key in the index directly as input to the
5978 HyperLogLog filter. As such, it will (incorrectly) consider keys that
5979 are not byte identical but that compare equal according to the Unicode
5980 collation rules to be distinct keys, and thus has the potential to
5981 overestimate the cardinality of the key space if a large number of such
5982 keys exist.
5983 _count
5985 Counts the number of values in the index with a given key. This could
5987 be implemented in JavaScript as:
5988 // could be replaced by _count
5990 function(keys, values, rereduce) {
5991 if (rereduce) {
5992 return sum(values);
5993 } else {
5994 return values.length;
5995 }
5996 }
5997 _stats
5999 Computes the following quantities for numeric values associated with
6001 each key: sum, min, max, count, and sumsqr. The behavior of the _stats
6002 function varies depending on the output of the map function. The sim‐
6003 plest case is when the map phase emits a single numeric value for each
6004 key. In this case the _stats function is equivalent to the following
6005 JavaScript:
6006 // could be replaced by _stats
6008 function(keys, values, rereduce) {
6009 if (rereduce) {
6010 return {
6011 'sum': values.reduce(function(a, b) { return a + b.sum }, 0),
6012 'min': values.reduce(function(a, b) { return Math.min(a, b.min) }, Infinity),
6013 'max': values.reduce(function(a, b) { return Math.max(a, b.max) }, -Infinity),
6014 'count': values.reduce(function(a, b) { return a + b.count }, 0),
6015 'sumsqr': values.reduce(function(a, b) { return a + b.sumsqr }, 0)
6016 }
6017 } else {
6018 return {
6019 'sum': sum(values),
6020 'min': Math.min.apply(null, values),
6021 'max': Math.max.apply(null, values),
6022 'count': values.length,
6023 'sumsqr': (function() {
6024 var sumsqr = 0;
6025 values.forEach(function (value) {
6027 sumsqr += value * value;
6028 });
6029 return sumsqr;
6031 })(),
6032 }
6033 }
6034 }
6035 The _stats function will also work with “pre-aggregated” values from a
6037 map phase. A map function that emits an object containing sum, min,
6038 max, count, and sumsqr keys and numeric values for each can use the
6039 _stats function to combine these results with the data from other docu‐
6040 ments. The emitted object may contain other keys (these are ignored by
6041 the reducer), and it is also possible to mix raw numeric values and
6042 pre-aggregated objects in a single view and obtain the correct aggre‐
6043 gated statistics.
6044 Finally, _stats can operate on key-value pairs where each value is an
6046 array comprised of numbers or pre-aggregated objects. In this case
6047 every value emitted from the map function must be an array, and the
6048 arrays must all be the same length, as _stats will compute the statis‐
6049 tical quantities above independently for each element in the array.
6050 Users who want to compute statistics on multiple values from a single
6051 document should either emit each value into the index separately, or
6052 compute the statistics for the set of values using the JavaScript exam‐
6053 ple above and emit a pre-aggregated object.
6054 _sum
6056 In its simplest variation, _sum sums the numeric values associated with
6058 each key, as in the following JavaScript:
6059 // could be replaced by _sum
6061 function(keys, values) {
6062 return sum(values);
6063 }
6064 As with _stats, the _sum function offers a number of extended capabili‐
6066 ties. The _sum function requires that map values be numbers, arrays of
6067 numbers, or objects. When presented with array output from a map func‐
6068 tion, _sum will compute the sum for every element of the array. A bare
6069 numeric value will be treated as an array with a single element, and
6070 arrays with fewer elements will be treated as if they contained zeroes
6071 for every additional element in the longest emitted array. As an exam‐
6072 ple, consider the following map output:
6073 {"total_rows":5, "offset":0, "rows": [
6075 {"id":"id1", "key":"abc", "value": 2},
6076 {"id":"id2", "key":"abc", "value": [3,5,7]},
6077 {"id":"id2", "key":"def", "value": [0,0,0,42]},
6078 {"id":"id2", "key":"ghi", "value": 1},
6079 {"id":"id1", "key":"ghi", "value": 3}
6080 ]}
6081 The _sum for this output without any grouping would be:
6083 {"rows": [
6085 {"key":null, "value": [9,5,7,42]}
6086 ]}
6087 while the grouped output would be
6089 {"rows": [
6091 {"key":"abc", "value": [5,5,7]},
6092 {"key":"def", "value": [0,0,0,42]},
6093 {"key":"ghi", "value": 4
6094 ]}
6095 This is in contrast to the behavior of the _stats function which
6097 requires that all emitted values be arrays of identical length if any
6098 array is emitted.
6099 It is also possible to have _sum recursively descend through an emitted
6101 object and compute the sums for every field in the object. Objects can‐
6102 not be mixed with other data structures. Objects can be arbitrarily
6103 nested, provided that the values for all fields are themselves numbers,
6104 arrays of numbers, or objects.
6105 NOTE:
6107 Why don’t reduce functions support CommonJS modules?
6108 While map functions have limited access to stored modules through
6110 require(), there is no such feature for reduce functions. The rea‐
6111 son lies deep inside the way map and reduce functions are processed
6112 by the Query Server. Let’s take a look at map functions first:
6113 1. CouchDB sends all map functions in a processed design document to
6115 the Query Server.
6116 2. the Query Server handles them one by one, compiles and puts them
6118 onto an internal stack.
6119 3. after all map functions have been processed, CouchDB will send
6121 the remaining documents for indexing, one by one.
6122 4. the Query Server receives the document object and applies it to
6124 every function from the stack. The emitted results are then
6125 joined into a single array and sent back to CouchDB.
6126 Now let’s see how reduce functions are handled:
6128 1. CouchDB sends as a single command the list of available reduce
6130 functions with the result list of key-value pairs that were pre‐
6131 viously returned from the map functions.
6132 2. the Query Server compiles the reduce functions and applies them
6134 to the key-value lists. The reduced result is sent back to
6135 CouchDB.
6136 As you may note, reduce functions are applied in a single shot to
6138 the map results while map functions are applied to documents one by
6139 one. This means that it’s possible for map functions to precompile
6140 CommonJS libraries and use them during the entire view processing,
6141 but for reduce functions they would be compiled again and again for
6142 each view result reduction, which would lead to performance degrada‐
6143 tion.
6144 Show Functions
6146 WARNING:
6147 Show functions are deprecated in CouchDB 3.0, and will be removed in
6148 CouchDB 4.0.
6149 showfun(doc, req)
6151 Arguments
6153 · doc – The document that is being processed; may be
6155 omitted.
6156 · req – Request object.
6158 Returns
6160 Response object
6161 Return type
6163 object or string
6164 Show functions are used to represent documents in various formats, com‐
6166 monly as HTML pages with nice formatting. They can also be used to run
6167 server-side functions without requiring a pre-existing document.
6168 Basic example of show function could be:
6170 function(doc, req){
6172 if (doc) {
6173 return "Hello from " + doc._id + "!";
6174 } else {
6175 return "Hello, world!";
6176 }
6177 }
6178 Also, there is more simple way to return json encoded data:
6180 function(doc, req){
6182 return {
6183 'json': {
6184 'id': doc['_id'],
6185 'rev': doc['_rev']
6186 }
6187 }
6188 }
6189 and even files (this one is CouchDB logo):
6191 function(doc, req){
6193 return {
6194 'headers': {
6195 'Content-Type' : 'image/png',
6196 },
6197 'base64': ''.concat(
6198 'iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAMAAAAoLQ9TAAAAsV',
6199 'BMVEUAAAD////////////////////////5ur3rEBn////////////////wDBL/',
6200 'AADuBAe9EB3IEBz/7+//X1/qBQn2AgP/f3/ilpzsDxfpChDtDhXeCA76AQH/v7',
6201 '/84eLyWV/uc3bJPEf/Dw/uw8bRWmP1h4zxSlD6YGHuQ0f6g4XyQkXvCA36MDH6',
6202 'wMH/z8/yAwX64ODeh47BHiv/Ly/20dLQLTj98PDXWmP/Pz//39/wGyJ7Iy9JAA',
6203 'AADHRSTlMAbw8vf08/bz+Pv19jK/W3AAAAg0lEQVR4Xp3LRQ4DQRBD0QqTm4Y5',
6204 'zMxw/4OleiJlHeUtv2X6RbNO1Uqj9g0RMCuQO0vBIg4vMFeOpCWIWmDOw82fZx',
6205 'vaND1c8OG4vrdOqD8YwgpDYDxRgkSm5rwu0nQVBJuMg++pLXZyr5jnc1BaH4GT',
6206 'LvEliY253nA3pVhQqdPt0f/erJkMGMB8xucAAAAASUVORK5CYII=')
6207 }
6208 }
6209 But what if you need to represent data in different formats via a sin‐
6211 gle function? Functions registerType() and provides() are your the best
6212 friends in that question:
6213 function(doc, req){
6215 provides('json', function(){
6216 return {'json': doc}
6217 });
6218 provides('html', function(){
6219 return '<pre>' + toJSON(doc) + '</pre>'
6220 })
6221 provides('xml', function(){
6222 return {
6223 'headers': {'Content-Type': 'application/xml'},
6224 'body' : ''.concat(
6225 '<?xml version="1.0" encoding="utf-8"?>\n',
6226 '<doc>',
6227 (function(){
6228 escape = function(s){
6229 return s.replace(/"/g, '"')
6230 .replace(/>/g, '>')
6231 .replace(/</g, '<')
6232 .replace(/&/g, '&');
6233 };
6234 var content = '';
6235 for(var key in doc){
6236 if(!doc.hasOwnProperty(key)) continue;
6237 var value = escape(toJSON(doc[key]));
6238 var key = escape(key);
6239 content += ''.concat(
6240 '<' + key + '>',
6241 value
6242 '</' + key + '>'
6243 )
6244 }
6245 return content;
6246 })(),
6247 '</doc>'
6248 )
6249 }
6250 })
6251 registerType('text-json', 'text/json')
6252 provides('text-json', function(){
6253 return toJSON(doc);
6254 })
6255 }
6256 This function may return html, json , xml or our custom text json for‐
6258 mat representation of same document object with same processing rules.
6259 Probably, the xml provider in our function needs more care to handle
6260 nested objects correctly, and keys with invalid characters, but you’ve
6261 got the idea!
6262 SEE ALSO:
6264 CouchDB Guide:
6266 · Show Functions
6268 List Functions
6270 WARNING:
6271 List functions are deprecated in CouchDB 3.0, and will be removed in
6272 CouchDB 4.0.
6273 listfun(head, req)
6275 Arguments
6277 · head – view_head_info_object
6279 · req – Request object.
6281 Returns
6283 Last chunk.
6284 Return type
6286 string
6287 While Show Functions are used to customize document presentation, List
6289 Functions are used for the same purpose, but on View Functions results.
6290 The following list function formats the view and represents it as a
6292 very simple HTML page:
6293 function(head, req){
6295 start({
6296 'headers': {
6297 'Content-Type': 'text/html'
6298 }
6299 });
6300 send('<html><body><table>');
6301 send('<tr><th>ID</th><th>Key</th><th>Value</th></tr>');
6302 while(row = getRow()){
6303 send(''.concat(
6304 '<tr>',
6305 '<td>' + toJSON(row.id) + '</td>',
6306 '<td>' + toJSON(row.key) + '</td>',
6307 '<td>' + toJSON(row.value) + '</td>',
6308 '</tr>'
6309 ));
6310 }
6311 send('</table></body></html>');
6312 }
6313 Templates and styles could obviously be used to present data in a nicer
6315 fashion, but this is an excellent starting point. Note that you may
6316 also use registerType() and provides() functions in a similar way as
6317 for Show Functions! However, note that provides() expects the return
6318 value to be a string when used inside a list function, so you’ll need
6319 to use start() to set any custom headers and stringify your JSON before
6320 returning it.
6321 SEE ALSO:
6323 CouchDB Guide:
6325 · Transforming Views with List Functions
6327 Update Functions
6329 updatefun(doc, req)
6330 Arguments
6332 · doc – The document that is being processed.
6334 · req – request_object
6336 Returns
6338 Two-element array: the first element is the (updated or
6339 new) document, which is committed to the database. If the
6340 first element is null no document will be committed to
6341 the database. If you are updating an existing document,
6342 it should already have an _id set, and if you are creat‐
6343 ing a new document, make sure to set its _id to some‐
6344 thing, either generated based on the input or the
6345 req.uuid provided. The second element is the response
6346 that will be sent back to the caller.
6347 Update handlers are functions that clients can request to invoke
6349 server-side logic that will create or update a document. This feature
6350 allows a range of use cases such as providing a server-side last modi‐
6351 fied timestamp, updating individual fields in a document without first
6352 getting the latest revision, etc.
6353 When the request to an update handler includes a document ID in the
6355 URL, the server will provide the function with the most recent version
6356 of that document. You can provide any other values needed by the
6357 update handler function via the POST/PUT entity body or query string
6358 parameters of the request.
6359 A basic example that demonstrates all use-cases of update handlers:
6361 function(doc, req){
6363 if (!doc){
6364 if ('id' in req && req['id']){
6365 // create new document
6366 return [{'_id': req['id']}, 'New World']
6367 }
6368 // change nothing in database
6369 return [null, 'Empty World']
6370 }
6371 doc['world'] = 'hello';
6372 doc['edited_by'] = req['userCtx']['name']
6373 return [doc, 'Edited World!']
6374 }
6375 Filter Functions
6377 filterfun(doc, req)
6378 Arguments
6380 · doc – The document that is being processed
6382 · req – request_object
6384 Returns
6386 Boolean value: true means that doc passes the filter
6387 rules, false means that it does not.
6388 Filter functions mostly act like Show Functions and List Functions:
6390 they format, or filter the changes feed.
6391 Classic Filters
6393 By default the changes feed emits all database documents changes. But
6394 if you’re waiting for some special changes, processing all documents is
6395 inefficient.
6396 Filters are special design document functions that allow the changes
6398 feed to emit only specific documents that pass filter rules.
6399 Let’s assume that our database is a mailbox and we need to handle only
6401 new mail events (documents with the status new). Our filter function
6402 would look like this:
6403 function(doc, req){
6405 // we need only `mail` documents
6406 if (doc.type != 'mail'){
6407 return false;
6408 }
6409 // we're interested only in `new` ones
6410 if (doc.status != 'new'){
6411 return false;
6412 }
6413 return true; // passed!
6414 }
6415 Filter functions must return true if a document passed all the rules.
6417 Now, if you apply this function to the changes feed it will emit only
6418 changes about “new mails”:
6419 GET /somedatabase/_changes?filter=mailbox/new_mail HTTP/1.1
6421 {"results":[
6423 {"seq":"1-g1AAAAF9eJzLYWBg4MhgTmHgz8tPSTV0MDQy1zMAQsMcoARTIkOS_P___7MymBMZc4EC7MmJKSmJqWaYynEakaQAJJPsoaYwgE1JM0o1TjQ3T2HgLM1LSU3LzEtNwa3fAaQ_HqQ_kQG3qgSQqnoCqvJYgCRDA5ACKpxPWOUCiMr9hFUegKi8T1jlA4hKkDuzAC2yZRo","id":"df8eca9da37dade42ee4d7aa3401f1dd","changes":[{"rev":"1-c2e0085a21d34fa1cecb6dc26a4ae657"}]},
6424 {"seq":"9-g1AAAAIreJyVkEsKwjAURUMrqCOXoCuQ5MU0OrI70XyppcaRY92J7kR3ojupaSPUUgqWwAu85By4t0AITbJYo5k7aUNSAnyJ_SGFf4gEkvOyLPMsFtHRL8ZKaC1M0v3eq5ALP-X2a0G1xYKhgnONpmenjT04o_v5tOJ3LV5itTES_uP3FX9ppcAACaVsQAo38hNd_eVFt8ZklVljPqSPYLoH06PJhG0Cxq7-yhQcz-B4_fQCjFuqBjjewVF3E9cORoExSrpU_gHBTo5m","id":"df8eca9da37dade42ee4d7aa34024714","changes":[{"rev":"1-29d748a6e87b43db967fe338bcb08d74"}]},
6425 ],
6426 "last_seq":"10-g1AAAAIreJyVkEsKwjAURR9tQR25BF2B5GMaHdmdaNIk1FLjyLHuRHeiO9Gd1LQRaimFlsALvOQcuLcAgGkWKpjbs9I4wYSvkDu4cA-BALkoyzLPQhGc3GKSCqWEjrvfexVy6abc_SxQWwzRVHCuYHaxSpuj1aqfTyp-3-IlSrdakmH8oeKvrRSIkJhSNiKFjdyEm7uc6N6YTKo3iI_pw5se3vRsMiETE23WgzJ5x8s73n-9EMYNTUc4Pt5RdxPVDkYJYxR3qfwLwW6OZw"}
6427 Note that the value of last_seq is 10-.., but we received only two
6429 records. Seems like any other changes were for documents that haven’t
6430 passed our filter.
6431 We probably need to filter the changes feed of our mailbox by more than
6433 a single status value. We’re also interested in statuses like “spam” to
6434 update spam-filter heuristic rules, “outgoing” to let a mail daemon
6435 actually send mails, and so on. Creating a lot of similar functions
6436 that actually do similar work isn’t good idea - so we need a dynamic
6437 filter.
6438 You may have noticed that filter functions take a second argument named
6440 request. This allows the creation of dynamic filters based on query
6441 parameters, user context and more.
6442 The dynamic version of our filter looks like this:
6444 function(doc, req){
6446 // we need only `mail` documents
6447 if (doc.type != 'mail'){
6448 return false;
6449 }
6450 // we're interested only in requested status
6451 if (doc.status != req.query.status){
6452 return false;
6453 }
6454 return true; // passed!
6455 }
6456 and now we have passed the status query parameter in the request to let
6458 our filter match only the required documents:
6459 GET /somedatabase/_changes?filter=mailbox/by_status&status=new HTTP/1.1
6461 {"results":[
6463 {"seq":"1-g1AAAAF9eJzLYWBg4MhgTmHgz8tPSTV0MDQy1zMAQsMcoARTIkOS_P___7MymBMZc4EC7MmJKSmJqWaYynEakaQAJJPsoaYwgE1JM0o1TjQ3T2HgLM1LSU3LzEtNwa3fAaQ_HqQ_kQG3qgSQqnoCqvJYgCRDA5ACKpxPWOUCiMr9hFUegKi8T1jlA4hKkDuzAC2yZRo","id":"df8eca9da37dade42ee4d7aa3401f1dd","changes":[{"rev":"1-c2e0085a21d34fa1cecb6dc26a4ae657"}]},
6464 {"seq":"9-g1AAAAIreJyVkEsKwjAURUMrqCOXoCuQ5MU0OrI70XyppcaRY92J7kR3ojupaSPUUgqWwAu85By4t0AITbJYo5k7aUNSAnyJ_SGFf4gEkvOyLPMsFtHRL8ZKaC1M0v3eq5ALP-X2a0G1xYKhgnONpmenjT04o_v5tOJ3LV5itTES_uP3FX9ppcAACaVsQAo38hNd_eVFt8ZklVljPqSPYLoH06PJhG0Cxq7-yhQcz-B4_fQCjFuqBjjewVF3E9cORoExSrpU_gHBTo5m","id":"df8eca9da37dade42ee4d7aa34024714","changes":[{"rev":"1-29d748a6e87b43db967fe338bcb08d74"}]},
6465 ],
6466 "last_seq":"10-g1AAAAIreJyVkEsKwjAURR9tQR25BF2B5GMaHdmdaNIk1FLjyLHuRHeiO9Gd1LQRaimFlsALvOQcuLcAgGkWKpjbs9I4wYSvkDu4cA-BALkoyzLPQhGc3GKSCqWEjrvfexVy6abc_SxQWwzRVHCuYHaxSpuj1aqfTyp-3-IlSrdakmH8oeKvrRSIkJhSNiKFjdyEm7uc6N6YTKo3iI_pw5se3vRsMiETE23WgzJ5x8s73n-9EMYNTUc4Pt5RdxPVDkYJYxR3qfwLwW6OZw"}
6467 and we can easily change filter behavior with:
6469 GET /somedatabase/_changes?filter=mailbox/by_status&status=spam HTTP/1.1
6471 {"results":[
6473 {"seq":"6-g1AAAAIreJyVkM0JwjAYQD9bQT05gk4gaWIaPdlNNL_UUuPJs26im-gmuklMjVClFFoCXyDJe_BSAsA4jxVM7VHpJEswWyC_ktJfRBzEzDlX5DGPDv5gJLlSXKfN560KMfdTbL4W-FgM1oQzpmByskqbvdWqnc8qfvvHCyTXWuBu_K7iz38VCOOUENqjwg79hIvfvOhamQahROoVYn3-I5huwXSvm5BJsTbLTk3B8QiO58-_YMoMkT0cr-BwdRElmFKSNKniDcAcjmM","id":"8960e91220798fc9f9d29d24ed612e0d","changes":[{"rev":"3-cc6ff71af716ddc2ba114967025c0ee0"}]},
6474 ],
6475 "last_seq":"10-g1AAAAIreJyVkEsKwjAURR9tQR25BF2B5GMaHdmdaNIk1FLjyLHuRHeiO9Gd1LQRaimFlsALvOQcuLcAgGkWKpjbs9I4wYSvkDu4cA-BALkoyzLPQhGc3GKSCqWEjrvfexVy6abc_SxQWwzRVHCuYHaxSpuj1aqfTyp-3-IlSrdakmH8oeKvrRSIkJhSNiKFjdyEm7uc6N6YTKo3iI_pw5se3vRsMiETE23WgzJ5x8s73n-9EMYNTUc4Pt5RdxPVDkYJYxR3qfwLwW6OZw"}
6476 Combining filters with a continuous feed allows creating powerful
6478 event-driven systems.
6479 View Filters
6481 View filters are the same as classic filters above, with one small dif‐
6482 ference: they use the map instead of the filter function of a view, to
6483 filter the changes feed. Each time a key-value pair is emitted from the
6484 map function, a change is returned. This allows avoiding filter func‐
6485 tions that mostly do the same work as views.
6486 To use them just pass filter=_view and view=designdoc/viewname as
6488 request parameters to the changes feed:
6489 GET /somedatabase/_changes?filter=_view&view=dname/viewname HTTP/1.1
6491 NOTE:
6493 Since view filters use map functions as filters, they can’t show any
6494 dynamic behavior since request object is not available.
6495 SEE ALSO:
6497 CouchDB Guide:
6499 · Guide to filter change notification
6501 Validate Document Update Functions
6503 validatefun(newDoc, oldDoc, userCtx, secObj)
6504 Arguments
6506 · newDoc – New version of document that will be stored.
6508 · oldDoc – Previous version of document that is already
6510 stored.
6511 · userCtx – userctx_object
6513 · secObj – security_object
6515 Throws forbidden error to gracefully prevent document storing.
6517 Throws unauthorized error to prevent storage and allow the user
6519 to re-auth.
6520 A design document may contain a function named validate_doc_update
6522 which can be used to prevent invalid or unauthorized document update
6523 requests from being stored. The function is passed the new document
6524 from the update request, the current document stored in the database, a
6525 userctx_object containing information about the user writing the docu‐
6526 ment (if present), and a security_object with lists of database secu‐
6527 rity roles.
6528 Validation functions typically examine the structure of the new docu‐
6530 ment to ensure that required fields are present and to verify that the
6531 requesting user should be allowed to make changes to the document prop‐
6532 erties. For example, an application may require that a user must be
6533 authenticated in order to create a new document or that specific docu‐
6534 ment fields be present when a document is updated. The validation func‐
6535 tion can abort the pending document write by throwing one of two error
6536 objects:
6537 // user is not authorized to make the change but may re-authenticate
6539 throw({ unauthorized: 'Error message here.' });
6540 // change is not allowed
6542 throw({ forbidden: 'Error message here.' });
6543 Document validation is optional, and each design document in the data‐
6545 base may have at most one validation function. When a write request is
6546 received for a given database, the validation function in each design
6547 document in that database is called in an unspecified order. If any of
6548 the validation functions throw an error, the write will not succeed.
6549 Example: The _design/_auth ddoc from _users database uses a validation
6551 function to ensure that documents contain some required fields and are
6552 only modified by a user with the _admin role:
6553 function(newDoc, oldDoc, userCtx, secObj) {
6555 if (newDoc._deleted === true) {
6556 // allow deletes by admins and matching users
6557 // without checking the other fields
6558 if ((userCtx.roles.indexOf('_admin') !== -1) ||
6559 (userCtx.name == oldDoc.name)) {
6560 return;
6561 } else {
6562 throw({forbidden: 'Only admins may delete other user docs.'});
6563 }
6564 }
6565 if ((oldDoc && oldDoc.type !== 'user') || newDoc.type !== 'user') {
6567 throw({forbidden : 'doc.type must be user'});
6568 } // we only allow user docs for now
6569 if (!newDoc.name) {
6571 throw({forbidden: 'doc.name is required'});
6572 }
6573 if (!newDoc.roles) {
6575 throw({forbidden: 'doc.roles must exist'});
6576 }
6577 if (!isArray(newDoc.roles)) {
6579 throw({forbidden: 'doc.roles must be an array'});
6580 }
6581 if (newDoc._id !== ('org.couchdb.user:' + newDoc.name)) {
6583 throw({
6584 forbidden: 'Doc ID must be of the form org.couchdb.user:name'
6585 });
6586 }
6587 if (oldDoc) { // validate all updates
6589 if (oldDoc.name !== newDoc.name) {
6590 throw({forbidden: 'Usernames can not be changed.'});
6591 }
6592 }
6593 if (newDoc.password_sha && !newDoc.salt) {
6595 throw({
6596 forbidden: 'Users with password_sha must have a salt.' +
6597 'See /_utils/script/couch.js for example code.'
6598 });
6599 }
6600 var is_server_or_database_admin = function(userCtx, secObj) {
6602 // see if the user is a server admin
6603 if(userCtx.roles.indexOf('_admin') !== -1) {
6604 return true; // a server admin
6605 }
6606 // see if the user a database admin specified by name
6608 if(secObj && secObj.admins && secObj.admins.names) {
6609 if(secObj.admins.names.indexOf(userCtx.name) !== -1) {
6610 return true; // database admin
6611 }
6612 }
6613 // see if the user a database admin specified by role
6615 if(secObj && secObj.admins && secObj.admins.roles) {
6616 var db_roles = secObj.admins.roles;
6617 for(var idx = 0; idx < userCtx.roles.length; idx++) {
6618 var user_role = userCtx.roles[idx];
6619 if(db_roles.indexOf(user_role) !== -1) {
6620 return true; // role matches!
6621 }
6622 }
6623 }
6624 return false; // default to no admin
6626 }
6627 if (!is_server_or_database_admin(userCtx, secObj)) {
6629 if (oldDoc) { // validate non-admin updates
6630 if (userCtx.name !== newDoc.name) {
6631 throw({
6632 forbidden: 'You may only update your own user document.'
6633 });
6634 }
6635 // validate role updates
6636 var oldRoles = oldDoc.roles.sort();
6637 var newRoles = newDoc.roles.sort();
6638 if (oldRoles.length !== newRoles.length) {
6640 throw({forbidden: 'Only _admin may edit roles'});
6641 }
6642 for (var i = 0; i < oldRoles.length; i++) {
6644 if (oldRoles[i] !== newRoles[i]) {
6645 throw({forbidden: 'Only _admin may edit roles'});
6646 }
6647 }
6648 } else if (newDoc.roles.length > 0) {
6649 throw({forbidden: 'Only _admin may set roles'});
6650 }
6651 }
6652 // no system roles in users db
6654 for (var i = 0; i < newDoc.roles.length; i++) {
6655 if (newDoc.roles[i][0] === '_') {
6656 throw({
6657 forbidden:
6658 'No system roles (starting with underscore) in users db.'
6659 });
6660 }
6661 }
6662 // no system names as names
6664 if (newDoc.name[0] === '_') {
6665 throw({forbidden: 'Username may not start with underscore.'});
6666 }
6667 var badUserNameChars = [':'];
6669 for (var i = 0; i < badUserNameChars.length; i++) {
6671 if (newDoc.name.indexOf(badUserNameChars[i]) >= 0) {
6672 throw({forbidden: 'Character `' + badUserNameChars[i] +
6673 '` is not allowed in usernames.'});
6674 }
6675 }
6676 }
6677 NOTE:
6679 The return statement is used only for function, it has no impact on
6680 the validation process.
6681 SEE ALSO:
6683 CouchDB Guide:
6685 · Validation Functions
6687 Guide to Views
6689 Views are the primary tool used for querying and reporting on CouchDB
6690 documents. There you’ll learn how they works and how to use them to
6691 build effective applications with CouchDB
6692 Introduction to Views
6694 Views are useful for many purposes:
6695 · Filtering the documents in your database to find those relevant to a
6697 particular process.
6698 · Extracting data from your documents and presenting it in a specific
6700 order.
6701 · Building efficient indexes to find documents by any value or struc‐
6703 ture that resides in them.
6704 · Use these indexes to represent relationships among documents.
6706 · Finally, with views you can make all sorts of calculations on the
6708 data in your documents. For example, if documents represent your com‐
6709 pany’s financial transactions, a view can answer the question of what
6710 the spending was in the last week, month, or year.
6711 What Is a View?
6713 Let’s go through the different use cases. First is extracting data that
6714 you might need for a special purpose in a specific order. For a front
6715 page, we want a list of blog post titles sorted by date. We’ll work
6716 with a set of example documents as we walk through how views work:
6717 {
6719 "_id":"biking",
6720 "_rev":"AE19EBC7654",
6721 "title":"Biking",
6723 "body":"My biggest hobby is mountainbiking. The other day...",
6724 "date":"2009/01/30 18:04:11"
6725 }
6726 {
6728 "_id":"bought-a-cat",
6729 "_rev":"4A3BBEE711",
6730 "title":"Bought a Cat",
6732 "body":"I went to the the pet store earlier and brought home a little kitty...",
6733 "date":"2009/02/17 21:13:39"
6734 }
6735 {
6737 "_id":"hello-world",
6738 "_rev":"43FBA4E7AB",
6739 "title":"Hello World",
6741 "body":"Well hello and welcome to my new blog...",
6742 "date":"2009/01/15 15:52:20"
6743 }
6744 Three will do for the example. Note that the documents are sorted by
6746 “_id”, which is how they are stored in the database. Now we define a
6747 view. Bear with us without an explanation while we show you some code:
6748 function(doc) {
6750 if(doc.date && doc.title) {
6751 emit(doc.date, doc.title);
6752 }
6753 }
6754 This is a map function, and it is written in JavaScript. If you are not
6756 familiar with JavaScript but have used C or any other C-like language
6757 such as Java, PHP, or C#, this should look familiar. It is a simple
6758 function definition.
6759 You provide CouchDB with view functions as strings stored inside the
6761 views field of a design document. You don’t run it yourself. Instead,
6762 when you query your view, CouchDB takes the source code and runs it for
6763 you on every document in the database your view was defined in. You
6764 query your view to retrieve the view result.
6765 All map functions have a single parameter doc. This is a single docu‐
6767 ment in the database. Our map function checks whether our document has
6768 a date and a title attribute — luckily, all of our documents have them
6769 — and then calls the built-in emit() function with these two attributes
6770 as arguments.
6771 The emit() function always takes two arguments: the first is key, and
6773 the second is value. The emit(key, value) function creates an entry in
6774 our view result. One more thing: the emit() function can be called mul‐
6775 tiple times in the map function to create multiple entries in the view
6776 results from a single document, but we are not doing that yet.
6777 CouchDB takes whatever you pass into the emit() function and puts it
6779 into a list (see Table 1, “View results” below). Each row in that list
6780 includes the key and value. More importantly, the list is sorted by key
6781 (by doc.date in our case). The most important feature of a view result
6782 is that it is sorted by key. We will come back to that over and over
6783 again to do neat things. Stay tuned.
6784 Table 1. View results:
6786 ┌──────────────────────┬────────────────┐
6788 │Key │ Value │
6789 ├──────────────────────┼────────────────┤
6790 │“2009/01/15 15:52:20” │ “Hello World” │
6791 ├──────────────────────┼────────────────┤
6792 │“2009/01/30 18:04:11” │ “Biking” │
6793 ├──────────────────────┼────────────────┤
6794 │“2009/02/17 21:13:39” │ “Bought a Cat” │
6795 └──────────────────────┴────────────────┘
6796 When you query your view, CouchDB takes the source code and runs it for
6798 you on every document in the database. If you have a lot of documents,
6799 that takes quite a bit of time and you might wonder if it is not horri‐
6800 bly inefficient to do this. Yes, it would be, but CouchDB is designed
6801 to avoid any extra costs: it only runs through all documents once, when
6802 you first query your view. If a document is changed, the map function
6803 is only run once, to recompute the keys and values for that single doc‐
6804 ument.
6805 The view result is stored in a B-tree, just like the structure that is
6807 responsible for holding your documents. View B-trees are stored in
6808 their own file, so that for high-performance CouchDB usage, you can
6809 keep views on their own disk. The B-tree provides very fast lookups of
6810 rows by key, as well as efficient streaming of rows in a key range. In
6811 our example, a single view can answer all questions that involve time:
6812 “Give me all the blog posts from last week” or “last month” or “this
6813 year.” Pretty neat.
6814 When we query our view, we get back a list of all documents sorted by
6816 date. Each row also includes the post title so we can construct links
6817 to posts. Table 1 is just a graphical representation of the view
6818 result. The actual result is JSON-encoded and contains a little more
6819 metadata:
6820 {
6822 "total_rows": 3,
6823 "offset": 0,
6824 "rows": [
6825 {
6826 "key": "2009/01/15 15:52:20",
6827 "id": "hello-world",
6828 "value": "Hello World"
6829 },
6830 {
6832 "key": "2009/01/30 18:04:11",
6833 "id": "biking",
6834 "value": "Biking"
6835 },
6836 {
6838 "key": "2009/02/17 21:13:39",
6839 "id": "bought-a-cat",
6840 "value": "Bought a Cat"
6841 }
6842 ]
6844 }
6845 Now, the actual result is not as nicely formatted and doesn’t include
6847 any superfluous whitespace or newlines, but this is better for you (and
6848 us!) to read and understand. Where does that “id” member in the result
6849 rows come from? That wasn’t there before. That’s because we omitted it
6850 earlier to avoid confusion. CouchDB automatically includes the document
6851 ID of the document that created the entry in the view result. We’ll use
6852 this as well when constructing links to the blog post pages.
6853 WARNING:
6855 Do not emit the entire document as the value of your emit(key,
6856 value) statement unless you’re sure you know you want it. This
6857 stores an entire additional copy of your document in the view’s sec‐
6858 ondary index. Views with emit(key, doc) take longer to update,
6859 longer to write to disk, and consume significantly more disk space.
6860 The only advantage is that they are faster to query than using the
6861 ?include_docs=true parameter when querying a view.
6862 Consider the trade-offs before emitting the entire document. Often
6864 it is sufficient to emit only a portion of the document, or just a
6865 single key / value pair, in your views.
6866 Efficient Lookups
6868 Let’s move on to the second use case for views: “building efficient
6869 indexes to find documents by any value or structure that resides in
6870 them.” We already explained the efficient indexing, but we skipped a
6871 few details. This is a good time to finish this discussion as we are
6872 looking at map functions that are a little more complex.
6873 First, back to the B-trees! We explained that the B-tree that backs the
6875 key-sorted view result is built only once, when you first query a view,
6876 and all subsequent queries will just read the B-tree instead of execut‐
6877 ing the map function for all documents again. What happens, though,
6878 when you change a document, add a new one, or delete one? Easy: CouchDB
6879 is smart enough to find the rows in the view result that were created
6880 by a specific document. It marks them invalid so that they no longer
6881 show up in view results. If the document was deleted, we’re good — the
6882 resulting B-tree reflects the state of the database. If a document got
6883 updated, the new document is run through the map function and the
6884 resulting new lines are inserted into the B-tree at the correct spots.
6885 New documents are handled in the same way. The B-tree is a very effi‐
6886 cient data structure for our needs, and the crash-only design of
6887 CouchDB databases is carried over to the view indexes as well.
6888 To add one more point to the efficiency discussion: usually multiple
6890 documents are updated between view queries. The mechanism explained in
6891 the previous paragraph gets applied to all changes in the database
6892 since the last time the view was queried in a batch operation, which
6893 makes things even faster and is generally a better use of your
6894 resources.
6895 Find One
6897 On to more complex map functions. We said “find documents by any value
6898 or structure that resides in them.” We already explained how to extract
6899 a value by which to sort a list of views (our date field). The same
6900 mechanism is used for fast lookups. The URI to query to get a view’s
6901 result is /database/_design/designdocname/_view/viewname. This gives
6902 you a list of all rows in the view. We have only three documents, so
6903 things are small, but with thousands of documents, this can get long.
6904 You can add view parameters to the URI to constrain the result set. Say
6905 we know the date of a blog post. To find a single document, we would
6906 use /blog/_design/docs/_view/by_date?key="2009/01/30 18:04:11" to get
6907 the “Biking” blog post. Remember that you can place whatever you like
6908 in the key parameter to the emit() function. Whatever you put in there,
6909 we can now use to look up exactly — and fast.
6910 Note that in the case where multiple rows have the same key (perhaps we
6912 design a view where the key is the name of the post’s author), key
6913 queries can return more than one row.
6914 Find Many
6916 We talked about “getting all posts for last month.” If it’s February
6917 now, this is as easy as:
6918 /blog/_design/docs/_view/by_date?startkey="2010/01/01 00:00:00"&endkey="2010/02/00 00:00:00"
6920 The startkey and endkey parameters specify an inclusive range on which
6922 we can search.
6923 To make things a little nicer and to prepare for a future example, we
6925 are going to change the format of our date field. Instead of a string,
6926 we are going to use an array, where individual members are part of a
6927 timestamp in decreasing significance. This sounds fancy, but it is
6928 rather easy. Instead of:
6929 {
6931 "date": "2009/01/31 00:00:00"
6932 }
6933 we use:
6935 {
6937 "date": [2009, 1, 31, 0, 0, 0]
6938 }
6939 Our map function does not have to change for this, but our view result
6941 looks a little different:
6942 Table 2. New view results:
6944 ┌──────────────────────────┬────────────────┐
6946 │Key │ Value │
6947 ├──────────────────────────┼────────────────┤
6948 │[2009, 1, 15, 15, 52, 20] │ “Hello World” │
6949 ├──────────────────────────┼────────────────┤
6950 │[2009, 2, 17, 21, 13, 39] │ “Biking” │
6951 ├──────────────────────────┼────────────────┤
6952 │[2009, 1, 30, 18, 4, 11] │ “Bought a Cat” │
6953 └──────────────────────────┴────────────────┘
6954 And our queries change to:
6956 /blog/_design/docs/_view/by_date?startkey=[2010, 1, 1, 0, 0, 0]&endkey=[2010, 2, 1, 0, 0, 0]
6958 For all you care, this is just a change in syntax, not meaning. But it
6960 shows you the power of views. Not only can you construct an index with
6961 scalar values like strings and integers, you can also use JSON struc‐
6962 tures as keys for your views. Say we tag our documents with a list of
6963 tags and want to see all tags, but we don’t care for documents that
6964 have not been tagged.
6965 {
6967 ...
6968 tags: ["cool", "freak", "plankton"],
6969 ...
6970 }
6971 {
6973 ...
6974 tags: [],
6975 ...
6976 }
6977 function(doc) {
6979 if(doc.tags.length > 0) {
6980 for(var idx in doc.tags) {
6981 emit(doc.tags[idx], null);
6982 }
6983 }
6984 }
6985 This shows a few new things. You can have conditions on structure
6987 (if(doc.tags.length > 0)) instead of just values. This is also an exam‐
6988 ple of how a map function calls emit() multiple times per document.
6989 And finally, you can pass null instead of a value to the value parame‐
6990 ter. The same is true for the key parameter. We’ll see in a bit how
6991 that is useful.
6992 Reversed Results
6994 To retrieve view results in reverse order, use the descending=true
6995 query parameter. If you are using a startkey parameter, you will find
6996 that CouchDB returns different rows or no rows at all. What’s up with
6997 that?
6998 It’s pretty easy to understand when you see how view query options work
7000 under the hood. A view is stored in a tree structure for fast lookups.
7001 Whenever you query a view, this is how CouchDB operates:
7002 1. Starts reading at the top, or at the position that startkey speci‐
7004 fies, if present.
7005 2. Returns one row at a time until the end or until it hits endkey, if
7007 present.
7008 If you specify descending=true, the reading direction is reversed, not
7010 the sort order of the rows in the view. In addition, the same two-step
7011 procedure is followed.
7012 Say you have a view result that looks like this:
7014 ┌────┬───────┐
7016 │Key │ Value │
7017 ├────┼───────┤
7018 │0 │ “foo” │
7019 ├────┼───────┤
7020 │1 │ “bar” │
7021 ├────┼───────┤
7022 │2 │ “baz” │
7023 └────┴───────┘
7024 Here are potential query options: ?startkey=1&descending=true. What
7026 will CouchDB do? See #1 above: it jumps to startkey, which is the row
7027 with the key 1, and starts reading backward until it hits the end of
7028 the view. So the particular result would be:
7029 ┌────┬───────┐
7031 │Key │ Value │
7032 ├────┼───────┤
7033 │1 │ “bar” │
7034 ├────┼───────┤
7035 │0 │ “foo” │
7036 └────┴───────┘
7037 This is very likely not what you want. To get the rows with the indexes
7039 1 and 2 in reverse order, you need to switch the startkey to endkey:
7040 endkey=1&descending=true:
7041 ┌────┬───────┐
7043 │Key │ Value │
7044 ├────┼───────┤
7045 │2 │ “baz” │
7046 ├────┼───────┤
7047 │1 │ “bar” │
7048 └────┴───────┘
7049 Now that looks a lot better. CouchDB started reading at the bottom of
7051 the view and went backward until it hit endkey.
7052 The View to Get Comments for Posts
7054 We use an array key here to support the group_level reduce query param‐
7055 eter. CouchDB’s views are stored in the B-tree file structure. Because
7056 of the way B-trees are structured, we can cache the intermediate reduce
7057 results in the non-leaf nodes of the tree, so reduce queries can be
7058 computed along arbitrary key ranges in logarithmic time. See Figure 1,
7059 “Comments map function”.
7060 In the blog app, we use group_level reduce queries to compute the count
7062 of comments both on a per-post and total basis, achieved by querying
7063 the same view index with different methods. With some array keys, and
7064 assuming each key has the value 1:
7065 ["a","b","c"]
7067 ["a","b","e"]
7068 ["a","c","m"]
7069 ["b","a","c"]
7070 ["b","a","g"]
7071 the reduce view:
7073 function(keys, values, rereduce) {
7075 return sum(values)
7076 }
7077 or:
7079 _sum
7081 which is a built-in CouchDB reduce function (the others are _count and
7083 _stats). _sum here returns the total number of rows between the start
7084 and end key. So with startkey=["a","b"]&endkey=["b"] (which includes
7085 the first three of the above keys) the result would equal 3. The effect
7086 is to count rows. If you’d like to count rows without depending on the
7087 row value, you can switch on the rereduce parameter:
7088 function(keys, values, rereduce) {
7090 if (rereduce) {
7091 return sum(values);
7092 } else {
7093 return values.length;
7094 }
7095 }
7096 NOTE:
7098 The JavaScript function above could be effectively replaced by the
7099 built-in _count.
7100 [image: Comments map function] [image] Figure 1. Comments map func‐
7101 tion.UNINDENT
7102 This is the reduce view used by the example app to count comments,
7104 while utilizing the map to output the comments, which are more useful
7105 than just 1 over and over. It pays to spend some time playing around
7106 with map and reduce functions. Fauxton is OK for this, but it doesn’t
7107 give full access to all the query parameters. Writing your own test
7108 code for views in your language of choice is a great way to explore
7109 the nuances and capabilities of CouchDB’s incremental MapReduce sys‐
7110 tem.
7111 Anyway, with a group_level query, you’re basically running a series
7113 of reduce range queries: one for each group that shows up at the
7114 level you query. Let’s reprint the key list from earlier, grouped at
7115 level 1:
7116 ["a"] 3
7118 ["b"] 2
7119 And at group_level=2:
7121 ["a","b"] 2
7123 ["a","c"] 1
7124 ["b","a"] 2
7125 Using the parameter group=true makes it behave as though it were
7127 group_level=999, so in the case of our current example, it would give
7128 the number 1 for each key, as there are no exactly duplicated keys.
7129 Reduce/Rereduce
7131 We briefly talked about the rereduce parameter to the reduce function.
7132 We’ll explain what’s up with it in this section. By now, you should
7133 have learned that your view result is stored in B-tree index structure
7134 for efficiency. The existence and use of the rereduce parameter is
7135 tightly coupled to how the B-tree index works.
7136 Consider the map result are:
7138 "afrikaans", 1
7140 "afrikaans", 1
7141 "chinese", 1
7142 "chinese", 1
7143 "chinese", 1
7144 "chinese", 1
7145 "french", 1
7146 "italian", 1
7147 "italian", 1
7148 "spanish", 1
7149 "vietnamese", 1
7150 "vietnamese", 1
7151 Example 1. Example view result (mmm, food)
7153 When we want to find out how many dishes there are per origin, we can
7155 reuse the simple reduce function shown earlier:
7156 function(keys, values, rereduce) {
7158 return sum(values);
7159 }
7160 Figure 2, “The B-tree index” shows a simplified version of what the
7162 B-tree index looks like. We abbreviated the key strings.
7163 [image: The B-tree index] [image] Figure 2. The B-tree index.UNINDENT
7164 The view result is what computer science grads call a “pre-order”
7166 walk through the tree. We look at each element in each node starting
7167 from the left. Whenever we see that there is a subnode to descend
7168 into, we descend and start reading the elements in that subnode. When
7169 we have walked through the entire tree, we’re done.
7170 You can see that CouchDB stores both keys and values inside each leaf
7172 node. In our case, it is simply always 1, but you might have a value
7173 where you count other results and then all rows have a different
7174 value. What’s important is that CouchDB runs all elements that are
7175 within a node into the reduce function (setting the rereduce parame‐
7176 ter to false) and stores the result inside the parent node along with
7177 the edge to the subnode. In our case, each edge has a 3 representing
7178 the reduce value for the node it points to.
7179 NOTE:
7181 In reality, nodes have more than 1,600 elements in them. CouchDB
7182 computes the result for all the elements in multiple iterations over
7183 the elements in a single node, not all at once (which would be dis‐
7184 astrous for memory consumption).
7185 Now let’s see what happens when we run a query. We want to know how
7187 many “chinese” entries we have. The query option is simple: ?key="chi‐
7188 nese". See Figure 3, “The B-tree index reduce result”.
7189 [image: The B-tree index reduce result] [image] Figure 3. The B-tree
7190 index reduce result.UNINDENT
7191 CouchDB detects that all values in the subnode include the “chinese”
7193 key. It concludes that it can take just the 3 values associated with
7194 that node to compute the final result. It then finds the node left to
7195 it and sees that it’s a node with keys outside the requested range
7196 (key= requests a range where the beginning and the end are the same
7197 value). It concludes that it has to use the “chinese” element’s value
7198 and the other node’s value and run them through the reduce function
7199 with the rereduce parameter set to true.
7200 The reduce function effectively calculates 3 + 1 at query time and
7202 returns the desired result. The next example shows some pseudocode
7203 that shows the last invocation of the reduce function with actual
7204 values:
7205 function(null, [3, 1], true) {
7207 return sum([3, 1]);
7208 }
7209 Now, we said your reduce function must actually reduce your values. If
7211 you see the B-tree, it should become obvious what happens when you
7212 don’t reduce your values. Consider the following map result and reduce
7213 function. This time we want to get a list of all the unique labels in
7214 our view:
7215 "abc", "afrikaans"
7217 "cef", "afrikaans"
7218 "fhi", "chinese"
7219 "hkl", "chinese"
7220 "ino", "chinese"
7221 "lqr", "chinese"
7222 "mtu", "french"
7223 "owx", "italian"
7224 "qza", "italian"
7225 "tdx", "spanish"
7226 "xfg", "vietnamese"
7227 "zul", "vietnamese"
7228 We don’t care for the key here and only list all the labels we have.
7230 Our reduce function removes duplicates:
7231 function(keys, values, rereduce) {
7233 var unique_labels = {};
7234 values.forEach(function(label) {
7235 if(!unique_labels[label]) {
7236 unique_labels[label] = true;
7237 }
7238 });
7239 return unique_labels;
7241 }
7242 This translates to Figure 4, “An overflowing reduce index”.
7244 We hope you get the picture. The way the B-tree storage works means
7246 that if you don’t actually reduce your data in the reduce function, you
7247 end up having CouchDB copy huge amounts of data around that grow lin‐
7248 early, if not faster, with the number of rows in your view.
7249 CouchDB will be able to compute the final result, but only for views
7251 with a few rows. Anything larger will experience a ridiculously slow
7252 view build time. To help with that, CouchDB since version 0.10.0 will
7253 throw an error if your reduce function does not reduce its input val‐
7254 ues.
7255 [image: An overflowing reduce index] [image] Figure 4. An overflowing
7256 reduce index.UNINDENT
7257 One vs. Multiple Design Documents
7259 A common question is: when should I split multiple views into multiple
7260 design documents, or keep them together?
7261 Each view you create corresponds to one B-tree. All views in a single
7263 design document will live in the same set of index files on disk (one
7264 file per database shard; in 2.0+ by default, 8 files per node).
7265 The most practical consideration for separating views into separate
7267 documents is how often you change those views. Views that change often,
7268 and are in the same design document as other views, will invalidate
7269 those other views’ indexes when the design document is written, forcing
7270 them all to rebuild from scratch. Obviously you will want to avoid this
7271 in production!
7272 However, when you have multiple views with the same map function in the
7274 same design document, CouchDB will optimize and only calculate that map
7275 function once. This lets you have two views with different reduce func‐
7276 tions (say, one with _sum and one with _stats) but build only a single
7277 copy of the mapped index. It also saves disk space and the time to
7278 write multiple copies to disk.
7279 Another benefit of having multiple views in the same design document is
7281 that the index files can keep a single index of backwards references
7282 from docids to rows. CouchDB needs these “back refs” to invalidate rows
7283 in a view when a document is deleted (otherwise, a delete would force a
7284 total rebuild!)
7285 One other consideration is that each separate design document will
7287 spawn another (set of) couchjs processes to generate the view, one per
7288 shard. Depending on the number of cores on your server(s), this may be
7289 efficient (using all of the idle cores you have) or inefficient (over‐
7290 loading the CPU on your servers). The exact situation will depend on
7291 your deployment architecture.
7292 So, should you use one or multiple design documents? The choice is
7294 yours.
7295 Lessons Learned
7297 · If you don’t use the key field in the map function, you are probably
7298 doing it wrong.
7299 · If you are trying to make a list of values unique in the reduce func‐
7301 tions, you are probably doing it wrong.
7302 · If you don’t reduce your values to a single scalar value or a small
7304 fixed-sized object or array with a fixed number of scalar values of
7305 small sizes, you are probably doing it wrong.
7306 Wrapping Up
7308 Map functions are side effect–free functions that take a document as
7309 argument and emit key/value pairs. CouchDB stores the emitted rows by
7310 constructing a sorted B-tree index, so row lookups by key, as well as
7311 streaming operations across a range of rows, can be accomplished in a
7312 small memory and processing footprint, while writes avoid seeks. Gener‐
7313 ating a view takes O(N), where N is the total number of rows in the
7314 view. However, querying a view is very quick, as the B-tree remains
7315 shallow even when it contains many, many keys.
7316 Reduce functions operate on the sorted rows emitted by map view func‐
7318 tions. CouchDB’s reduce functionality takes advantage of one of the
7319 fundamental properties of B-tree indexes: for every leaf node (a sorted
7320 row), there is a chain of internal nodes reaching back to the root.
7321 Each leaf node in the B-tree carries a few rows (on the order of tens,
7322 depending on row size), and each internal node may link to a few leaf
7323 nodes or other internal nodes.
7324 The reduce function is run on every node in the tree in order to calcu‐
7326 late the final reduce value. The end result is a reduce function that
7327 can be incrementally updated upon changes to the map function, while
7328 recalculating the reduction values for a minimum number of nodes. The
7329 initial reduction is calculated once per each node (inner and leaf) in
7330 the tree.
7331 When run on leaf nodes (which contain actual map rows), the reduce
7333 function’s third parameter, rereduce, is false. The arguments in this
7334 case are the keys and values as output by the map function. The func‐
7335 tion has a single returned reduction value, which is stored on the
7336 inner node that a working set of leaf nodes have in common, and is used
7337 as a cache in future reduce calculations.
7338 When the reduce function is run on inner nodes, the rereduce flag is
7340 true. This allows the function to account for the fact that it will be
7341 receiving its own prior output. When rereduce is true, the values
7342 passed to the function are intermediate reduction values as cached from
7343 previous calculations. When the tree is more than two levels deep, the
7344 rereduce phase is repeated, consuming chunks of the previous level’s
7345 output until the final reduce value is calculated at the root node.
7346 A common mistake new CouchDB users make is attempting to construct com‐
7348 plex aggregate values with a reduce function. Full reductions should
7349 result in a scalar value, like 5, and not, for instance, a JSON hash
7350 with a set of unique keys and the count of each. The problem with this
7351 approach is that you’ll end up with a very large final value. The num‐
7352 ber of unique keys can be nearly as large as the number of total keys,
7353 even for a large set. It is fine to combine a few scalar calculations
7354 into one reduce function; for instance, to find the total, average, and
7355 standard deviation of a set of numbers in a single function.
7356 If you’re interested in pushing the edge of CouchDB’s incremental
7358 reduce functionality, have a look at Google’s paper on Sawzall, which
7359 gives examples of some of the more exotic reductions that can be accom‐
7360 plished in a system with similar constraints.
7361 Views Collation
7363 Basics
7364 View functions specify a key and a value to be returned for each row.
7365 CouchDB collates the view rows by this key. In the following example,
7366 the LastName property serves as the key, thus the result will be sorted
7367 by LastName:
7368 function(doc) {
7370 if (doc.Type == "customer") {
7371 emit(doc.LastName, {FirstName: doc.FirstName, Address: doc.Address});
7372 }
7373 }
7374 CouchDB allows arbitrary JSON structures to be used as keys. You can
7376 use JSON arrays as keys for fine-grained control over sorting and
7377 grouping.
7378 Examples
7380 The following clever trick would return both customer and order docu‐
7381 ments. The key is composed of a customer _id and a sorting token.
7382 Because the key for order documents begins with the _id of a customer
7383 document, all the orders will be sorted by customer. Because the sort‐
7384 ing token for customers is lower than the token for orders, the cus‐
7385 tomer document will come before the associated orders. The values 0 and
7386 1 for the sorting token are arbitrary.
7387 function(doc) {
7389 if (doc.Type == "customer") {
7390 emit([doc._id, 0], null);
7391 } else if (doc.Type == "order") {
7392 emit([doc.customer_id, 1], null);
7393 }
7394 }
7395 To list a specific customer with _id XYZ, and all of that customer’s
7397 orders, limit the startkey and endkey ranges to cover only documents
7398 for that customer’s _id:
7399 startkey=["XYZ"]&endkey=["XYZ", {}]
7401 It is not recommended to emit the document itself in the view. Instead,
7403 to include the bodies of the documents when requesting the view,
7404 request the view with ?include_docs=true.
7405 Sorting by Dates
7407 It maybe be convenient to store date attributes in a human readable
7408 format (i.e. as a string), but still sort by date. This can be done by
7409 converting the date to a number in the emit() function. For example,
7410 given a document with a created_at attribute of 'Wed Jul 23 16:29:21
7411 +0100 2013', the following emit function would sort by date:
7412 emit(Date.parse(doc.created_at).getTime(), null);
7414 Alternatively, if you use a date format which sorts lexicographically,
7416 such as "2013/06/09 13:52:11 +0000" you can just
7417 emit(doc.created_at, null);
7419 and avoid the conversion. As a bonus, this date format is compatible
7421 with the JavaScript date parser, so you can use new Date(doc.cre‐
7422 ated_at) in your client side JavaScript to make date sorting easy in
7423 the browser.
7424 String Ranges
7426 If you need start and end keys that encompass every string with a given
7427 prefix, it is better to use a high value Unicode character, than to use
7428 a 'ZZZZ' suffix.
7429 That is, rather than:
7431 startkey="abc"&endkey="abcZZZZZZZZZ"
7433 You should use:
7435 startkey="abc"&endkey="abc\ufff0"
7437 Collation Specification
7439 This section is based on the view_collation function in
7440 view_collation.js:
7441 // special values sort before all other types
7443 null
7444 false
7445 true
7446 // then numbers
7448 1
7449 2
7450 3.0
7451 4
7452 // then text, case sensitive
7454 "a"
7455 "A"
7456 "aa"
7457 "b"
7458 "B"
7459 "ba"
7460 "bb"
7461 // then arrays. compared element by element until different.
7463 // Longer arrays sort after their prefixes
7464 ["a"]
7465 ["b"]
7466 ["b","c"]
7467 ["b","c", "a"]
7468 ["b","d"]
7469 ["b","d", "e"]
7470 // then object, compares each key value in the list until different.
7472 // larger objects sort after their subset objects.
7473 {a:1}
7474 {a:2}
7475 {b:1}
7476 {b:2}
7477 {b:2, a:1} // Member order does matter for collation.
7478 // CouchDB preserves member order
7479 // but doesn't require that clients will.
7480 // this test might fail if used with a js engine
7481 // that doesn't preserve order
7482 {b:2, c:2}
7483 Comparison of strings is done using ICU which implements the Unicode
7485 Collation Algorithm, giving a dictionary sorting of keys. This can
7486 give surprising results if you were expecting ASCII ordering. Note
7487 that:
7488 · All symbols sort before numbers and letters (even the “high” symbols
7490 like tilde, 0x7e)
7491 · Differing sequences of letters are compared without regard to case,
7493 so a < aa but also A < aa and a < AA
7494 · Identical sequences of letters are compared with regard to case, with
7496 lowercase before uppercase, so a < A
7497 You can demonstrate the collation sequence for 7-bit ASCII characters
7499 like this:
7500 require 'rubygems'
7502 require 'restclient'
7503 require 'json'
7504 DB="http://127.0.0.1:5984/collator"
7506 RestClient.delete DB rescue nil
7508 RestClient.put "#{DB}",""
7509 (32..126).each do |c|
7511 RestClient.put "#{DB}/#{c.to_s(16)}", {"x"=>c.chr}.to_json
7512 end
7513 RestClient.put "#{DB}/_design/test", <<EOS
7515 {
7516 "views":{
7517 "one":{
7518 "map":"function (doc) { emit(doc.x,null); }"
7519 }
7520 }
7521 }
7522 EOS
7523 puts RestClient.get("#{DB}/_design/test/_view/one")
7525 This shows the collation sequence to be:
7527 ` ^ _ - , ; : ! ? . ' " ( ) [ ] { } @ * / \ & # % + < = > | ~ $ 0 1 2 3 4 5 6 7 8 9
7529 a A b B c C d D e E f F g G h H i I j J k K l L m M n N o O p P q Q r R s S t T u U v V w W x X y Y z Z
7530 Key ranges
7532 Take special care when querying key ranges. For example: the query:
7533 startkey="Abc"&endkey="AbcZZZZ"
7535 will match “ABC” and “abc1”, but not “abc”. This is because UCA sorts
7537 as:
7538 abc < Abc < ABC < abc1 < AbcZZZZZ
7540 For most applications, to avoid problems you should lowercase the
7542 startkey:
7543 startkey="abc"&endkey="abcZZZZZZZZ"
7545 will match all keys starting with [aA][bB][cC]
7547 Complex keys
7549 The query startkey=["foo"]&endkey=["foo",{}] will match most array keys
7550 with “foo” in the first element, such as ["foo","bar"] and
7551 ["foo",["bar","baz"]]. However it will not match
7552 ["foo",{"an":"object"}]
7553 _all_docs
7555 The _all_docs view is a special case because it uses ASCII collation
7556 for doc ids, not UCA:
7557 startkey="_design/"&endkey="_design/ZZZZZZZZ"
7559 will not find _design/abc because ‘Z’ comes before ‘a’ in the ASCII
7561 sequence. A better solution is:
7562 startkey="_design/"&endkey="_design0"
7564 Raw collation
7566 To squeeze a little more performance out of views, you can specify
7567 "options":{"collation":"raw"} within the view definition for native
7568 Erlang collation, especially if you don’t require UCA. This gives a
7569 different collation sequence:
7570 1
7572 false
7573 null
7574 true
7575 {"a":"a"},
7576 ["a"]
7577 "a"
7578 Beware that {} is no longer a suitable “high” key sentinel value. Use a
7580 string like "\ufff0" instead.
7581 Joins With Views
7583 Linked Documents
7584 If your map function emits an object value which has {'_id': XXX} and
7585 you query view with include_docs=true parameter, then CouchDB will
7586 fetch the document with id XXX rather than the document which was pro‐
7587 cessed to emit the key/value pair.
7588 This means that if one document contains the ids of other documents, it
7590 can cause those documents to be fetched in the view too, adjacent to
7591 the same key if required.
7592 For example, if you have the following hierarchically-linked documents:
7594 [
7596 { "_id": "11111" },
7597 { "_id": "22222", "ancestors": ["11111"], "value": "hello" },
7598 { "_id": "33333", "ancestors": ["22222","11111"], "value": "world" }
7599 ]
7600 You can emit the values with the ancestor documents adjacent to them in
7602 the view like this:
7603 function(doc) {
7605 if (doc.value) {
7606 emit([doc.value, 0], null);
7607 if (doc.ancestors) {
7608 for (var i in doc.ancestors) {
7609 emit([doc.value, Number(i)+1], {_id: doc.ancestors[i]});
7610 }
7611 }
7612 }
7613 }
7614 The result you get is:
7616 {
7618 "total_rows": 5,
7619 "offset": 0,
7620 "rows": [
7621 {
7622 "id": "22222",
7623 "key": [
7624 "hello",
7625 0
7626 ],
7627 "value": null,
7628 "doc": {
7629 "_id": "22222",
7630 "_rev": "1-0eee81fecb5aa4f51e285c621271ff02",
7631 "ancestors": [
7632 "11111"
7633 ],
7634 "value": "hello"
7635 }
7636 },
7637 {
7638 "id": "22222",
7639 "key": [
7640 "hello",
7641 1
7642 ],
7643 "value": {
7644 "_id": "11111"
7645 },
7646 "doc": {
7647 "_id": "11111",
7648 "_rev": "1-967a00dff5e02add41819138abb3284d"
7649 }
7650 },
7651 {
7652 "id": "33333",
7653 "key": [
7654 "world",
7655 0
7656 ],
7657 "value": null,
7658 "doc": {
7659 "_id": "33333",
7660 "_rev": "1-11e42b44fdb3d3784602eca7c0332a43",
7661 "ancestors": [
7662 "22222",
7663 "11111"
7664 ],
7665 "value": "world"
7666 }
7667 },
7668 {
7669 "id": "33333",
7670 "key": [
7671 "world",
7672 1
7673 ],
7674 "value": {
7675 "_id": "22222"
7676 },
7677 "doc": {
7678 "_id": "22222",
7679 "_rev": "1-0eee81fecb5aa4f51e285c621271ff02",
7680 "ancestors": [
7681 "11111"
7682 ],
7683 "value": "hello"
7684 }
7685 },
7686 {
7687 "id": "33333",
7688 "key": [
7689 "world",
7690 2
7691 ],
7692 "value": {
7693 "_id": "11111"
7694 },
7695 "doc": {
7696 "_id": "11111",
7697 "_rev": "1-967a00dff5e02add41819138abb3284d"
7698 }
7699 }
7700 ]
7701 }
7702 which makes it very cheap to fetch a document plus all its ancestors in
7704 one query.
7705 Note that the "id" in the row is still that of the originating docu‐
7707 ment. The only difference is that include_docs fetches a different
7708 doc.
7709 The current revision of the document is resolved at query time, not at
7711 the time the view is generated. This means that if a new revision of
7712 the linked document is added later, it will appear in view queries even
7713 though the view itself hasn’t changed. To force a specific revision of
7714 a linked document to be used, emit a "_rev" property as well as "_id".
7715 Using View Collation
7717 Author Christopher Lenz
7718 Date 2007-10-05
7720 Source http://www.cmlenz.net/archives/2007/10/couchdb-joins
7722 Just today, there was a discussion on IRC on how you’d go about model‐
7724 ing a simple blogging system with “post” and “comment” entities, where
7725 any blog post might have N comments. If you’d be using an SQL database,
7726 you’d obviously have two tables with foreign keys and you’d be using
7727 joins. (At least until you needed to add some denormalization).
7728 But what would the “obvious” approach in CouchDB look like?
7730 Approach #1: Comments Inlined
7732 A simple approach would be to have one document per blog post, and
7733 store the comments inside that document:
7734 {
7736 "_id": "myslug",
7737 "_rev": "123456",
7738 "author": "john",
7739 "title": "My blog post",
7740 "content": "Bla bla bla …",
7741 "comments": [
7742 {"author": "jack", "content": "…"},
7743 {"author": "jane", "content": "…"}
7744 ]
7745 }
7746 NOTE:
7748 Of course the model of an actual blogging system would be more
7749 extensive, you’d have tags, timestamps, etc, etc. This is just to
7750 demonstrate the basics.
7751 The obvious advantage of this approach is that the data that belongs
7753 together is stored in one place. Delete the post, and you automatically
7754 delete the corresponding comments, and so on.
7755 You may be thinking that putting the comments inside the blog post doc‐
7757 ument would not allow us to query for the comments themselves, but
7758 you’d be wrong. You could trivially write a CouchDB view that would
7759 return all comments across all blog posts, keyed by author:
7760 function(doc) {
7762 for (var i in doc.comments) {
7763 emit(doc.comments[i].author, doc.comments[i].content);
7764 }
7765 }
7766 Now you could list all comments by a particular user by invoking the
7768 view and passing it a ?key="username" query string parameter.
7769 However, this approach has a drawback that can be quite significant for
7771 many applications: To add a comment to a post, you need to:
7772 · Fetch the blog post document
7774 · Add the new comment to the JSON structure
7776 · Send the updated document to the server
7778 Now if you have multiple client processes adding comments at roughly
7780 the same time, some of them will get a HTTP 409 Conflict error on step
7781 3 (that’s optimistic concurrency in action). For some applications this
7782 makes sense, but in many other apps, you’d want to append new related
7783 data regardless of whether other data has been added in the meantime.
7784 The only way to allow non-conflicting addition of related data is by
7786 putting that related data into separate documents.
7787 Approach #2: Comments Separate
7789 Using this approach you’d have one document per blog post, and one doc‐
7790 ument per comment. The comment documents would have a “backlink” to the
7791 post they belong to.
7792 The blog post document would look similar to the above, minus the com‐
7794 ments property. Also, we’d now have a type property on all our docu‐
7795 ments so that we can tell the difference between posts and comments:
7796 {
7798 "_id": "myslug",
7799 "_rev": "123456",
7800 "type": "post",
7801 "author": "john",
7802 "title": "My blog post",
7803 "content": "Bla bla bla …"
7804 }
7805 The comments themselves are stored in separate documents, which also
7807 have a type property (this time with the value “comment”), and addi‐
7808 tionally feature a post property containing the ID of the post document
7809 they belong to:
7810 {
7812 "_id": "ABCDEF",
7813 "_rev": "123456",
7814 "type": "comment",
7815 "post": "myslug",
7816 "author": "jack",
7817 "content": "…"
7818 }
7819 {
7821 "_id": "DEFABC",
7822 "_rev": "123456",
7823 "type": "comment",
7824 "post": "myslug",
7825 "author": "jane",
7826 "content": "…"
7827 }
7828 To list all comments per blog post, you’d add a simple view, keyed by
7830 blog post ID:
7831 function(doc) {
7833 if (doc.type == "comment") {
7834 emit(doc.post, {author: doc.author, content: doc.content});
7835 }
7836 }
7837 And you’d invoke that view passing it a ?key="post_id" query string
7839 parameter.
7840 Viewing all comments by author is just as easy as before:
7842 function(doc) {
7844 if (doc.type == "comment") {
7845 emit(doc.author, {post: doc.post, content: doc.content});
7846 }
7847 }
7848 So this is better in some ways, but it also has a disadvantage. Imag‐
7850 ine you want to display a blog post with all the associated comments on
7851 the same web page. With our first approach, we needed just a single
7852 request to the CouchDB server, namely a GET request to the document.
7853 With this second approach, we need two requests: a GET request to the
7854 post document, and a GET request to the view that returns all comments
7855 for the post.
7856 That is okay, but not quite satisfactory. Just imagine you wanted to
7858 add threaded comments: you’d now need an additional fetch per comment.
7859 What we’d probably want then would be a way to join the blog post and
7860 the various comments together to be able to retrieve them with a single
7861 HTTP request.
7862 This was when Damien Katz, the author of CouchDB, chimed in to the dis‐
7864 cussion on IRC to show us the way.
7865 Optimization: Using the Power of View Collation
7867 Obvious to Damien, but not at all obvious to the rest of us: it’s
7868 fairly simple to make a view that includes both the content of the blog
7869 post document, and the content of all the comments associated with that
7870 post. The way you do that is by using complex keys. Until now we’ve
7871 been using simple string values for the view keys, but in fact they can
7872 be arbitrary JSON values, so let’s make some use of that:
7873 function(doc) {
7875 if (doc.type == "post") {
7876 emit([doc._id, 0], null);
7877 } else if (doc.type == "comment") {
7878 emit([doc.post, 1], null);
7879 }
7880 }
7881 Okay, this may be confusing at first. Let’s take a step back and look
7883 at what views in CouchDB are really about.
7884 CouchDB views are basically highly efficient on-disk dictionaries that
7886 map keys to values, where the key is automatically indexed and can be
7887 used to filter and/or sort the results you get back from your views.
7888 When you “invoke” a view, you can say that you’re only interested in a
7889 subset of the view rows by specifying a ?key=foo query string parame‐
7890 ter. Or you can specify ?startkey=foo and/or ?endkey=bar query string
7891 parameters to fetch rows over a range of keys. Finally, by adding
7892 ?include_docs=true to the query, the result will include the full body
7893 of each emitted document.
7894 It’s also important to note that keys are always used for collating
7896 (i.e. sorting) the rows. CouchDB has well defined (but as of yet
7897 undocumented) rules for comparing arbitrary JSON objects for collation.
7898 For example, the JSON value ["foo", 2] is sorted after (considered
7899 “greater than”) the values ["foo"] or ["foo", 1, "bar"], but before
7900 e.g. ["foo", 2, "bar"]. This feature enables a whole class of tricks
7901 that are rather non-obvious…
7902 SEE ALSO:
7904 views/collation
7905 With that in mind, let’s return to the view function above. First note
7907 that, unlike the previous view functions we’ve used here, this view
7908 handles both “post” and “comment” documents, and both of them end up as
7909 rows in the same view. Also, the key in this view is not just a simple
7910 string, but an array. The first element in that array is always the ID
7911 of the post, regardless of whether we’re processing an actual post doc‐
7912 ument, or a comment associated with a post. The second element is 0 for
7913 post documents, and 1 for comment documents.
7914 Let’s assume we have two blog posts in our database. Without limiting
7916 the view results via key, startkey, or endkey, we’d get back something
7917 like the following:
7918 {
7920 "total_rows": 5, "offset": 0, "rows": [{
7921 "id": "myslug",
7922 "key": ["myslug", 0],
7923 "value": null
7924 }, {
7925 "id": "ABCDEF",
7926 "key": ["myslug", 1],
7927 "value": null
7928 }, {
7929 "id": "DEFABC",
7930 "key": ["myslug", 1],
7931 "value": null
7932 }, {
7933 "id": "other_slug",
7934 "key": ["other_slug", 0],
7935 "value": null
7936 }, {
7937 "id": "CDEFAB",
7938 "key": ["other_slug", 1],
7939 "value": null
7940 },
7941 ]
7942 }
7943 NOTE:
7945 The ... placeholders here would contain the complete JSON encoding
7946 of the corresponding documents
7947 Now, to get a specific blog post and all associated comments, we’d
7949 invoke that view with the query string:
7950 ?startkey=["myslug"]&endkey=["myslug", 2]&include_docs=true
7952 We’d get back the first three rows, those that belong to the myslug
7954 post, but not the others, along with the full bodies of each document.
7955 Et voila, we now have the data we need to display a post with all asso‐
7956 ciated comments, retrieved via a single GET request.
7957 You may be asking what the 0 and 1 parts of the keys are for. They’re
7959 simply to ensure that the post document is always sorted before the the
7960 associated comment documents. So when you get back the results from
7961 this view for a specific post, you’ll know that the first row contains
7962 the data for the blog post itself, and the remaining rows contain the
7963 comment data.
7964 One remaining problem with this model is that comments are not ordered,
7966 but that’s simply because we don’t have date/time information associ‐
7967 ated with them. If we had, we’d add the timestamp as third element of
7968 the key array, probably as ISO date/time strings. Now we would continue
7969 using the query string ?startkey=["myslug"]&endkey=["myslug",
7970 2]&include_docs=true to fetch the blog post and all associated com‐
7971 ments, only now they’d be in chronological order.
7972 View Cookbook for SQL Jockeys
7974 This is a collection of some common SQL queries and how to get the same
7975 result in CouchDB. The key to remember here is that CouchDB does not
7976 work like an SQL database at all, and that best practices from the SQL
7977 world do not translate well or at all to CouchDB. This document’s
7978 “cookbook” assumes that you are familiar with the CouchDB basics such
7979 as creating and updating databases and documents.
7980 Using Views
7982 How you would do this in SQL:
7983 CREATE TABLE
7985 or:
7987 ALTER TABLE
7989 How you can do this in CouchDB?
7991 Using views is a two-step process. First you define a view; then you
7993 query it. This is analogous to defining a table structure (with
7994 indexes) using CREATE TABLE or ALTER TABLE and querying it using an SQL
7995 query.
7996 Defining a View
7998 Defining a view is done by creating a special document in a CouchDB
7999 database. The only real specialness is the _id of the document, which
8000 starts with _design/ — for example, _design/application. Other than
8001 that, it is just a regular CouchDB document. To make sure CouchDB
8002 understands that you are defining a view, you need to prepare the con‐
8003 tents of that design document in a special format. Here is an example:
8004 {
8006 "_id": "_design/application",
8007 "_rev": "1-C1687D17",
8008 "views": {
8009 "viewname": {
8010 "map": "function(doc) { ... }",
8011 "reduce": "function(keys, values) { ... }"
8012 }
8013 }
8014 }
8015 We are defining a view viewname. The definition of the view consists of
8017 two functions: the map function and the reduce function. Specifying a
8018 reduce function is optional. We’ll look at the nature of the functions
8019 later. Note that viewname can be whatever you like: users, by-name, or
8020 by-date are just some examples.
8021 A single design document can also include multiple view definitions,
8023 each identified by a unique name:
8024 {
8026 "_id": "_design/application",
8027 "_rev": "1-C1687D17",
8028 "views": {
8029 "viewname": {
8030 "map": "function(doc) { ... }",
8031 "reduce": "function(keys, values) { ... }"
8032 },
8033 "anotherview": {
8034 "map": "function(doc) { ... }",
8035 "reduce": "function(keys, values) { ... }"
8036 }
8037 }
8038 }
8039 Querying a View
8041 The name of the design document and the name of the view are signifi‐
8042 cant for querying the view. To query the view viewname, you perform an
8043 HTTP GET request to the following URI:
8044 /database/_design/application/_view/viewname
8046 database is the name of the database you created your design document
8048 in. Next up is the design document name, and then the view name pre‐
8049 fixed with _view/. To query anotherview, replace viewname in that URI
8050 with anotherview. If you want to query a view in a different design
8051 document, adjust the design document name.
8052 MapReduce Functions
8054 MapReduce is a concept that solves problems by applying a two-step
8055 process, aptly named the map phase and the reduce phase. The map phase
8056 looks at all documents in CouchDB separately one after the other and
8057 creates a map result. The map result is an ordered list of key/value
8058 pairs. Both key and value can be specified by the user writing the map
8059 function. A map function may call the built-in emit(key, value) func‐
8060 tion 0 to N times per document, creating a row in the map result per
8061 invocation.
8062 CouchDB is smart enough to run a map function only once for every docu‐
8064 ment, even on subsequent queries on a view. Only changes to documents
8065 or new documents need to be processed anew.
8066 Map functions
8068 Map functions run in isolation for every document. They can’t modify
8069 the document, and they can’t talk to the outside world—they can’t have
8070 side effects. This is required so that CouchDB can guarantee correct
8071 results without having to recalculate a complete result when only one
8072 document gets changed.
8073 The map result looks like this:
8075 {"total_rows":3,"offset":0,"rows":[
8077 {"id":"fc2636bf50556346f1ce46b4bc01fe30","key":"Lena","value":5},
8078 {"id":"1fb2449f9b9d4e466dbfa47ebe675063","key":"Lisa","value":4},
8079 {"id":"8ede09f6f6aeb35d948485624b28f149","key":"Sarah","value":6}
8080 ]}
8081 It is a list of rows sorted by the value of key. The id is added auto‐
8083 matically and refers back to the document that created this row. The
8084 value is the data you’re looking for. For example purposes, it’s the
8085 girl’s age.
8086 The map function that produces this result is:
8088 function(doc) {
8090 if(doc.name && doc.age) {
8091 emit(doc.name, doc.age);
8092 }
8093 }
8094 It includes the if statement as a sanity check to ensure that we’re
8096 operating on the right fields and calls the emit function with the name
8097 and age as the key and value.
8098 Look Up by Key
8100 How you would do this in SQL:
8101 SELECT field FROM table WHERE value="searchterm"
8103 How you can do this in CouchDB?
8105 Use case: get a result (which can be a record or set of records) asso‐
8107 ciated with a key (“searchterm”).
8108 To look something up quickly, regardless of the storage mechanism, an
8110 index is needed. An index is a data structure optimized for quick
8111 search and retrieval. CouchDB’s map result is stored in such an index,
8112 which happens to be a B+ tree.
8113 To look up a value by “searchterm”, we need to put all values into the
8115 key of a view. All we need is a simple map function:
8116 function(doc) {
8118 if(doc.value) {
8119 emit(doc.value, null);
8120 }
8121 }
8122 This creates a list of documents that have a value field sorted by the
8124 data in the value field. To find all the records that match
8125 “searchterm”, we query the view and specify the search term as a query
8126 parameter:
8127 /database/_design/application/_view/viewname?key="searchterm"
8129 Consider the documents from the previous section, and say we’re index‐
8131 ing on the age field of the documents to find all the five-year-olds:
8132 function(doc) {
8134 if(doc.age && doc.name) {
8135 emit(doc.age, doc.name);
8136 }
8137 }
8138 Query:
8140 /ladies/_design/ladies/_view/age?key=5
8142 Result:
8144 {"total_rows":3,"offset":1,"rows":[
8146 {"id":"fc2636bf50556346f1ce46b4bc01fe30","key":5,"value":"Lena"}
8147 ]}
8148 Easy.
8150 Note that you have to emit a value. The view result includes the asso‐
8152 ciated document ID in every row. We can use it to look up more data
8153 from the document itself. We can also use the ?include_docs=true param‐
8154 eter to have CouchDB fetch the individual documents for us.
8155 Look Up by Prefix
8157 How you would do this in SQL:
8158 SELECT field FROM table WHERE value LIKE "searchterm%"
8160 How you can do this in CouchDB?
8162 Use case: find all documents that have a field value that starts with
8164 searchterm. For example, say you stored a MIME type (like text/html or
8165 image/jpg) for each document and now you want to find all documents
8166 that are images according to the MIME type.
8167 The solution is very similar to the previous example: all we need is a
8169 map function that is a little more clever than the first one. But
8170 first, an example document:
8171 {
8173 "_id": "Hugh Laurie",
8174 "_rev": "1-9fded7deef52ac373119d05435581edf",
8175 "mime-type": "image/jpg",
8176 "description": "some dude"
8177 }
8178 The clue lies in extracting the prefix that we want to search for from
8180 our document and putting it into our view index. We use a regular
8181 expression to match our prefix:
8182 function(doc) {
8184 if(doc["mime-type"]) {
8185 // from the start (^) match everything that is not a slash ([^\/]+) until
8186 // we find a slash (\/). Slashes needs to be escaped with a backslash (\/)
8187 var prefix = doc["mime-type"].match(/^[^\/]+\//);
8188 if(prefix) {
8189 emit(prefix, null);
8190 }
8191 }
8192 }
8193 We can now query this view with our desired MIME type prefix and not
8195 only find all images, but also text, video, and all other formats:
8196 /files/_design/finder/_view/by-mime-type?key="image/"
8198 Aggregate Functions
8200 How you would do this in SQL:
8201 SELECT COUNT(field) FROM table
8203 How you can do this in CouchDB?
8205 Use case: calculate a derived value from your data.
8207 We haven’t explained reduce functions yet. Reduce functions are similar
8209 to aggregate functions in SQL. They compute a value over multiple docu‐
8210 ments.
8211 To explain the mechanics of reduce functions, we’ll create one that
8213 doesn’t make a whole lot of sense. But this example is easy to under‐
8214 stand. We’ll explore more useful reductions later.
8215 Reduce functions operate on the output of the map function (also called
8217 the map result or intermediate result). The reduce function’s job,
8218 unsurprisingly, is to reduce the list that the map function produces.
8219 Here’s what our summing reduce function looks like:
8221 function(keys, values) {
8223 var sum = 0;
8224 for(var idx in values) {
8225 sum = sum + values[idx];
8226 }
8227 return sum;
8228 }
8229 Here’s an alternate, more idiomatic JavaScript version:
8231 function(keys, values) {
8233 var sum = 0;
8234 values.forEach(function(element) {
8235 sum = sum + element;
8236 });
8237 return sum;
8238 }
8239 NOTE:
8241 Don’t miss effective built-in reduce functions like _sum and _count
8242 This reduce function takes two arguments: a list of keys and a list of
8244 values. For our summing purposes we can ignore the keys-list and con‐
8245 sider only the value list. We’re looping over the list and add each
8246 item to a running total that we’re returning at the end of the func‐
8247 tion.
8248 You’ll see one difference between the map and the reduce function. The
8250 map function uses emit() to create its result, whereas the reduce func‐
8251 tion returns a value.
8252 For example, from a list of integer values that specify the age, calcu‐
8254 late the sum of all years of life for the news headline, “786 life
8255 years present at event.” A little contrived, but very simple and thus
8256 good for demonstration purposes. Consider the documents and the map
8257 view we used earlier in this document.
8258 The reduce function to calculate the total age of all girls is:
8260 function(keys, values) {
8262 return sum(values);
8263 }
8264 Note that, instead of the two earlier versions, we use CouchDB’s prede‐
8266 fined sum() function. It does the same thing as the other two, but it
8267 is such a common piece of code that CouchDB has it included.
8268 The result for our reduce view now looks like this:
8270 {"rows":[
8272 {"key":null,"value":15}
8273 ]}
8274 The total sum of all age fields in all our documents is 15. Just what
8276 we wanted. The key member of the result object is null, as we can’t
8277 know anymore which documents took part in the creation of the reduced
8278 result. We’ll cover more advanced reduce cases later on.
8279 As a rule of thumb, the reduce function should reduce to a single
8281 scalar value. That is, an integer; a string; or a small, fixed-size
8282 list or object that includes an aggregated value (or values) from the
8283 values argument. It should never just return values or similar.
8284 CouchDB will give you a warning if you try to use reduce “the wrong
8285 way”:
8286 {
8288 "error":"reduce_overflow_error",
8289 "message":"Reduce output must shrink more rapidly: Current output: ..."
8290 }
8291 Get Unique Values
8293 How you would do this in SQL:
8294 SELECT DISTINCT field FROM table
8296 How you can do this in CouchDB?
8298 Getting unique values is not as easy as adding a keyword. But a reduce
8300 view and a special query parameter give us the same result. Let’s say
8301 you want a list of tags that your users have tagged themselves with and
8302 no duplicates.
8303 First, let’s look at the source documents. We punt on _id and _rev
8305 attributes here:
8306 {
8308 "name":"Chris",
8309 "tags":["mustache", "music", "couchdb"]
8310 }
8311 {
8313 "name":"Noah",
8314 "tags":["hypertext", "philosophy", "couchdb"]
8315 }
8316 {
8318 "name":"Jan",
8319 "tags":["drums", "bike", "couchdb"]
8320 }
8321 Next, we need a list of all tags. A map function will do the trick:
8323 function(doc) {
8325 if(doc.name && doc.tags) {
8326 doc.tags.forEach(function(tag) {
8327 emit(tag, null);
8328 });
8329 }
8330 }
8331 The result will look like this:
8333 {"total_rows":9,"offset":0,"rows":[
8335 {"id":"3525ab874bc4965fa3cda7c549e92d30","key":"bike","value":null},
8336 {"id":"3525ab874bc4965fa3cda7c549e92d30","key":"couchdb","value":null},
8337 {"id":"53f82b1f0ff49a08ac79a9dff41d7860","key":"couchdb","value":null},
8338 {"id":"da5ea89448a4506925823f4d985aabbd","key":"couchdb","value":null},
8339 {"id":"3525ab874bc4965fa3cda7c549e92d30","key":"drums","value":null},
8340 {"id":"53f82b1f0ff49a08ac79a9dff41d7860","key":"hypertext","value":null},
8341 {"id":"da5ea89448a4506925823f4d985aabbd","key":"music","value":null},
8342 {"id":"da5ea89448a4506925823f4d985aabbd","key":"mustache","value":null},
8343 {"id":"53f82b1f0ff49a08ac79a9dff41d7860","key":"philosophy","value":null}
8344 ]}
8345 As promised, these are all the tags, including duplicates. Since each
8347 document gets run through the map function in isolation, it cannot know
8348 if the same key has been emitted already. At this stage, we need to
8349 live with that. To achieve uniqueness, we need a reduce:
8350 function(keys, values) {
8352 return true;
8353 }
8354 This reduce doesn’t do anything, but it allows us to specify a special
8356 query parameter when querying the view:
8357 /dudes/_design/dude-data/_view/tags?group=true
8359 CouchDB replies:
8361 {"rows":[
8363 {"key":"bike","value":true},
8364 {"key":"couchdb","value":true},
8365 {"key":"drums","value":true},
8366 {"key":"hypertext","value":true},
8367 {"key":"music","value":true},
8368 {"key":"mustache","value":true},
8369 {"key":"philosophy","value":true}
8370 ]}
8371 In this case, we can ignore the value part because it is always true,
8373 but the result includes a list of all our tags and no duplicates!
8374 With a small change we can put the reduce to good use, too. Let’s see
8376 how many of the non-unique tags are there for each tag. To calculate
8377 the tag frequency, we just use the summing up we already learned about.
8378 In the map function, we emit a 1 instead of null:
8379 function(doc) {
8381 if(doc.name && doc.tags) {
8382 doc.tags.forEach(function(tag) {
8383 emit(tag, 1);
8384 });
8385 }
8386 }
8387 In the reduce function, we return the sum of all values:
8389 function(keys, values) {
8391 return sum(values);
8392 }
8393 Now, if we query the view with the ?group=true parameter, we get back
8395 the count for each tag:
8396 {"rows":[
8398 {"key":"bike","value":1},
8399 {"key":"couchdb","value":3},
8400 {"key":"drums","value":1},
8401 {"key":"hypertext","value":1},
8402 {"key":"music","value":1},
8403 {"key":"mustache","value":1},
8404 {"key":"philosophy","value":1}
8405 ]}
8406 Enforcing Uniqueness
8408 How you would do this in SQL:
8409 UNIQUE KEY(column)
8411 How you can do this in CouchDB?
8413 Use case: your applications require that a certain value exists only
8415 once in a database.
8416 This is an easy one: within a CouchDB database, each document must have
8418 a unique _id field. If you require unique values in a database, just
8419 assign them to a document’s _id field and CouchDB will enforce unique‐
8420 ness for you.
8421 There’s one caveat, though: in the distributed case, when you are run‐
8423 ning more than one CouchDB node that accepts write requests, uniqueness
8424 can be guaranteed only per node or outside of CouchDB. CouchDB will
8425 allow two identical IDs to be written to two different nodes. On repli‐
8426 cation, CouchDB will detect a conflict and flag the document accord‐
8427 ingly.
8428 Pagination Recipe
8430 This recipe explains how to paginate over view results. Pagination is
8431 a user interface (UI) pattern that allows the display of a large number
8432 of rows (the result set) without loading all the rows into the UI at
8433 once. A fixed-size subset, the page, is displayed along with next and
8434 previous links or buttons that can move the viewport over the result
8435 set to an adjacent page.
8436 We assume you’re familiar with creating and querying documents and
8438 views as well as the multiple view query options.
8439 Example Data
8441 To have some data to work with, we’ll create a list of bands, one docu‐
8442 ment per band:
8443 { "name":"Biffy Clyro" }
8445 { "name":"Foo Fighters" }
8447 { "name":"Tool" }
8449 { "name":"Nirvana" }
8451 { "name":"Helmet" }
8453 { "name":"Tenacious D" }
8455 { "name":"Future of the Left" }
8457 { "name":"A Perfect Circle" }
8459 { "name":"Silverchair" }
8461 { "name":"Queens of the Stone Age" }
8463 { "name":"Kerub" }
8465 A View
8467 We need a simple map function that gives us an alphabetical list of
8468 band names. This should be easy, but we’re adding extra smarts to fil‐
8469 ter out “The” and “A” in front of band names to put them into the right
8470 position:
8471 function(doc) {
8473 if(doc.name) {
8474 var name = doc.name.replace(/^(A|The) /, "");
8475 emit(name, null);
8476 }
8477 }
8478 The views result is an alphabetical list of band names. Now say we want
8480 to display band names five at a time and have a link pointing to the
8481 next five names that make up one page, and a link for the previous
8482 five, if we’re not on the first page.
8483 We learned how to use the startkey, limit, and skip parameters in ear‐
8485 lier documents. We’ll use these again here. First, let’s have a look at
8486 the full result set:
8487 {"total_rows":11,"offset":0,"rows":[
8489 {"id":"a0746072bba60a62b01209f467ca4fe2","key":"Biffy Clyro","value":null},
8490 {"id":"b47d82284969f10cd1b6ea460ad62d00","key":"Foo Fighters","value":null},
8491 {"id":"45ccde324611f86ad4932555dea7fce0","key":"Tenacious D","value":null},
8492 {"id":"d7ab24bb3489a9010c7d1a2087a4a9e4","key":"Future of the Left","value":null},
8493 {"id":"ad2f85ef87f5a9a65db5b3a75a03cd82","key":"Helmet","value":null},
8494 {"id":"a2f31cfa68118a6ae9d35444fcb1a3cf","key":"Nirvana","value":null},
8495 {"id":"67373171d0f626b811bdc34e92e77901","key":"Kerub","value":null},
8496 {"id":"3e1b84630c384f6aef1a5c50a81e4a34","key":"Perfect Circle","value":null},
8497 {"id":"84a371a7b8414237fad1b6aaf68cd16a","key":"Queens of the Stone Age","value":null},
8498 {"id":"dcdaf08242a4be7da1a36e25f4f0b022","key":"Silverchair","value":null},
8499 {"id":"fd590d4ad53771db47b0406054f02243","key":"Tool","value":null}
8500 ]}
8501 Setup
8503 The mechanics of paging are very simple:
8504 · Display first page
8506 · If there are more rows to show, show next link
8508 · Draw subsequent page
8510 · If this is not the first page, show a previous link
8512 · If there are more rows to show, show next link
8514 Or in a pseudo-JavaScript snippet:
8516 var result = new Result();
8518 var page = result.getPage();
8519 page.display();
8521 if(result.hasPrev()) {
8523 page.display_link('prev');
8524 }
8525 if(result.hasNext()) {
8527 page.display_link('next');
8528 }
8529 Paging
8531 To get the first five rows from the view result, you use the ?limit=5
8532 query parameter:
8533 curl -X GET http://127.0.0.1:5984/artists/_design/artists/_view/by-name?limit=5
8535 The result:
8537 {"total_rows":11,"offset":0,"rows":[
8539 {"id":"a0746072bba60a62b01209f467ca4fe2","key":"Biffy Clyro","value":null},
8540 {"id":"b47d82284969f10cd1b6ea460ad62d00","key":"Foo Fighters","value":null},
8541 {"id":"45ccde324611f86ad4932555dea7fce0","key":"Tenacious D","value":null},
8542 {"id":"d7ab24bb3489a9010c7d1a2087a4a9e4","key":"Future of the Left","value":null},
8543 {"id":"ad2f85ef87f5a9a65db5b3a75a03cd82","key":"Helmet","value":null}
8544 ]}
8545 By comparing the total_rows value to our limit value, we can determine
8547 if there are more pages to display. We also know by the offset member
8548 that we are on the first page. We can calculate the value for skip= to
8549 get the results for the next page:
8550 var rows_per_page = 5;
8552 var page = (offset / rows_per_page) + 1; // == 1
8553 var skip = page * rows_per_page; // == 5 for the first page, 10 for the second ...
8554 So we query CouchDB with:
8556 curl -X GET 'http://127.0.0.1:5984/artists/_design/artists/_view/by-name?limit=5&skip=5'
8558 Note we have to use ' (single quotes) to escape the & character that is
8560 special to the shell we execute curl in.
8561 The result:
8563 {"total_rows":11,"offset":5,"rows":[
8565 {"id":"a2f31cfa68118a6ae9d35444fcb1a3cf","key":"Nirvana","value":null},
8566 {"id":"67373171d0f626b811bdc34e92e77901","key":"Kerub","value":null},
8567 {"id":"3e1b84630c384f6aef1a5c50a81e4a34","key":"Perfect Circle","value":null},
8568 {"id":"84a371a7b8414237fad1b6aaf68cd16a","key":"Queens of the Stone Age",
8569 "value":null},
8570 {"id":"dcdaf08242a4be7da1a36e25f4f0b022","key":"Silverchair","value":null}
8571 ]}
8572 Implementing the hasPrev() and hasNext() method is pretty straightfor‐
8574 ward:
8575 function hasPrev()
8577 {
8578 return page > 1;
8579 }
8580 function hasNext()
8582 {
8583 var last_page = Math.floor(total_rows / rows_per_page) +
8584 (total_rows % rows_per_page);
8585 return page != last_page;
8586 }
8587 Paging (Alternate Method)
8589 The method described above performed poorly with large skip values
8590 until CouchDB 1.2. Additionally, some use cases may call for the fol‐
8591 lowing alternate method even with newer versions of CouchDB. One such
8592 case is when duplicate results should be prevented. Using skip alone it
8593 is possible for new documents to be inserted during pagination which
8594 could change the offset of the start of the subsequent page.
8595 A correct solution is not much harder. Instead of slicing the result
8597 set into equally sized pages, we look at 10 rows at a time and use
8598 startkey to jump to the next 10 rows. We even use skip, but only with
8599 the value 1.
8600 Here is how it works:
8602 · Request rows_per_page + 1 rows from the view
8604 · Display rows_per_page rows, store + 1 row as next_startkey and
8606 next_startkey_docid
8607 · As page information, keep startkey and next_startkey
8609 · Use the next_* values to create the next link, and use the others to
8611 create the previous link
8612 The trick to finding the next page is pretty simple. Instead of
8614 requesting 10 rows for a page, you request 11 rows, but display only 10
8615 and use the values in the 11th row as the startkey for the next page.
8616 Populating the link to the previous page is as simple as carrying the
8617 current startkey over to the next page. If there’s no previous
8618 startkey, we are on the first page. We stop displaying the link to the
8619 next page if we get rows_per_page or less rows back. This is called
8620 linked list pagination, as we go from page to page, or list item to
8621 list item, instead of jumping directly to a pre-computed page. There is
8622 one caveat, though. Can you spot it?
8623 CouchDB view keys do not have to be unique; you can have multiple index
8625 entries read. What if you have more index entries for a key than rows
8626 that should be on a page? startkey jumps to the first row, and you’d be
8627 screwed if CouchDB didn’t have an additional parameter for you to use.
8628 All view keys with the same value are internally sorted by docid, that
8629 is, the ID of the document that created that view row. You can use the
8630 startkey_docid and endkey_docid parameters to get subsets of these
8631 rows. For pagination, we still don’t need endkey_docid, but
8632 startkey_docid is very handy. In addition to startkey and limit, you
8633 also use startkey_docid for pagination if, and only if, the extra row
8634 you fetch to find the next page has the same key as the current
8635 startkey.
8636 It is important to note that the *_docid parameters only work in addi‐
8638 tion to the *key parameters and are only useful to further narrow down
8639 the result set of a view for a single key. They do not work on their
8640 own (the one exception being the built-in _all_docs view that already
8641 sorts by document ID).
8642 The advantage of this approach is that all the key operations can be
8644 performed on the super-fast B-tree index behind the view. Looking up a
8645 page doesn’t include scanning through hundreds and thousands of rows
8646 unnecessarily.
8647 Jump to Page
8649 One drawback of the linked list style pagination is that you can’t
8650 pre-compute the rows for a particular page from the page number and the
8651 rows per page. Jumping to a specific page doesn’t really work. Our gut
8652 reaction, if that concern is raised, is, “Not even Google is doing
8653 that!” and we tend to get away with it. Google always pretends on the
8654 first page to find 10 more pages of results. Only if you click on the
8655 second page (something very few people actually do) might Google dis‐
8656 play a reduced set of pages. If you page through the results, you get
8657 links for the previous and next 10 pages, but no more. Pre-computing
8658 the necessary startkey and startkey_docid for 20 pages is a feasible
8659 operation and a pragmatic optimization to know the rows for every page
8660 in a result set that is potentially tens of thousands of rows long, or
8661 more.
8662 If you really do need to jump to a page over the full range of docu‐
8664 ments (we have seen applications that require that), you can still
8665 maintain an integer value index as the view index and take a hybrid
8666 approach at solving pagination.
8667 Search
8669 Search indexes enable you to query a database by using the Lucene Query
8670 Parser Syntax. A search index uses one, or multiple, fields from your
8671 documents. You can use a search index to run queries, find documents
8672 based on the content they contain, or work with groups, facets, or geo‐
8673 graphical searches.
8674 WARNING:
8676 Search cannot function unless it has a functioning, cluster-con‐
8677 nected Clouseau instance. See Search Plugin Installation for
8678 details.
8679 To create a search index, you add a JavaScript function to a design
8681 document in the database. An index builds after processing one search
8682 request or after the server detects a document update. The index func‐
8683 tion takes the following parameters:
8684 1. Field name - The name of the field you want to use when you query
8686 the index. If you set this parameter to default, then this field is
8687 queried if no field is specified in the query syntax.
8688 2. Data that you want to index, for example, doc.address.country.
8690 3. (Optional) The third parameter includes the following fields:
8692 boost, facet, index, and store. These fields are described in more
8693 detail later.
8694 By default, a search index response returns 25 rows. The number of rows
8696 that is returned can be changed by using the limit parameter. Each
8697 response includes a bookmark field. You can include the value of the
8698 bookmark field in later queries to look through the responses.
8699 Example design document that defines a search index:
8701 {
8703 "_id": "_design/search_example",
8704 "indexes": {
8705 "animals": {
8706 "index": "function(doc){ ... }"
8707 }
8708 }
8709 }
8710 A search index will inherit the partitioning type from the options.par‐
8712 titioned field of the design document that contains it.
8713 Index functions
8715 Attempting to index by using a data field that does not exist fails. To
8716 avoid this problem, use the appropriate guard clause.
8717 NOTE:
8719 Your indexing functions operate in a memory-constrained environment
8720 where the document itself forms a part of the memory that is used in
8721 that environment. Your code’s stack and document must fit inside
8722 this memory. In other words, a document must be loaded in order to
8723 be indexed. Documents are limited to a maximum size of 64 MB.
8724 NOTE:
8726 Within a search index, do not index the same field name with more
8727 than one data type. If the same field name is indexed with different
8728 data types in the same search index function, you might get an error
8729 when querying the search index that says the field “was indexed
8730 without position data.” For example, do not include both of these
8731 lines in the same search index function, as they index the myfield
8732 field as two different data types: a string "this is a string" and a
8733 number 123.
8734 index("myfield", "this is a string");
8736 index("myfield", 123);
8737 The function that is contained in the index field is a JavaScript func‐
8739 tion that is called for each document in the database. The function
8740 takes the document as a parameter, extracts some data from it, and then
8741 calls the function that is defined in the index field to index that
8742 data.
8743 The index function takes three parameters, where the third parameter is
8745 optional.
8746 The first parameter is the name of the field you intend to use when
8748 querying the index, and which is specified in the Lucene syntax portion
8749 of subsequent queries. An example appears in the following query:
8750 query=color:red
8752 The Lucene field name color is the first parameter of the index func‐
8754 tion.
8755 The query parameter can be abbreviated to q, so another way of writing
8757 the query is as follows:
8758 q=color:red
8760 If the special value "default" is used when you define the name, you do
8762 not have to specify a field name at query time. The effect is that the
8763 query can be simplified:
8764 query=red
8766 The second parameter is the data to be indexed. Keep the following
8768 information in mind when you index your data:
8769 · This data must be only a string, number, or boolean. Other types will
8771 cause an error to be thrown by the index function call.
8772 · If an error is thrown when running your function, for this reason or
8774 others, the document will not be added to that search index.
8775 The third, optional, parameter is a JavaScript object with the follow‐
8777 ing fields:
8778 Index function (optional parameter)
8780 · boost - A number that specifies the relevance in search results. Con‐
8782 tent that is indexed with a boost value greater than 1 is more rele‐
8783 vant than content that is indexed without a boost value. Content with
8784 a boost value less than one is not so relevant. Value is a positive
8785 floating point number. Default is 1 (no boosting).
8786 · facet - Creates a faceted index. See Faceting. Values are true or
8788 false. Default is false.
8789 · index - Whether the data is indexed, and if so, how. If set to false,
8791 the data cannot be used for searches, but can still be retrieved from
8792 the index if store is set to true. See Analyzers. Values are true or
8793 false. Default is true
8794 · store - If true, the value is returned in the search result; other‐
8796 wise, the value is not returned. Values are true or false. Default is
8797 false.
8798 NOTE:
8800 If you do not set the store parameter, the index data results for
8801 the document are not returned in response to a query.
8802 Example search index function:
8804 function(doc) {
8806 index("default", doc._id);
8807 if (doc.min_length) {
8808 index("min_length", doc.min_length, {"store": true});
8809 }
8810 if (doc.diet) {
8811 index("diet", doc.diet, {"store": true});
8812 }
8813 if (doc.latin_name) {
8814 index("latin_name", doc.latin_name, {"store": true});
8815 }
8816 if (doc.class) {
8817 index("class", doc.class, {"store": true});
8818 }
8819 }
8820 Index guard clauses
8822 The index function requires the name of the data field to index as the
8823 second parameter. However, if that data field does not exist for the
8824 document, an error occurs. The solution is to use an appropriate
8825 ‘guard clause’ that checks if the field exists, and contains the
8826 expected type of data, before any attempt to create the corresponding
8827 index.
8828 Example of failing to check whether the index data field exists:
8830 if (doc.min_length) {
8832 index("min_length", doc.min_length, {"store": true});
8833 }
8834 You might use the JavaScript typeof function to implement the guard
8836 clause test. If the field exists and has the expected type, the correct
8837 type name is returned, so the guard clause test succeeds and it is safe
8838 to use the index function. If the field does not exist, you would not
8839 get back the expected type of the field, therefore you would not
8840 attempt to index the field.
8841 JavaScript considers a result to be false if one of the following val‐
8843 ues is tested:
8844 · ‘undefined’
8846 · null
8848 · The number +0
8850 · The number -0
8852 · NaN (not a number)
8854 · “” (the empty string)
8856 Using a guard clause to check whether the required data field exists,
8858 and holds a number, before an attempt to index:
8859 if (typeof(doc.min_length) === 'number') {
8861 index("min_length", doc.min_length, {"store": true});
8862 }
8863 Use a generic guard clause test to ensure that the type of the candi‐
8865 date data field is defined.
8866 Example of a ‘generic’ guard clause:
8868 if (typeof(doc.min_length) !== 'undefined') {
8870 // The field exists, and does have a type, so we can proceed to index using it.
8871 ...
8872 }
8873 Analyzers
8875 Analyzers are settings that define how to recognize terms within text.
8876 Analyzers can be helpful if you need to index multiple languages.
8877 Here’s the list of generic analyzers, and their descriptions, that are
8879 supported by search:
8880 · classic - The standard Lucene analyzer, circa release 3.1.
8882 · email - Like the standard analyzer, but tries harder to match an
8884 email address as a complete token.
8885 · keyword - Input is not tokenized at all.
8887 · simple - Divides text at non-letters.
8889 · standard - The default analyzer. It implements the Word Break rules
8891 from the Unicode Text Segmentation algorithm
8892 · whitespace - Divides text at white space boundaries.
8894 Example analyzer document:
8896 {
8898 "_id": "_design/analyzer_example",
8899 "indexes": {
8900 "INDEX_NAME": {
8901 "index": "function (doc) { ... }",
8902 "analyzer": "$ANALYZER_NAME"
8903 }
8904 }
8905 }
8906 Language-specific analyzers
8908 These analyzers omit common words in the specific language, and many
8909 also remove prefixes and suffixes. The name of the language is also
8910 the name of the analyzer. See package org.apache.lucene.analysis for
8911 more information.
8912 ┌───────────┬───────────────────────────┐
8914 │Language │ Analyzer │
8915 ├───────────┼───────────────────────────┤
8916 │arabic │ org.apache.lucene.analy‐ │
8917 │ │ sis.ar.ArabicAnalyzer │
8918 ├───────────┼───────────────────────────┤
8919 │armenian │ org.apache.lucene.analy‐ │
8920 │ │ sis.hy.ArmenianAnalyzer │
8921 ├───────────┼───────────────────────────┤
8922 │basque │ org.apache.lucene.analy‐ │
8923 │ │ sis.eu.BasqueAnalyzer │
8924 ├───────────┼───────────────────────────┤
8925 │bulgarian │ org.apache.lucene.analy‐ │
8926 │ │ sis.bg.BulgarianAnalyzer │
8927 ├───────────┼───────────────────────────┤
8928 │brazilian │ org.apache.lucene.analy‐ │
8929 │ │ sis.br.BrazilianAnalyzer │
8930 ├───────────┼───────────────────────────┤
8931 │catalan │ org.apache.lucene.analy‐ │
8932 │ │ sis.ca.CatalanAnalyzer │
8933 ├───────────┼───────────────────────────┤
8934 │cjk │ org.apache.lucene.analy‐ │
8935 │ │ sis.cjk.CJKAnalyzer │
8936 ├───────────┼───────────────────────────┤
8937 │chinese │ org.apache.lucene.analy‐ │
8938 │ │ sis.cn.smart.SmartChine‐ │
8939 │ │ seAnalyzer │
8940 ├───────────┼───────────────────────────┤
8941 │czech │ org.apache.lucene.analy‐ │
8942 │ │ sis.cz.CzechAnalyzer │
8943 ├───────────┼───────────────────────────┤
8944 │danish │ org.apache.lucene.analy‐ │
8945 │ │ sis.da.DanishAnalyzer │
8946 ├───────────┼───────────────────────────┤
8947 │dutch │ org.apache.lucene.analy‐ │
8948 │ │ sis.nl.DutchAnalyzer │
8949 ├───────────┼───────────────────────────┤
8950 │english │ org.apache.lucene.analy‐ │
8951 │ │ sis.en.EnglishAnalyzer │
8952 ├───────────┼───────────────────────────┤
8953 │finnish │ org.apache.lucene.analy‐ │
8954 │ │ sis.fi.FinnishAnalyzer │
8955 ├───────────┼───────────────────────────┤
8956 │french │ org.apache.lucene.analy‐ │
8957 │ │ sis.fr.FrenchAnalyzer │
8958 ├───────────┼───────────────────────────┤
8959 │german │ org.apache.lucene.analy‐ │
8960 │ │ sis.de.GermanAnalyzer │
8961 ├───────────┼───────────────────────────┤
8962 │greek │ org.apache.lucene.analy‐ │
8963 │ │ sis.el.GreekAnalyzer │
8964 ├───────────┼───────────────────────────┤
8965 │galician │ org.apache.lucene.analy‐ │
8966 │ │ sis.gl.GalicianAnalyzer │
8967 ├───────────┼───────────────────────────┤
8968 │hindi │ org.apache.lucene.analy‐ │
8969 │ │ sis.hi.HindiAnalyzer │
8970 ├───────────┼───────────────────────────┤
8971 │hungarian │ org.apache.lucene.analy‐ │
8972 │ │ sis.hu.HungarianAnalyzer │
8973 ├───────────┼───────────────────────────┤
8974 │indonesian │ org.apache.lucene.analy‐ │
8975 │ │ sis.id.IndonesianAnalyzer │
8976 ├───────────┼───────────────────────────┤
8977 │irish │ org.apache.lucene.analy‐ │
8978 │ │ sis.ga.IrishAnalyzer │
8979 └───────────┴───────────────────────────┘
8980 │italian │ org.apache.lucene.analy‐ │
8983 │ │ sis.it.ItalianAnalyzer │
8984 ├───────────┼───────────────────────────┤
8985 │japanese │ org.apache.lucene.analy‐ │
8986 │ │ sis.ja.JapaneseAnalyzer │
8987 ├───────────┼───────────────────────────┤
8988 │japanese │ org.apache.lucene.analy‐ │
8989 │ │ sis.ja.JapaneseTokenizer │
8990 ├───────────┼───────────────────────────┤
8991 │latvian │ org.apache.lucene.analy‐ │
8992 │ │ sis.lv.LatvianAnalyzer │
8993 ├───────────┼───────────────────────────┤
8994 │norwegian │ org.apache.lucene.analy‐ │
8995 │ │ sis.no.NorwegianAnalyzer │
8996 ├───────────┼───────────────────────────┤
8997 │persian │ org.apache.lucene.analy‐ │
8998 │ │ sis.fa.PersianAnalyzer │
8999 ├───────────┼───────────────────────────┤
9000 │polish │ org.apache.lucene.analy‐ │
9001 │ │ sis.pl.PolishAnalyzer │
9002 ├───────────┼───────────────────────────┤
9003 │portuguese │ org.apache.lucene.analy‐ │
9004 │ │ sis.pt.PortugueseAnalyzer │
9005 ├───────────┼───────────────────────────┤
9006 │romanian │ org.apache.lucene.analy‐ │
9007 │ │ sis.ro.RomanianAnalyzer │
9008 ├───────────┼───────────────────────────┤
9009 │russian │ org.apache.lucene.analy‐ │
9010 │ │ sis.ru.RussianAnalyzer │
9011 ├───────────┼───────────────────────────┤
9012 │spanish │ org.apache.lucene.analy‐ │
9013 │ │ sis.es.SpanishAnalyzer │
9014 ├───────────┼───────────────────────────┤
9015 │swedish │ org.apache.lucene.analy‐ │
9016 │ │ sis.sv.SwedishAnalyzer │
9017 ├───────────┼───────────────────────────┤
9018 │thai │ org.apache.lucene.analy‐ │
9019 │ │ sis.th.ThaiAnalyzer │
9020 ├───────────┼───────────────────────────┤
9021 │turkish │ org.apache.lucene.analy‐ │
9022 │ │ sis.tr.TurkishAnalyzer │
9023 └───────────┴───────────────────────────┘
9024 NOTE:
9026 The japanese analyzer, org.apache.lucene.analysis.ja.JapaneseTok‐
9027 enizer, includes DEFAULT_MODE and defaultStopTags.
9028 NOTE:
9030 Language-specific analyzers are optimized for the specified lan‐
9031 guage. You cannot combine a generic analyzer with a language-spe‐
9032 cific analyzer. Instead, you might use a per field analyzer to
9033 select different analyzers for different fields within the docu‐
9034 ments.
9035 Per-field analyzers
9037 The perfield analyzer configures multiple analyzers for different
9038 fields.
9039 Example of defining different analyzers for different fields:
9041 {
9043 "_id": "_design/analyzer_example",
9044 "indexes": {
9045 "INDEX_NAME": {
9046 "analyzer": {
9047 "name": "perfield",
9048 "default": "english",
9049 "fields": {
9050 "spanish": "spanish",
9051 "german": "german"
9052 }
9053 },
9054 "index": "function (doc) { ... }"
9055 }
9056 }
9057 }
9058 Stop words
9060 Stop words are words that do not get indexed. You define them within a
9061 design document by turning the analyzer string into an object.
9062 NOTE:
9064 The keyword, simple, and whitespace analyzers do not support stop
9065 words.
9066 The default stop words for the standard analyzer are included below:
9068 "a", "an", "and", "are", "as", "at", "be", "but", "by", "for", "if",
9070 "in", "into", "is", "it", "no", "not", "of", "on", "or", "such",
9071 "that", "the", "their", "then", "there", "these", "they", "this",
9072 "to", "was", "will", "with"
9073 Example of defining non-indexed (‘stop’) words:
9075 {
9077 "_id": "_design/stop_words_example",
9078 "indexes": {
9079 "INDEX_NAME": {
9080 "analyzer": {
9081 "name": "portuguese",
9082 "stopwords": [
9083 "foo",
9084 "bar",
9085 "baz"
9086 ]
9087 },
9088 "index": "function (doc) { ... }"
9089 }
9090 }
9091 }
9092 Testing analyzer tokenization
9094 You can test the results of analyzer tokenization by posting sample
9095 data to the _search_analyze endpoint.
9096 Example of using HTTP to test the keyword analyzer:
9098 POST /_search_analyze HTTP/1.1
9100 Content-Type: application/json
9101 {"analyzer":"keyword", "text":"ablanks@renovations.com"}
9102 Example of using the command line to test the keyword analyzer:
9104 curl 'https://$HOST:5984/_search_analyze' -H 'Content-Type: application/json'
9106 -d '{"analyzer":"keyword", "text":"ablanks@renovations.com"}'
9107 Result of testing the keyword analyzer:
9109 {
9111 "tokens": [
9112 "ablanks@renovations.com"
9113 ]
9114 }
9115 Example of using HTTP to test the standard analyzer:
9117 POST /_search_analyze HTTP/1.1
9119 Content-Type: application/json
9120 {"analyzer":"standard", "text":"ablanks@renovations.com"}
9121 Example of using the command line to test the standard analyzer:
9123 curl 'https://$HOST:5984/_search_analyze' -H 'Content-Type: application/json'
9125 -d '{"analyzer":"standard", "text":"ablanks@renovations.com"}'
9126 Result of testing the standard analyzer:
9128 {
9130 "tokens": [
9131 "ablanks",
9132 "renovations.com"
9133 ]
9134 }
9135 Queries
9137 After you create a search index, you can query it.
9138 · Issue a partition query using: GET /$DATABASE/_partition/$PARTI‐
9140 TION_KEY/_design/$DDOC/_search/$INDEX_NAME
9141 · Issue a global query using: GET /$DATA‐
9143 BASE/_design/$DDOC/_search/$INDEX_NAME
9144 Specify your search by using the query parameter.
9146 Example of using HTTP to query a partitioned index:
9148 GET /$DATABASE/_partition/$PARTITION_KEY/_design/$DDOC/_search/$INDEX_NAME?include_docs=true&query="*:*"&limit=1 HTTP/1.1
9150 Content-Type: application/json
9151 Example of using HTTP to query a global index:
9153 GET /$DATABASE/_design/$DDOC/_search/$INDEX_NAME?include_docs=true&query="*:*"&limit=1 HTTP/1.1
9155 Content-Type: application/json
9156 Example of using the command line to query a partitioned index:
9158 curl https://$HOST:5984/$DATABASE/_partition/$PARTITION_KEY/_design/$DDOC/
9160 _search/$INDEX_NAME?include_docs=true\&query="*:*"\&limit=1 \
9161 Example of using the command line to query a global index:
9163 curl https://$HOST:5984/$DATABASE/_design/$DDOC/_search/$INDEX_NAME?
9165 include_docs=true\&query="*:*"\&limit=1 \
9166 Query Parameters
9168 A full list of query parameters can be found in the API Reference.
9169 You must enable faceting before you can use the following parameters:
9171 · counts
9173 · drilldown
9175 · ranges
9177 NOTE:
9179 Do not combine the bookmark and stale options. These options con‐
9180 strain the choice of shard replicas to use for the response. When
9181 used together, the options might cause problems when contact is
9182 attempted with replicas that are slow or not available.
9183 Relevance
9185 When more than one result might be returned, it is possible for them to
9186 be sorted. By default, the sorting order is determined by ‘relevance’.
9187 Relevance is measured according to Apache Lucene Scoring. As an exam‐
9189 ple, if you search a simple database for the word example, two docu‐
9190 ments might contain the word. If one document mentions the word example
9191 10 times, but the second document mentions it only twice, then the
9192 first document is considered to be more ‘relevant’.
9193 If you do not provide a sort parameter, relevance is used by default.
9195 The highest scoring matches are returned first.
9196 If you provide a sort parameter, then matches are returned in that
9198 order, ignoring relevance.
9199 If you want to use a sort parameter, and also include ordering by rele‐
9201 vance in your search results, use the special fields -<score> or
9202 <score> within the sort parameter.
9203 POSTing search queries
9205 Instead of using the GET HTTP method, you can also use POST. The main
9206 advantage of POST queries is that they can have a request body, so you
9207 can specify the request as a JSON object. Each parameter in the query
9208 string of a GET request corresponds to a field in the JSON object in
9209 the request body.
9210 Example of using HTTP to POST a search request:
9212 POST /db/_design/ddoc/_search/searchname HTTP/1.1
9214 Content-Type: application/json
9215 Example of using the command line to POST a search request:
9217 curl 'https://$HOST:5984/db/_design/ddoc/_search/searchname' -X POST -H 'Content-Type: application/json' -d @search.json
9219 Example JSON document that contains a search request:
9221 {
9223 "q": "index:my query",
9224 "sort": "foo",
9225 "limit": 3
9226 }
9227 Query syntax
9229 The CouchDB search query syntax is based on the Lucene syntax. Search
9230 queries take the form of name:value unless the name is omitted, in
9231 which case they use the default field, as demonstrated in the following
9232 examples:
9233 Example search query expressions:
9235 // Birds
9237 class:bird
9238 // Animals that begin with the letter "l"
9240 l*
9241 // Carnivorous birds
9243 class:bird AND diet:carnivore
9244 // Herbivores that start with letter "l"
9246 l* AND diet:herbivore
9247 // Medium-sized herbivores
9249 min_length:[1 TO 3] AND diet:herbivore
9250 // Herbivores that are 2m long or less
9252 diet:herbivore AND min_length:[-Infinity TO 2]
9253 // Mammals that are at least 1.5m long
9255 class:mammal AND min_length:[1.5 TO Infinity]
9256 // Find "Meles meles"
9258 latin_name:"Meles meles"
9259 // Mammals who are herbivore or carnivore
9261 diet:(herbivore OR omnivore) AND class:mammal
9262 // Return all results
9264 *:*
9265 Queries over multiple fields can be logically combined, and groups and
9267 fields can be further grouped. The available logical operators are
9268 case-sensitive and are AND, +, OR, NOT and -. Range queries can run
9269 over strings or numbers.
9270 If you want a fuzzy search, you can run a query with ~ to find terms
9272 like the search term. For instance, look~ finds the terms book and
9273 took.
9274 NOTE:
9276 If the lower and upper bounds of a range query are both strings that
9277 contain only numeric digits, the bounds are treated as numbers not
9278 as strings. For example, if you search by using the query
9279 mod_date:["20170101" TO "20171231"], the results include documents
9280 for which mod_date is between the numeric values 20170101 and
9281 20171231, not between the strings “20170101” and “20171231”.
9282 You can alter the importance of a search term by adding ^ and a posi‐
9284 tive number. This alteration makes matches containing the term more or
9285 less relevant, proportional to the power of the boost value. The
9286 default value is 1, which means no increase or decrease in the strength
9287 of the match. A decimal value of 0 - 1 reduces importance. making the
9288 match strength weaker. A value greater than one increases importance,
9289 making the match strength stronger.
9290 Wildcard searches are supported, for both single (?) and multiple (*)
9292 character searches. For example, dat? would match date and data,
9293 whereas dat* would match date, data, database, and dates. Wildcards
9294 must come after the search term.
9295 Use *:* to return all results.
9297 If the search query does not specify the "group_field" argument, the
9299 response contains a bookmark. If this bookmark is later provided as a
9300 URL parameter, the response skips the rows that were seen already, mak‐
9301 ing it quick and easy to get the next set of results.
9302 NOTE:
9304 The response never includes a bookmark if the "group_field" parame‐
9305 ter is included in the search query. See group_field parameter.
9306 NOTE:
9308 The group_field, group_limit, and group_sort options are only avail‐
9309 able when making global queries.
9310 The following characters require escaping if you want to search on
9312 them:
9313 + - && || ! ( ) { } [ ] ^ " ~ * ? : \ /
9315 To escape one of these characters, use a preceding backslash character
9317 (\).
9318 The response to a search query contains an order field for each of the
9320 results. The order field is an array where the first element is the
9321 field or fields that are specified in the sort parameter. See the sort
9322 parameter. If no sort parameter is included in the query, then the
9323 order field contains the Lucene relevance score. If you use the ‘sort
9324 by distance’ feature as described in geographical searches, then the
9325 first element is the distance from a point. The distance is measured by
9326 using either kilometers or miles.
9327 NOTE:
9329 The second element in the order array can be ignored. It is used
9330 for troubleshooting purposes only.
9331 Faceting
9333 CouchDB Search also supports faceted searching, enabling discovery of
9334 aggregate information about matches quickly and easily. You can match
9335 all documents by using the special ?q=*:* query syntax, and use the
9336 returned facets to refine your query. To indicate that a field must be
9337 indexed for faceted queries, set {"facet": true} in its options.
9338 Example of search query, specifying that faceted search is enabled:
9340 function(doc) {
9342 index("type", doc.type, {"facet": true});
9343 index("price", doc.price, {"facet": true});
9344 }
9345 To use facets, all the documents in the index must include all the
9347 fields that have faceting enabled. If your documents do not include all
9348 the fields, you receive a bad_request error with the following reason,
9349 “The field_name does not exist.” If each document does not contain all
9350 the fields for facets, create separate indexes for each field. If you
9351 do not create separate indexes for each field, you must include only
9352 documents that contain all the fields. Verify that the fields exist in
9353 each document by using a single if statement.
9354 Example if statement to verify that the required fields exist in each
9356 document:
9357 if (typeof doc.town == "string" && typeof doc.name == "string") {
9359 index("town", doc.town, {facet: true});
9360 index("name", doc.name, {facet: true});
9361 }
9362 Counts
9364 NOTE:
9365 The counts option is only available when making global queries.
9366 The counts facet syntax takes a list of fields, and returns the number
9368 of query results for each unique value of each named field.
9369 NOTE:
9371 The count operation works only if the indexed values are strings.
9372 The indexed values cannot be mixed types. For example, if 100
9373 strings are indexed, and one number, then the index cannot be used
9374 for count operations. You can check the type by using the typeof
9375 operator, and convert it by using the parseInt, parseFloat, or
9376 .toString() functions.
9377 Example of a query using the counts facet syntax:
9379 ?q=*:*&counts=["type"]
9381 Example response after using of the counts facet syntax:
9383 {
9385 "total_rows":100000,
9386 "bookmark":"g...",
9387 "rows":[...],
9388 "counts":{
9389 "type":{
9390 "sofa": 10,
9391 "chair": 100,
9392 "lamp": 97
9393 }
9394 }
9395 }
9396 Drilldown
9398 NOTE:
9399 The drilldown option is only available when making global queries.
9400 You can restrict results to documents with a dimension equal to the
9402 specified label. Restrict the results by adding drilldown=["dimen‐
9403 sion","label"] to a search query. You can include multiple drilldown
9404 parameters to restrict results along multiple dimensions.
9405 GET /things/_design/inventory/_search/fruits?q=*:*&drilldown=["state","old"]&drilldown=["item","apple"]&include_docs=true HTTP/1.1
9407 For better language interoperability, you can achieve the same by sup‐
9409 plying a list of lists:
9410 GET /things/_design/inventory/_search/fruits?q=*:*&drilldown=[["state","old"],["item","apple"]]&include_docs=true HTTP/1.1
9412 You can also supply a list of lists for drilldown in bodies of POST
9414 requests.
9415 Note that, multiple values for a single key in a drilldown means an OR
9417 relation between them and there is an AND relation between multiple
9418 keys.
9419 Using a drilldown parameter is similar to using key:value in the q
9421 parameter, but the drilldown parameter returns values that the analyzer
9422 might skip.
9423 For example, if the analyzer did not index a stop word like "a", using
9425 drilldown returns it when you specify drilldown=["key","a"].
9426 Ranges
9428 NOTE:
9429 The ranges option is only available when making global queries.
9430 The range facet syntax reuses the standard Lucene syntax for ranges to
9432 return counts of results that fit into each specified category. Inclu‐
9433 sive range queries are denoted by brackets ([, ]). Exclusive range
9434 queries are denoted by curly brackets ({, }).
9435 NOTE:
9437 The range operation works only if the indexed values are numbers.
9438 The indexed values cannot be mixed types. For example, if 100
9439 strings are indexed, and one number, then the index cannot be used
9440 for range operations. You can check the type by using the typeof
9441 operator, and convert it by using the parseInt, parseFloat, or
9442 .toString() functions.
9443 Example of a request that uses faceted search for matching ranges:
9445 ?q=*:*&ranges={"price":{"cheap":"[0 TO 100]","expensive":"{100 TO Infinity}"}}
9447 Example results after a ranges check on a faceted search:
9449 {
9451 "total_rows":100000,
9452 "bookmark":"g...",
9453 "rows":[...],
9454 "ranges": {
9455 "price": {
9456 "expensive": 278682,
9457 "cheap": 257023
9458 }
9459 }
9460 }
9461 Geographical searches
9463 In addition to searching by the content of textual fields, you can also
9464 sort your results by their distance from a geographic coordinate using
9465 Lucene’s built-in geospatial capabilities.
9466 To sort your results in this way, you must index two numeric fields,
9468 representing the longitude and latitude.
9469 NOTE:
9471 You can also sort your results by their distance from a geographic
9472 coordinate using Lucene’s built-in geospatial capabilities.
9473 You can then query by using the special <distance...> sort field, which
9475 takes five parameters:
9476 · Longitude field name: The name of your longitude field (mylon in the
9478 example).
9479 · Latitude field name: The name of your latitude field (mylat in the
9481 example).
9482 · Longitude of origin: The longitude of the place you want to sort by
9484 distance from.
9485 · Latitude of origin: The latitude of the place you want to sort by
9487 distance from.
9488 · Units: The units to use: km for kilometers or mi for miles. The dis‐
9490 tance is returned in the order field.
9491 You can combine sorting by distance with any other search query, such
9493 as range searches on the latitude and longitude, or queries that
9494 involve non-geographical information.
9495 That way, you can search in a bounding box, and narrow down the search
9497 with extra criteria.
9498 Example geographical data:
9500 {
9502 "name":"Aberdeen, Scotland",
9503 "lat":57.15,
9504 "lon":-2.15,
9505 "type":"city"
9506 }
9507 Example of a design document that contains a search index for the geo‐
9509 graphic data:
9510 function(doc) {
9512 if (doc.type && doc.type == 'city') {
9513 index('city', doc.name, {'store': true});
9514 index('lat', doc.lat, {'store': true});
9515 index('lon', doc.lon, {'store': true});
9516 }
9517 }
9518 An example of using HTTP for a query that sorts cities in the northern
9520 hemisphere by their distance to New York:
9521 GET /examples/_design/cities-designdoc/_search/cities?q=lat:[0+TO+90]&sort="<distance,lon,lat,-74.0059,40.7127,km>" HTTP/1.1
9523 An example of using the command line for a query that sorts cities in
9525 the northern hemisphere by their distance to New York:
9526 curl 'https://$HOST:5984/examples/_design/cities-designdoc/_search/cities?q=lat:[0+TO+90]&sort="<distance,lon,lat,-74.0059,40.7127,km>"'
9528 Example (abbreviated) response, containing a list of northern hemi‐
9530 sphere cities sorted by distance to New York:
9531 {
9533 "total_rows": 205,
9534 "bookmark": "g1A...XIU",
9535 "rows": [
9536 {
9537 "id": "city180",
9538 "order": [
9539 8.530665755719783,
9540 18
9541 ],
9542 "fields": {
9543 "city": "New York, N.Y.",
9544 "lat": 40.78333333333333,
9545 "lon": -73.96666666666667
9546 }
9547 },
9548 {
9549 "id": "city177",
9550 "order": [
9551 13.756343205985946,
9552 17
9553 ],
9554 "fields": {
9555 "city": "Newark, N.J.",
9556 "lat": 40.733333333333334,
9557 "lon": -74.16666666666667
9558 }
9559 },
9560 {
9561 "id": "city178",
9562 "order": [
9563 113.53603438866077,
9564 26
9565 ],
9566 "fields": {
9567 "city": "New Haven, Conn.",
9568 "lat": 41.31666666666667,
9569 "lon": -72.91666666666667
9570 }
9571 }
9572 ]
9573 }
9574 Highlighting search terms
9576 Sometimes it is useful to get the context in which a search term was
9577 mentioned so that you can display more emphasized results to a user.
9578 To get more emphasized results, add the highlight_fields parameter to
9580 the search query. Specify the field names for which you would like
9581 excerpts, with the highlighted search term returned.
9582 By default, the search term is placed in <em> tags to highlight it, but
9584 the highlight can be overridden by using the highlights_pre_tag and
9585 highlights_post_tag parameters.
9586 The length of the fragments is 100 characters by default. A different
9588 length can be requested with the highlights_size parameter.
9589 The highlights_number parameter controls the number of fragments that
9591 are returned, and defaults to 1.
9592 In the response, a highlights field is added, with one subfield per
9594 field name.
9595 For each field, you receive an array of fragments with the search term
9597 highlighted.
9598 NOTE:
9600 For highlighting to work, store the field in the index by using the
9601 store: true option.
9602 Example of using HTTP to search with highlighting enabled:
9604 GET /movies/_design/searches/_search/movies?q=movie_name:Azazel&highlight_fields=["movie_name"]&highlight_pre_tag="**"&highlight_post_tag="**"&highlights_size=30&highlights_number=2 HTTP/1.1
9606 Authorization: ...
9607 Example of using the command line to search with highlighting enabled:
9609 curl "https://$HOST:5984/movies/_design/searches/_search/movies?q=movie_name:Azazel&highlight_fields=\[\"movie_name\"\]&highlight_pre_tag=\"**\"&highlight_post_tag=\"**\"&highlights_size=30&highlights_number=2
9611 Example of highlighted search results:
9613 {
9615 "highlights": {
9616 "movie_name": [
9617 " on the Azazel Orient Express",
9618 " Azazel manuals, you"
9619 ]
9620 }
9621 }
9622 Note: Previously, the functionality provided by CouchDB’s design docu‐
9624 ments, in combination with document attachments, was referred to as
9625 “CouchApps.” The general principle was that entire web applications
9626 could be hosted in CouchDB, without need for an additional application
9627 server.
9628 Use of CouchDB as a combined standalone database and application server
9630 is no longer recommended. There are significant limitations to a pure
9631 CouchDB web server application stack, including but not limited to:
9632 fully-fledged fine-grained security, robust templating and scaffolding,
9633 complete developer tooling, and most importantly, a thriving ecosystem
9634 of developers, modules and frameworks to choose from.
9635 The developers of CouchDB believe that web developers should pick “the
9637 right tool for the right job”. Use CouchDB as your database layer, in
9638 conjunction with any number of other server-side web application frame‐
9639 works, such as the entire Node.JS ecosystem, Python’s Django and Flask,
9640 PHP’s Drupal, Java’s Apache Struts, and more.
9641
9643 In this chapter, we present some of the best ways to use Apache
9644 CouchDB. These usage patterns reflect many years of real-world use. We
9645 hope that these will jump-start your next project, or improve the per‐
9646 formance of your current system.
9647 Document Design Considerations
9649 When designing your database, and your document structure, there are a
9650 number of best practices to take into consideration. Especially for
9651 people accustomed to relational databases, some of these techniques may
9652 be non-obvious.
9653 Don’t rely on CouchDB’s auto-UUID generation
9655 While CouchDB will generate a unique identifier for the _id field of
9656 any doc that you create, in most cases you are better off generating
9657 them yourself for a few reasons:
9658 · If for any reason you miss the 200 OK reply from CouchDB, and storing
9660 the document is attempted again, you would end up with the same docu‐
9661 ment content stored under multiple _ids. This could easily happen
9662 with intermediary proxies and cache systems that may not inform
9663 developers that the failed transaction is being retried.
9664 · _ids are the only unique enforced value within CouchDB so you might
9666 as well make use of this. CouchDB stores its documents in a B+ tree.
9667 Each additional or updated document is stored as a leaf node, and may
9668 require re-writing intermediary and parent nodes. You may be able to
9669 take advantage of sequencing your own ids more effectively than the
9670 automatically generated ids if you can arrange them to be sequential
9671 yourself.
9672 Alternatives to auto-incrementing sequences
9674 Because of replication, as well as the distributed nature of CouchDB,
9675 it is not practical to use auto-incrementing sequences with CouchDB.
9676 These are often used to ensure unique identifiers for each row in a
9677 database table. CouchDB generates unique ids on its own and you can
9678 specify your own as well, so you don’t really need a sequence here. If
9679 you use a sequence for something else, you will be better off finding
9680 another way to express it in CouchDB in another way.
9681 Pre-aggregating your data
9683 If your intent for CouchDB is as a collect-and-report model, not a
9684 real-time view, you may not need to store a single document for every
9685 event you’re recording. In this case, pre-aggregating your data may be
9686 a good idea. You probably don’t need 1000 documents per second if all
9687 you are trying to do is to track summary statistics about those docu‐
9688 ments. This reduces the computational pressure on CouchDB’s MapReduce
9689 engine(s), as well as reduces its storage requirements.
9690 In this case, using an in-memory store to summarize your statistical
9692 information, then writing out to CouchDB every 10 seconds / 1 minute /
9693 whatever level of granularity you need would greatly reduce the number
9694 of documents you’ll put in your database.
9695 Later, you can then further decimate your data by walking the entire
9697 database and generating documents to be stored in a new database with a
9698 lower level of granularity (say, 1 document a day). You can then delete
9699 the older, more fine-grained database when you’re done with it.
9700 Designing an application to work with replication
9702 Whilst CouchDB includes replication and a conflict-flagging mechanism,
9703 this is not the whole story for building an application which repli‐
9704 cates in a way which users expect.
9705 Here we consider a simple example of a bookmarks application. The idea
9707 is that a user can replicate their own bookmarks, work with them on
9708 another machine, and then synchronise their changes later.
9709 Let’s start with a very simple definition of bookmarks: an ordered,
9711 nestable mapping of name to URL. Internally the application might rep‐
9712 resent it like this:
9713 [
9715 {"name":"Weather", "url":"http://www.bbc.co.uk/weather"},
9716 {"name":"News", "url":"http://news.bbc.co.uk/"},
9717 {"name":"Tech", "bookmarks": [
9718 {"name":"Register", "url":"http://www.theregister.co.uk/"},
9719 {"name":"CouchDB", "url":"http://couchdb.apache.org/"}
9720 ]}
9721 ]
9722 It can then present the bookmarks menu and sub-menus by traversing this
9724 structure.
9725 Now consider this scenario: the user has a set of bookmarks on her PC,
9727 and then replicates it to her laptop. On the laptop, she changes the
9728 News link to point to CNN, renames “Register” to “The Register”, and
9729 adds a new link to slashdot just after it. On the desktop, her husband
9730 deletes the Weather link, and adds a new link to CNET in the Tech
9731 folder.
9732 So after these changes, the laptop has:
9734 [
9736 {"name":"Weather", "url":"http://www.bbc.co.uk/weather"},
9737 {"name":"News", "url":"http://www.cnn.com/"},
9738 {"name":"Tech", "bookmarks": [
9739 {"name":"The Register", "url":"http://www.theregister.co.uk/"},
9740 {"name":"Slashdot", "url":"http://www.slashdot.new/"},
9741 {"name":"CouchDB", "url":"http://couchdb.apache.org/"}
9742 ]}
9743 ]
9744 and the PC has:
9746 [
9748 {"name":"News", "url":"http://www.cnn.com/"},
9749 {"name":"Tech", "bookmarks": [
9750 {"name":"Register", "url":"http://www.theregister.co.uk/"},
9751 {"name":"CouchDB", "url":"http://couchdb.apache.org/"},
9752 {"name":"CNET", "url":"http://news.cnet.com/"}
9753 ]}
9754 ]
9755 Upon the next synchronisation, we want the expected merge to take
9757 place. That is: links which were changed, added or deleted on one side
9758 are also changed, added or deleted on the other side - with no human
9759 intervention required unless absolutely necessary.
9760 We will also assume that both sides are doing a CouchDB “compact” oper‐
9762 ation periodically, and are disconnected for more than this time before
9763 they resynchronise.
9764 All of the approaches below which allow automated merging of changes
9766 rely on having some sort of history, back to the point where the repli‐
9767 cas diverged.
9768 CouchDB does not provide a mechanism for this itself. It stores arbi‐
9770 trary numbers of old _ids for one document (trunk now has a mechanism
9771 for pruning the _id history), for the purposes of replication. However
9772 it will not keep the documents themselves through a compaction cycle,
9773 except where there are conflicting versions of a document.
9774 Do not rely on the CouchDB revision history mechanism to help you build
9776 an application-level version history. Its sole purpose is to ensure
9777 eventually consistent replication between databases. It is up to you to
9778 maintain history explicitly in whatever form makes sense for your
9779 application, and to prune it to avoid excessive storage utilisation,
9780 whilst not pruning past the point where live replicas last diverged.
9781 Approach 1: Single JSON doc
9783 The above structure is already valid JSON, and so could be represented
9784 in CouchDB just by wrapping it in an object and storing as a single
9785 document:
9786 {
9788 "bookmarks":
9789 // ... same as above
9790 }
9791 This makes life very easy for the application, as the ordering and
9793 nesting is all taken care of. The trouble here is that on replication,
9794 only two sets of bookmarks will be visible: example B and example C.
9795 One will be chosen as the main revision, and the other will be stored
9796 as a conflicting revision.
9797 At this point, the semantics are very unsatisfactory from the user’s
9799 point of view. The best that can be offered is a choice saying “Which
9800 of these two sets of bookmarks do you wish to keep: B or C?” However
9801 neither represents the desired outcome. There is also insufficient data
9802 to be able to correctly merge them, since the base revision A is lost.
9803 This is going to be highly unsatisfactory for the user, who will have
9805 to apply one set of changes again manually.
9806 Approach 2: Separate document per bookmark
9808 An alternative solution is to make each field (bookmark) a separate
9809 document in its own right. Adding or deleting a bookmark is then just a
9810 case of adding or deleting a document, which will never conflict
9811 (although if the same bookmark is added on both sides, then you will
9812 end up with two copies of it). Changing a bookmark will only conflict
9813 if both sides made changes to the same one, and then it is reasonable
9814 to ask the user to choose between them.
9815 Since there will now be lots of small documents, you may either wish to
9817 keep a completely separate database for bookmarks, or else add an
9818 attribute to distinguish bookmarks from other kinds of document in the
9819 database. In the latter case, a view can be made to return only book‐
9820 mark documents.
9821 Whilst replication is now fixed, care is needed with the “ordered” and
9823 “nestable” properties of bookmarks.
9824 For ordering, one suggestion is to give each item a floating-point
9826 index, and then when inserting an object between A and B, give it an
9827 index which is the average of A and B’s indices. Unfortunately, this
9828 will fail after a while when you run out of precision, and the user
9829 will be bemused to find that their most recent bookmarks no longer
9830 remember the exact position they were put in.
9831 A better way is to keep a string representation of index, which can
9833 grow as the tree is subdivided. This will not suffer the above problem,
9834 but it may result in this string becoming arbitrarily long after time.
9835 They could be renumbered, but the renumbering operation could introduce
9836 a lot of conflicts, especially if attempted by both sides indepen‐
9837 dently.
9838 For “nestable”, you can have a separate doc which represents a list of
9840 bookmarks, and each bookmark can have a “belongs to” field which iden‐
9841 tifies the list. It may be useful anyway to be able to have multiple
9842 top-level bookmark sets (Bob’s bookmarks, Jill’s bookmarks etc). Some
9843 care is needed when deleting a list or sub-list, to ensure that all
9844 associated bookmarks are also deleted, otherwise they will become
9845 orphaned.
9846 Building the entire bookmark set can be performed through the use of
9848 emitting a compound key that describes the path to the document, then
9849 using group levels to retrieve the position of the tree in the docu‐
9850 ment. The following code excerpt describes a tree of files, where the
9851 path to the file is stored in the document under the "path" key:
9852 // map function
9854 function(doc) {
9855 if (doc.type === "file") {
9856 if (doc.path.substr(-1) === "/") {
9857 var raw_path = doc.path.slice(0, -1);
9858 } else {
9859 var raw_path = doc.path;
9860 }
9861 emit (raw_path.split('/'), 1);
9862 }
9863 }
9864 // reduce
9866 _sum
9867 This will emit rows into the view of the form ["opt", "couchdb", "etc",
9869 "local.ini"] for a doc.path of /opt/couchdb/etc/local.ini. You can then
9870 query a list of files in the /opt/couchdb/etc directory by specifying a
9871 startkey of ["opt", "couchdb", "etc"] and an endkey of ["opt",
9872 "couchdb", "etc", {}].
9873 Approach 3: Immutable history / event sourcing
9875 Another approach to consider is Event Sourcing or Command Logging, as
9876 implemented in many NoSQL databases and as used in many operational
9877 transformation systems.
9878 In this model, instead of storing individual bookmarks, you store
9880 records of changes made - “Bookmark added”, “Bookmark changed”, “Book‐
9881 mark moved”, “Bookmark deleted”. These are stored in an append-only
9882 fashion. Since records are never modified or deleted, only added to,
9883 there are never any replication conflicts.
9884 These records can also be stored as an array in a single CouchDB docu‐
9886 ment. Replication can cause a conflict, but in this case it is easy to
9887 resolve by simply combining elements from the two arrays.
9888 In order to see the full set of bookmarks, you need to start with a
9890 baseline set (initially empty) and run all the change records since the
9891 baseline was created; and/or you need to maintain a most-recent version
9892 and update it with changes not yet seen.
9893 Care is needed after replication when merging together history from
9895 multiple sources. You may get different results depending on how you
9896 order them - consider taking all A’s changes before B’s, taking all B’s
9897 before A’s, or interleaving them (e.g. if each change has a timestamp).
9898 Also, over time the amount of storage used can grow arbitrarily large,
9900 even if the set of bookmarks itself is small. This can be controlled by
9901 moving the baseline version forwards and then keeping only the changes
9902 after that point. However, care is needed not to move the baseline
9903 version forward so far that there are active replicas out there which
9904 last synchronised before that time, as this may result in conflicts
9905 which cannot be resolved automatically.
9906 If there is any uncertainty, it is best to present the user with a
9908 prompt to assist with merging the content in the application itself.
9909 Approach 4: Keep historic versions explicitly
9911 If you are going to keep a command log history, then it may be simpler
9912 just to keep old revisions of the bookmarks list itself around. The
9913 intention is to subvert CouchDB’s automatic behaviour of purging old
9914 revisions, by keeping these revisions as separate documents.
9915 You can keep a pointer to the ‘most current’ revision, and each revi‐
9917 sion can point to its predecessor. On replication, merging can take
9918 place by diffing each of the previous versions (in effect synthesising
9919 the command logs) back to a common ancestor.
9920 This is the sort of behaviour which revision control systems such as
9922 Git implement as a matter of routine, although generally comparing text
9923 files line-by-line rather than comparing JSON objects field-by-field.
9924 Systems like Git will accumulate arbitrarily large amounts of history
9926 (although they will attempt to compress it by packing multiple revi‐
9927 sions so that only their diffs are stored). With Git you can use “his‐
9928 tory rewriting” to remove old history, but this may prohibit merging if
9929 history doesn’t go back far enough in time.
9930 Adding client-side security with a translucent database
9932 Many applications do not require a thick layer of security at the
9933 server. It is possible to use a modest amount of encryption and one-way
9934 functions to obscure the sensitive columns or key-value pairs, a tech‐
9935 nique often called a translucent database. (See a description.)
9936 The simplest solutions use a one-way function like SHA-256 at the
9938 client to scramble the name and password before storing the informa‐
9939 tion. This solution gives the client control of the data in the data‐
9940 base without requiring a thick layer on the database to test each
9941 transaction. Some advantages are:
9942 · Only the client or someone with the knowledge of the name and pass‐
9944 word can compute the value of SHA256 and recover the data.
9945 · Some columns are still left in the clear, an advantage for computing
9947 aggregated statistics.
9948 · Computation of SHA256 is left to the client side computer which usu‐
9950 ally has cycles to spare.
9951 · The system prevents server-side snooping by insiders and any attacker
9953 who might penetrate the OS or any of the tools running upon it.
9954 There are limitations:
9956 · There is no root password. If the person forgets their name and pass‐
9958 word, their access is gone forever. This limits its use to databases
9959 that can continue by issuing a new user name and password.
9960 There are many variations on this theme detailed in the book
9962 Translucent Databases, including:
9963 · Adding a backdoor with public-key cryptography.
9965 · Adding a second layer with steganography.
9967 · Dealing with typographical errors.
9969 · Mixing encryption with one-way functions.
9971 Document submission using HTML Forms
9973 It is possible to write to a CouchDB document directly from an HTML
9974 form by using a document update function. Here’s how:
9975 The HTML form
9977 First, write an HTML form. Here’s a simple “Contact Us” form excerpt:
9978 <form action="/dbname/_design/ddocname/_update/contactform" method="post">
9980 <div>
9981 <label for="name">Name:</label>
9982 <input type="text" id="name" name="name" />
9983 </div>
9984 <div>
9985 <label for="mail">Email:</label>
9986 <input type="text" id="mail" name="email" />
9987 </div>
9988 <div>
9989 <label for="msg">Message:</label>
9990 <textarea id="msg" name="message"></textarea>
9991 </div>
9992 </form>
9993 Customize the /dbname/_design/ddocname/_update/contactform portion of
9995 the form action URL to reflect the exact path to your database, design
9996 document and update function (see below).
9997 As CouchDB no longer recommends the use of CouchDB-hosted web applica‐
9999 tions , you may want to use a reverse proxy to expose CouchDB as a sub‐
10000 directory of your web application. If so, add that prefix to the
10001 action destination in the form.
10002 Another option is to alter CouchDB’s CORS settings and use a
10004 cross-domain POST. Be sure you understand all security implications
10005 before doing this!
10006 The update function
10008 Then, write an update function. This is the server-side JavaScript
10009 function that will receive the POST-ed data.
10010 The first argument to the function will be the document that is being
10012 processed (if it exists). Because we are using POST and not PUT, this
10013 should be empty in our scenario - but we should check to be sure. The
10014 POST-ed data will be passed as the second parameter to the function,
10015 along with any query parameters and the full request headers.
10016 Here’s a sample handler that extracts the form data, generates a docu‐
10018 ment _id based on the email address and timestamp, and saves the docu‐
10019 ment. It then returns a JSON success response back to the browser.
10020 function(doc, req) {
10022 if (doc) {
10024 return [doc, toJSON({"error": "request already filed"})]
10025 }
10026 if !(req.form && req.form.email) {
10028 return [null, toJSON({"error": "incomplete form"})]
10029 }
10030 var date = new Date()
10032 var newdoc = req.form
10033 newdoc._id = req.form.email + "_" + date.toISOString()
10034 return [newdoc, toJSON({"success":"ok"})]
10036 }
10037 Place the above function in your design document under the updates key.
10039 Note that this function does not attempt any sort of input validation
10041 or sanitization. That is best handled by a validate document update
10042 function instead. (A “VDU” will validate any document written to the
10043 database, not just those that use your update function.)
10044 If the first element passed to return is a document, the HTTP response
10046 headers will include X-Couch-Id, the _id value for the newly created
10047 document, and X-Couch-Update-NewRev, the _rev value for the newly cre‐
10048 ated document. This is handy if your client-side code wants to access
10049 or update the document in a future call.
10050 Example output
10052 Here’s the worked sample above, using curl to simulate the form POST.
10053 $ curl -X PUT localhost:5984/testdb/_design/myddoc -d '{ "updates": { "contactform": "function(doc, req) { ... }" } }'
10055 {"ok":true,"id":"_design/myddoc","rev":"1-2a2b0951fcaf7287817573b03bba02ed"}
10056 $ curl --data "name=Lin&email=lin@example.com&message=I Love CouchDB" http://localhost:5984/testdb/_design/myddoc/_update/contactform
10058 * Trying 127.0.0.1...
10059 * TCP_NODELAY set
10060 * Connected to localhost (127.0.0.1) port 5984 (#1)
10061 > POST /testdb/_design/myddoc/_update/contactform HTTP/1.1
10062 > Host: localhost:5984
10063 > User-Agent: curl/7.59.0
10064 > Accept: */*
10065 > Content-Length: 53
10066 > Content-Type: application/x-www-form-urlencoded
10067 >
10068 * upload completely sent off: 53 out of 53 bytes
10069 < HTTP/1.1 201 Created
10070 < Content-Length: 16
10071 < Content-Type: text/html; charset=utf-8
10072 < Date: Thu, 05 Apr 2018 19:56:42 GMT
10073 < Server: CouchDB/2.2.0-948a1311c (Erlang OTP/19)
10074 < X-Couch-Id: lin%40example.com_2018-04-05T19:51:22.278Z
10075 < X-Couch-Request-ID: 03a5f4fbe0
10076 < X-Couch-Update-NewRev: 1-34483732407fcc6cfc5b60ace48b9da9
10077 < X-CouchDB-Body-Time: 0
10078 <
10079 * Connection #1 to host localhost left intact
10080 {"success":"ok"}
10081 $ curl http://localhost:5984/testdb/lin\@example.com_2018-04-05T19:51:22.278Z
10083 {"_id":"lin@example.com_2018-04-05T19:51:22.278Z","_rev":"1-34483732407fcc6cfc5b60ace48b9da9","name":"Lin","email":"lin@example.com","message":"I Love CouchDB"}
10084 Using an ISO Formatted Date for Document IDs
10086 The ISO 8601 date standard describes a useful scheme for representing a
10087 date string in a Year-Month-DayTHour:Minute:Second.microsecond format.
10088 For time-bound documents in a CouchDB database this can be a very handy
10089 way to create a unique identifier, since JavaScript can directly use it
10090 to create a Date object. Using this sample map function:
10091 function(doc) {
10093 var dt = new Date(doc._id);
10094 emit([dt.getDate(), doc.widget], 1);
10095 }
10096 simply use group_level to zoom in on whatever time you wish to use.
10098 curl -X GET "http://localhost:5984/transactions/_design/widget_count/_view/toss?group_level=1"
10100 {"rows":[
10102 {"key":[20],"value":10},
10103 {"key":[21],"value":20}
10104 ]}
10105 curl -X GET "http://localhost:5984/transactions/_design/widget_count/_view/toss?group_level=2"
10107 {"rows":[
10109 {"key":[20,widget],"value":10},
10110 {"key":[21,widget],"value":10},
10111 {"key":[21,thing],"value":10}
10112 ]}
10113 Another method is using parseint() and datetime.substr() to cut out
10115 useful values for a return key:
10116 function (doc) {
10118 var datetime = doc._id;
10119 var year = parseInt(datetime.substr(0, 4));
10120 var month = parseInt(datetime.substr(5, 2), 10);
10121 var day = parseInt(datetime.substr(8, 2), 10);
10122 var hour = parseInt(datetime.substr(11, 2), 10);
10123 var minute = parseInt(datetime.substr(14, 2), 10);
10124 emit([doc.widget, year, month, day, hour, minute], 1);
10125 }
10126 JavaScript development tips
10128 Working with Apache CouchDB’s JavaScript environment is a lot different
10129 than working with traditional JavaScript development environments. Here
10130 are some tips and tricks that will ease the difficulty.
10131 · Remember that CouchDB’s JavaScript engine is old, only supporting the
10133 ECMA-262 5th edition (“ES5”) of the language. ES6/2015 and newer con‐
10134 structs cannot be used.
10135 Fortunately, there are many tools available for transpiling modern
10137 JavaScript into code compatible with older JS engines. The Babel
10138 Project website, for example, offers an in-browser text editor which
10139 transpiles JavaScript in real-time. Configuring CouchDB-compatibility
10140 is as easy as enabling the ENV PRESET option, and typing “firefox
10141 4.0” into the Browsers field.
10142 · The log() function will log output to the CouchDB log file or stream.
10144 You can log strings, objects, and arrays directly, without first con‐
10145 verting to JSON. Use this in conjunction with a local CouchDB
10146 instance for best results.
10147 · Be sure to guard all document accesses to avoid exceptions when
10149 fields or subfields are missing: if (doc && doc.myarray &&
10150 doc.myarray.length)...
10151 View recommendations
10153 Here are some tips and tricks for working with CouchDB’s
10154 (JavaScript-based) views.
10155 Deploying a view change in a live environment
10157 It is possible to change the definition of a view, build the index,
10158 then make those changes go live without causing downtime for your
10159 application. The trick to making this work is that CouchDB’s JavaScript
10160 view index files are based on the contents of the design document - not
10161 its name, _id or revision. This means that two design documents with
10162 identical view code will share the same on-disk view index files.
10163 Here is a worked example, assuming your /db/_design/ddoc needs tobe
10165 updated.
10166 1. Upload the old design doc to /db/_design/ddoc-old (or copy the docu‐
10168 ment) if you want an easy way to rollback in case of problems. The
10169 ddoc-old document will reference the same view indexes already built
10170 for _design/ddoc.
10171 2. Upload the updated design doc to /db/_design/ddoc-new.
10173 3. Query a view in the new design document to trigger secondary index
10175 generation. You can track the indexing progress via the
10176 /_active_tasks endpoint, or through the fauxton web interface.
10177 4. When the index is done being built, re-upload the updated design
10179 document to /db/_design/ddoc (or copy the document). The ddoc docu‐
10180 ment will now reference the same view indexes already built for
10181 _design/ddoc-new.
10182 5. Delete /db/_design/ddoc-new and/or /db/_design/ddoc-old at your dis‐
10184 cretion. Don’t forget to trigger compact/views/cleanup to reclaim
10185 disk space after deleting ddoc-old.
10186 The COPY HTTP verb can be used to copy the design document with a sin‐
10188 gle command:
10189 curl -X COPY <URL of source design document> -H "Destination: <ID of destination design document>"
10191 Reverse Proxies
10193 Reverse proxying with HAProxy
10194 CouchDB recommends the use of HAProxy as a load balancer and reverse
10195 proxy. The team’s experience with using it in production has shown it
10196 to be superior for configuration and monitoring capabilities, as well
10197 as overall performance.
10198 CouchDB’s sample haproxy configuration is present in the code reposi‐
10200 tory and release tarball as rel/haproxy.cfg. It is included below. This
10201 example is for a 3 node CouchDB cluster:
10202 global
10204 maxconn 512
10205 spread-checks 5
10206 defaults
10208 mode http
10209 log global
10210 monitor-uri /_haproxy_health_check
10211 option log-health-checks
10212 option httplog
10213 balance roundrobin
10214 option forwardfor
10215 option redispatch
10216 retries 4
10217 option http-server-close
10218 timeout client 150000
10219 timeout server 3600000
10220 timeout connect 500
10221 stats enable
10223 stats uri /_haproxy_stats
10224 # stats auth admin:admin # Uncomment for basic auth
10225 frontend http-in
10227 # This requires HAProxy 1.5.x
10228 # bind *:$HAPROXY_PORT
10229 bind *:5984
10230 default_backend couchdbs
10231 backend couchdbs
10233 option httpchk GET /_up
10234 http-check disable-on-404
10235 server couchdb1 x.x.x.x:5984 check inter 5s
10236 server couchdb2 x.x.x.x:5984 check inter 5s
10237 server couchdb2 x.x.x.x:5984 check inter 5s
10238 Reverse proxying with nginx
10240 Basic Configuration
10241 Here’s a basic excerpt from an nginx config file in <nginx config
10242 directory>/sites-available/default. This will proxy all requests from
10243 http://domain.com/... to http://localhost:5984/...
10244 location / {
10246 proxy_pass http://localhost:5984;
10247 proxy_redirect off;
10248 proxy_buffering off;
10249 proxy_set_header Host $host;
10250 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
10251 }
10252 Proxy buffering must be disabled, or continuous replication will not
10254 function correctly behind nginx.
10255 Reverse proxying CouchDB in a subdirectory with nginx
10257 It can be useful to provide CouchDB as a subdirectory of your overall
10258 domain, especially to avoid CORS concerns. Here’s an excerpt of a basic
10259 nginx configuration that proxies the URL http://domain.com/couchdb to
10260 http://localhost:5984 so that requests appended to the subdirectory,
10261 such as http://domain.com/couchdb/db1/doc1 are proxied to http://local‐
10262 host:5984/db1/doc1.
10263 location /couchdb {
10265 rewrite /couchdb/(.*) /$1 break;
10266 proxy_pass http://localhost:5984;
10267 proxy_redirect off;
10268 proxy_buffering off;
10269 proxy_set_header Host $host;
10270 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
10271 }
10272 Session based replication is default functionality since CouchDB 2.3.0.
10274 To enable session based replication with reverse proxied CouchDB in a
10275 subdirectory.
10276 location /_session {
10278 proxy_pass http://localhost:5984/_session;
10279 proxy_redirect off;
10280 proxy_buffering off;
10281 proxy_set_header Host $host;
10282 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
10283 }
10284 Authentication with nginx as a reverse proxy
10286 Here’s a sample config setting with basic authentication enabled, plac‐
10287 ing CouchDB in the /couchdb subdirectory:
10288 location /couchdb {
10290 auth_basic "Restricted";
10291 auth_basic_user_file htpasswd;
10292 rewrite /couchdb/(.*) /$1 break;
10293 proxy_pass http://localhost:5984;
10294 proxy_redirect off;
10295 proxy_buffering off;
10296 proxy_set_header Host $host;
10297 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
10298 proxy_set_header Authorization "";
10299 }
10300 This setup leans entirely on nginx performing authorization, and for‐
10302 warding requests to CouchDB with no authentication (with CouchDB in
10303 Admin Party mode), which isn’t sufficient in CouchDB 3.0 anymore as
10304 Admin Party has been removed. You’d need to at the very least
10305 hard-code user credentials into this version with headers.
10306 For a better solution, see api/auth/proxy.
10308 SSL with nginx
10310 In order to enable SSL, just enable the nginx SSL module, and add
10311 another proxy header:
10312 ssl on;
10314 ssl_certificate PATH_TO_YOUR_PUBLIC_KEY.pem;
10315 ssl_certificate_key PATH_TO_YOUR_PRIVATE_KEY.key;
10316 ssl_protocols SSLv3;
10317 ssl_session_cache shared:SSL:1m;
10318 location / {
10320 proxy_pass http://localhost:5984;
10321 proxy_redirect off;
10322 proxy_set_header Host $host;
10323 proxy_buffering off;
10324 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
10325 proxy_set_header X-Forwarded-Ssl on;
10326 }
10327 The X-Forwarded-Ssl header tells CouchDB that it should use the https
10329 scheme instead of the http scheme. Otherwise, all CouchDB-generated
10330 redirects will fail.
10331 Reverse Proxying with Caddy 2
10333 Caddy is https-by-default, and will automatically acquire, install,
10334 activate and, when necessary, renew a trusted SSL certificate for you -
10335 all in the background. Certificates are issued by the Let’s Encrypt
10336 certificate authority.
10337 Basic configuration
10339 Here’s a basic excerpt from a Caddyfile in /etc/caddy/Caddyfile. This
10340 will proxy all requests from http(s)://domain.com/... to http://local‐
10341 host:5984/...
10342 domain.com {
10344 reverse_proxy localhost:5984
10346 }
10348 Reverse proxying CouchDB in a subdirectory with Caddy 2
10350 It can be useful to provide CouchDB as a subdirectory of your overall
10351 domain, especially to avoid CORS concerns. Here’s an excerpt of a basic
10352 Caddy configuration that proxies the URL http(s)://domain.com/couchdb
10353 to http://localhost:5984 so that requests appended to the subdirectory,
10354 such as http(s)://domain.com/couchdb/db1/doc1 are proxied to
10355 http://localhost:5984/db1/doc1.
10356 domain.com {
10358 reverse_proxy /couchdb/* localhost:5984
10360 }
10362 Reverse proxying + load balancing for CouchDB clusters
10364 Here’s a basic excerpt from a Caddyfile in /<path>/<to>/<site>/Caddy‐
10365 file. This will proxy and evenly distribute all requests from
10366 http(s)://domain.com/... among 3 CouchDB cluster nodes at local‐
10367 host:15984, localhost:25984 and localhost:35984.
10368 Caddy will check the status, i.e. health, of each node every 5 seconds;
10370 if a node goes down, Caddy will avoid proxying requests to that node
10371 until it comes back online.
10372 domain.com {
10374 reverse_proxy http://localhost:15984 http://localhost:25984 http://localhost:35984 {
10376 lb_policy round_robin
10377 lb_try_interval 500ms
10378 health_interval 5s
10380 }
10381 }
10383 Authentication with Caddy 2 as a reverse proxy
10385 Here’s a sample config setting with basic authentication enabled, plac‐
10386 ing CouchDB in the /couchdb subdirectory:
10387 domain.com {
10389 basicauth /couchdb/* {
10391 couch_username couchdb_hashed_password_base64
10392 }
10393 reverse_proxy /couchdb/* localhost:5984
10395 }
10397 This setup leans entirely on nginx performing authorization, and for‐
10399 warding requests to CouchDB with no authentication (with CouchDB in
10400 Admin Party mode), which isn’t sufficient in CouchDB 3.0 anymore as
10401 Admin Party has been removed. You’d need to at the very least
10402 hard-code user credentials into this version with headers.
10403 For a better solution, see api/auth/proxy.
10405 Reverse Proxying with Apache HTTP Server
10407 WARNING:
10408 As of this writing, there is no way to fully disable the buffering
10409 between Apache HTTPD Server and CouchDB. This may present problems
10410 with continuous replication. The Apache CouchDB team strongly recom‐
10411 mend the use of an alternative reverse proxy such as haproxy or
10412 nginx, as described earlier in this section.
10413 Basic Configuration
10415 Here’s a basic excerpt for using a VirtualHost block config to use
10416 Apache as a reverse proxy for CouchDB. You need at least to configure
10417 Apache with the --enable-proxy --enable-proxy-http options and use a
10418 version equal to or higher than Apache 2.2.7 in order to use the
10419 nocanon option in the ProxyPass directive. The ProxyPass directive adds
10420 the X-Forwarded-For header needed by CouchDB, and the ProxyPreserveHost
10421 directive ensures the original client Host header is preserved.
10422 <VirtualHost *:80>
10424 ServerAdmin webmaster@dummy-host.example.com
10425 DocumentRoot "/opt/websites/web/www/dummy"
10426 ServerName couchdb.localhost
10427 AllowEncodedSlashes On
10428 ProxyRequests Off
10429 KeepAlive Off
10430 <Proxy *>
10431 Order deny,allow
10432 Deny from all
10433 Allow from 127.0.0.1
10434 </Proxy>
10435 ProxyPass / http://localhost:5984 nocanon
10436 ProxyPassReverse / http://localhost:5984
10437 ProxyPreserveHost On
10438 ErrorLog "logs/couchdb.localhost-error_log"
10439 CustomLog "logs/couchdb.localhost-access_log" common
10440 </VirtualHost>
10441
10443 Installation on Unix-like systems
10444 WARNING:
10445 CouchDB 3.0+ will not run without an admin user being created first.
10446 Be sure to create an admin user before starting CouchDB!
10447 Installation using the Apache CouchDB convenience binary packages
10449 If you are running one of the following operating systems, the easiest
10450 way to install CouchDB is to use the convenience binary packages:
10451 · CentOS/RHEL 6
10453 · CentOS/RHEL 7
10455 · Debian 9 (stretch)
10457 · Debian 10 (buster)
10459 · Ubuntu 16.04 (xenial)
10461 · Ubuntu 18.04 (bionic)
10463 · Ubuntu 20.04 (focal)
10465 These RedHat-style rpm packages and Debian-style deb packages will
10467 install CouchDB at /opt/couchdb and ensure CouchDB is run at system
10468 startup by the appropriate init subsystem (SysV-style initd or sys‐
10469 temd).
10470 The Debian-style deb packages also pre-configure CouchDB as a stand‐
10472 alone or clustered node, prompt for the address to which it will bind,
10473 and a password for the admin user. Responses to these prompts may be
10474 pre-seeded using standard debconf tools. Further details are in the
10475 README.Debian file.
10476 Apache CouchDB also provides packages for the SpiderMonkey 1.8.5
10478 JavaScript dependency, as the upstream packages for this shared library
10479 are starting to disappear or become unreliable.
10480 Enabling the Apache CouchDB package repository
10482 Debian 9 (stretch): Run the following commands:
10483 $ sudo apt-get install -y apt-transport-https gnupg ca-certificates
10485 $ echo "deb https://apache.bintray.com/couchdb-deb stretch main" \
10486 | sudo tee /etc/apt/sources.list.d/couchdb.list
10487 Debian 10 (buster): Run the following commands:
10489 $ sudo apt-get install -y gnupg ca-certificates
10491 $ echo "deb https://apache.bintray.com/couchdb-deb buster main" \
10492 | sudo tee /etc/apt/sources.list.d/couchdb.list
10493 Ubuntu 16.04 (Xenial): Run the following commands:
10495 $ sudo apt-get install -y apt-transport-https gnupg ca-certificates
10497 $ echo "deb https://apache.bintray.com/couchdb-deb xenial main" \
10498 | sudo tee /etc/apt/sources.list.d/couchdb.list
10499 Ubuntu 18.04 (Bionic): Run the following commands:
10501 $ sudo apt-get install -y gnupg ca-certificates
10503 $ echo "deb https://apache.bintray.com/couchdb-deb bionic main" \
10504 | sudo tee /etc/apt/sources.list.d/couchdb.list
10505 Ubuntu 20.04 (Focal): Run the following commands:
10507 $ sudo apt-get install -y gnupg ca-certificates
10509 $ echo "deb https://apache.bintray.com/couchdb-deb focal main" \
10510 | sudo tee /etc/apt/sources.list.d/couchdb.list
10511 CentOS: Place the following text into /etc/yum.repos.d/bin‐
10513 tray-apache-couchdb-rpm.repo:
10514 [bintray--apache-couchdb-rpm]
10516 name=bintray--apache-couchdb-rpm
10517 baseurl=http://apache.bintray.com/couchdb-rpm/el$releasever/$basearch/
10518 gpgcheck=0
10519 repo_gpgcheck=0
10520 enabled=1
10521 RedHat 6: Place the following text into /etc/yum.repos.d/bin‐
10523 tray-apache-couchdb-rpm.repo:
10524 [bintray--apache-couchdb-rpm]
10526 name=bintray--apache-couchdb-rpm
10527 baseurl=http://apache.bintray.com/couchdb-rpm/el6/$basearch/
10528 gpgcheck=0
10529 repo_gpgcheck=0
10530 enabled=1
10531 RedHat 7: Place the following text into /etc/yum.repos.d/bin‐
10533 tray-apache-couchdb-rpm.repo:
10534 [bintray--apache-couchdb-rpm]
10536 name=bintray--apache-couchdb-rpm
10537 baseurl=http://apache.bintray.com/couchdb-rpm/el7/$basearch/
10538 gpgcheck=0
10539 repo_gpgcheck=0
10540 enabled=1
10541 RedHat 8: Place the following text into /etc/yum.repos.d/bin‐
10543 tray-apache-couchdb-rpm.repo:
10544 [bintray--apache-couchdb-rpm]
10546 name=bintray--apache-couchdb-rpm
10547 baseurl=http://apache.bintray.com/couchdb-rpm/el8/$basearch/
10548 gpgcheck=0
10549 repo_gpgcheck=0
10550 enabled=1
10551 Installing the Apache CouchDB packages
10553 Debian/Ubuntu: First, install the CouchDB repository key:
10554 $ sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys \
10556 8756C4F765C9AC3CB6B85D62379CE192D401AB61
10557 Then update the repository cache and install the package:
10559 $ sudo apt update
10561 $ sudo apt install -y couchdb
10562 Debian/Ubuntu installs from binaries can be pre-configured for single
10564 node or clustered installations. For clusters, multiple nodes will
10565 still need to be joined together and configured consistently across all
10566 machines; follow the Cluster Setup walkthrough to complete the process.
10567 RedHat/CentOS: Run the command:
10569 $ sudo yum -y install epel-release && sudo yum -y install couchdb
10571 Once installed, create an admin user by hand before starting CouchDB,
10573 if your installer didn’t do this for you already.
10574 You can now start the service.
10576 Your installation is not complete. Be sure to complete the Setup steps
10578 for a single node or clustered installation.
10579 Relax! CouchDB is installed and running.
10581 Installation from source
10583 The remainder of this document describes the steps required to install
10584 CouchDB directly from source code.
10585 This guide, as well as the INSTALL.Unix document in the official tar‐
10587 ball release are the canonical sources of installation information.
10588 However, many systems have gotchas that you need to be aware of. In
10589 addition, dependencies frequently change as distributions update their
10590 archives.
10591 Dependencies
10593 You should have the following installed:
10594 · Erlang OTP (19.x, 20.x >= 21.3.8.5, 21.x >= 21.2.3, 22.x >= 22.0.5)
10596 · ICU
10598 · OpenSSL
10600 · Mozilla SpiderMonkey (1.8.5)
10602 · GNU Make
10604 · GNU Compiler Collection
10606 · libcurl
10608 · help2man
10610 · Python (>=2.7) for docs
10612 · Python Sphinx (>=1.1.3)
10614 It is recommended that you install Erlang OTP R16B03-1 or above where
10616 possible. You will only need libcurl if you plan to run the JavaScript
10617 test suite. And help2man is only need if you plan on installing the
10618 CouchDB man pages. Python and Sphinx are only required for building
10619 the online documentation. Documentation build can be disabled by
10620 adding the --disable-docs flag to the configure script.
10621 Debian-based Systems
10623 You can install the dependencies by running:
10624 sudo apt-get --no-install-recommends -y install \
10626 build-essential pkg-config erlang \
10627 libicu-dev libmozjs185-dev libcurl4-openssl-dev
10628 Be sure to update the version numbers to match your system’s available
10630 packages.
10631 RedHat-based (Fedora, Centos, RHEL) Systems
10633 You can install the dependencies by running:
10634 sudo yum install autoconf autoconf-archive automake \
10636 curl-devel erlang-asn1 erlang-erts erlang-eunit gcc-c++ \
10637 erlang-os_mon erlang-xmerl erlang-erl_interface help2man \
10638 js-devel-1.8.5 libicu-devel libtool perl-Test-Harness
10639 While CouchDB builds against the default js-devel-1.7.0 included in
10641 some distributions, it’s recommended to use a more recent
10642 js-devel-1.8.5.
10643 Warning: To build a release for CouchDB the erlang-reltool package is
10645 required, yet on CentOS/RHEL this package depends on erlang-wx which
10646 pulls in wxGTK and several X11 libraries. If CouchDB is being built on
10647 a console only server it might be a good idea to install this in a sep‐
10648 arate step to the rest of the dependencies, so that the package and all
10649 its dependencies can be removed using the yum history tool after the
10650 release is built. (reltool is needed only during release build but not
10651 for CouchDB functioning)
10652 The package can be installed by running:
10654 sudo yum install erlang-reltool
10656 Mac OS X
10658 Follow install/mac/homebrew reference for Mac App installation.
10659 If you are installing from source, you will need to install the Command
10661 Line Tools:
10662 xcode-select --install
10664 You can then install the other dependencies by running:
10666 brew install autoconf autoconf-archive automake libtool \
10668 erlang icu4c spidermonkey curl pkg-config
10669 You will need Homebrew installed to use the brew command.
10671 Some versions of Mac OS X ship a problematic OpenSSL library. If you’re
10673 experiencing troubles with CouchDB crashing intermittently with a seg‐
10674 mentation fault or a bus error, you will need to install your own ver‐
10675 sion of OpenSSL. See the wiki, mentioned above, for more information.
10676 SEE ALSO:
10678 · Homebrew
10680 FreeBSD
10682 FreeBSD requires the use of GNU Make. Where make is specified in this
10683 documentation, substitute gmake.
10684 You can install this by running:
10686 pkg install gmake
10688 Installing
10690 Once you have satisfied the dependencies you should run:
10691 ./configure
10693 If you wish to customize the installation, pass --help to this script.
10695 If everything was successful you should see the following message:
10697 You have configured Apache CouchDB, time to relax.
10699 Relax.
10701 To build CouchDB you should run:
10703 make release
10705 Try gmake if make is giving you any problems.
10707 If include paths or other compiler options must be specified, they can
10709 be passed to rebar, which compiles CouchDB, with the ERL_CFLAGS envi‐
10710 ronment variable. Likewise, options may be passed to the linker with
10711 the ERL_LDFLAGS environment variable:
10712 make release ERL_CFLAGS="-I/usr/local/include/js -I/usr/local/lib/erlang/usr/include"
10714 If everything was successful you should see the following message:
10716 ... done
10718 You can now copy the rel/couchdb directory anywhere on your system.
10719 Start CouchDB with ./bin/couchdb from within that directory.
10720 Relax.
10722 Note: a fully-fledged ./configure with the usual GNU Autotools options
10724 for package managers and a corresponding make install are in develop‐
10725 ment, but not part of the 2.0.0 release.
10726 User Registration and Security
10728 For OS X, in the steps below, substitute /Users/couchdb for
10729 /home/couchdb.
10730 You should create a special couchdb user for CouchDB.
10732 On many Unix-like systems you can run:
10734 adduser --system \
10736 --shell /bin/bash \
10737 --group --gecos \
10738 "CouchDB Administrator" couchdb
10739 On Mac OS X you can use the Workgroup Manager to create users up to
10741 version 10.9, and dscl or sysadminctl after version 10.9. Search
10742 Apple’s support site to find the documentation appropriate for your
10743 system. As of recent versions of OS X, this functionality is also
10744 included in Server.app, available through the App Store only as part of
10745 OS X Server.
10746 You must make sure that the user has a working POSIX shell and a
10748 writable home directory.
10749 You can test this by:
10751 · Trying to log in as the couchdb user
10753 · Running pwd and checking the present working directory
10755 As a recommendation, copy the rel/couchdb directory into /home/couchdb
10757 or /Users/couchdb.
10758 Ex: copy the built couchdb release to the new user’s home directory:
10760 cp -R /path/to/couchdb/rel/couchdb /home/couchdb
10762 Change the ownership of the CouchDB directories by running:
10764 chown -R couchdb:couchdb /home/couchdb
10766 Change the permission of the CouchDB directories by running:
10768 find /home/couchdb -type d -exec chmod 0770 {} \;
10770 Update the permissions for your ini files:
10772 chmod 0644 /home/couchdb/etc/*
10774 First Run
10776 NOTE:
10777 Be sure to create an admin user before trying to start CouchDB!
10778 You can start the CouchDB server by running:
10780 sudo -i -u couchdb /home/couchdb/bin/couchdb
10782 This uses the sudo command to run the couchdb command as the couchdb
10784 user.
10785 When CouchDB starts it should eventually display following messages:
10787 {database_does_not_exist,[{mem3_shards,load_shards_from_db,"_users" ...
10789 Don’t be afraid, we will fix this in a moment.
10791 To check that everything has worked, point your web browser to:
10793 http://127.0.0.1:5984/_utils/index.html
10795 From here you should verify your installation by pointing your web
10797 browser to:
10798 http://localhost:5984/_utils/index.html#verifyinstall
10800 Your installation is not complete. Be sure to complete the Setup steps
10802 for a single node or clustered installation.
10803 Running as a Daemon
10805 CouchDB no longer ships with any daemonization scripts.
10806 The CouchDB team recommends runit to run CouchDB persistently and reli‐
10808 ably. According to official site:
10809 runit is a cross-platform Unix init scheme with service supervision,
10810 a replacement for sysvinit, and other init schemes. It runs on
10811 GNU/Linux, *BSD, MacOSX, Solaris, and can easily be adapted to other
10812 Unix operating systems.
10813 Configuration of runit is straightforward; if you have questions, con‐
10815 tact the CouchDB user mailing list or IRC-channel #couchdb in FreeNode
10816 network.
10817 Let’s consider configuring runit on Ubuntu 16.04. The following steps
10819 should be considered only as an example. Details will vary by operating
10820 system and distribution. Check your system’s package management tools
10821 for specifics.
10822 Install runit:
10824 sudo apt-get install runit
10826 Create a directory where logs will be written:
10828 sudo mkdir /var/log/couchdb
10830 sudo chown couchdb:couchdb /var/log/couchdb
10831 Create directories that will contain runit configuration for CouchDB:
10833 sudo mkdir /etc/sv/couchdb
10835 sudo mkdir /etc/sv/couchdb/log
10836 Create /etc/sv/couchdb/log/run script:
10838 #!/bin/sh
10840 exec svlogd -tt /var/log/couchdb
10841 Basically it determines where and how exactly logs will be written.
10843 See man svlogd for more details.
10844 Create /etc/sv/couchdb/run:
10846 #!/bin/sh
10848 export HOME=/home/couchdb
10849 exec 2>&1
10850 exec chpst -u couchdb /home/couchdb/bin/couchdb
10851 This script determines how exactly CouchDB will be launched. Feel free
10853 to add any additional arguments and environment variables here if nec‐
10854 essary.
10855 Make scripts executable:
10857 sudo chmod u+x /etc/sv/couchdb/log/run
10859 sudo chmod u+x /etc/sv/couchdb/run
10860 Then run:
10862 sudo ln -s /etc/sv/couchdb/ /etc/service/couchdb
10864 In a few seconds runit will discover a new symlink and start CouchDB.
10866 You can control CouchDB service like this:
10867 sudo sv status couchdb
10869 sudo sv stop couchdb
10870 sudo sv start couchdb
10871 Naturally now CouchDB will start automatically shortly after system
10873 starts.
10874 You can also configure systemd, launchd or SysV-init daemons to launch
10876 CouchDB and keep it running using standard configuration files. Consult
10877 your system documentation for more information.
10878 Installation on Windows
10880 There are two ways to install CouchDB on Windows.
10881 Installation from binaries
10883 This is the simplest way to go.
10884 WARNING:
10886 Windows 8, 8.1, and 10 require the .NET Framework v3.5 to be
10887 installed.
10888 1. Get the latest Windows binaries from the CouchDB web site. Old
10890 releases are available at archive.
10891 2. Follow the installation wizard steps. Be sure to install CouchDB to
10893 a path with no spaces, such as C:\CouchDB.
10894 3. Your installation is not complete. Be sure to complete the Setup
10896 steps for a single node or clustered installation.
10897 4. Open up Fauxton
10899 5. It’s time to Relax!
10901 NOTE:
10903 In some cases you might been asked to reboot Windows to complete
10904 installation process, because of using on different Microsoft Visual
10905 C++ runtimes by CouchDB.
10906 NOTE:
10908 Upgrading note
10909 It’s recommended to uninstall previous CouchDB version before
10911 upgrading, especially if the new one is built against different
10912 Erlang release. The reason is simple: there may be leftover
10913 libraries with alternative or incompatible versions from old Erlang
10914 release that may create conflicts, errors and weird crashes.
10915 In this case, make sure you backup of your local.ini config and
10917 CouchDB database/index files.
10918 Silent Install
10920 The Windows installer supports silent installs. Here are some sample
10921 commands, supporting the new features of the 3.0 installer.
10922 Install CouchDB without a service, but with an admin user:password of
10924 admin:hunter2:
10925 msiexec /i apache-couchdb-3.0.0.msi /quiet ADMINUSER=admin ADMINPASSWORD=hunter2 /norestart
10927 The same as above, but also install and launch CouchDB as a service:
10929 msiexec /i apache-couchdb-3.0.0.msi /quiet INSTALLSERVICE=1 ADMINUSER=admin ADMINPASSWORD=hunter2 /norestart
10931 Unattended uninstall of CouchDB to target directory D:CouchDB:
10933 msiexec /x apache-couchdb-3.0.0.msi INSTALLSERVICE=1 APPLICATIONFOLDER="D:\CouchDB" ADMINUSER=admin ADMINPASSWORD=hunter2 /quiet /norestart
10935 Unattended uninstall if the installer file is unavailable:
10937 msiexec /x {4CD776E0-FADF-4831-AF56-E80E39F34CFC} /quiet /norestart
10939 Add /l* log.txt to any of the above to generate a useful logfile for
10941 debugging.
10942 Installation from sources
10944 SEE ALSO:
10945 Glazier: Automate building of CouchDB from source on Windows
10946 Installation on macOS
10948 Installation using the Apache CouchDB native application
10949 The easiest way to run CouchDB on macOS is through the native macOS
10950 application. Just follow the below instructions:
10951 1. Download Apache CouchDB for macOS. Old releases are available at
10953 archive.
10954 2. Double click on the Zip file
10956 3. Drag and drop the Apache CouchDB.app into Applications folder
10958 That’s all, now CouchDB is installed on your Mac:
10960 1. Run Apache CouchDB application
10962 2. Open up Fauxton, the CouchDB admin interface
10964 3. Verify the install by clicking on Verify, then Verify Installation.
10966 4. Your installation is not complete. Be sure to complete the Setup
10968 steps for a single node or clustered installation.
10969 5. Time to Relax!
10971 Installation with Homebrew
10973 The Homebrew build of CouchDB 2.x is still in development. Check back
10974 often for updates.
10975 Installation from source
10977 Installation on macOS is possible from source. Download the source tar‐
10978 ball, extract it, and follow the instructions in the INSTALL.Unix.md
10979 file.
10980 Running as a Daemon
10982 CouchDB itself no longer ships with any daemonization scripts.
10983 The CouchDB team recommends runit to run CouchDB persistently and reli‐
10985 ably. Configuration of runit is straightforward; if you have questions,
10986 reach out to the CouchDB user mailing list.
10987 Naturally, you can configure launchd or other init daemons to launch
10989 CouchDB and keep it running using standard configuration files.
10990 Consult your system documentation for more information.
10992 Installation on FreeBSD
10994 Installation from ports
10995 cd /usr/ports/databases/couchdb
10996 make install clean
10997 This will install CouchDB from the ports collection.
10999 NOTE:
11001 Be sure to create an admin user before starting CouchDB for the
11002 first time!
11003 Start script
11005 The following options for /etc/rc.conf or /etc/rc.conf.local are sup‐
11006 ported by the start script (defaults shown):
11007 couchdb_enable="NO"
11009 couchdb_enablelogs="YES"
11010 couchdb_user="couchdb"
11011 After enabling the couchdb rc service use the following command to
11013 start CouchDB:
11014 /usr/local/etc/rc.d/couchdb start
11016 This script responds to the arguments start, stop, status, rcvar etc..
11018 The start script will also use settings from the following config
11020 files:
11021 · /usr/local/etc/couchdb/default.ini
11023 · /usr/local/etc/couchdb/local.ini
11025 Administrators should use default.ini as reference and only modify the
11027 local.ini file.
11028 Post install
11030 Your installation is not complete. Be sure to complete the Setup steps
11031 for a single node or clustered installation.
11032 In case the install script fails to install a non-interactive user
11034 “couchdb” to be used for the database, the user needs to be created
11035 manually:
11036 I used the pw command to add a user “couchdb” in group “couchdb”:
11038 pw user add couchdb
11040 pw user mod couchdb -c 'CouchDB, time to relax' -s /usr/sbin/nologin -d /var/lib/couchdb
11041 pw group add couchdb
11042 The user is added to /etc/passwd and should look similar to the follow‐
11044 ing:
11045 shell# grep couchdb /etc/passwd
11047 couchdb:*:1013:1013:Couchdb, time to relax:/var/lib/couchdb/:/usr/sbin/nologin
11048 To change any of these settings, please refrain from editing
11050 /etc/passwd and instead use pw user mod ... or vipw. Make sure that the
11051 user has no shell, but instead uses /usr/sbin/nologin. The ‘*’ in the
11052 second field means that this user can not login via password authoriza‐
11053 tion. For details use man 5 passwd.
11054 Installation via Docker
11056 Apache CouchDB provides ‘convenience binary’ Docker images through
11057 Docker Hub at apache/couchdb. This is our upstream release; it is usu‐
11058 ally mirrored downstream at Docker’s top-level couchdb as well.
11059 At least these tags are always available on the image:
11061 · latest - always the latest
11063 · 3: always the latest 3.x version
11065 · 2: always the latest 2.x version
11067 · 1, 1.7, 1.7.2: CouchDB 1.7.2 (convenience only; no longer supported)
11069 · 1-couchperuser, 1.7-couchperuser, 1.7.2-couchperuser: CouchDB 1.7.2
11071 with couchperuser plugin (convenience only; no longer supported)
11072 These images expose CouchDB on port 5984 of the container, run every‐
11074 thing as user couchdb (uid 5984), and support use of a Docker volume
11075 for data at /opt/couchdb/data.
11076 Your installation is not complete. Be sure to complete the Setup steps
11078 for a single node or clustered installation.
11079 Further details on the Docker configuration are available in our
11081 couchdb-docker git repository.
11082 Installation via Snap
11084 Apache CouchDB provides ‘convenience binary’ Snap builds through the
11085 Ubuntu snapcraft repository under the name couchdb. Only snaps built
11086 from official stable CouchDB releases (2.0, 2.1, etc.) are available
11087 through this channel. There are separate snap channels for each major
11088 release stream, e.g. 2.x, 3.x, as well as a latest stream.
11089 After installing snapd, the CouchDB snap can be installed via:
11091 $ sudo snap install couchdb
11093 CouchDB will be installed at /snap/couchdb. Data will be stored at
11095 /var/snap/couchdb/.
11096 Please note that all other file system paths are relative to the snap
11098 `chroot` instead of the system root. In addition, the exact path
11099 depends on your system. For example, when you normally want to refer‐
11100 ence /opt/couchdb/etc/local.ini, under snap, this could live at
11101 /snap/couchdb/5/opt/couchdb/etc/local.ini.
11102 Your installation is not complete. Be sure to complete the Setup steps
11104 for a single node or clustered installation.
11105 Further details on the snap build process are available in our
11107 couchdb-pkg git repository.
11108 Installation on Kubernetes
11110 Apache CouchDB provides a Helm chart to enable deployment to Kuber‐
11111 netes.
11112 To install the chart with the release name my-release:
11114 helm repo add couchdb https://apache.github.io/couchdb-helm
11116 helm repo update
11118 helm install --name my-release couchdb/couchdb
11120 Further details on the configuration options are available in the Helm
11122 chart readme.
11123 Search Plugin Installation
11125 New in version 3.0.
11126 CouchDB can build and query full-text search indexes using an external
11129 Java service that embeds Apache Lucene. Typically, this service is
11130 installed on the same host as CouchDB and communicates with it over the
11131 loopback network.
11132 The search plugin is runtime-compatible with Java JDKs 6, 7 and 8.
11134 Building a release from source requires JDK 6.
11135 Installation of Binary Packages
11137 Binary packages that bundle all the necessary dependencies of the
11138 search plugin are available on GitHub. The files in each release
11139 should be unpacked into a directory on the Java classpath. If you do
11140 not have a classpath already set, or you wish to explicitly set the
11141 classpath location for Clouseau, then add the line:
11142 -classpath '/path/to/clouseau/*'
11144 to the server command below. If clouseau is installed in /opt/clouseau
11146 the line would be:
11147 -classpath '/opt/clouseau/*'
11149 The service expects to find a couple of configuration files convention‐
11151 ally called clouseau.ini and log4j.properties with the following con‐
11152 tent:
11153 clouseau.ini:
11155 [clouseau]
11157 ; the name of the Erlang node created by the service, leave this unchanged
11159 name=clouseau@127.0.0.1
11160 ; set this to the same distributed Erlang cookie used by the CouchDB nodes
11162 cookie=monster
11163 ; the path where you would like to store the search index files
11165 dir=/path/to/index/storage
11166 ; the number of search indexes that can be open simultaneously
11168 max_indexes_open=500
11169 log4j.properties:
11171 log4j.rootLogger=debug, CONSOLE
11173 log4j.appender.CONSOLE=org.apache.log4j.ConsoleAppender
11174 log4j.appender.CONSOLE.layout=org.apache.log4j.PatternLayout
11175 log4j.appender.CONSOLE.layout.ConversionPattern=%d{ISO8601} %c [%p] %m%n
11176 Once these files are in place the service can be started with an invo‐
11178 cation like the following:
11179 java -server \
11181 -Xmx2G \
11182 -Dsun.net.inetaddr.ttl=30 \
11183 -Dsun.net.inetaddr.negative.ttl=30 \
11184 -Dlog4j.configuration=file:/path/to/log4j.properties \
11185 -XX:OnOutOfMemoryError=\"kill -9 %p\" \
11186 -XX:+UseConcMarkSweepGC \
11187 -XX:+CMSParallelRemarkEnabled \
11188 com.cloudant.clouseau.Main \
11189 /path/to/clouseau.ini
11190 Chef
11192 The CouchDB cookbook can build the search plugin from source and
11193 install it on a server alongside CouchDB.
11194 Kubernetes
11196 Users running CouchDB on Kubernetes via the Helm chart can add the
11197 search service to each CouchDB Pod by setting enableSearch: true in the
11198 chart values.
11199 Additional Details
11201 The Search User Guide provides detailed information on creating and
11202 querying full-text indexes using this plugin.
11203 The source code for the plugin and additional configuration documenta‐
11205 tion is available on GitHub at
11206 https://github.com/cloudant-labs/clouseau.
11207 Upgrading from prior CouchDB releases
11209 Important Notes
11210 · Always back up your data/ and etc/ directories prior to upgrading
11211 CouchDB.
11212 · We recommend that you overwrite your etc/default.ini file with the
11214 version provided by the new release. New defaults sometimes contain
11215 mandatory changes to enable default functionality. Always places your
11216 customizations in etc/local.ini or any etc/local.d/*.ini file.
11217 Upgrading from CouchDB 2.x
11219 If you are coming from a prior release of CouchDB 2.x, upgrading is
11220 simple.
11221 Standalone (single) node upgrades
11223 If you are running a standalone (single) CouchDB node:
11224 1. Plan for downtime.
11226 2. Backup everything.
11228 3. Check for new recommended settings in the shipped etc/local.ini
11230 file, and merge any changes desired into your own local settings
11231 file(s).
11232 4. Stop CouchDB.
11234 5. Upgrade CouchDB in place.
11236 6. Be sure to create an admin user if you do not have one. CouchDB 3.0+
11238 require an admin user to start (the admin party has ended).
11239 7. Start CouchDB.
11241 8. Relax! You’re done.
11243 Cluster upgrades
11245 CouchDB 2.x and 3.x are explicitly designed to allow “mixed clusters”
11246 during the upgrade process. This allows you to perform a rolling
11247 restart across a cluster, upgrading one node at a time, for a zero
11248 downtime upgrade. The process is also entirely scriptable within your
11249 configuration management tool of choice.
11250 We’re proud of this feature, and you should be, too!
11252 If you are running a CouchDB cluster:
11254 1. Backup everything.
11256 2. Check for new recommended settings in the shipped etc/local.ini
11258 file, and merge any changes desired into your own local settings
11259 file(s), staging these changes to occur as you upgrade the node.
11260 3. Stop CouchDB on a single node.
11262 4. Upgrade that CouchDB install in place.
11264 5. Start CouchDB.
11266 6. Double-check that the node has re-joined the cluster through the
11268 /_membership <api/server/membership> endpoint. If your load balancer
11269 has health check functionality driven by the /_up <api/server/up>
11270 endpoint, check whether it thinks the node is healthy as well.
11271 7. Repeat the last 4 steps on the remaining nodes in the cluster.
11273 8. Relax! You’re done.
11275 Upgrading from CouchDB 1.x
11277 To upgrade from CouchDB 1.x, first upgrade to a version of CouchDB 2.x.
11278 You will need to convert all databases to CouchDB 2.x format first; see
11279 the Upgrade Notes there for instructions. Then, upgrade to CouchDB 3.x.
11280 Troubleshooting an Installation
11282 First Install
11283 If your CouchDB doesn’t start after you’ve just installed, check the
11284 following things:
11285 · On UNIX-like systems, this is usually this is a permissions issue.
11287 Ensure that you’ve followed the install/unix/security chown/chmod
11288 commands. This problem is indicated by the presence of the keyword
11289 eacces somewhere in the error output from CouchDB itself.
11290 · Some Linux distributions split up Erlang into multiple packages. For
11292 your distribution, check that you really installed all the required
11293 Erlang modules. This varies from platform to platform, so you’ll just
11294 have to work it out for yourself. For example, on recent versions of
11295 Ubuntu/Debian, the erlang package includes all Erlang modules.
11296 · Confirm that Erlang itself starts up with crypto (SSL) support:
11298 ## what version of erlang are you running? Ensure it is supported
11300 erl -noshell -eval 'io:put_chars(erlang:system_info(otp_release)).' -s erlang halt
11301 ## are the erlang crypto (SSL) libraries working?
11302 erl -noshell -eval 'case application:load(crypto) of ok -> io:put_chars("yay_crypto\n") ; _ -> exit(no_crypto) end.' -s init stop
11303 · Next, identify where your Erlang CouchDB libraries are installed.
11305 This will typically be the lib/ subdirectory of the release that you
11306 have installed.
11307 · Use this to start up Erlang with the CouchDB libraries in its path:
11309 erl -env ERL_LIBS $ERL_LIBS:/path/to/couchdb/lib -couch_ini -s crypto
11311 · In that Erlang shell, let’s check that the key libraries are running.
11313 The %% lines are comments, so you can skip them:
11314 %% test SSL support. If this fails, ensure you have the OTP erlang-crypto library installed
11316 crypto:md5_init().
11317 %% test Snappy compression. If this fails, check your CouchDB configure script output or alternatively
11319 %% if your distro comes with erlang-snappy make sure you're using only the CouchDB supplied version
11320 snappy:compress("gogogogogogogogogogogogogogo").
11321 %% test the CouchDB JSON encoder. CouchDB uses different encoders in each release, this one matches
11323 %% what is used in 2.0.x.
11324 jiffy:decode(jiffy:encode(<<"[1,2,3,4,5]">>)).
11325 %% this is how you quit the erlang shell.
11327 q().
11328 · The output should resemble this, or an error will be thrown:
11330 Erlang/OTP 17 [erts-6.2] [source] [64-bit] [smp:2:2] [async-threads:10] [kernel-poll:false]
11332 Eshell V6.2 (abort with ^G)
11334 1> crypto:md5_init().
11335 <<1,35,69,103,137,171,205,239,254,220,186,152,118,84,50,
11336 16,0,0,0,0,0,0,0,0,0,0,0,0,0,...>>
11337 2> snappy:compress("gogogogogogogogogogogogogogo").
11338 {ok,<<28,4,103,111,102,2,0>>}
11339 3> jiffy:decode(jiffy:encode(<<"[1,2,3,4,5]">>)).
11340 <<"[1,2,3,4,5]">>
11341 4> q().
11342 · At this point the only remaining dependency is your system’s Unicode
11344 support library, ICU, and the Spidermonkey Javascript VM from
11345 Mozilla. Make sure that your LD_LIBRARY_PATH or equivalent for
11346 non-Linux systems (DYLD_LIBRARY_PATH on macOS) makes these available
11347 to CouchDB. Linux example running as normal user:
11348 LD_LIBRARY_PATH=/usr/local/lib:/usr/local/spidermonkey/lib couchdb
11350 Linux example running as couchdb user:
11352 echo LD_LIBRARY_PATH=/usr/local/lib:/usr/local/spidermonkey/lib couchdb | sudo -u couchdb sh
11354 · If you receive an error message including the key word eaddrinuse,
11356 such as this:
11357 Failure to start Mochiweb: eaddrinuse
11359 edit your ``etc/default.ini`` or ``etc/local.ini`` file and change the
11361 ``[chttpd] port = 5984`` line to an available port.
11362 · If you receive an error including the string:
11364 … OS Process Error … {os_process_error,{exit_status,127}}
11366 then it is likely your SpiderMonkey JavaScript VM installation is not
11368 correct. Please recheck your build dependencies and try again.
11369 · If you receive an error including the string:
11371 … OS Process Error … {os_process_error,{exit_status,139}}
11373 this is caused by the fact that SELinux blocks access to certain areas
11375 of the file system. You must re-configure SELinux, or you can fully
11376 disable SELinux using the command:
11377 setenforce 0
11379 · If you are still not able to get CouchDB to start at this point, keep
11381 reading.
11382 Quick Build
11384 Having problems getting CouchDB to run for the first time? Follow this
11385 simple procedure and report back to the user mailing list or IRC with
11386 the output of each step. Please put the output of these steps into a
11387 paste service (such as https://paste.ee/) rather than including the
11388 output of your entire run in IRC or the mailing list directly.
11389 1. Note down the name and version of your operating system and your
11391 processor architecture.
11392 2. Note down the installed versions of CouchDB’s dependencies.
11394 3. Follow the checkout instructions to get a fresh copy of CouchDB’s
11396 trunk.
11397 4. Configure from the couchdb directory:
11399 ./configure
11401 5. Build the release:
11403 make release
11405 6. Run the couchdb command and log the output:
11407 cd rel/couchdb
11409 bin/couchdb
11410 7. Use your system’s kernel trace tool and log the output of the above
11412 command.
11413 a. For example, linux systems should use strace:
11415 strace bin/couchdb 2> strace.out
11417 8. Report back to the mailing list (or IRC) with the output of each
11419 step.
11420 Upgrading
11422 Are you upgrading from CouchDB 1.x? Install CouchDB into a fresh direc‐
11423 tory. CouchDB’s directory layout has changed and may be confused by
11424 libraries present from previous releases.
11425 Runtime Errors
11427 Erlang stack trace contains system_limit, open_port, or emfile
11428 Modern Erlang has a default limit of 65536 ports (8196 on Windows),
11429 where each open file handle, tcp connection, and linked-in driver uses
11430 one port. OSes have different soft and hard limits on the number of
11431 open handles per process, often as low as 1024 or 4096 files. You’ve
11432 probably exceeded this.
11433 There are two settings that need changing to increase this value. Con‐
11435 sult your OS documentation for how to increase the limit for your
11436 process. Under Linux and systemd, this setting can be adjusted via sys‐
11437 temctl edit couchdb and adding the lines:
11438 [Service]
11440 LimitNOFILE=65536
11441 to the file in the editor.
11443 To increase this value higher than 65536, you must also add the Erlang
11445 +Q parameter to your etc/vm.args file by adding the line:
11446 +Q 102400
11448 The old ERL_MAX_PORTS environment variable is ignored by the version of
11450 Erlang supplied with CouchDB.
11451 Lots of memory being used on startup
11453 Is your CouchDB using a lot of memory (several hundred MB) on startup?
11454 This one seems to especially affect Dreamhost installs. It’s really an
11455 issue with the Erlang VM pre-allocating data structures when ulimit is
11456 very large or unlimited. A detailed discussion can be found on the
11457 erlang-questions list, but the short answer is that you should decrease
11458 ulimit -n or lower the vm.args parameter +Q to something reasonable
11459 like 1024.
11460 function raised exception (Cannot encode ‘undefined’ value as JSON)
11462 If you see this in the CouchDB error logs, the JavaScript code you are
11463 using for either a map or reduce function is referencing an object mem‐
11464 ber that is not defined in at least one document in your database. Con‐
11465 sider this document:
11466 {
11468 "_id":"XYZ123",
11469 "_rev":"1BB2BB",
11470 "field":"value"
11471 }
11472 and this map function:
11474 function(doc) {
11476 emit(doc.name, doc.address);
11477 }
11478 This will fail on the above document, as it does not contain a name or
11480 address member. Instead, use guarding to make sure the function only
11481 accesses members when they exist in a document:
11482 function(doc) {
11484 if(doc.name && doc.address) {
11485 emit(doc.name, doc.address);
11486 }
11487 }
11488 While the above guard will work in most cases, it’s worth bearing
11490 JavaScript’s understanding of ‘false’ values in mind. Testing against a
11491 property with a value of 0 (zero), '' (empty String), false or null
11492 will return false. If this is undesired, a guard of the form if
11493 (doc.foo !== undefined) should do the trick.
11494 This error can also be caused if a reduce function does not return a
11496 value. For example, this reduce function will cause an error:
11497 function(key, values) {
11499 sum(values);
11500 }
11501 The function needs to return a value:
11503 function(key, values) {
11505 return sum(values);
11506 }
11507 erlang stack trace contains bad_utf8_character_code
11509 CouchDB 1.1.1 and later contain stricter handling of UTF8 encoding. If
11510 you are replicating from older versions to newer versions, then this
11511 error may occur during replication.
11512 A number of work-arounds exist; the simplest is to do an in-place
11514 upgrade of the relevant CouchDB and then compact prior to replicating.
11515 Alternatively, if the number of documents impacted is small, use fil‐
11517 tered replication to exclude only those documents.
11518 FIPS mode
11520 Operating systems can be configured to disallow the use of OpenSSL MD5
11521 hash functions in order to prevent use of MD5 for cryptographic pur‐
11522 poses. CouchDB makes use of MD5 hashes for verifying the integrity of
11523 data (and not for cryptography) and will not run without the ability to
11524 use MD5 hashes.
11525 The message below indicates that the operating system is running in
11527 “FIPS mode,” which, among other restrictions, does not allow the use of
11528 OpenSSL’s MD5 functions:
11529 md5_dgst.c(82): OpenSSL internal error, assertion failed: Digest MD5 forbidden in FIPS mode!
11531 [os_mon] memory supervisor port (memsup): Erlang has closed
11532 [os_mon] cpu supervisor port (cpu_sup): Erlang has closed
11533 Aborted
11534 A workaround for this is provided with the --erlang-md5 compile flag.
11536 Use of the flag results in CouchDB substituting the OpenSSL MD5 func‐
11537 tion calls with equivalent calls to Erlang’s built-in library
11538 erlang:md5. NOTE: there may be a performance penalty associated with
11539 this workaround.
11540 Because CouchDB does not make use of MD5 hashes for cryptographic pur‐
11542 poses, this workaround does not defeat the purpose of “FIPS mode,” pro‐
11543 vided that the system owner is aware of and consents to its use.
11544 Debugging startup
11546 If you’ve compiled from scratch and are having problems getting CouchDB
11547 to even start up, you may want to see more detail. Start by enabling
11548 logging at the debug level:
11549 [log]
11551 level = debug
11552 You can then pass the -init_debug +W i +v +V -emu_args flags in the
11554 ERL_FLAGS environment variable to turn on additional debugging informa‐
11555 tion that CouchDB developers can use to help you.
11556 Then, reach out to the CouchDB development team using the links pro‐
11558 vided on the CouchDB home page for assistance.
11559 macOS Known Issues
11561 undefined error, exit_status 134
11562 Sometimes the Verify Installation fails with an undefined error. This
11563 could be due to a missing dependency with Mac. In the logs, you will
11564 find couchdb exit_status,134.
11565 Installing the missing nspr via brew install nspr resolves the issue.
11567 (see: https://github.com/apache/couchdb/issues/979)
11568
11570 CouchDB 2.x can be deployed in either a single-node or a clustered con‐
11571 figuration. This section covers the first-time setup steps required for
11572 each of these configurations.
11573 Single Node Setup
11575 Many users simply need a single-node CouchDB 2.x installation. Opera‐
11576 tionally, it is roughly equivalent to the CouchDB 1.x series. Note that
11577 a single-node setup obviously doesn’t take any advantage of the new
11578 scaling and fault-tolerance features in CouchDB 2.x.
11579 After installation and initial startup, visit Fauxton at
11581 http://127.0.0.1:5984/_utils#setup. You will be asked to set up CouchDB
11582 as a single-node instance or set up a cluster. When you click “Sin‐
11583 gle-Node-Setup”, you will get asked for an admin username and password.
11584 Choose them well and remember them.
11585 You can also bind CouchDB to a public address, so it is accessible
11587 within your LAN or the public, if you are doing this on a public VM.
11588 Or, you can keep the installation private by binding only to 127.0.0.1
11589 (localhost). Binding to 0.0.0.0 will bind to all addresses. The wizard
11590 then configures your admin username and password and creates the three
11591 system databases _users, _replicator and _global_changes for you.
11592 Another option is to set the configuration parameter [couchdb] sin‐
11594 gle_node=true in your local.ini file. When doing this, CouchDB will
11595 create the system database for you on restart.
11596 Alternatively, if you don’t want to use the Setup Wizard or set that
11598 value, and run 3.x as a single node with a server administrator already
11599 configured via config file, make sure to create the three system data‐
11600 bases manually on startup:
11601 curl -X PUT http://127.0.0.1:5984/_users
11603 curl -X PUT http://127.0.0.1:5984/_replicator
11605 curl -X PUT http://127.0.0.1:5984/_global_changes
11607 Note that the last of these is not necessary if you do not expect to be
11609 using the global changes feed. Feel free to delete this database if you
11610 have created it, it has grown in size, and you do not need the function
11611 (and do not wish to waste system resources on compacting it regularly.)
11612 Cluster Set Up
11614 This section describes everything you need to know to prepare, install,
11615 and set up your first CouchDB 2.x cluster.
11616 Ports and Firewalls
11618 CouchDB uses the following ports:
11619 ┌─────────────────┬──────────┬──────────────────┬──────────────────┐
11621 │Port Number │ Protocol │ Recommended │ Usage │
11622 │ │ │ binding │ │
11623 ├─────────────────┼──────────┼──────────────────┼──────────────────┤
11624 │5984 │ tcp │ As desired, by │ Standard clus‐ │
11625 │ │ │ default local‐ │ tered port for │
11626 │ │ │ host │ all HTTP API │
11627 │ │ │ │ requests │
11628 ├─────────────────┼──────────┼──────────────────┼──────────────────┤
11629 │4369 │ tcp │ All interfaces │ Erlang port map‐ │
11630 │ │ │ by default │ per daemon │
11631 │ │ │ │ (epmd) │
11632 ├─────────────────┼──────────┼──────────────────┼──────────────────┤
11633 │Random above │ tcp │ Automatic │ Communication │
11634 │1024 (see below) │ │ │ with other │
11635 │ │ │ │ CouchDB nodes in │
11636 │ │ │ │ the cluster │
11637 └─────────────────┴──────────┴──────────────────┴──────────────────┘
11638 CouchDB in clustered mode uses the port 5984, just as in a standalone
11640 configuration. Port 5986, previously used in CouchDB 2.x, has been
11641 removed in CouchDB 3.0. All endpoints previously accessible at that
11642 port are now available under the /_node/{node-name}/... hierarchy via
11643 the primary 5984 port.
11644 CouchDB uses Erlang-native clustering functionality to achieve a clus‐
11646 tered installation. Erlang uses TCP port 4369 (EPMD) to find other
11647 nodes, so all servers must be able to speak to each other on this port.
11648 In an Erlang cluster, all nodes are connected to all other nodes, in a
11649 mesh network configuration.
11650 WARNING:
11652 If you expose the port 4369 to the Internet or any other untrusted
11653 network, then the only thing protecting you is the Erlang cookie.
11654 Every Erlang application running on that machine (such as CouchDB) then
11656 uses automatically assigned ports for communciation with other nodes.
11657 Yes, this means random ports. This will obviously not work with a fire‐
11658 wall, but it is possible to force an Erlang application to use a spe‐
11659 cific port range.
11660 This documentation will use the range TCP 9100-9200, but this range is
11662 unnecessarily broad. If you only have a single Erlang application run‐
11663 ning on a machine, the range can be limited to a single port:
11664 9100-9100, since the ports epmd assign are for inbound connections
11665 only. Three CouchDB nodes running on a single machine, as in a develop‐
11666 ment cluster scenario, would need three ports in this range.
11667 Configure and Test the Communication with Erlang
11669 Make CouchDB use correct IP|FQDN and the open ports
11670 In file etc/vm.args change the line -name couchdb@127.0.0.1 to -name
11671 couchdb@<reachable-ip-address|fully-qualified-domain-name> which
11672 defines the name of the node. Each node must have an identifier that
11673 allows remote systems to talk to it. The node name is of the form
11674 <name>@<reachable-ip-address|fully-qualified-domain-name>.
11675 The name portion can be couchdb on all nodes, unless you are running
11677 more than 1 CouchDB node on the same server with the same IP address or
11678 domain name. In that case, we recommend names of couchdb1, couchdb2,
11679 etc.
11680 The second portion of the node name must be an identifier by which
11682 other nodes can access this node – either the node’s fully qualified
11683 domain name (FQDN) or the node’s IP address. The FQDN is preferred so
11684 that you can renumber the node’s IP address without disruption to the
11685 cluster. (This is common in cloud-hosted environments.)
11686 WARNING:
11688 Tricks with /etc/hosts and libresolv don’t work with Erlang. Either
11689 properly set up DNS and use fully-qualified domain names, or use IP
11690 addresses. DNS and FQDNs are preferred.
11691 Open etc/vm.args, on all nodes, and add -kernel inet_dist_listen_min
11693 9100 and -kernel inet_dist_listen_max 9200 like below:
11694 -name ...
11696 -setcookie ...
11697 ...
11698 -kernel inet_dist_listen_min 9100
11699 -kernel inet_dist_listen_max 9200
11700 Again, a small range is fine, down to a single port (set both to 9100)
11702 if you only ever run a single CouchDB node on each machine.
11703 Confirming connectivity between nodes
11705 For this test, you need 2 servers with working hostnames. Let us call
11706 them server1.test.com and server2.test.com. They reside at 192.168.0.1
11707 and 192.168.0.2, respectively.
11708 On server1.test.com:
11710 erl -name bus@192.168.0.1 -setcookie 'brumbrum' -kernel inet_dist_listen_min 9100 -kernel inet_dist_listen_max 9200
11712 Then on server2.test.com:
11714 erl -name car@192.168.0.2 -setcookie 'brumbrum' -kernel inet_dist_listen_min 9100 -kernel inet_dist_listen_max 9200
11716 An explanation to the commands:
11718 · erl the Erlang shell.
11720 · -name bus@192.168.0.1 the name of the Erlang node and its IP
11722 address or FQDN.
11723 · -setcookie 'brumbrum' the “password” used when nodes connect
11725 to each other.
11726 · -kernel inet_dist_listen_min 9100 the lowest port in the
11728 range.
11729 · -kernel inet_dist_listen_max 9200 the highest port in the
11731 range.
11732 This gives us 2 Erlang shells. shell1 on server1, shell2 on server2.
11734 Time to connect them. Enter the following, being sure to end the line
11735 with a period (.):
11736 In shell1:
11738 net_kernel:connect_node(car@192.168.0.2).
11740 This will connect to the node called car on the server called
11742 192.168.0.2.
11743 If that returns true, then you have an Erlang cluster, and the fire‐
11745 walls are open. This means that 2 CouchDB nodes on these two servers
11746 will be able to communicate with each other successfully. If you get
11747 false or nothing at all, then you have a problem with the firewall,
11748 DNS, or your settings. Try again.
11749 If you’re concerned about firewall issues, or having trouble connecting
11751 all nodes of your cluster later on, repeat the above test between all
11752 pairs of servers to confirm connectivity and system configuration is
11753 correct.
11754 Preparing CouchDB nodes to be joined into a cluster
11756 Before you can add nodes to form a cluster, you must have them listen‐
11757 ing on an IP address accessible from the other nodes in the cluster.
11758 You should also ensure that a few critical settings are identical
11759 across all nodes before joining them.
11760 The settings we recommend you set now, before joining the nodes into a
11762 cluster, are:
11763 1. etc/vm.args settings as described in the previous two sections
11765 2. At least one server administrator user (and password)
11767 3. Bind the node’s clustered interface (port 5984) to a reachable IP
11769 address
11770 4. A consistent UUID. The UUID is used in identifying the cluster when
11772 replicating. If this value is not consistent across all nodes in the
11773 cluster, replications may be forced to rewind the changes feed to
11774 zero, leading to excessive memory, CPU and network use.
11775 5. A consistent httpd secret. The secret is used in calculating and
11777 evaluating cookie and proxy authentication, and should be set con‐
11778 sistently to avoid unnecessary repeated session cookie requests.
11779 As of CouchDB 3.0, steps 4 and 5 above are automatically performed for
11781 you when using the setup API endpoints described below.
11782 If you use a configuration management tool, such as Chef, Ansible, Pup‐
11784 pet, etc., then you can place these settings in a .ini file and dis‐
11785 tribute them to all nodes ahead of time. Be sure to pre-encrypt the
11786 password (cutting and pasting from a test instance is easiest) if you
11787 use this route to avoid CouchDB rewriting the file.
11788 If you do not use configuration management, or are just experimenting
11790 with CouchDB for the first time, use these commands once per server to
11791 perform steps 2-4 above. Be sure to change the password to something
11792 secure, and again, use the same password on all nodes. You may have to
11793 run these commands locally on each node; if so, replace
11794 <server-IP|FQDN> below with 127.0.0.1.
11795 # First, get two UUIDs to use later on. Be sure to use the SAME UUIDs on all nodes.
11797 curl http://<server-IP|FQDN>:5984/_uuids?count=2
11798 # CouchDB will respond with something like:
11800 # {"uuids":["60c9e8234dfba3e2fdab04bf92001142","60c9e8234dfba3e2fdab04bf92001cc2"]}
11801 # Copy the provided UUIDs into your clipboard or a text editor for later use.
11802 # Use the first UUID as the cluster UUID.
11803 # Use the second UUID as the cluster shared http secret.
11804 # Create the admin user and password:
11806 curl -X PUT http://<server-IP|FQDN>:5984/_node/_local/_config/admins/admin -d '"password"'
11807 # Now, bind the clustered interface to all IP addresses availble on this machine
11809 curl -X PUT http://<server-IP|FQDN>:5984/_node/_local/_config/chttpd/bind_address -d '"0.0.0.0"'
11810 # If not using the setup wizard / API endpoint, the following 2 steps are required:
11812 # Set the UUID of the node to the first UUID you previously obtained:
11813 curl -X PUT http://<server-IP|FQDN>:5984/_node/_local/_config/couchdb/uuid -d '"FIRST-UUID-GOES-HERE"'
11814 # Finally, set the shared http secret for cookie creation to the second UUID:
11816 curl -X PUT http://<server-IP|FQDN>:5984/_node/_local/_config/couch_httpd_auth/secret -d '"SECOND-UUID-GOES-HERE"'
11817 The Cluster Setup Wizard
11819 CouchDB 2.x comes with a convenient Cluster Setup Wizard as part of the
11820 Fauxton web administration interface. For first-time cluster setup, and
11821 for experimentation, this is your best option.
11822 It is strongly recommended that the minimum number of nodes in a clus‐
11824 ter is 3. For more explanation, see the Cluster Theory section of this
11825 documentation.
11826 After installation and initial start-up of all nodes in your cluster,
11828 ensuring all nodes are reachable, and the pre-configuration steps
11829 listed above, visit Fauxton at http://<server1>:5984/_utils#setup. You
11830 will be asked to set up CouchDB as a single-node instance or set up a
11831 cluster.
11832 When you click “Setup Cluster” you are asked for admin credentials
11834 again, and then to add nodes by IP address. To get more nodes, go
11835 through the same install procedure on other machines. Be sure to spec‐
11836 ify the total number of nodes you expect to add to the cluster before
11837 adding nodes.
11838 Now enter each node’s IP address or FQDN in the setup wizard, ensuring
11840 you also enter the previously set server admin username and password.
11841 Once you have added all nodes, click “Setup” and Fauxton will finish
11843 the cluster configuration for you.
11844 To check that all nodes have been joined correctly, visit
11846 http://<server-IP|FQDN>:5984/_membership on each node. The returned
11847 list should show all of the nodes in your cluster:
11848 {
11850 "all_nodes": [
11851 "couchdb@server1.test.com",
11852 "couchdb@server2.test.com",
11853 "couchdb@server3.test.com"
11854 ],
11855 "cluster_nodes": [
11856 "couchdb@server1.test.com",
11857 "couchdb@server2.test.com",
11858 "couchdb@server3.test.com"
11859 ]
11860 }
11861 The all_nodes section is the list of expected nodes; the cluster_nodes
11863 section is the list of actually connected nodes. Be sure the two lists
11864 match.
11865 Now your cluster is ready and available! You can send requests to any
11867 one of the nodes, and all three will respond as if you are working with
11868 a single CouchDB cluster.
11869 For a proper production setup, you’d now set up an HTTP reverse proxy
11871 in front of the cluster, for load balancing and SSL termination. We
11872 recommend HAProxy, but others can be used. Sample configurations are
11873 available in the best-practices section.
11874 The Cluster Setup API
11876 If you would prefer to manually configure your CouchDB cluster, CouchDB
11877 exposes the _cluster_setup endpoint for that purpose. After installa‐
11878 tion and initial setup/config, we can set up the cluster. On each node
11879 we need to run the following command to set up the node:
11880 curl -X POST -H "Content-Type: application/json" http://admin:password@127.0.0.1:5984/_cluster_setup -d '{"action": "enable_cluster", "bind_address":"0.0.0.0", "username": "admin", "password":"password", "node_count":"3"}'
11882 After that we can join all the nodes together. Choose one node as the
11884 “setup coordination node” to run all these commands on. This “setup
11885 coordination node” only manages the setup and requires all other nodes
11886 to be able to see it and vice versa. It has no special purpose beyond
11887 the setup process; CouchDB does not have the concept of a “master” node
11888 in a cluster.
11889 Setup will not work with unavailable nodes. All nodes must be online
11891 and properly preconfigured before the cluster setup process can begin.
11892 To join a node to the cluster, run these commands for each node you
11894 want to add:
11895 curl -X POST -H "Content-Type: application/json" http://admin:password@<setup-coordination-node>:5984/_cluster_setup -d '{"action": "enable_cluster", "bind_address":"0.0.0.0", "username": "admin", "password":"password", "port": 5984, "node_count": "3", "remote_node": "<remote-node-ip>", "remote_current_user": "<remote-node-username>", "remote_current_password": "<remote-node-password>" }'
11897 curl -X POST -H "Content-Type: application/json" http://admin:password@<setup-coordination-node>:5984/_cluster_setup -d '{"action": "add_node", "host":"<remote-node-ip>", "port": <remote-node-port>, "username": "admin", "password":"password"}'
11898 This will join the two nodes together. Keep running the above commands
11900 for each node you want to add to the cluster. Once this is done run the
11901 following command to complete the cluster setup and add the system
11902 databases:
11903 curl -X POST -H "Content-Type: application/json" http://admin:password@<setup-coordination-node>:5984/_cluster_setup -d '{"action": "finish_cluster"}'
11905 Verify install:
11907 curl http://admin:password@<setup-coordination-node>:5984/_cluster_setup
11909 Response:
11911 {"state":"cluster_finished"}
11913 Verify all cluster nodes are connected:
11915 curl http://admin:password@<setup-coordination-node>:5984/_membership
11917 Response:
11919 {
11921 "all_nodes": [
11922 "couchdb@couch1.test.com",
11923 "couchdb@couch2.test.com",
11924 "couchdb@couch3.test.com",
11925 ],
11926 "cluster_nodes": [
11927 "couchdb@couch1.test.com",
11928 "couchdb@couch2.test.com",
11929 "couchdb@couch3.test.com",
11930 ]
11931 }
11932 Ensure the all_nodes and cluster_nodes lists match.
11934 You CouchDB cluster is now set up.
11936
11938 Introduction To Configuring
11939 Configuration files
11940 By default, CouchDB reads configuration files from the following loca‐
11941 tions, in the following order:
11942 1. etc/default.ini
11944 2. etc/default.d/*.ini
11946 3. etc/local.ini
11948 4. etc/local.d/*.ini
11950 All paths are specified relative to the CouchDB installation directory:
11952 /opt/couchdb recommended on UNIX-like systems, C:\CouchDB recommended
11953 on Windows systems, and a combination of two directories on macOS:
11954 Applications/Apache CouchDB.app/Contents/Resources/couchdbx-core/etc
11955 for the default.ini and default.d directories, and /Users/you‐
11956 ruser/Library/Application Support/CouchDB2/etc/couchdb for the
11957 local.ini and local.d directories.
11958 Settings in successive documents override the settings in earlier
11960 entries. For example, setting the chttpd/bind_address parameter in
11961 local.ini would override any setting in default.ini.
11962 WARNING:
11964 The default.ini file may be overwritten during an upgrade or
11965 re-installation, so localised changes should be made to the
11966 local.ini file or files within the local.d directory.
11967 The configuration file chain may be changed by setting the ERL_FLAGS
11969 environment variable:
11970 export ERL_FLAGS="-couch_ini /path/to/my/default.ini /path/to/my/local.ini"
11972 or by placing the -couch_ini .. flag directly in the etc/vm.args file.
11974 Passing -couch_ini .. as a command-line argument when launching couchdb
11975 is the same as setting the ERL_FLAGS environment variable.
11976 WARNING:
11978 The environment variable/command-line flag overrides any -couch_ini
11979 option specified in the etc/vm.args file. And, BOTH of these options
11980 completely override CouchDB from searching in the default locations.
11981 Use these options only when necessary, and be sure to track the con‐
11982 tents of etc/default.ini, which may change in future releases.
11983 If there is a need to use different vm.args or sys.config files, for
11985 example, in different locations to the ones provided by CouchDB, or you
11986 don’t want to edit the original files, the default locations may be
11987 changed by setting the COUCHDB_ARGS_FILE or COUCHDB_SYSCONFIG_FILE
11988 environment variables:
11989 export COUCHDB_ARGS_FILE="/path/to/my/vm.args"
11991 export COUCHDB_SYSCONFIG_FILE="/path/to/my/sys.config"
11992 Parameter names and values
11994 All parameter names are case-sensitive. Every parameter takes a value
11995 of one of five types: boolean, integer, string, tuple and proplist.
11996 Boolean values can be written as true or false.
11997 Parameters with value type of tuple or proplist are following the
11999 Erlang requirement for style and naming.
12000 Setting parameters via the configuration file
12002 The common way to set some parameters is to edit the local.ini file
12003 (location explained above).
12004 For example:
12006 ; This is a comment
12008 [section]
12009 param = value ; inline comments are allowed
12010 Each configuration file line may contains section definition, parameter
12012 specification, empty (space and newline characters only) or commented
12013 line. You can set up inline commentaries for sections or parameters.
12014 The section defines group of parameters that are belongs to some spe‐
12016 cific CouchDB subsystem. For instance, httpd section holds not only
12017 HTTP server parameters, but also others that directly interacts with
12018 it.
12019 The parameter specification contains two parts divided by the equal
12021 sign (=): the parameter name on the left side and the parameter value
12022 on the right one. The leading and following whitespace for = is an
12023 optional to improve configuration readability.
12024 NOTE:
12026 In case when you’d like to remove some parameter from the
12027 default.ini without modifying that file, you may override in
12028 local.ini, but without any value:
12029 [compactions]
12031 _default =
12032 This could be read as: “remove the _default parameter from the com‐
12034 pactions section if it was ever set before”.
12035 The semicolon (;) signals the start of a comment. Everything after this
12037 character is ignored by CouchDB.
12038 After editing the configuration file, CouchDB should be restarted to
12040 apply any changes.
12041 Setting parameters via the HTTP API
12043 Alternatively, configuration parameters can be set via the HTTP API.
12044 This API allows changing CouchDB configuration on-the-fly without
12045 requiring a server restart:
12046 curl -X PUT http://localhost:5984/_node/<name@host>/_config/uuids/algorithm -d '"random"'
12048 The old parameter’s value is returned in the response:
12050 "sequential"
12052 You should be careful changing configuration via the HTTP API since
12054 it’s possible to make CouchDB unreachable, for example, by changing
12055 the chttpd/bind_address:
12056 curl -X PUT http://localhost:5984/_node/<name@host>/_config/chttpd/bind_address -d '"10.10.0.128"'
12058 If you make a typo or the specified IP address is not available from
12060 your network, CouchDB will be unreachable. The only way to resolve this
12061 will be to remote into the server, correct the config file, and restart
12062 CouchDB. To protect yourself against such accidents you may set the
12063 httpd/config_whitelist of permitted configuration parameters for
12064 updates via the HTTP API. Once this option is set, further changes to
12065 non-whitelisted parameters must take place via the configuration file,
12066 and in most cases, will also require a server restart before taking
12067 effect.
12068 Configuring the local node
12070 While the HTTP API allows configuring all nodes in the cluster, as a
12071 convenience, you can use the literal string _local in place of the node
12072 name, to interact with the local node’s configuration. For example:
12073 curl -X PUT http://localhost:5984/_node/_local/_config/uuids/algorithm -d '"random"'
12075 Base Configuration
12077 Base CouchDB Options
12078 [couchdb]
12079 attachment_stream_buffer_size
12081 Higher values may result in better read performance due
12082 to fewer read operations and/or more OS page cache hits.
12083 However, they can also increase overall response time for
12084 writes when there are many attachment write requests in
12085 parallel.
12086 [couchdb]
12088 attachment_stream_buffer_size = 4096
12089 database_dir
12091 Specifies location of CouchDB database files (*.couch
12092 named). This location should be writable and readable for
12093 the user the CouchDB service runs as (couchdb by
12094 default).
12095 [couchdb]
12097 database_dir = /var/lib/couchdb
12098 default_security
12100 Changed in version 3.0: admin_only is now the default.
12101 Default security object for databases if not explicitly
12104 set. When set to everyone, anyone can performs reads and
12105 writes. When set to admin_only, only admins can read and
12106 write. When set to admin_local, sharded databases can be
12107 read and written by anyone but the shards can only be
12108 read and written by admins.
12109 [couchdb] default_security = admin_only
12110 enable_database_recovery
12112 Enable this to only “soft-delete” databases when DELETE
12113 /{db} DELETE requests are made. This will place a
12114 .recovery directory in your data directory and move
12115 deleted databases/shards there instead. You can then man‐
12116 ually delete these files later, as desired.
12117 Default is false.
12119 [couchdb]
12121 enable_database_recovery = false
12122 file_compression
12124 Changed in version 1.2: Added Google Snappy compression
12125 algorithm.
12126 Method used to compress everything that is appended to
12129 database and view index files, except for attachments
12130 (see the attachments section). Available methods are:
12131 · none: no compression
12133 · snappy: use Google Snappy, a very fast compres‐
12135 sor/decompressor
12136 · deflate_N: use zlib’s deflate; N is the compression
12138 level which ranges from 1 (fastest, lowest compression
12139 ratio) to 9 (slowest, highest compression ratio)
12140 [couchdb]
12142 file_compression = snappy
12143 maintenance_mode
12145 A CouchDB node may be put into two distinct maintenance
12146 modes by setting this configuration parameter.
12147 · true: The node will not respond to clustered requests
12149 from other nodes and the /_up endpoint will return a
12150 404 response.
12151 · nolb: The /_up endpoint will return a 404 response.
12153 · false: The node responds normally, /_up returns a 200
12155 response.
12156 It is expected that the administrator has configured a
12158 load balancer in front of the CouchDB nodes in the clus‐
12159 ter. This load balancer should use the /_up endpoint to
12160 determine whether or not to send HTTP requests to any
12161 particular node. For HAProxy, the following config is
12162 appropriate:
12163 http-check disable-on-404
12165 option httpchk GET /_up
12166 max_dbs_open
12168 This option places an upper bound on the number of data‐
12169 bases that can be open at once. CouchDB reference counts
12170 database accesses internally and will close idle data‐
12171 bases as needed. Sometimes it is necessary to keep more
12172 than the default open at once, such as in deployments
12173 where many databases will be replicating continuously.
12174 [couchdb]
12176 max_dbs_open = 100
12177 max_document_size
12179 Changed in version 3.0.0.
12180 Limit maximum document body size. Size is calculated
12183 based on the serialized Erlang representation of the JSON
12184 document body, because that reflects more accurately the
12185 amount of storage consumed on disk. In particular, this
12186 limit does not include attachments.
12187 HTTP requests which create or update documents will fail
12189 with error code 413 if one or more documents is larger
12190 than this configuration value.
12191 In case of _update handlers, document size is checked
12193 after the transformation and right before being inserted
12194 into the database.
12195 [couchdb]
12197 max_document_size = 8000000 ; bytes
12198 WARNING:
12200 Before version 2.1.0 this setting was implemented by
12201 simply checking http request body sizes. For individ‐
12202 ual document updates via PUT that approximation was
12203 close enough, however that is not the case for
12204 _bulk_docs endpoint. After 2.1.0 a separate configura‐
12205 tion parameter was defined:
12206 httpd/max_http_request_size, which can be used to
12207 limit maximum http request sizes. After upgrade, it is
12208 advisable to review those settings and adjust them
12209 accordingly.
12210 os_process_timeout
12212 If an external process, such as a query server or exter‐
12213 nal process, runs for this amount of milliseconds without
12214 returning any results, it will be terminated. Keeping
12215 this value smaller ensures you get expedient errors, but
12216 you may want to tweak it for your specific needs.
12217 [couchdb]
12219 os_process_timeout = 5000 ; 5 sec
12220 single_node
12222 New in version 3.0.0.
12223 When this configuration setting is set to true, automati‐
12226 cally create the system databases on startup. Must be set
12227 false for a clustered CouchDB installation.
12228 uri_file
12230 This file contains the full URI that can be used to
12231 access this instance of CouchDB. It is used to help dis‐
12232 cover the port CouchDB is running on (if it was set to 0
12233 (e.g. automatically assigned any free one). This file
12234 should be writable and readable for the user that runs
12235 the CouchDB service (couchdb by default).
12236 [couchdb]
12238 uri_file = /var/run/couchdb/couchdb.uri
12239 users_db_security_editable
12241 New in version 3.0.0.
12242 When this configuration setting is set to false, reject
12245 any attempts to modify the _users database security
12246 object. Modification of this object is deprecated in 3.x
12247 and will be completely disallowed in CouchDB 4.x.
12248 users_db_suffix
12250 Specifies the suffix (last component of a name) of the
12251 system database for storing CouchDB users.
12252 [couchdb]
12254 users_db_suffix = _users
12255 WARNING:
12257 If you change the database name, do not forget to
12258 remove or clean up the old database, since it will no
12259 longer be protected by CouchDB.
12260 util_driver_dir
12262 Specifies location of binary drivers (icu, ejson, etc.).
12263 This location and its contents should be readable for the
12264 user that runs the CouchDB service.
12265 [couchdb]
12267 util_driver_dir = /usr/lib/couchdb/erlang/lib/couch-1.5.0/priv/lib
12268 uuid New in version 1.3.
12270 Unique identifier for this CouchDB server instance.
12273 [couchdb]
12275 uuid = 0a959b9b8227188afc2ac26ccdf345a6
12276 view_index_dir
12278 Specifies location of CouchDB view index files. This
12279 location should be writable and readable for the user
12280 that runs the CouchDB service (couchdb by default).
12281 [couchdb]
12283 view_index_dir = /var/lib/couchdb
12284 Configuring Clustering
12286 Cluster Options
12287 [cluster]
12288 q
12290 Sets the default number of shards for newly created databases.
12292 The default value, 2, splits a database into 2 separate parti‐
12293 tions.
12294 [cluster]
12296 q = 2
12297 For systems with only a few, heavily accessed, large databases,
12299 or for servers with many CPU cores, consider increasing this
12300 value to 4 or 8.
12301 The value of q can also be overridden on a per-DB basis, at DB
12303 creation time.
12304 SEE ALSO:
12306 PUT /{db}
12307 n
12309 Sets the number of replicas of each document in a cluster.
12311 CouchDB will only place one replica per node in a cluster. When
12312 set up through the Cluster Setup Wizard, a standalone single
12313 node will have n = 1, a two node cluster will have n = 2, and
12314 any larger cluster will have n = 3. It is recommended not to set
12315 n greater than 3.
12316 [cluster]
12318 n = 3
12319 placement
12321 WARNING:
12323 Use of this option will override the n option for replica
12324 cardinality. Use with care.
12325 Sets the cluster-wide replica placement policy when creating new
12327 databases. The value must be a comma-delimited list of strings
12328 of the format zone_name:#, where zone_name is a zone as speci‐
12329 fied in the nodes database and # is an integer indicating the
12330 number of replicas to place on nodes with a matching zone_name.
12331 This parameter is not specified by default.
12333 [cluster]
12335 placement = metro-dc-a:2,metro-dc-b:1
12336 SEE ALSO:
12338 cluster/databases/placement
12339 seedlist
12341 An optional, comma-delimited list of node names that this node
12343 should contact in order to join a cluster. If a seedlist is con‐
12344 figured the _up endpoint will return a 404 until the node has
12345 successfully contacted at least one of the members of the
12346 seedlist and replicated an up-to-date copy of the _nodes, _dbs,
12347 and _users system databases.
12348 [cluster] seedlist =
12349 couchdb@node1.example.com,couchdb@node2.example.com
12350 RPC Performance Tuning
12352 [rexi] CouchDB uses distributed Erlang to communicate between nodes in
12353 a cluster. The rexi library provides an optimized RPC mechanism
12354 over this communication channel. There are a few configuration
12355 knobs for this system, although in general the defaults work
12356 well.
12357 buffer_count
12359 The local RPC server will buffer messages if a remote node goes
12361 unavailable. This flag determines how many messages will be
12362 buffered before the local server starts dropping messages.
12363 Default value is 2000.
12364 server_per_node
12366 By default, rexi will spawn one local gen_server process for
12368 each node in the cluster. Disabling this flag will cause CouchDB
12369 to use a single process for all RPC communication, which is not
12370 recommended in high throughput deployments.
12371 stream_limit
12373 New in version 3.0.
12374 This flag comes into play during streaming operations like views
12377 and change feeds. It controls how many messages a remote worker
12378 process can send to a coordinator without waiting for an
12379 acknowledgement from the coordinator process. If this value is
12380 too large the coordinator can become overwhelmed by messages
12381 from the worker processes and actually deliver lower overall
12382 throughput to the client. In CouchDB 2.x this value was
12383 hard-coded to 10. In the 3.x series it is configurable and
12384 defaults to 5. Databases with a high q value are especially
12385 sensitive to this setting.
12386 couch_peruser
12388 couch_peruser Options
12389 [couch_peruser]
12390 enable
12392 If set to true, couch_peruser ensures that a private per-user
12394 database exists for each document in _users. These databases are
12395 writable only by the corresponding user. Databases are in the
12396 following form: userdb-{hex encoded username}.
12397 [couch_peruser]
12399 enable = false
12400 NOTE:
12402 The _users database must exist before couch_peruser can be
12403 enabled.
12404 delete_dbs
12406 If set to true and a user is deleted, the respective database
12408 gets deleted as well.
12409 [couch_peruser]
12411 delete_dbs = false
12412 CouchDB HTTP Server
12414 HTTP Server Options
12415 [chttpd]
12416 NOTE:
12418 In CouchDB 2.x, the chttpd section refers to the standard, clustered
12419 port. All use of CouchDB, aside from a few specific maintenance
12420 tasks as described in this documentation, should be performed over
12421 this port.
12422 bind_address
12424 Defines the IP address by which the clustered port is avail‐
12425 able:
12426 [chttpd]
12428 bind_address = 127.0.0.1
12429 To let CouchDB listen any available IP address, use 0.0.0.0:
12431 [chttpd]
12433 bind_address = 0.0.0.0
12434 For IPv6 support you need to set ::1 if you want to let
12436 CouchDB listen correctly:
12437 [chttpd]
12439 bind_address = ::1
12440 or :: for any available:
12442 [chttpd]
12444 bind_address = ::
12445 port Defines the port number to listen:
12447 [chttpd]
12449 port = 5984
12450 To let CouchDB use any free port, set this option to 0:
12452 [chttpd]
12454 port = 0
12455 prefer_minimal
12457 If a request has the header “Prefer”: “return=minimal”,
12458 CouchDB will only send the headers that are listed for the
12459 prefer_minimal configuration.:
12460 [chttpd]
12462 prefer_minimal = Cache-Control, Content-Length, Content-Range, Content-Type, ETag, Server, Transfer-Encoding, Vary
12463 WARNING:
12465 Removing the Server header from the settings will mean
12466 that the CouchDB server header is replaced with the
12467 MochiWeb server header.
12468 authentication_handlers
12470 List of authentication handlers used by CouchDB. You may
12471 extend them via third-party plugins or remove some of them if
12472 you won’t let users to use one of provided methods:
12473 [chttpd]
12475 authentication_handlers = {chttpd_auth, cookie_authentication_handler}, {chttpd_auth, default_authentication_handler}
12476 · {chttpd_auth, cookie_authentication_handler}: used for
12478 Cookie auth;
12479 · {chttpd_auth, proxy_authentication_handler}: used for Proxy
12481 auth;
12482 · {chttpd_auth, jwt_authentication_handler}: used for JWT
12484 auth;
12485 · {chttpd_auth, default_authentication_handler}: used for
12487 Basic auth;
12488 · {couch_httpd_auth, null_authentication_handler}: disables
12490 auth, breaks CouchDB.
12491 [httpd]
12493 allow_jsonp
12495 The true value of this option enables JSONP support (it’s
12496 false by default):
12497 [httpd]
12499 allow_jsonp = false
12500 changes_timeout
12502 Specifies default timeout value for Changes Feed in mil‐
12503 liseconds (60000 by default):
12504 [httpd]
12506 changes_timeout = 60000 ; 60 seconds
12507 config_whitelist
12509 Sets the configuration modification whitelist. Only
12510 whitelisted values may be changed via the config API. To
12511 allow the admin to change this value over HTTP, remember
12512 to include {httpd,config_whitelist} itself. Excluding it
12513 from the list would require editing this file to update
12514 the whitelist:
12515 [httpd]
12517 config_whitelist = [{httpd,config_whitelist}, {log,level}, {etc,etc}]
12518 enable_cors
12520 New in version 1.3.
12521 Controls CORS feature:
12524 [httpd]
12526 enable_cors = false
12527 server_options
12529 Server options for the MochiWeb component of CouchDB can
12530 be added to the configuration files:
12531 [httpd]
12533 server_options = [{backlog, 128}, {acceptor_pool_size, 16}]
12534 The options supported are a subset of full options sup‐
12536 ported by the TCP/IP stack. A list of the supported
12537 options are provided in the Erlang inet documentation.
12538 secure_rewrites
12540 This option allow to isolate databases via subdomains:
12541 [httpd]
12543 secure_rewrites = true
12544 socket_options
12546 The socket options for the listening socket in CouchDB,
12547 as set at the beginning of ever request, can be specified
12548 as a list of tuples. For example:
12549 [httpd]
12551 socket_options = [{sndbuf, 262144}]
12552 The options supported are a subset of full options sup‐
12554 ported by the TCP/IP stack. A list of the supported
12555 options are provided in the Erlang inet documentation.
12556 x_forwarded_host
12558 The x_forwarded_host header (X-Forwarded-Host by default)
12559 is used to forward the original value of the Host header
12560 field in case, for example, if a reverse proxy is rewrit‐
12561 ing the “Host” header field to some internal host name
12562 before forward the request to CouchDB:
12563 [httpd]
12565 x_forwarded_host = X-Forwarded-Host
12566 This header has higher priority above Host one, if only
12568 it exists in the request.
12569 x_forwarded_proto
12571 x_forwarded_proto header (X-Forwarder-Proto by default)
12572 is used for identifying the originating protocol of an
12573 HTTP request, since a reverse proxy may communicate with
12574 CouchDB instance using HTTP even if the request to the
12575 reverse proxy is HTTPS:
12576 [httpd]
12578 x_forwarded_proto = X-Forwarded-Proto
12579 x_forwarded_ssl
12581 The x_forwarded_ssl header (X-Forwarded-Ssl by default)
12582 tells CouchDB that it should use the https scheme instead
12583 of the http. Actually, it’s a synonym for X-For‐
12584 warded-Proto: https header, but used by some reverse
12585 proxies:
12586 [httpd]
12588 x_forwarded_ssl = X-Forwarded-Ssl
12589 enable_xframe_options
12591 Controls Enables or disabled feature:
12592 [httpd]
12594 enable_xframe_options = false
12595 max_http_request_size
12597 Changed in version 2.1.0.
12598 Limit the maximum size of the HTTP request body. This
12601 setting applies to all requests and it doesn’t discrimi‐
12602 nate between single vs. multi-document operations. So
12603 setting it to 1MB would block a PUT of a document larger
12604 than 1MB, but it might also block a _bulk_docs update of
12605 1000 1KB documents, or a multipart/related update of a
12606 small document followed by two 512KB attachments. This
12607 setting is intended to be used as a protection against
12608 maliciously large HTTP requests rather than for limiting
12609 maximum document sizes.
12610 [httpd]
12612 max_http_request_size = 4294967296 ; 4 GB
12613 WARNING:
12615 Before version 2.1.0 couchdb/max_document_size was
12616 implemented effectively as max_http_request_size. That
12617 is, it checked HTTP request bodies instead of document
12618 sizes. After the upgrade, it is advisable to review
12619 the usage of these configuration settings.
12620 HTTPS (SSL/TLS) Options
12622 [ssl] CouchDB supports TLS/SSL natively, without the use of a proxy
12623 server.
12624 HTTPS setup can be tricky, but the configuration in CouchDB was
12626 designed to be as easy as possible. All you need is two files; a
12627 certificate and a private key. If you have an official certifi‐
12628 cate from a certificate authority, both should be in your pos‐
12629 session already.
12630 If you just want to try this out and don’t want to go through
12632 the hassle of obtaining an official certificate, you can create
12633 a self-signed certificate. Everything will work the same, but
12634 clients will get a warning about an insecure certificate.
12635 You will need the OpenSSL command line tool installed. It proba‐
12637 bly already is.
12638 shell> mkdir /etc/couchdb/cert
12640 shell> cd /etc/couchdb/cert
12641 shell> openssl genrsa > privkey.pem
12642 shell> openssl req -new -x509 -key privkey.pem -out couchdb.pem -days 1095
12643 shell> chmod 600 privkey.pem couchdb.pem
12644 shell> chown couchdb privkey.pem couchdb.pem
12645 Now, you need to edit CouchDB’s configuration, by editing your
12647 local.ini file. Here is what you need to do.
12648 Under the [ssl] section, enable HTTPS and set up the newly gen‐
12650 erated certificates:
12651 [ssl]
12653 enable = true
12654 cert_file = /etc/couchdb/cert/couchdb.pem
12655 key_file = /etc/couchdb/cert/privkey.pem
12656 For more information please read certificates HOWTO.
12658 Now start (or restart) CouchDB. You should be able to connect to
12660 it using HTTPS on port 6984:
12661 shell> curl https://127.0.0.1:6984/
12663 curl: (60) SSL certificate problem, verify that the CA cert is OK. Details:
12664 error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
12665 More details here: http://curl.haxx.se/docs/sslcerts.html
12666 curl performs SSL certificate verification by default, using a "bundle"
12668 of Certificate Authority (CA) public keys (CA certs). If the default
12669 bundle file isn't adequate, you can specify an alternate file
12670 using the --cacert option.
12671 If this HTTPS server uses a certificate signed by a CA represented in
12672 the bundle, the certificate verification probably failed due to a
12673 problem with the certificate (it might be expired, or the name might
12674 not match the domain name in the URL).
12675 If you'd like to turn off curl's verification of the certificate, use
12676 the -k (or --insecure) option.
12677 Oh no! What happened?! Remember, clients will notify their users
12679 that your certificate is self signed. curl is the client in this
12680 case and it notifies you. Luckily you trust yourself (don’t
12681 you?) and you can specify the -k option as the message reads:
12682 shell> curl -k https://127.0.0.1:6984/
12684 {"couchdb":"Welcome","version":"1.5.0"}
12685 All done.
12687 For performance reasons, and for ease of setup, you may still
12689 wish to terminate HTTPS connections at your load balancer /
12690 reverse proxy, then use unencrypted HTTP between it and your
12691 CouchDB cluster. This is a recommended approach.
12692 cacert_file
12694 The path to a file containing PEM encoded CA certifi‐
12695 cates. The CA certificates are used to build the server
12696 certificate chain, and for client authentication. Also
12697 the CAs are used in the list of acceptable client CAs
12698 passed to the client when a certificate is requested. May
12699 be omitted if there is no need to verify the client and
12700 if there are not any intermediate CAs for the server cer‐
12701 tificate:
12702 [ssl]
12704 cacert_file = /etc/ssl/certs/ca-certificates.crt
12705 cert_file
12707 Path to a file containing the user’s certificate:
12708 [ssl]
12710 cert_file = /etc/couchdb/cert/couchdb.pem
12711 key_file
12713 Path to file containing user’s private PEM encoded key:
12714 [ssl]
12716 key_file = /etc/couchdb/cert/privkey.pem
12717 password
12719 String containing the user’s password. Only used if the
12720 private key file is password protected:
12721 [ssl]
12723 password = somepassword
12724 ssl_certificate_max_depth
12726 Maximum peer certificate depth (must be set even if cer‐
12727 tificate validation is off):
12728 [ssl]
12730 ssl_certificate_max_depth = 1
12731 verify_fun
12733 The verification fun (optional) if not specified, the
12734 default verification fun will be used:
12735 [ssl]
12737 verify_fun = {Module, VerifyFun}
12738 verify_ssl_certificates
12740 Set to true to validate peer certificates:
12741 [ssl]
12743 verify_ssl_certificates = false
12744 fail_if_no_peer_cert
12746 Set to true to terminate the TLS/SSL handshake with a
12747 handshake_failure alert message if the client does not
12748 send a certificate. Only used if verify_ssl_certificates
12749 is true. If set to false it will only fail if the client
12750 sends an invalid certificate (an empty certificate is
12751 considered valid):
12752 [ssl]
12754 fail_if_no_peer_cert = false
12755 secure_renegotiate
12757 Set to true to reject renegotiation attempt that does not
12758 live up to RFC 5746:
12759 [ssl]
12761 secure_renegotiate = true
12762 ciphers
12764 Set to the cipher suites that should be supported which
12765 can be specified in erlang format
12766 “{ecdhe_ecdsa,aes_128_cbc,sha256}” or in OpenSSL format
12767 “ECDHE-ECDSA-AES128-SHA256”.
12768 [ssl]
12770 ciphers = ["ECDHE-ECDSA-AES128-SHA256", "ECDHE-ECDSA-AES128-SHA"]
12771 tls_versions
12773 Set to a list of permitted SSL/TLS protocol versions:
12774 [ssl]
12776 tls_versions = [tlsv1 | 'tlsv1.1' | 'tlsv1.2']
12777 Cross-Origin Resource Sharing
12779 [cors] New in version 1.3: added CORS support, see JIRA COUCHDB-431
12780 CORS, or “Cross-Origin Resource Sharing”, allows a resource such
12783 as a web page running JavaScript inside a browser, to make AJAX
12784 requests (XMLHttpRequests) to a different domain, without com‐
12785 promising the security of either party.
12786 A typical use case is to have a static website hosted on a CDN
12788 make requests to another resource, such as a hosted CouchDB
12789 instance. This avoids needing an intermediary proxy, using JSONP
12790 or similar workarounds to retrieve and host content.
12791 While CouchDB’s integrated HTTP server has support for document
12793 attachments makes this less of a constraint for pure CouchDB
12794 projects, there are many cases where separating the static con‐
12795 tent from the database access is desirable, and CORS makes this
12796 very straightforward.
12797 By supporting CORS functionality, a CouchDB instance can accept
12799 direct connections to protected databases and instances, without
12800 the browser functionality being blocked due to same-origin con‐
12801 straints. CORS is supported today on over 90% of recent
12802 browsers.
12803 CORS support is provided as experimental functionality in 1.3,
12805 and as such will need to be enabled specifically in CouchDB’s
12806 configuration. While all origins are forbidden from making
12807 requests by default, support is available for simple requests,
12808 preflight requests and per-vhost configuration.
12809 This section requires httpd/enable_cors option have true value:
12811 [httpd]
12813 enable_cors = true
12814 credentials
12816 By default, neither authentication headers nor cookies
12817 are included in requests and responses. To do so requires
12818 both setting XmlHttpRequest.withCredentials = true on the
12819 request object in the browser and enabling credentials
12820 support in CouchDB.
12821 [cors]
12823 credentials = true
12824 CouchDB will respond to a credentials-enabled CORS
12826 request with an additional header, Access-Con‐
12827 trol-Allow-Credentials=true.
12828 origins
12830 List of origins separated by a comma, * means accept all.
12831 You can’t set origins = * and credentials = true option
12832 at the same time:
12833 [cors]
12835 origins = *
12836 Access can be restricted by protocol, host and optionally
12838 by port. Origins must follow the scheme:
12839 http://example.com:80:
12840 [cors]
12842 origins = http://localhost, https://localhost, http://couch.mydev.name:8080
12843 Note that by default, no origins are accepted. You must
12845 define them explicitly.
12846 headers
12848 List of accepted headers separated by a comma:
12849 [cors]
12851 headers = X-Couch-Id, X-Couch-Rev
12852 methods
12854 List of accepted methods:
12855 [cors]
12857 methods = GET,POST
12858 max_age
12860 Sets the Access-Control-Max-Age header in seconds. Use it
12861 to avoid repeated OPTIONS requests.
12862 [cors] max_age = 3600
12863 SEE ALSO:
12865 Original JIRA implementation ticket
12866 Standards and References:
12868 · IETF RFCs relating to methods: RFC 2618, RFC 2817, RFC 5789
12870 · IETF RFC for Web Origins: RFC 6454
12872 · W3C CORS standard
12874 Mozilla Developer Network Resources:
12876 · Same origin policy for URIs
12878 · HTTP Access Control
12880 · Server-side Access Control
12882 · JavaScript same origin policy
12884 Client-side CORS support and usage:
12886 · CORS browser support matrix
12888 · COS tutorial
12890 · XHR with CORS
12892 Per Virtual Host Configuration
12894 WARNING:
12895 Virtual Hosts are deprecated in CouchDB 3.0, and will be removed in
12896 CouchDB 4.0.
12897 To set the options for a vhosts, you will need to create a section with
12899 the vhost name prefixed by cors:. Example case for the vhost exam‐
12900 ple.com:
12901 [cors:example.com]
12903 credentials = false
12904 ; List of origins separated by a comma
12905 origins = *
12906 ; List of accepted headers separated by a comma
12907 headers = X-CouchDB-Header
12908 ; List of accepted methods
12909 methods = HEAD, GET
12910 A video from 2010 on vhost and rewrite configuration is available, but
12912 is not guaranteed to match current syntax or behaviour.
12913 Virtual Hosts
12915 WARNING:
12916 Virtual Hosts are deprecated in CouchDB 3.0, and will be removed in
12917 CouchDB 4.0.
12918 [vhosts]
12920 CouchDB can map requests to different locations based on the
12921 Host header, even if they arrive on the same inbound IP address.
12922 This allows different virtual hosts on the same machine to map
12924 to different databases or design documents, etc. The most common
12925 use case is to map a virtual host to a Rewrite Handler, to pro‐
12926 vide full control over the application’s URIs.
12927 To add a virtual host, add a CNAME pointer to the DNS for your
12929 domain name. For development and testing, it is sufficient to
12930 add an entry in the hosts file, typically /etc/hosts` on
12931 Unix-like operating systems:
12932 # CouchDB vhost definitions, refer to local.ini for further details
12934 127.0.0.1 couchdb.local
12935 Test that this is working:
12937 $ ping -n 2 couchdb.local
12939 PING couchdb.local (127.0.0.1) 56(84) bytes of data.
12940 64 bytes from localhost (127.0.0.1): icmp_req=1 ttl=64 time=0.025 ms
12941 64 bytes from localhost (127.0.0.1): icmp_req=2 ttl=64 time=0.051 ms
12942 Finally, add an entry to your configuration file in the [vhosts]
12944 section:
12945 [vhosts]
12947 couchdb.local:5984 = /example
12948 *.couchdb.local:5984 = /example
12949 If your CouchDB is listening on the the default HTTP port (80),
12951 or is sitting behind a proxy, then you don’t need to specify a
12952 port number in the vhost key.
12953 The first line will rewrite the request to display the content
12955 of the example database. This rule works only if the Host header
12956 is couchdb.local and won’t work for CNAMEs. The second rule, on
12957 the other hand, matches all CNAMEs to example db, so that both
12958 www.couchdb.local and db.couchdb.local will work.
12959 Rewriting Hosts to a Path
12961 Like in the _rewrite handler you can match some variable and use them
12962 to create the target path. Some examples:
12963 [vhosts]
12965 *.couchdb.local = /*
12966 :dbname. = /:dbname
12967 :ddocname.:dbname.example.com = /:dbname/_design/:ddocname/_rewrite
12968 The first rule passes the wildcard as dbname. The second one does the
12970 same, but uses a variable name. And the third one allows you to use any
12971 URL with ddocname in any database with dbname.
12972 X-Frame-Options
12974 X-Frame-Options is a response header that controls whether a http
12975 response can be embedded in a <frame>, <iframe> or <object>. This is a
12976 security feature to help against clickjacking.
12977 [x_frame_options] ; Settings same-origin will return
12978 X-Frame-Options: SAMEORIGIN. ; If same origin is set, it will
12979 ignore the hosts setting ; same_origin = true ; Settings hosts will
12980 ; return X-Frame-Options: ALLOW-FROM https://example.com/ ; List of
12981 hosts separated by a comma. * means accept all ; hosts =
12982 If xframe_options is enabled it will return X-Frame-Options: DENY by
12984 default. If same_origin is enabled it will return X-Frame-Options:
12985 SAMEORIGIN. A X-FRAME-OPTIONS: ALLOW-FROM url will be returned when
12986 same_origin is false, and the HOST header matches one of the urls in
12987 the hosts config. Otherwise a X-Frame-Options: DENY will be returned.
12988 Authentication and Authorization
12990 Server Administrators
12991 [admins]
12992 Changed in version 3.0.0: CouchDB requires an admin account to start.
12994 If an admin account has not been created, CouchDB will print an error
12995 message and terminate.
12996 CouchDB server administrators and passwords are not stored in the
12998 _users database, but in the last [admins] section that CouchDB finds
12999 when loading its ini files. See :config:intro for details on config
13000 file order and behaviour. This file (which could be something like
13001 etc/local.ini or etc/local.d/10-admins.ini on a Debian/Ubuntu system
13002 installed from packages) should be appropriately secured and readable
13003 only by system administrators:
13004 [admins]
13006 ;admin = mysecretpassword
13007 admin = -hashed-6d3c30241ba0aaa4e16c6ea99224f915687ed8cd,7f4a3e05e0cbc6f48a0035e3508eef90
13008 architect = -pbkdf2-43ecbd256a70a3a2f7de40d2374b6c3002918834,921a12f74df0c1052b3e562a23cd227f,10000
13009 Administrators can be added directly to the [admins] section, and when
13011 CouchDB is restarted, the passwords will be salted and encrypted. You
13012 may also use the HTTP interface to create administrator accounts; this
13013 way, you don’t need to restart CouchDB, and there’s no need to tempo‐
13014 rarily store or transmit passwords in plaintext. The HTTP
13015 /_node/{node-name}/_config/admins endpoint supports querying, deleting
13016 or creating new admin accounts:
13017 GET /_node/nonode@nohost/_config/admins HTTP/1.1
13019 Accept: application/json
13020 Host: localhost:5984
13021 HTTP/1.1 200 OK
13023 Cache-Control: must-revalidate
13024 Content-Length: 196
13025 Content-Type: application/json
13026 Date: Fri, 30 Nov 2012 11:37:18 GMT
13027 Server: CouchDB (Erlang/OTP)
13028 {
13030 "admin": "-hashed-6d3c30241ba0aaa4e16c6ea99224f915687ed8cd,7f4a3e05e0cbc6f48a0035e3508eef90",
13031 "architect": "-pbkdf2-43ecbd256a70a3a2f7de40d2374b6c3002918834,921a12f74df0c1052b3e562a23cd227f,10000"
13032 }
13033 If you already have a salted, encrypted password string (for example,
13035 from an old ini file, or from a different CouchDB server), then you can
13036 store the “raw” encrypted string, without having CouchDB doubly encrypt
13037 it.
13038 PUT /_node/nonode@nohost/_config/admins/architect?raw=true HTTP/1.1
13040 Accept: application/json
13041 Content-Type: application/json
13042 Content-Length: 89
13043 Host: localhost:5984
13044 "-pbkdf2-43ecbd256a70a3a2f7de40d2374b6c3002918834,921a12f74df0c1052b3e562a23cd227f,10000"
13046 HTTP/1.1 200 OK
13048 Cache-Control: must-revalidate
13049 Content-Length: 89
13050 Content-Type: application/json
13051 Date: Fri, 30 Nov 2012 11:39:18 GMT
13052 Server: CouchDB (Erlang/OTP)
13053 "-pbkdf2-43ecbd256a70a3a2f7de40d2374b6c3002918834,921a12f74df0c1052b3e562a23cd227f,10000"
13055 Further details are available in security, including configuring the
13057 work factor for PBKDF2, and the algorithm itself at PBKDF2 (RFC-2898).
13058 Changed in version 1.4: PBKDF2 server-side hashed salted password sup‐
13060 port added, now as a synchronous call for the _config/admins API.
13061 Authentication Configuration
13065 [chttpd]
13066 require_valid_user
13068 When this option is set to true, no requests are allowed
13069 from anonymous users. Everyone must be authenticated.
13070 [chttpd]
13072 require_valid_user = false
13073 require_valid_user_except_for_up
13075 When this option is set to true, no requests are allowed
13076 from anonymous users, except for the /_up endpoint.
13077 Everyone else must be authenticated.
13078 [chttpd]
13080 require_valid_user_except_for_up = false
13081 [couch_httpd_auth]
13083 allow_persistent_cookies
13085 When set to true, CouchDB will set the Max-Age and
13086 Expires attributes on the cookie, which causes user
13087 agents (like browsers) to preserve the cookie over
13088 restarts.
13089 [couch_httpd_auth]
13091 allow_persistent_cookies = true
13092 cookie_domain
13094 New in version 2.1.1.
13095 Configures the domain attribute of the AuthSession
13098 cookie. By default the domain attribute is empty, result‐
13099 ing in the cookie being set on CouchDB’s domain.
13100 [couch_httpd_auth]
13102 cookie_domain = example.com
13103 same_site
13105 New in version 3.0.0.
13106 When this option is set to a non-empty value, a SameSite
13109 attribute is added to the AuthSession cookie. Valid val‐
13110 ues are none, lax or strict.:
13111 [couch_httpd_auth]
13113 same_site = strict
13114 auth_cache_size
13116 Number of userctx_object to cache in memory, to reduce
13117 disk lookups.
13118 [couch_httpd_auth]
13120 auth_cache_size = 50
13121 authentication_redirect
13123 Specifies the location for redirection on successful
13124 authentication if a text/html response is accepted by the
13125 client (via an Accept header).
13126 [couch_httpd_auth]
13128 authentication_redirect = /_utils/session.html
13129 iterations
13131 New in version 1.3.
13132 The number of iterations for password hashing by the
13135 PBKDF2 algorithm. A higher number provides better hash
13136 durability, but comes at a cost in performance for each
13137 request that requires authentication.
13138 [couch_httpd_auth]
13140 iterations = 10000
13141 min_iterations
13143 New in version 1.6.
13144 The minimum number of iterations allowed for passwords
13147 hashed by the PBKDF2 algorithm. Any user with fewer iter‐
13148 ations is forbidden.
13149 [couch_httpd_auth]
13151 min_iterations = 100
13152 max_iterations
13154 New in version 1.6.
13155 The maximum number of iterations allowed for passwords
13158 hashed by the PBKDF2 algorithm. Any user with greater
13159 iterations is forbidden.
13160 [couch_httpd_auth]
13162 max_iterations = 100000
13163 proxy_use_secret
13165 When this option is set to true, the
13166 couch_httpd_auth/secret option is required for
13167 api/auth/proxy.
13168 [couch_httpd_auth]
13170 proxy_use_secret = false
13171 public_fields
13173 New in version 1.4.
13174 A comma-separated list of field names in user documents
13177 (in couchdb/users_db_suffix) that can be read by any
13178 user. If unset or not specified, authenticated users can
13179 only retrieve their own document.
13180 [couch_httpd_auth]
13182 public_fields = first_name, last_name, contacts, url
13183 NOTE:
13185 Using the public_fields whitelist for user document
13186 properties requires setting the
13187 couch_httpd_auth/users_db_public option to true (the
13188 latter option has no other purpose):
13189 [couch_httpd_auth]
13191 users_db_public = true
13192 require_valid_user
13194 When this option is set to true, no requests are allowed
13195 from anonymous users. Everyone must be authenticated.
13196 [couch_httpd_auth]
13198 require_valid_user = false
13199 secret The secret token is used for api/auth/proxy and for
13201 api/auth/cookie.
13202 [couch_httpd_auth]
13204 secret = 92de07df7e7a3fe14808cef90a7cc0d91
13205 timeout
13207 Number of seconds since the last request before sessions
13208 will be expired.
13209 [couch_httpd_auth]
13211 timeout = 600
13212 users_db_public
13214 New in version 1.4.
13215 Allow all users to view user documents. By default, only
13218 admins may browse all users documents, while users may
13219 browse only their own document.
13220 [couch_httpd_auth]
13222 users_db_public = false
13223 x_auth_roles
13225 The HTTP header name (X-Auth-CouchDB-Roles by default)
13226 that contains the list of a user’s roles, separated by a
13227 comma. Used for api/auth/proxy.
13228 [couch_httpd_auth]
13230 x_auth_roles = X-Auth-CouchDB-Roles
13231 x_auth_token
13233 The HTTP header name (X-Auth-CouchDB-Token by default)
13234 containing the token used to authenticate the authoriza‐
13235 tion. This token is an HMAC-SHA1 created from the
13236 couch_httpd_auth/secret and
13237 couch_httpd_auth/x_auth_username. The secret key should
13238 be the same on the client and the CouchDB node. This
13239 token is optional if the value of the
13240 couch_httpd_auth/proxy_use_secret option is not true.
13241 Used for api/auth/proxy.
13242 [couch_httpd_auth]
13244 x_auth_token = X-Auth-CouchDB-Token
13245 x_auth_username
13247 The HTTP header name (X-Auth-CouchDB-UserName by default)
13248 containing the username. Used for api/auth/proxy.
13249 [couch_httpd_auth]
13251 x_auth_username = X-Auth-CouchDB-UserName
13252 [jwt_auth]
13254 required_claims
13256 This parameter is a comma-separated list of additional
13257 mandatory JWT claims that must be present in any pre‐
13258 sented JWT token. A :code 400:Bad Request is sent if any
13259 are missing.
13260 [jwt_auth]
13262 required_claims = exp,iat
13263 Compaction
13265 Database Compaction Options
13266 [database_compaction]
13267 doc_buffer_size
13269 Specifies the copy buffer’s maximum size in bytes:
13270 [database_compaction]
13272 doc_buffer_size = 524288
13273 checkpoint_after
13275 Triggers a checkpoint after the specified amount of bytes
13276 were successfully copied to the compacted database:
13277 [database_compaction]
13279 checkpoint_after = 5242880
13280 View Compaction Options
13282 [view_compaction]
13283 keyvalue_buffer_size
13285 Specifies maximum copy buffer size in bytes used during
13286 compaction:
13287 [view_compaction]
13289 keyvalue_buffer_size = 2097152
13290 Compaction Daemon
13292 CouchDB ships with an automated, event-driven daemon internally known
13293 as “smoosh” that continuously re-prioritizes the database and secondary
13294 index files on each node and automatically compacts the files that will
13295 recover the most free space according to the following parameters.
13296 [smoosh]
13298 db_channels
13300 A comma-delimited list of channels that are sent the
13301 names of database files when those files are updated.
13302 Each channel can choose whether to enqueue the database
13303 for compaction; once a channel has enqueued the database,
13304 no additional channel in the list will be given the
13305 opportunity to do so.
13306 view_channels
13308 A comma-delimited list of channels that are sent the
13309 names of secondary index files when those files are
13310 updated. Each channel can choose whether to enqueue the
13311 index for compaction; once a channel has enqueued the
13312 index, no additional channel in the list will be given
13313 the opportunity to do so.
13314 staleness
13316 The number of minutes that the (expensive) priority cal‐
13317 culation on an individual can be stale for before it is
13318 recalculated. Defaults to 5.
13319 cleanup_index_files
13321 If set to true, the compaction daemon will delete the
13322 files for indexes that are no longer associated with any
13323 design document. Defaults to false and probably shouldn’t
13324 be changed unless the node is running low on disk space,
13325 and only after considering the ramifications.
13326 wait_secs
13328 The time a channel waits before starting compactions to
13329 allow time to observe the system and make a smarter deci‐
13330 sion about what to compact first. Hardly ever changed
13331 from the default of 30 (seconds).
13332 [smoosh.<channel>]
13334 The following settings control the resource allocation for a given com‐
13336 paction channel.
13337 capacity
13339 The maximum number of items the channel can hold (lowest pri‐
13340 ority item is removed to make room for new items). Defaults
13341 to 9999.
13342 concurrency
13344 The maximum number of jobs that can run concurrently in this
13345 channel. Defaults to 1.
13346 There are also several settings that collectively control whether a
13348 channel will enqueue a file for compaction and how it prioritizes files
13349 within its queue:
13350 max_priority
13352 Each item must have a priority lower than this to be
13353 enqueued. Defaults to infinity.
13354 max_size
13356 The item must be no larger than this many bytes in length to
13357 be enqueued. Defaults to infinity.
13358 min_priority
13360 The item must have a priority at least this high to be
13361 enqueued. Defaults to 5.0 for ratio and 16 MB for slack.
13362 min_changes
13364 The minimum number of changes since last compaction before
13365 the item will be enqueued. Defaults to 0. Currently only
13366 works for databases.
13367 min_size
13369 The item must be at least this many bytes in length to be
13370 enqueued. Defaults to 1mb (1048576 bytes).
13371 priority
13373 The method used to calculate priority. Can be ratio (calcu‐
13374 lated as sizes.file/sizes.active) or slack (calculated as
13375 sizes.file - sizes.active). Defaults to ratio.
13376 Background Indexing
13378 Secondary indexes in CouchDB are not updated during document write
13379 operations. In order to avoid high latencies when reading indexes fol‐
13380 lowing a large block of writes, CouchDB automatically kicks off back‐
13381 ground jobs to keep secondary indexes “warm”. The daemon responsible
13382 for this process is internally known as “ken” and can be configured
13383 using the following settings.
13384 [ken]
13386 batch_channels
13388 This setting controls the number of background view
13389 builds that can be running in parallel at any given time.
13390 The default is 20.
13391 incremental_channels
13393 It is possible for all the slots in the normal build sys‐
13394 tem to be occupied by long-running index rebuilds (e.g.
13395 if new design documents are posted to several databases
13396 simultaneously). In order to avoid already-built indexes
13397 from falling behind when this occurs, CouchDB will allow
13398 for a number of short background indexing jobs to run
13399 even when all slots are full. This setting controls how
13400 many additional short jobs are allowed to run concur‐
13401 rently with the main jobs. The default is 80.
13402 max_incremental_updates
13404 CouchDB estimates whether an indexing job is “incremen‐
13405 tal” or not by looking at the difference in sequence num‐
13406 bers between the current index and the main database. If
13407 the difference is larger than the threshold defined here
13408 the background job will only be allowed to run in the
13409 main queue. Defaults to 1000.
13410 [ken.ignore]
13412 Entries in this configuration section can be used to tell the back‐
13414 ground indexer to skip over specific database shard files. The key must
13415 be the exact name of the shard with the .couch suffix omitted, for
13416 example:
13417 [ken.ignore]
13419 shards/00000000-1fffffff/mydb.1567719095 = true
13420 NOTE:
13422 In case when you’d like to skip all views from a ddoc, you may
13423 add autoupdate: false to the ddoc. All views of that ddoc will
13424 then be skipped.
13425 More at PUT /{db}/_design/{ddoc}.
13427 IO Queue
13429 CouchDB has an internal subsystem that can prioritize IO associated
13430 with certain classes of operations. This subsystem can be configured to
13431 limit the resources devoted to background operations like internal
13432 replication and compaction according to the settings described below.
13433 [ioq]
13435 concurrency
13437 Specifies the maximum number of concurrent in-flight IO
13438 requests that the queueing system will submit:
13439 [ioq]
13441 concurrency = 10
13442 ratio The fraction of the time that a background IO request
13444 will be selected over an interactive IO request when both
13445 queues are non-empty:
13446 [ioq]
13448 ratio = 0.01
13449 [ioq.bypass]
13451 System administrators can choose to submit specific classes of
13452 IO directly to the underlying file descriptor or OS process,
13453 bypassing the queues altogether. Installing a bypass can yield
13454 higher throughput and lower latency, but relinquishes some con‐
13455 trol over prioritization. The following classes are recognized:
13456 os_process
13458 Messages on their way to an external process (e.g.,
13459 couchjs).
13460 read Disk IO fulfilling interactive read requests.
13462 write Disk IO required to update a database.
13464 view_update
13466 Disk IO required to update views and other secondary
13467 indexes.
13468 shard_sync
13470 Disk IO issued by the background replication processes
13471 that fix any inconsistencies between shard copies.
13472 compaction
13474 Disk IO issued by compaction jobs.
13475 Without any configuration CouchDB will enqueue all classes of
13477 IO. The default.ini configuration file that ships with CouchDB
13478 activates a bypass for each of the interactive IO classes and
13479 only background IO goes into the queueing system:
13480 [ioq.bypass]
13482 os_process = true
13483 read = true
13484 write = true
13485 view_update = true
13486 shard_sync = false
13487 compaction = false
13488 Recommendations
13490 The default configuration protects against excessive IO from background
13491 operations like compaction disrupting the latency of interactive opera‐
13492 tions, while maximizing the overall IO throughput devoted to those
13493 interactive requests. There are certain situations where this configu‐
13494 ration could be sub-optimal:
13495 · An administrator may want to devote a larger portion of the overall
13497 IO bandwidth to compaction in order to stay ahead of the incoming
13498 write load. In this it may be necessary to disable the bypass for
13499 write (to help with database compaction) and/or view_update (to help
13500 with view index compaction) and then increase the ratio to give com‐
13501 paction a higher priority.
13502 · A server with a large number of views that do not need to be
13504 comlpetely up-to-date may benefit from removing the bypass on
13505 view_update in order to optimize the latency for regular document
13506 read and write operations, and build the views during quieter peri‐
13507 ods.
13508 Logging
13510 Logging options
13511 [log] CouchDB logging configuration.
13512 writer Current writers include:
13514 · stderr: Logs are sent to stderr.
13516 · file: Logs are sent to the file set in log file.
13518 · syslog: Logs are sent to the syslog daemon.
13520 · journald: Logs are sent to stderr without timestamp and
13522 log levels compatible with sd-daemon.
13523 You can also specify a full module name here if implement
13525 your own writer:
13526 [log]
13528 writer = stderr
13529 file Specifies the location of file for logging output. Only
13531 used by the file writer:
13532 [log]
13534 file = /var/log/couchdb/couch.log
13535 This path should be readable and writable for user that
13537 runs CouchDB service (couchdb by default).
13538 write_buffer
13540 Specifies the size of the file log write buffer in bytes,
13541 to enable delayed log writes. Only used by the file
13542 writer:
13543 [log]
13545 write_buffer = 0
13546 write_delay
13548 Specifies the wait in milliseconds before committing logs
13549 to disk, to enable delayed log writes. Only used by the
13550 file writer:
13551 [log]
13553 write_delay = 0
13554 level Changed in version 1.3: Added warning level.
13556 Logging level defines how verbose and detailed logging
13559 will be:
13560 [log]
13562 level = info
13563 Available levels:
13565 · debug: Detailed debug logging.
13567 · info: Informative logging. Includes HTTP requests head‐
13569 lines, startup of an external processes etc.
13570 · notice
13572 · warning or warn: Warning messages are alerts about edge
13574 situations that may lead to errors. For instance, com‐
13575 paction daemon alerts about low or insufficient disk
13576 space at this level.
13577 · error or err: Error level includes only things that go
13579 wrong, like crash reports and HTTP error responses (5xx
13580 codes).
13581 · critical or crit
13583 · alert
13585 · emergency or emerg
13587 · none: Disables logging any messages.
13589 include_sasl
13591 Includes SASL information in logs:
13592 [log]
13594 include_sasl = true
13595 syslog_host
13597 Specifies the syslog host to send logs to. Only used by
13598 the syslog writer:
13599 [log]
13601 syslog_host = localhost
13602 syslog_port
13604 Specifies the syslog port to connect to when sending
13605 logs. Only used by the syslog writer:
13606 [log]
13608 syslog_port = 514
13609 syslog_appid
13611 Specifies application name to the syslog writer:
13612 [log]
13614 syslog_appid = couchdb
13615 syslog_facility
13617 Specifies the syslog facility to use with the syslog
13618 writer:
13619 [log]
13621 syslog_facility = local2
13622 Replicator
13624 Replicator Database Configuration
13625 [replicator]
13626 max_jobs
13628 New in version 2.1.
13629 Number of actively running replications. This value rep‐
13632 resents the threshold to trigger the automatic replica‐
13633 tion scheduler. The system will check every interval
13634 milliseconds how many replication jobs are running, and
13635 if there are more than max_jobs active jobs, the sched‐
13636 uler will pause-and-restart up to max_churn jobs in the
13637 scheduler queue. Making this value too high could cause
13638 performance issues, while making it too low could mean
13639 replications jobs might not have enough time to make
13640 progress before getting unscheduled again. This parame‐
13641 ter can be adjusted at runtime and will take effect dur‐
13642 ing next rescheduling cycle:
13643 [replicator]
13645 max_jobs = 500
13646 interval
13648 New in version 2.1.
13649 Scheduling interval in milliseconds. During each
13652 reschedule cycle the scheduler might start or stop up to
13653 max_churn number of jobs:
13654 [replicator]
13656 interval = 60000
13657 max_churn
13659 New in version 2.1.
13660 Maximum number of replication jobs to start and stop dur‐
13663 ing rescheduling. This parameter, along with interval,
13664 defines the rate of job replacement. During startup,
13665 however, a much larger number of jobs could be started
13666 (up to max_jobs) in a short period of time:
13667 [replicator]
13669 max_churn = 20
13670 max_history
13672 Maximum number of events recorded for each job. This
13673 parameter defines an upper bound on the consecutive fail‐
13674 ure count for a job, and in turn the maximum backoff fac‐
13675 tor used when determining the delay before the job is
13676 restarted. The longer the length of the crash count, the
13677 longer the possible length of the delay:
13678 [replicator]
13680 max_history = 20
13681 update_docs
13683 New in version 2.1.
13684 When set to true replicator will update replication docu‐
13687 ment with error and triggered states. This approximates
13688 pre-2.1 replicator behavior:
13689 [replicator]
13691 update_docs = false
13692 worker_batch_size
13694 With lower batch sizes checkpoints are done more fre‐
13695 quently. Lower batch sizes also reduce the total amount
13696 of used RAM memory:
13697 [replicator]
13699 worker_batch_size = 500
13700 worker_processes
13702 More worker processes can give higher network throughput
13703 but can also imply more disk and network IO:
13704 [replicator]
13706 worker_processes = 4
13707 http_connections
13709 Maximum number of HTTP connections per replication:
13710 [replicator]
13712 http_connections = 20
13713 connection_timeout
13715 HTTP connection timeout per replication. This is divided
13716 by three (3) when the replicator makes changes feed
13717 requests. Even for very fast/reliable networks it might
13718 need to be increased if a remote database is too busy:
13719 [replicator]
13721 connection_timeout = 30000
13722 retries_per_request
13724 Changed in version 2.1.1.
13725 If a request fails, the replicator will retry it up to N
13728 times. The default value for N is 5 (before version 2.1.1
13729 it was 10). The requests are retried with a doubling
13730 exponential backoff starting at 0.25 seconds. So by
13731 default requests would be retried in 0.25, 0.5, 1, 2, 4
13732 second intervals. When number of retires is exhausted,
13733 the whole replication job is stopped and will retry again
13734 later:
13735 [replicator]
13737 retries_per_request = 5
13738 socket_options
13740 Some socket options that might boost performance in some
13741 scenarios:
13742 · {nodelay, boolean()}
13744 · {sndbuf, integer()}
13746 · {recbuf, integer()}
13748 · {priority, integer()}
13750 See the inet Erlang module’s man page for the full list
13752 of options:
13753 [replicator]
13755 socket_options = [{keepalive, true}, {nodelay, false}]
13756 checkpoint_interval
13758 New in version 1.6.
13759 Defines replication checkpoint interval in milliseconds.
13762 Replicator will requests from the Source database at the
13763 specified interval:
13764 [replicator]
13766 checkpoint_interval = 5000
13767 Lower intervals may be useful for frequently changing
13769 data, while higher values will lower bandwidth and make
13770 fewer requests for infrequently updated databases.
13771 use_checkpoints
13773 New in version 1.6.
13774 If use_checkpoints is set to true, CouchDB will make
13777 checkpoints during replication and at the completion of
13778 replication. CouchDB can efficiently resume replication
13779 from any of these checkpoints:
13780 [replicator]
13782 use_checkpoints = true
13783 NOTE:
13785 Checkpoints are stored in local documents on both the
13786 source and target databases (which requires write
13787 access).
13788 WARNING:
13790 Disabling checkpoints is not recommended as CouchDB
13791 will scan the Source database’s changes feed from the
13792 beginning.
13793 cert_file
13795 Path to a file containing the user’s certificate:
13796 [replicator]
13798 cert_file = /full/path/to/server_cert.pem
13799 key_file
13801 Path to file containing user’s private PEM encoded key:
13802 [replicator]
13804 key_file = /full/path/to/server_key.pem
13805 password
13807 String containing the user’s password. Only used if the
13808 private key file is password protected:
13809 [replicator]
13811 password = somepassword
13812 verify_ssl_certificates
13814 Set to true to validate peer certificates:
13815 [replicator]
13817 verify_ssl_certificates = false
13818 ssl_trusted_certificates_file
13820 File containing a list of peer trusted certificates (in
13821 the PEM format):
13822 [replicator]
13824 ssl_trusted_certificates_file = /etc/ssl/certs/ca-certificates.crt
13825 ssl_certificate_max_depth
13827 Maximum peer certificate depth (must be set even if cer‐
13828 tificate validation is off):
13829 [replicator]
13831 ssl_certificate_max_depth = 3
13832 auth_plugins
13834 New in version 2.2.
13835 List of replicator client authentication plugins. Plugins
13838 will be tried in order and the first to initialize suc‐
13839 cessfully will be used. By default there are two plugins
13840 available: couch_replicator_auth_session implementing
13841 session (cookie) authentication, and couch_replica‐
13842 tor_auth_noop implementing basic authentication. For
13843 backwards compatibility, the no-op plugin should be used
13844 at the end of the plugin list:
13845 [replicator]
13847 auth_plugins = couch_replicator_auth_session,couch_replicator_auth_noop
13848 NOTE:
13850 In version 2.2, the session plugin is considered
13851 experimental and is not enabled by default.
13852 Query Servers
13854 Query Servers Definition
13855 Changed in version 2.3: Changed configuration method for Query Servers
13856 and Native Query Servers.
13857 CouchDB delegates computation of design documents functions to external
13860 query servers. The external query server is a special OS process which
13861 communicates with CouchDB over standard input/output using a very sim‐
13862 ple line-based protocol with JSON messages.
13863 An external query server may be defined with environment variables fol‐
13865 lowing this pattern:
13866 COUCHDB_QUERY_SERVER_LANGUAGE="PATH ARGS"
13868 Where:
13870 · LANGUAGE: is a programming language which code this query server may
13872 execute. For instance, there are PYTHON, RUBY, CLOJURE and other
13873 query servers in the wild. This value in lowercase is also used for
13874 ddoc field language to determine which query server processes the
13875 functions.
13876 Note, that you may set up multiple query servers for the same pro‐
13878 gramming language, but you have to name them differently (like
13879 PYTHONDEV etc.).
13880 · PATH: is a system path to the executable binary program that runs the
13882 query server.
13883 · ARGS: optionally, you may specify additional command line arguments
13885 for the executable PATH.
13886 The default query server is written in JavaScript, running via Mozilla
13888 SpiderMonkey. It requires no special environment settings to enable,
13889 but is the equivalent of these two variables:
13890 COUCHDB_QUERY_SERVER_JAVASCRIPT="/opt/couchdb/bin/couchjs /opt/couchdb/share/server/main.js"
13892 COUCHDB_QUERY_SERVER_COFFEESCRIPT="/opt/couchdb/bin/couchjs /opt/couchdb/share/server/main-coffee.js"
13893 By default, couchjs limits the max runtime allocation to 64MiB. If you
13895 run into out of memory issue in your ddoc functions, you can adjust the
13896 memory limitation (here, increasing to 512 MiB):
13897 COUCHDB_QUERY_SERVER_JAVASCRIPT="/usr/bin/couchjs -S 536870912 /usr/share/server/main.js"
13899 For more info about the available options, please consult couchjs -h.
13901 SEE ALSO:
13903 The Mango Query Server is a declarative language that requires no
13904 programming, allowing for easier indexing and finding of data in
13905 documents.
13906 The Native Erlang Query Server allows running ddocs written in
13908 Erlang natively, bypassing stdio communication and JSON serializa‐
13909 tion/deserialization round trip overhead.
13910 Query Servers Configuration
13912 [query_server_config]
13913 commit_freq
13915 Specifies the delay in seconds before view index changes
13916 are committed to disk. The default value is 5:
13917 [query_server_config]
13919 commit_freq = 5
13920 os_process_limit
13922 limit Hard limit on the number of OS processes usable by Query
13924 Servers. The default value is 100:
13925 [query_server_config]
13927 os_process_limit = 100
13928 Setting os_process_limit too low can result in starvation
13930 of Query Servers, and manifest in os_process_timeout
13931 errors, while setting it too high can potentially use too
13932 many system resources. Production settings are typically
13933 10-20 times the default value.
13934 os_process_soft_limit
13936 soft limit
13938 Soft limit on the number of OS processes usable by Query
13939 Servers. The default value is 100:
13940 [query_server_config]
13942 os_process_soft_limit = 100
13943 Idle OS processes are closed until the total reaches the
13945 soft limit.
13946 For example, if the hard limit is 200 and the soft limit
13948 is 100, the total number of OS processes will never
13949 exceed 200, and CouchDB will close all idle OS processes
13950 until it reaches 100, at which point it will leave the
13951 rest intact, even if some are idle.
13952 reduce_limit
13954 Controls Reduce overflow error that raises when output of
13955 reduce functions is too big:
13956 [query_server_config]
13958 reduce_limit = true
13959 Normally, you don’t have to disable (by setting false
13961 value) this option since main propose of reduce functions
13962 is to reduce the input.
13963 Native Erlang Query Server
13965 [native_query_servers]
13966 WARNING:
13967 Due to security restrictions, the Erlang query server is dis‐
13968 abled by default.
13969 Unlike the JavaScript query server, the Erlang one does not
13971 runs in a sandbox mode. This means that Erlang code has full
13972 access to your OS, file system and network, which may lead to
13973 security issues. While Erlang functions are faster than
13974 JavaScript ones, you need to be careful about running them,
13975 especially if they were written by someone else.
13976 CouchDB has a native Erlang query server, allowing you to write
13978 your map/reduce functions in Erlang.
13979 First, you’ll need to edit your local.ini to include a
13981 [native_query_servers] section:
13982 [native_query_servers]
13984 enable_erlang_query_server = true
13985 To see these changes you will also need to restart the server.
13987 Let’s try an example of map/reduce functions which count the
13989 total documents at each number of revisions (there are x many
13990 documents at version “1”, and y documents at “2”… etc). Add a
13991 few documents to the database, then enter the following func‐
13992 tions as a view:
13993 %% Map Function
13995 fun({Doc}) ->
13996 <<K,_/binary>> = proplists:get_value(<<"_rev">>, Doc, null),
13997 V = proplists:get_value(<<"_id">>, Doc, null),
13998 Emit(<<K>>, V)
13999 end.
14000 %% Reduce Function
14002 fun(Keys, Values, ReReduce) -> length(Values) end.
14003 If all has gone well, after running the view you should see a
14005 list of the total number of documents at each revision number.
14006 Additional examples are on the users@couchdb.apache.org mailing
14008 list.
14009 Search
14011 CouchDB’s search subsystem can be configured via the dreyfus configura‐
14012 tion section.
14013 [dreyfus]
14015 name The name and location of the Clouseau Java service
14017 required to enable Search functionality. Defaults to
14018 clouseau@127.0.0.1.
14019 retry_limit
14021 CouchDB will try to reconnect to Clouseau using a bounded
14022 exponential backoff with the following number of itera‐
14023 tions. Defaults to 5.
14024 limit The number of results returned from a global search query
14026 if no limit is specified. Defaults to 25.
14027 limit_partitions
14029 The number of results returned from a search on a parti‐
14030 tion of a database if no limit is specified. Defaults to
14031 2000.
14032 max_limit
14034 The maximum number of results that can be returned from a
14035 global search query (or any search query on a database
14036 without user-defined partitions). Attempts to set
14037 ?limit=N higher than this value will be rejected.
14038 Defaults to 200.
14039 max_limit_partitions
14041 The maximum number of results that can be returned when
14042 searching a partition of a database. Attempts to set
14043 ?limit=N higher than this value will be rejected. If this
14044 config setting is not defined, CouchDB will use the value
14045 of max_limit instead. If neither is defined, the default
14046 is 2000.
14047 Mango
14049 Mango is the Query Engine that services the _find, endpoint.
14050 [mango]
14052 index_all_disabled
14054 Set to true to disable the “index all fields” text index.
14055 This can lead to out of memory issues when there are doc‐
14056 uments with nested array fields. Defaults to false.
14057 [mango] index_all_disabled = false
14058 default_limit
14060 Sets the default number of results that will be returned
14061 in a _find response. Individual requests can override
14062 this by setting limit directly in the query parameters.
14063 Defaults to 25.
14064 [mango] default_limit = 25
14065 index_scan_warning_threshold
14067 index scan warning
14069 This sets the ratio between documents scanned and
14070 results matched that will generate a warning in the
14071 _find response. For example, if a query requires read‐
14072 ing 100 documents to return 10 rows, a warning will be
14073 generated if this value is 10.
14074 Defaults to 10. Setting the value to 0 disables the
14076 warning.
14077 [mango] index_scan_warning_threshold = 10
14078 Miscellaneous Parameters
14080 Configuration of Attachment Storage
14081 [attachments]
14082 compression_level
14084 Defines zlib compression level for the attachments from 1
14085 (lowest, fastest) to 9 (highest, slowest). A value of 0
14086 disables compression:
14087 [attachments]
14089 compression_level = 8
14090 compressible_types
14092 Since compression is ineffective for some types of files,
14093 it is possible to let CouchDB compress only some types of
14094 attachments, specified by their MIME type:
14095 [attachments]
14097 compressible_types = text/*, application/javascript, application/json, application/xml
14098 Statistic Calculation
14100 [stats]
14101 interval
14103 Interval between gathering statistics in seconds:
14104 [stats]
14106 interval = 10
14107 UUIDs Configuration
14109 [uuids]
14110 algorithm
14112 Changed in version 1.3: Added utc_id algorithm.
14113 CouchDB provides various algorithms to generate the UUID
14116 values that are used for document _id’s by default:
14117 [uuids]
14119 algorithm = sequential
14120 Available algorithms:
14122 · random: 128 bits of random awesome. All awesome, all
14124 the time:
14125 {
14127 "uuids": [
14128 "5fcbbf2cb171b1d5c3bc6df3d4affb32",
14129 "9115e0942372a87a977f1caf30b2ac29",
14130 "3840b51b0b81b46cab99384d5cd106e3",
14131 "b848dbdeb422164babf2705ac18173e1",
14132 "b7a8566af7e0fc02404bb676b47c3bf7",
14133 "a006879afdcae324d70e925c420c860d",
14134 "5f7716ee487cc4083545d4ca02cd45d4",
14135 "35fdd1c8346c22ccc43cc45cd632e6d6",
14136 "97bbdb4a1c7166682dc026e1ac97a64c",
14137 "eb242b506a6ae330bda6969bb2677079"
14138 ]
14139 }
14140 · sequential: Monotonically increasing ids with random
14142 increments. The first 26 hex characters are random,
14143 the last 6 increment in random amounts until an over‐
14144 flow occurs. On overflow, the random prefix is regener‐
14145 ated and the process starts over.
14146 {
14148 "uuids": [
14149 "4e17c12963f4bee0e6ec90da54804894",
14150 "4e17c12963f4bee0e6ec90da5480512f",
14151 "4e17c12963f4bee0e6ec90da54805c25",
14152 "4e17c12963f4bee0e6ec90da54806ba1",
14153 "4e17c12963f4bee0e6ec90da548072b3",
14154 "4e17c12963f4bee0e6ec90da54807609",
14155 "4e17c12963f4bee0e6ec90da54807718",
14156 "4e17c12963f4bee0e6ec90da54807754",
14157 "4e17c12963f4bee0e6ec90da54807e5d",
14158 "4e17c12963f4bee0e6ec90da54808d28"
14159 ]
14160 }
14161 · utc_random: The time since Jan 1, 1970 UTC, in
14163 microseconds. The first 14 characters are the time in
14164 hex. The last 18 are random.
14165 {
14167 "uuids": [
14168 "04dd32b3af699659b6db9486a9c58c62",
14169 "04dd32b3af69bb1c2ac7ebfee0a50d88",
14170 "04dd32b3af69d8591b99a8e86a76e0fb",
14171 "04dd32b3af69f4a18a76efd89867f4f4",
14172 "04dd32b3af6a1f7925001274bbfde952",
14173 "04dd32b3af6a3fe8ea9b120ed906a57f",
14174 "04dd32b3af6a5b5c518809d3d4b76654",
14175 "04dd32b3af6a78f6ab32f1e928593c73",
14176 "04dd32b3af6a99916c665d6bbf857475",
14177 "04dd32b3af6ab558dd3f2c0afacb7d66"
14178 ]
14179 }
14180 · utc_id: The time since Jan 1, 1970 UTC, in microsec‐
14182 onds, plus the utc_id_suffix string. The first 14 char‐
14183 acters are the time in hex. The uuids/utc_id_suffix
14184 string value is appended to these.
14185 {
14187 "uuids": [
14188 "04dd32bd5eabcc@mycouch",
14189 "04dd32bd5eabee@mycouch",
14190 "04dd32bd5eac05@mycouch",
14191 "04dd32bd5eac28@mycouch",
14192 "04dd32bd5eac43@mycouch",
14193 "04dd32bd5eac58@mycouch",
14194 "04dd32bd5eac6e@mycouch",
14195 "04dd32bd5eac84@mycouch",
14196 "04dd32bd5eac98@mycouch",
14197 "04dd32bd5eacad@mycouch"
14198 ]
14199 }
14200 NOTE:
14202 Impact of UUID choices: the choice of UUID has a sig‐
14203 nificant impact on the layout of the B-tree, prior to
14204 compaction.
14205 For example, using a sequential UUID algorithm while
14207 uploading a large batch of documents will avoid the
14208 need to rewrite many intermediate B-tree nodes. A ran‐
14209 dom UUID algorithm may require rewriting intermediate
14210 nodes on a regular basis, resulting in significantly
14211 decreased throughput and wasted disk space space due
14212 to the append-only B-tree design.
14213 It is generally recommended to set your own UUIDs, or
14215 use the sequential algorithm unless you have a spe‐
14216 cific need and take into account the likely need for
14217 compaction to re-balance the B-tree and reclaim wasted
14218 space.
14219 utc_id_suffix
14221 New in version 1.3.
14222 The utc_id_suffix value will be appended to UUIDs gener‐
14225 ated by the utc_id algorithm. Replicating instances
14226 should have unique utc_id_suffix values to ensure unique‐
14227 ness of utc_id ids.
14228 [uuid]
14230 utc_id_suffix = my-awesome-suffix
14231 max_count
14233 New in version 1.5.1.
14234 No more than this number of UUIDs will be sent in a sin‐
14237 gle request. If more UUIDs are requested, an HTTP error
14238 response will be thrown.
14239 [uuid]
14241 max_count = 1000
14242 Vendor information
14244 [vendor]
14245 New in version 1.3.
14246 CouchDB distributors have the option of customizing CouchDB’s
14249 welcome message. This is returned when requesting GET /.
14250 [vendor]
14252 name = The Apache Software Foundation
14253 version = 1.5.0
14254 Content-Security-Policy
14256 [csp] Experimental support of CSP Headers for /_utils (Fauxton).
14257 enable Enable the sending of the Header Content-Security-Policy:
14259 [csp]
14261 enable = true
14262 header_value
14264 You can change the default value for the Header which is
14265 sent:
14266 [csp]
14268 header_value = default-src 'self'; img-src *; font-src *;
14269 Configuration of Database Purge
14271 [purge]
14272 max_document_id_number
14274 New in version 3.0.
14275 Sets the maximum number of documents allowed in a single
14278 purge request:
14279 [purge]
14281 max_document_id_number = 100
14282 max_revisions_number
14284 New in version 3.0.
14285 Sets the maximum number of accumulated revisions allowed
14288 in a single purge request:
14289 [purge]
14291 max_revisions_number = 1000
14292 index_lag_warn_seconds
14294 New in version 3.0.
14295 Sets the allowed duration when index is not updated for
14298 local purge checkpoint document. Default is 24 hours:
14299 [purge]
14301 index_lag_warn_seconds = 86400
14302 Resharding
14304 Resharding Configuration
14305 [resharding]
14306 max_jobs
14308 Maximum number of resharding jobs per cluster node. This
14309 includes completed, failed, and running jobs. If the job
14310 appears in the _reshard/jobs HTTP API results it will be
14311 counted towards the limit. When more than max_jobs jobs
14312 have been created, subsequent requests will start to fail
14313 with the max_jobs_exceeded error:
14314 [reshard]
14316 max_jobs = 48
14317 max_history
14319 Each resharding job maintains a timestamped event log.
14320 This setting limits the maximum size of that log:
14321 [reshard]
14323 max_history = 20
14324 max_retries
14326 How many times to retry shard splitting steps if they
14327 fail. For example, if indexing or topping off fails, it
14328 will be retried up to this many times before the whole
14329 resharding job fails:
14330 [reshard]
14332 max_retries = 1
14333 retry_interval_sec
14335 How long to wait between subsequent retries:
14336 [reshard]
14338 retry_interval_sec = 10
14339 delete_source
14341 Indicates if the source shard should be deleted after
14342 resharding has finished. By default, it is true as that
14343 would recover the space utilized by the shard. When
14344 debugging or when extra safety is required, this can be
14345 switched to false:
14346 [reshard]
14348 delete_source = true
14349 update_shard_map_timeout_sec
14351 How many seconds to wait for the shard map update opera‐
14352 tion to complete. If there is a large number of shard db
14353 changes waiting to finish replicating, it might be bene‐
14354 ficial to increase this timeout:
14355 [reshard]
14357 update_shard_map_timeout_sec = 60
14358 source_close_timeout_sec
14360 How many seconds to wait for the source shard to close.
14361 “Close” in this context means that client requests which
14362 keep the database open have all finished:
14363 [reshard]
14365 source_close_timeout_sec = 600
14366 require_node_param
14368 Require users to specify a node parameter when creating
14369 resharding jobs. This can be used as a safety check to
14370 avoid inadvertently starting too many resharding jobs by
14371 accident:
14372 [reshard]
14374 require_node_param = false
14375 require_range_param
14377 Require users to specify a range parameter when creating
14378 resharding jobs. This can be used as a safety check to
14379 avoid inadvertently starting too many resharding jobs by
14380 accident:
14381 [reshard]
14383 require_range_param = false
14384
14386 As of CouchDB 2.0.0, CouchDB can be run in two different modes of oper‐
14387 ation:
14388 · Standalone
14390 · Cluster
14392 This section details the theory behind CouchDB clusters, and provides
14394 specific operational instructions on node, database and shard manage‐
14395 ment.
14396 Theory
14398 Before we move on, we need some theory.
14399 As you see in etc/default.ini there is a section called [cluster]
14401 [cluster]
14403 q=2
14404 n=3
14405 · q - The number of shards.
14407 · n - The number of copies there is of every document. Replicas.
14409 When creating a database you can send your own values with request and
14411 thereby override the defaults in default.ini.
14412 In clustered operation, a quorum must be reached before CouchDB returns
14414 a 200 for a fetch, or 201 for a write operation. A quorum is defined as
14415 one plus half the number of “relevant copies”. “Relevant copies” is
14416 defined slightly differently for read and write operations.
14417 For read operations, the number of relevant copies is the number of
14419 currently-accessible shards holding the requested data, meaning that in
14420 the case of a failure or network partition, the number of relevant
14421 copies may be lower than the number of replicas in the cluster. The
14422 number of read copies can be set with the r parameter.
14423 For write operations the number of relevant copies is always n, the
14425 number of replicas in the cluster. For write operations, the number of
14426 copies can be set using the w parameter. If fewer than this number of
14427 nodes is available, a 202 will be returned.
14428 We will focus on the shards and replicas for now.
14430 A shard is a part of a database. It can be replicated multiple times.
14432 The more copies of a shard, the more you can scale out. If you have 4
14433 replicas, that means that all 4 copies of this specific shard will live
14434 on at most 4 nodes. With one replica you can have only one node, just
14435 as with CouchDB 1.x. No node can have more than one copy of each shard
14436 replica. The default for CouchDB since 3.0.0 is q=2 and n=3, meaning
14437 each database (and secondary index) is split into 2 shards, with 3
14438 replicas per shard, for a total of 6 shard replica files. For a CouchDB
14439 cluster only hosting a single database with these default values, a
14440 maximum of 6 nodes can be used to scale horizontally.
14441 Replicas add failure resistance, as some nodes can be offline without
14443 everything crashing down.
14444 · n=1 All nodes must be up.
14446 · n=2 Any 1 node can be down.
14448 · n=3 Any 2 nodes can be down.
14450 · etc
14452 Computers go down and sysadmins pull out network cables in a furious
14454 rage from time to time, so using n<2 is asking for downtime. Having too
14455 high a value of n adds servers and complexity without any real benefit.
14456 The sweet spot is at n=3.
14457 Say that we have a database with 3 replicas and 4 shards. That would
14459 give us a maximum of 12 nodes: 4*3=12.
14460 We can lose any 2 nodes and still read and write all documents.
14462 What happens if we lose more nodes? It depends on how lucky we are. As
14464 long as there is at least one copy of every shard online, we can read
14465 and write all documents.
14466 So, if we are very lucky then we can lose 8 nodes at maximum.
14468 Node Management
14470 Adding a node
14471 Go to http://server1:5984/_membership to see the name of the node and
14472 all the nodes it is connected to and knows about.
14473 curl -X GET "http://xxx.xxx.xxx.xxx:5984/_membership" --user admin-user
14475 {
14477 "all_nodes":[
14478 "node1@xxx.xxx.xxx.xxx"],
14479 "cluster_nodes":[
14480 "node1@xxx.xxx.xxx.xxx"]
14481 }
14482 · all_nodes are all the nodes thats this node knows about.
14484 · cluster_nodes are the nodes that are connected to this node.
14486 To add a node simply do:
14488 curl -X PUT "http://xxx.xxx.xxx.xxx/_node/_local/_nodes/node2@yyy.yyy.yyy.yyy" -d {}
14490 Now look at http://server1:5984/_membership again.
14492 {
14494 "all_nodes":[
14495 "node1@xxx.xxx.xxx.xxx",
14496 "node2@yyy.yyy.yyy.yyy"
14497 ],
14498 "cluster_nodes":[
14499 "node1@xxx.xxx.xxx.xxx",
14500 "node2@yyy.yyy.yyy.yyy"
14501 ]
14502 }
14503 And you have a 2 node cluster :)
14505 http://yyy.yyy.yyy.yyy:5984/_membership will show the same thing, so
14507 you only have to add a node once.
14508 Removing a node
14510 Before you remove a node, make sure that you have moved all shards away
14511 from that node.
14512 To remove node2 from server yyy.yyy.yyy.yyy, you need to first know the
14514 revision of the document that signifies that node’s existence:
14515 curl "http://xxx.xxx.xxx.xxx/_node/_local/_nodes/node2@yyy.yyy.yyy.yyy"
14517 {"_id":"node2@yyy.yyy.yyy.yyy","_rev":"1-967a00dff5e02add41820138abb3284d"}
14518 With that _rev, you can now proceed to delete the node document:
14520 curl -X DELETE "http://xxx.xxx.xxx.xxx/_node/_local/_nodes/node2@yyy.yyy.yyy.yyy?rev=1-967a00dff5e02add41820138abb3284d"
14522 Database Management
14524 Creating a database
14525 This will create a database with 3 replicas and 8 shards.
14526 curl -X PUT "http://xxx.xxx.xxx.xxx:5984/database-name?n=3&q=8" --user admin-user
14528 The database is in data/shards. Look around on all the nodes and you
14530 will find all the parts.
14531 If you do not specify n and q the default will be used. The default is
14533 3 replicas and 8 shards.
14534 Deleting a database
14536 curl -X DELETE "http://xxx.xxx.xxx.xxx:5984/database-name --user admin-user
14537 Placing a database on specific nodes
14539 In BigCouch, the predecessor to CouchDB 2.0’s clustering functionality,
14540 there was the concept of zones. CouchDB 2.0 carries this forward with
14541 cluster placement rules.
14542 WARNING:
14544 Use of the placement argument will override the standard logic for
14545 shard replica cardinality (specified by [cluster] n.)
14546 First, each node must be labeled with a zone attribute. This defines
14548 which zone each node is in. You do this by editing the node’s document
14549 in the system _nodes database, which is accessed node-local via the GET
14550 /_node/_local/_nodes/{node-name} endpoint.
14551 Add a key value pair of the form:
14553 "zone": "metro-dc-a"
14555 Do this for all of the nodes in your cluster.
14557 In your config file (local.ini or default.ini) on each node, define a
14559 consistent cluster-wide setting like:
14560 [cluster]
14562 placement = metro-dc-a:2,metro-dc-b:1
14563 In this example, it will ensure that two replicas for a shard will be
14565 hosted on nodes with the zone attribute set to metro-dc-a and one
14566 replica will be hosted on a new with the zone attribute set to
14567 metro-dc-b.
14568 Note that you can also use this system to ensure certain nodes in the
14570 cluster do not host any replicas for newly created databases, by giving
14571 them a zone attribute that does not appear in the [cluster] placement
14572 string.
14573 Shard Management
14575 Introduction
14576 This document discusses how sharding works in CouchDB along with how to
14577 safely add, move, remove, and create placement rules for shards and
14578 shard replicas.
14579 A shard is a horizontal partition of data in a database. Partitioning
14581 data into shards and distributing copies of each shard (called “shard
14582 replicas” or just “replicas”) to different nodes in a cluster gives the
14583 data greater durability against node loss. CouchDB clusters automati‐
14584 cally shard databases and distribute the subsets of documents that com‐
14585 pose each shard among nodes. Modifying cluster membership and sharding
14586 behavior must be done manually.
14587 Shards and Replicas
14589 How many shards and replicas each database has can be set at the global
14590 level, or on a per-database basis. The relevant parameters are q and n.
14591 q is the number of database shards to maintain. n is the number of
14593 copies of each document to distribute. The default value for n is 3,
14594 and for q is 2. With q=2, the database is split into 2 shards. With
14595 n=3, the cluster distributes three replicas of each shard. Altogether,
14596 that’s 6 shard replicas for a single database.
14597 In a 3-node cluster with q=8, each node would receive 8 shards. In a
14599 4-node cluster, each node would receive 6 shards. We recommend in the
14600 general case that the number of nodes in your cluster should be a mul‐
14601 tiple of n, so that shards are distributed evenly.
14602 CouchDB nodes have a etc/default.ini file with a section named cluster
14604 which looks like this:
14605 [cluster]
14607 q=2
14608 n=3
14609 These settings specify the default sharding parameters for newly cre‐
14611 ated databases. These can be overridden in the etc/local.ini file by
14612 copying the text above, and replacing the values with your new
14613 defaults. The values can also be set on a per-database basis by speci‐
14614 fying the q and n query parameters when the database is created. For
14615 example:
14616 $ curl -X PUT "$COUCH_URL:5984/database-name?q=4&n=2"
14618 This creates a database that is split into 4 shards and 2 replicas,
14620 yielding 8 shard replicas distributed throughout the cluster.
14621 Quorum
14623 Depending on the size of the cluster, the number of shards per data‐
14624 base, and the number of shard replicas, not every node may have access
14625 to every shard, but every node knows where all the replicas of each
14626 shard can be found through CouchDB’s internal shard map.
14627 Each request that comes in to a CouchDB cluster is handled by any one
14629 random coordinating node. This coordinating node proxies the request to
14630 the other nodes that have the relevant data, which may or may not
14631 include itself. The coordinating node sends a response to the client
14632 once a quorum of database nodes have responded; 2, by default. The
14633 default required size of a quorum is equal to r=w=((n+1)/2) where r
14634 refers to the size of a read quorum, w refers to the size of a write
14635 quorum, and n refers to the number of replicas of each shard. In a
14636 default cluster where n is 3, ((n+1)/2) would be 2.
14637 NOTE:
14639 Each node in a cluster can be a coordinating node for any one
14640 request. There are no special roles for nodes inside the cluster.
14641 The size of the required quorum can be configured at request time by
14643 setting the r parameter for document and view reads, and the w parame‐
14644 ter for document writes. For example, here is a request that directs
14645 the coordinating node to send a response once at least two nodes have
14646 responded:
14647 $ curl "$COUCH_URL:5984/{db}/{doc}?r=2"
14649 Here is a similar example for writing a document:
14651 $ curl -X PUT "$COUCH_URL:5984/{db}/{doc}?w=2" -d '{...}'
14653 Setting r or w to be equal to n (the number of replicas) means you will
14655 only receive a response once all nodes with relevant shards have
14656 responded or timed out, and as such this approach does not guarantee
14657 ACIDic consistency. Setting r or w to 1 means you will receive a
14658 response after only one relevant node has responded.
14659 Examining database shards
14661 There are a few API endpoints that help you understand how a database
14662 is sharded. Let’s start by making a new database on a cluster, and
14663 putting a couple of documents into it:
14664 $ curl -X PUT $COUCH_URL:5984/mydb
14666 {"ok":true}
14667 $ curl -X PUT $COUCH_URL:5984/mydb/joan -d '{"loves":"cats"}'
14668 {"ok":true,"id":"joan","rev":"1-cc240d66a894a7ee7ad3160e69f9051f"}
14669 $ curl -X PUT $COUCH_URL:5984/mydb/robert -d '{"loves":"dogs"}'
14670 {"ok":true,"id":"robert","rev":"1-4032b428c7574a85bc04f1f271be446e"}
14671 First, the top level api/db endpoint will tell you what the sharding
14673 parameters are for your database:
14674 $ curl -s $COUCH_URL:5984/db | jq .
14676 {
14677 "db_name": "mydb",
14678 ...
14679 "cluster": {
14680 "q": 8,
14681 "n": 3,
14682 "w": 2,
14683 "r": 2
14684 },
14685 ...
14686 }
14687 So we know this database was created with 8 shards (q=8), and each
14689 shard has 3 replicas (n=3) for a total of 24 shard replicas across the
14690 nodes in the cluster.
14691 Now, let’s see how those shard replicas are placed on the cluster with
14693 the api/db/shards endpoint:
14694 $ curl -s $COUCH_URL:5984/mydb/_shards | jq .
14696 {
14697 "shards": {
14698 "00000000-1fffffff": [
14699 "node1@127.0.0.1",
14700 "node2@127.0.0.1",
14701 "node4@127.0.0.1"
14702 ],
14703 "20000000-3fffffff": [
14704 "node1@127.0.0.1",
14705 "node2@127.0.0.1",
14706 "node3@127.0.0.1"
14707 ],
14708 "40000000-5fffffff": [
14709 "node2@127.0.0.1",
14710 "node3@127.0.0.1",
14711 "node4@127.0.0.1"
14712 ],
14713 "60000000-7fffffff": [
14714 "node1@127.0.0.1",
14715 "node3@127.0.0.1",
14716 "node4@127.0.0.1"
14717 ],
14718 "80000000-9fffffff": [
14719 "node1@127.0.0.1",
14720 "node2@127.0.0.1",
14721 "node4@127.0.0.1"
14722 ],
14723 "a0000000-bfffffff": [
14724 "node1@127.0.0.1",
14725 "node2@127.0.0.1",
14726 "node3@127.0.0.1"
14727 ],
14728 "c0000000-dfffffff": [
14729 "node2@127.0.0.1",
14730 "node3@127.0.0.1",
14731 "node4@127.0.0.1"
14732 ],
14733 "e0000000-ffffffff": [
14734 "node1@127.0.0.1",
14735 "node3@127.0.0.1",
14736 "node4@127.0.0.1"
14737 ]
14738 }
14739 }
14740 Now we see that there are actually 4 nodes in this cluster, and CouchDB
14742 has spread those 24 shard replicas evenly across all 4 nodes.
14743 We can also see exactly which shard contains a given document with the
14745 api/db/shards/doc endpoint:
14746 $ curl -s $COUCH_URL:5984/mydb/_shards/joan | jq .
14748 {
14749 "range": "e0000000-ffffffff",
14750 "nodes": [
14751 "node1@127.0.0.1",
14752 "node3@127.0.0.1",
14753 "node4@127.0.0.1"
14754 ]
14755 }
14756 $ curl -s $COUCH_URL:5984/mydb/_shards/robert | jq .
14757 {
14758 "range": "60000000-7fffffff",
14759 "nodes": [
14760 "node1@127.0.0.1",
14761 "node3@127.0.0.1",
14762 "node4@127.0.0.1"
14763 ]
14764 }
14765 CouchDB shows us the specific shard into which each of the two sample
14767 documents is mapped.
14768 Moving a shard
14770 When moving shards or performing other shard manipulations on the clus‐
14771 ter, it is advisable to stop all resharding jobs on the cluster. See
14772 Stopping Resharding Jobs for more details.
14773 This section describes how to manually place and replace shards. These
14775 activities are critical steps when you determine your cluster is too
14776 big or too small, and want to resize it successfully, or you have
14777 noticed from server metrics that database/shard layout is non-optimal
14778 and you have some “hot spots” that need resolving.
14779 Consider a three-node cluster with q=8 and n=3. Each database has 24
14781 shards, distributed across the three nodes. If you add a fourth node to
14782 the cluster, CouchDB will not redistribute existing database shards to
14783 it. This leads to unbalanced load, as the new node will only host
14784 shards for databases created after it joined the cluster. To balance
14785 the distribution of shards from existing databases, they must be moved
14786 manually.
14787 Moving shards between nodes in a cluster involves the following steps:
14789 0. Ensure the target node has joined the cluster.
14791 1. Copy the shard(s) and any secondary index shard(s) onto the target
14793 node.
14794 2. Set the target node to maintenance mode.
14796 3. Update cluster metadata to reflect the new target shard(s).
14798 4. Monitor internal replication to ensure up-to-date shard(s).
14800 5. Clear the target node’s maintenance mode.
14802 6. Update cluster metadata again to remove the source shard(s)
14804 7. Remove the shard file(s) and secondary index file(s) from the
14806 source node.
14807 Copying shard files
14809 NOTE:
14810 Technically, copying database and secondary index shards is
14811 optional. If you proceed to the next step without performing this
14812 data copy, CouchDB will use internal replication to populate the
14813 newly added shard replicas. However, copying files is faster than
14814 internal replication, especially on a busy cluster, which is why we
14815 recommend performing this manual data copy first.
14816 Shard files live in the data/shards directory of your CouchDB install.
14818 Within those subdirectories are the shard files themselves. For
14819 instance, for a q=8 database called abc, here is its database shard
14820 files:
14821 data/shards/00000000-1fffffff/abc.1529362187.couch
14823 data/shards/20000000-3fffffff/abc.1529362187.couch
14824 data/shards/40000000-5fffffff/abc.1529362187.couch
14825 data/shards/60000000-7fffffff/abc.1529362187.couch
14826 data/shards/80000000-9fffffff/abc.1529362187.couch
14827 data/shards/a0000000-bfffffff/abc.1529362187.couch
14828 data/shards/c0000000-dfffffff/abc.1529362187.couch
14829 data/shards/e0000000-ffffffff/abc.1529362187.couch
14830 Secondary indexes (including JavaScript views, Erlang views and Mango
14832 indexes) are also sharded, and their shards should be moved to save the
14833 new node the effort of rebuilding the view. View shards live in
14834 data/.shards. For example:
14835 data/.shards
14837 data/.shards/e0000000-ffffffff/_replicator.1518451591_design
14838 data/.shards/e0000000-ffffffff/_replicator.1518451591_design/mrview
14839 data/.shards/e0000000-ffffffff/_replicator.1518451591_design/mrview/3e823c2a4383ac0c18d4e574135a5b08.view
14840 data/.shards/c0000000-dfffffff
14841 data/.shards/c0000000-dfffffff/_replicator.1518451591_design
14842 data/.shards/c0000000-dfffffff/_replicator.1518451591_design/mrview
14843 data/.shards/c0000000-dfffffff/_replicator.1518451591_design/mrview/3e823c2a4383ac0c18d4e574135a5b08.view
14844 ...
14845 Since they are files, you can use cp, rsync, scp or other file-copying
14847 command to copy them from one node to another. For example:
14848 # one one machine
14850 $ mkdir -p data/.shards/{range}
14851 $ mkdir -p data/shards/{range}
14852 # on the other
14853 $ scp {couch-dir}/data/.shards/{range}/{database}.{datecode}* \
14854 {node}:{couch-dir}/data/.shards/{range}/
14855 $ scp {couch-dir}/data/shards/{range}/{database}.{datecode}.couch \
14856 {node}:{couch-dir}/data/shards/{range}/
14857 NOTE:
14859 Remember to move view files before database files! If a view index
14860 is ahead of its database, the database will rebuild it from scratch.
14861 Set the target node to true maintenance mode
14863 Before telling CouchDB about these new shards on the node, the node
14864 must be put into maintenance mode. Maintenance mode instructs CouchDB
14865 to return a 404 Not Found response on the /_up endpoint, and ensures it
14866 does not participate in normal interactive clustered requests for its
14867 shards. A properly configured load balancer that uses GET /_up to check
14868 the health of nodes will detect this 404 and remove the node from cir‐
14869 culation, preventing requests from being sent to that node. For exam‐
14870 ple, to configure HAProxy to use the /_up endpoint, use:
14871 http-check disable-on-404
14873 option httpchk GET /_up
14874 If you do not set maintenance mode, or the load balancer ignores this
14876 maintenance mode status, after the next step is performed the cluster
14877 may return incorrect responses when consulting the node in question.
14878 You don’t want this! In the next steps, we will ensure that this shard
14879 is up-to-date before allowing it to participate in end-user requests.
14880 To enable maintenance mode:
14882 $ curl -X PUT -H "Content-type: application/json" \
14884 $COUCH_URL:5984/_node/{node-name}/_config/couchdb/maintenance_mode \
14885 -d "\"true\""
14886 Then, verify that the node is in maintenance mode by performing a GET
14888 /_up on that node’s individual endpoint:
14889 $ curl -v $COUCH_URL/_up
14891 …
14892 < HTTP/1.1 404 Object Not Found
14893 …
14894 {"status":"maintenance_mode"}
14895 Finally, check that your load balancer has removed the node from the
14897 pool of available backend nodes.
14898 Updating cluster metadata to reflect the new target shard(s)
14900 Now we need to tell CouchDB that the target node (which must already be
14901 joined to the cluster) should be hosting shard replicas for a given
14902 database.
14903 To update the cluster metadata, use the special /_dbs database, which
14905 is an internal CouchDB database that maps databases to shards and
14906 nodes. This database is automatically replicated between nodes. It is
14907 accessible only through the special /_node/_local/_dbs endpoint.
14908 First, retrieve the database’s current metadata:
14910 $ curl http://localhost/_node/_local/_dbs/{name}
14912 {
14913 "_id": "{name}",
14914 "_rev": "1-e13fb7e79af3b3107ed62925058bfa3a",
14915 "shard_suffix": [46, 49, 53, 51, 48, 50, 51, 50, 53, 50, 54],
14916 "changelog": [
14917 ["add", "00000000-1fffffff", "node1@xxx.xxx.xxx.xxx"],
14918 ["add", "00000000-1fffffff", "node2@xxx.xxx.xxx.xxx"],
14919 ["add", "00000000-1fffffff", "node3@xxx.xxx.xxx.xxx"],
14920 …
14921 ],
14922 "by_node": {
14923 "node1@xxx.xxx.xxx.xxx": [
14924 "00000000-1fffffff",
14925 …
14926 ],
14927 …
14928 },
14929 "by_range": {
14930 "00000000-1fffffff": [
14931 "node1@xxx.xxx.xxx.xxx",
14932 "node2@xxx.xxx.xxx.xxx",
14933 "node3@xxx.xxx.xxx.xxx"
14934 ],
14935 …
14936 }
14937 }
14938 Here is a brief anatomy of that document:
14940 · _id: The name of the database.
14942 · _rev: The current revision of the metadata.
14944 · shard_suffix: A timestamp of the database’s creation, marked as sec‐
14946 onds after the Unix epoch mapped to the codepoints for ASCII numer‐
14947 als.
14948 · changelog: History of the database’s shards.
14950 · by_node: List of shards on each node.
14952 · by_range: On which nodes each shard is.
14954 To reflect the shard move in the metadata, there are three steps:
14956 1. Add appropriate changelog entries.
14958 2. Update the by_node entries.
14960 3. Update the by_range entries.
14962 WARNING:
14964 Be very careful! Mistakes during this process can irreparably cor‐
14965 rupt the cluster!
14966 As of this writing, this process must be done manually.
14968 To add a shard to a node, add entries like this to the database meta‐
14970 data’s changelog attribute:
14971 ["add", "{range}", "{node-name}"]
14973 The {range} is the specific shard range for the shard. The {node-name}
14975 should match the name and address of the node as displayed in GET
14976 /_membership on the cluster.
14977 NOTE:
14979 When removing a shard from a node, specify remove instead of add.
14980 Once you have figured out the new changelog entries, you will need to
14982 update the by_node and by_range to reflect who is storing what shards.
14983 The data in the changelog entries and these attributes must match. If
14984 they do not, the database may become corrupted.
14985 Continuing our example, here is an updated version of the metadata
14987 above that adds shards to an additional node called node4:
14988 {
14990 "_id": "{name}",
14991 "_rev": "1-e13fb7e79af3b3107ed62925058bfa3a",
14992 "shard_suffix": [46, 49, 53, 51, 48, 50, 51, 50, 53, 50, 54],
14993 "changelog": [
14994 ["add", "00000000-1fffffff", "node1@xxx.xxx.xxx.xxx"],
14995 ["add", "00000000-1fffffff", "node2@xxx.xxx.xxx.xxx"],
14996 ["add", "00000000-1fffffff", "node3@xxx.xxx.xxx.xxx"],
14997 ...
14998 ["add", "00000000-1fffffff", "node4@xxx.xxx.xxx.xxx"]
14999 ],
15000 "by_node": {
15001 "node1@xxx.xxx.xxx.xxx": [
15002 "00000000-1fffffff",
15003 ...
15004 ],
15005 ...
15006 "node4@xxx.xxx.xxx.xxx": [
15007 "00000000-1fffffff"
15008 ]
15009 },
15010 "by_range": {
15011 "00000000-1fffffff": [
15012 "node1@xxx.xxx.xxx.xxx",
15013 "node2@xxx.xxx.xxx.xxx",
15014 "node3@xxx.xxx.xxx.xxx",
15015 "node4@xxx.xxx.xxx.xxx"
15016 ],
15017 ...
15018 }
15019 }
15020 Now you can PUT this new metadata:
15022 $ curl -X PUT http://localhost/_node/_local/_dbs/{name} -d '{...}'
15024 Forcing synchronization of the shard(s)
15026 New in version 2.4.0.
15027 Whether you pre-copied shards to your new node or not, you can force
15030 CouchDB to synchronize all replicas of all shards in a database with
15031 the api/db/sync_shards endpoint:
15032 $ curl -X POST $COUCH_URL:5984/{db}/_sync_shards
15034 {"ok":true}
15035 This starts the synchronization process. Note that this will put addi‐
15037 tional load onto your cluster, which may affect performance.
15038 It is also possible to force synchronization on a per-shard basis by
15040 writing to a document that is stored within that shard.
15041 NOTE:
15043 Admins may want to bump their [mem3] sync_concurrency value to a
15044 larger figure for the duration of the shards sync.
15045 Monitor internal replication to ensure up-to-date shard(s)
15047 After you complete the previous step, CouchDB will have started syn‐
15048 chronizing the shards. You can observe this happening by monitoring the
15049 /_node/{node-name}/_system endpoint, which includes the internal_repli‐
15050 cation_jobs metric.
15051 Once this metric has returned to the baseline from before you started
15053 the shard sync, or is 0, the shard replica is ready to serve data and
15054 we can bring the node out of maintenance mode.
15055 Clear the target node’s maintenance mode
15057 You can now let the node start servicing data requests by putting
15058 "false" to the maintenance mode configuration endpoint, just as in step
15059 2.
15060 Verify that the node is not in maintenance mode by performing a GET
15062 /_up on that node’s individual endpoint.
15063 Finally, check that your load balancer has returned the node to the
15065 pool of available backend nodes.
15066 Update cluster metadata again to remove the source shard
15068 Now, remove the source shard from the shard map the same way that you
15069 added the new target shard to the shard map in step 2. Be sure to add
15070 the ["remove", {range}, {source-shard}] entry to the end of the
15071 changelog as well as modifying both the by_node and by_range sections
15072 of the database metadata document.
15073 Remove the shard and secondary index files from the source node
15075 Finally, you can remove the source shard replica by deleting its file
15076 from the command line on the source host, along with any view shard
15077 replicas:
15078 $ rm {couch-dir}/data/shards/{range}/{db}.{datecode}.couch
15080 $ rm -r {couch-dir}/data/.shards/{range}/{db}.{datecode}*
15081 Congratulations! You have moved a database shard replica. By adding and
15083 removing database shard replicas in this way, you can change the clus‐
15084 ter’s shard layout, also known as a shard map.
15085 Specifying database placement
15087 You can configure CouchDB to put shard replicas on certain nodes at
15088 database creation time using placement rules.
15089 WARNING:
15091 Use of the placement option will override the n option, both in the
15092 .ini file as well as when specified in a URL.
15093 First, each node must be labeled with a zone attribute. This defines
15095 which zone each node is in. You do this by editing the node’s document
15096 in the special /_nodes database, which is accessed through the special
15097 node-local API endpoint at /_node/_local/_nodes/{node-name}. Add a key
15098 value pair of the form:
15099 "zone": "{zone-name}"
15101 Do this for all of the nodes in your cluster. For example:
15103 $ curl -X PUT http://localhost/_node/_local/_nodes/{node-name} \
15105 -d '{ \
15106 "_id": "{node-name}",
15107 "_rev": "{rev}",
15108 "zone": "{zone-name}"
15109 }'
15110 In the local config file (local.ini) of each node, define a consistent
15112 cluster-wide setting like:
15113 [cluster]
15115 placement = {zone-name-1}:2,{zone-name-2}:1
15116 In this example, CouchDB will ensure that two replicas for a shard will
15118 be hosted on nodes with the zone attribute set to {zone-name-1} and one
15119 replica will be hosted on a new with the zone attribute set to
15120 {zone-name-2}.
15121 This approach is flexible, since you can also specify zones on a per-
15123 database basis by specifying the placement setting as a query parameter
15124 when the database is created, using the same syntax as the ini file:
15125 curl -X PUT $COUCH_URL:5984/{db}?zone={zone}
15127 The placement argument may also be specified. Note that this will over‐
15129 ride the logic that determines the number of created replicas!
15130 Note that you can also use this system to ensure certain nodes in the
15132 cluster do not host any replicas for newly created databases, by giving
15133 them a zone attribute that does not appear in the [cluster] placement
15134 string.
15135 Splitting Shards
15137 The api/server/reshard is an HTTP API for shard manipulation. Currently
15138 it only supports shard splitting. To perform shard merging, refer to
15139 the manual process outlined in the Merging Shards section.
15140 The main way to interact with api/server/reshard is to create reshard‐
15142 ing jobs, monitor those jobs, wait until they complete, remove them,
15143 post new jobs, and so on. What follows are a few steps one might take
15144 to use this API to split shards.
15145 At first, it’s a good idea to call GET /_reshard to see a summary of
15147 resharding on the cluster.
15148 $ curl -s $COUCH_URL:5984/_reshard | jq .
15150 {
15151 "state": "running",
15152 "state_reason": null,
15153 "completed": 3,
15154 "failed": 0,
15155 "running": 0,
15156 "stopped": 0,
15157 "total": 3
15158 }
15159 Two important things to pay attention to are the total number of jobs
15161 and the state.
15162 The state field indicates the state of resharding on the cluster. Nor‐
15164 mally it would be running, however, another user could have disabled
15165 resharding temporarily. Then, the state would be stopped and hopefully,
15166 there would be a reason or a comment in the value of the state_reason
15167 field. See Stopping Resharding Jobs for more details.
15168 The total number of jobs is important to keep an eye on because there
15170 is a maximum number of resharding jobs per node, and creating new jobs
15171 after the limit has been reached will result in an error. Before star‐
15172 ing new jobs it’s a good idea to remove already completed jobs. See
15173 reshard configuration section for the default value of max_jobs parame‐
15174 ter and how to adjust if needed.
15175 For example, if the jobs have completed, to remove all the jobs run:
15177 $ curl -s $COUCH_URL:5984/_reshard/jobs | jq -r '.jobs[].id' |\
15179 while read -r jobid; do\
15180 curl -s -XDELETE $COUCH_URL:5984/_reshard/jobs/$jobid\
15181 done
15182 Then it’s a good idea to see what the db shard map looks like.
15184 $ curl -s $COUCH_URL:5984/db1/_shards | jq '.'
15186 {
15187 "shards": {
15188 "00000000-7fffffff": [
15189 "node1@127.0.0.1",
15190 "node2@127.0.0.1",
15191 "node3@127.0.0.1"
15192 ],
15193 "80000000-ffffffff": [
15194 "node1@127.0.0.1",
15195 "node2@127.0.0.1",
15196 "node3@127.0.0.1"
15197 ]
15198 }
15199 }
15200 In this example we’ll split all the copies of the 00000000-7fffffff
15202 range. The API allows a combination of parameters such as: splitting
15203 all the ranges on all the nodes, all the ranges on just one node, or
15204 one particular range on one particular node. These are specified via
15205 the db, node and range job parameters.
15206 To split all the copies of 00000000-7fffffff we issue a request like
15208 this:
15209 $ curl -s -H "Content-type: application/json" -XPOST $COUCH_URL:5984/_reshard/jobs \
15211 -d '{"type": "split", "db":"db1", "range":"00000000-7fffffff"}' | jq '.'
15212 [
15213 {
15214 "ok": true,
15215 "id": "001-ef512cfb502a1c6079fe17e9dfd5d6a2befcc694a146de468b1ba5339ba1d134",
15216 "node": "node1@127.0.0.1",
15217 "shard": "shards/00000000-7fffffff/db1.1554242778"
15218 },
15219 {
15220 "ok": true,
15221 "id": "001-cec63704a7b33c6da8263211db9a5c74a1cb585d1b1a24eb946483e2075739ca",
15222 "node": "node2@127.0.0.1",
15223 "shard": "shards/00000000-7fffffff/db1.1554242778"
15224 },
15225 {
15226 "ok": true,
15227 "id": "001-fc72090c006d9b059d4acd99e3be9bb73e986d60ca3edede3cb74cc01ccd1456",
15228 "node": "node3@127.0.0.1",
15229 "shard": "shards/00000000-7fffffff/db1.1554242778"
15230 }
15231 ]
15232 The request returned three jobs, one job for each of the three copies.
15234 To check progress of these jobs use GET /_reshard/jobs or GET
15236 /_reshard/jobs/{jobid}.
15237 Eventually, these jobs should complete and the shard map should look
15239 like this:
15240 $ curl -s $COUCH_URL:5984/db1/_shards | jq '.'
15242 {
15243 "shards": {
15244 "00000000-3fffffff": [
15245 "node1@127.0.0.1",
15246 "node2@127.0.0.1",
15247 "node3@127.0.0.1"
15248 ],
15249 "40000000-7fffffff": [
15250 "node1@127.0.0.1",
15251 "node2@127.0.0.1",
15252 "node3@127.0.0.1"
15253 ],
15254 "80000000-ffffffff": [
15255 "node1@127.0.0.1",
15256 "node2@127.0.0.1",
15257 "node3@127.0.0.1"
15258 ]
15259 }
15260 }
15261 Stopping Resharding Jobs
15263 Resharding at the cluster level could be stopped and then restarted.
15264 This can be helpful to allow external tools which manipulate the shard
15265 map to avoid interfering with resharding jobs. To stop all resharding
15266 jobs on a cluster issue a PUT to /_reshard/state endpoint with the
15267 "state": "stopped" key and value. You can also specify an optional note
15268 or reason for stopping.
15269 For example:
15271 $ curl -s -H "Content-type: application/json" \
15273 -XPUT $COUCH_URL:5984/_reshard/state \
15274 -d '{"state": "stopped", "reason":"Moving some shards"}'
15275 {"ok": true}
15276 This state will then be reflected in the global summary:
15278 $ curl -s $COUCH_URL:5984/_reshard | jq .
15280 {
15281 "state": "stopped",
15282 "state_reason": "Moving some shards",
15283 "completed": 74,
15284 "failed": 0,
15285 "running": 0,
15286 "stopped": 0,
15287 "total": 74
15288 }
15289 To restart, issue a PUT request like above with running as the state.
15291 That should resume all the shard splitting jobs since their last check‐
15292 point.
15293 See the API reference for more details: api/server/reshard.
15295 Merging Shards
15297 The q value for a database can be set when the database is created or
15298 it can be increased later by splitting some of the shards Splitting
15299 Shards. In order to decrease q and merge some shards together, the
15300 database must be regenerated. Here are the steps:
15301 1. If there are running shard splitting jobs on the cluster, stop them
15303 via the HTTP API Stopping Resharding Jobs.
15304 2. Create a temporary database with the desired shard settings, by
15306 specifying the q value as a query parameter during the PUT opera‐
15307 tion.
15308 3. Stop clients accessing the database.
15310 4. Replicate the primary database to the temporary one. Multiple repli‐
15312 cations may be required if the primary database is under active use.
15313 5. Delete the primary database. Make sure nobody is using it!
15315 6. Recreate the primary database with the desired shard settings.
15317 7. Clients can now access the database again.
15319 8. Replicate the temporary back to the primary.
15321 9. Delete the temporary database.
15323 Once all steps have completed, the database can be used again. The
15325 cluster will create and distribute its shards according to placement
15326 rules automatically.
15327 Downtime can be avoided in production if the client application(s) can
15329 be instructed to use the new database instead of the old one, and a
15330 cut- over is performed during a very brief outage window.
15331 Clustered Purge
15333 The primary purpose of clustered purge is to clean databases that have
15334 multiple deleted tombstones or single documents that contain large num‐
15335 bers of conflicts. But it can also be used to purge any document
15336 (deleted or non-deleted) with any number of revisions.
15337 Clustered purge is designed to maintain eventual consistency and pre‐
15339 vent unnecessary invalidation of secondary indexes. For this, every
15340 database keeps track of a certain number of historical purges requested
15341 in the database, as well as its current purge_seq. Internal replica‐
15342 tions and secondary indexes process database’s purges and periodically
15343 update their corresponding purge checkpoint documents to report
15344 purge_seq processed by them. To ensure eventual consistency, the data‐
15345 base will remove stored historical purge requests only after they have
15346 been processed by internal replication jobs and secondary indexes.
15347 Internal Structures
15349 To enable internal replication of purge information between nodes and
15350 secondary indexes, two internal purge trees were added to a database
15351 file to track historical purges.
15352 purge_tree: UUID -> {PurgeSeq, DocId, Revs}
15354 purge_seq_tree: PurgeSeq -> {UUID, DocId, Revs}
15355 Each interactive request to _purge API, creates an ordered set of pairs
15357 on increasing purge_seq and purge_request, where purge_request is a
15358 tuple that contains docid and list of revisions. For each purge_request
15359 uuid is generated. A purge request is added to internal purge trees: a
15360 tuple {UUID -> {PurgeSeq, DocId, Revs}} is added to purge_tree, a tuple
15361 is {PurgeSeq -> {UUID, DocId, Revs}} added to purge_seq_tree.
15362 Compaction of Purges
15364 During the compaction of the database the oldest purge requests are to
15365 be removed to store only purged_infos_limit number of purges in the
15366 database. But in order to keep the database consistent with indexes
15367 and other replicas, we can only remove purge requests that have already
15368 been processed by indexes and internal replications jobs. Thus, occa‐
15369 sionally purge trees may store more than purged_infos_limit purges. If
15370 the number of stored purges in the database exceeds purged_infos_limit
15371 by a certain threshold, a warning is produced in logs signaling a prob‐
15372 lem of synchronization of database’s purges with indexes and other
15373 replicas.
15374 Local Purge Checkpoint Documents
15376 Indexes and internal replications of the database with purges create
15377 and periodically update local checkpoint purge documents:
15378 _local/purge-$type-$hash. These documents report the last purge_seq
15379 processed by them and the timestamp of the last processing. An example
15380 of a local checkpoint purge document:
15381 {
15383 "_id": "_local/purge-mrview-86cacdfbaf6968d4ebbc324dd3723fe7",
15384 "type": "mrview",
15385 "purge_seq": 10,
15386 "updated_on": 1540541874,
15387 "ddoc_id": "_design/foo",
15388 "signature": "5d10247925f826ae3e00966ec24b7bf6"
15389 }
15390 The below image shows possible local checkpoint documents that a data‐
15392 base may have.
15393 [image: Local Purge Checkpoint Documents] [image] Local Purge Check‐
15394 point Documents.UNINDENT
15395 Internal Replication
15397 Purge requests are replayed across all nodes in an eventually consis‐
15398 tent manner. Internal replication of purges consists of two steps:
15399 1. Pull replication. Internal replication first starts by pulling
15401 purges from target and applying them on source to make sure we don’t
15402 reintroduce to target source’s docs/revs that have been already purged
15403 on target. In this step, we use purge checkpoint documents stored on
15404 target to keep track of the last target’s purge_seq processed by the
15405 source. We find purge requests occurred after this purge_seq, and
15406 replay them on source. This step is done by updating the target’s
15407 checkpoint purge documents with the latest process purge_seq and time‐
15408 stamp.
15409 2. Push replication. Then internal replication proceeds as usual with
15411 an extra step inserted to push source’s purge requests to target. In
15412 this step, we use local internal replication checkpoint documents, that
15413 are updated both on target and source.
15414 Under normal conditions, an interactive purge request is already sent
15416 to every node containing a database shard’s replica, and applied on
15417 every replica. Internal replication of purges between nodes is just an
15418 extra step to ensure consistency between replicas, where all purge
15419 requests on one node are replayed on another node. In order not to
15420 replay the same purge request on a replica, each interactive purge
15421 request is tagged with a unique uuid. Internal replication filters out
15422 purge requests with UUIDs that already exist in the replica’s
15423 purge_tree, and applies only purge requests with UUIDs that don’t exist
15424 in the purge_tree. This is the reason why we needed to have two inter‐
15425 nal purge trees: 1) purge_tree: {UUID -> {PurgeSeq, DocId, Revs}}
15426 allows to quickly find purge requests with UUIDs that already exist in
15427 the replica; 2) purge_seq_tree: {PurgeSeq -> {UUID, DocId, Revs}}
15428 allows to iterate from a given purge_seq to collect all purge requests
15429 happened after this purge_seq.
15430 Indexes
15432 Each purge request will bump up update_seq of the database, so that
15433 each secondary index is also updated in order to apply the purge
15434 requests to maintain consistency within the main database.
15435 Config Settings
15437 These settings can be updated in the default.ini or local.ini:
15438 ┌──────────────────────┬─────────────────────┬─────────┐
15440 │Field │ Description │ Default │
15441 ├──────────────────────┼─────────────────────┼─────────┤
15442 │max_docu‐ │ Allowed maximum │ 100 │
15443 │ment_id_number │ number of documents │ │
15444 │ │ in one purge │ │
15445 │ │ request │ │
15446 ├──────────────────────┼─────────────────────┼─────────┤
15447 │max_revisions_num‐ │ Allowed maximum │ 1000 │
15448 │ber │ number of accumu‐ │ │
15449 │ │ lated revisions in │ │
15450 │ │ one purge request │ │
15451 ├──────────────────────┼─────────────────────┼─────────┤
15452 │allowed_purge_seq_lag │ Beside │ 100 │
15453 │ │ purged_infos_limit, │ │
15454 │ │ allowed additional │ │
15455 │ │ buffer to store │ │
15456 │ │ purge requests │ │
15457 ├──────────────────────┼─────────────────────┼─────────┤
15458 │index_lag_warn_sec‐ │ Allowed durations │ 86400 │
15459 │onds │ when index is not │ │
15460 │ │ updated for local │ │
15461 │ │ purge checkpoint │ │
15462 │ │ document │ │
15463 └──────────────────────┴─────────────────────┴─────────┘
15464 During a database compaction, we check all checkpoint purge docs. A
15466 client (an index or internal replication job) is allowed to have the
15467 last reported purge_seq to be smaller than the current database shard’s
15468 purge_seq by the value of (purged_infos_limit + allowed_purge_seq_lag).
15469 If the client’s purge_seq is even smaller, and the client has not
15470 checkpointed within index_lag_warn_seconds, it prevents compaction of
15471 purge trees and we have to issue the following log warning for this
15472 client:
15473 Purge checkpoint '_local/purge-mrview-9152d15c12011288629bcffba7693fd4’
15475 not updated in 86400 seconds in
15476 <<"shards/00000000-1fffffff/testdb12.1491979089">>
15477 If this type of log warning occurs, check the client to see why the
15479 processing of purge requests is stalled in it.
15480 There is a mapping relationship between a design document of indexes
15482 and local checkpoint docs. If a design document of indexes is updated
15483 or deleted, the corresponding local checkpoint document should be also
15484 automatically deleted. But in an unexpected case, when a design doc
15485 was updated/deleted, but its checkpoint document still exists in a
15486 database, the following warning will be issued:
15487 "Invalid purge doc '<<"_design/bar">>' on database
15489 <<"shards/00000000-1fffffff/testdb12.1491979089">>
15490 with purge_seq '50'"
15491 If this type of log warning occurs, remove the local purge doc from a
15493 database.
15494
15496 Compaction
15497 The compaction operation is the way to reduce disk space usage by
15498 removing unused and old data from database or view index files. This
15499 operation is very similar to the vacuum (SQLite ex.) operation avail‐
15500 able for other database management systems.
15501 During compaction of the target CouchDB creates new file with the .com‐
15503 pact extension and transfers only actual data into. Because of this,
15504 CouchDB checks first for the available disk space - it should be twice
15505 greater than the compacted file’s data.
15506 When all actual data is successfully transferred to the compacted file
15508 CouchDB replaces the target with the compacted file.
15509 Database Compaction
15511 Database compaction compresses the database file by removing unused
15512 file sections created during updates. Old documents revisions are
15513 replaced with small amount of metadata called tombstone which are used
15514 for conflicts resolution during replication. The number of stored revi‐
15515 sions (and their tombstones) can be configured by using the _revs_limit
15516 URL endpoint.
15517 Compaction can be manually triggered per database and runs as a back‐
15519 ground task. To start it for specific database there is need to send
15520 HTTP POST /{db}/_compact sub-resource of the target database:
15521 curl -H "Content-Type: application/json" -X POST http://localhost:5984/my_db/_compact
15523 On success, HTTP status 202 Accepted is returned immediately:
15525 HTTP/1.1 202 Accepted
15527 Cache-Control: must-revalidate
15528 Content-Length: 12
15529 Content-Type: text/plain; charset=utf-8
15530 Date: Wed, 19 Jun 2013 09:43:52 GMT
15531 Server: CouchDB (Erlang/OTP)
15532 {"ok":true}
15534 Although the request body is not used you must still specify
15536 Content-Type header with application/json value for the request. If you
15537 don’t, you will be aware about with HTTP status 415 Unsupported Media
15538 Type response:
15539 HTTP/1.1 415 Unsupported Media Type
15541 Cache-Control: must-revalidate
15542 Content-Length: 78
15543 Content-Type: application/json
15544 Date: Wed, 19 Jun 2013 09:43:44 GMT
15545 Server: CouchDB (Erlang/OTP)
15546 {"error":"bad_content_type","reason":"Content-Type must be application/json"}
15548 When the compaction is successful started and running it is possible to
15550 get information about it via database information resource:
15551 curl http://localhost:5984/my_db
15553 HTTP/1.1 200 OK
15555 Cache-Control: must-revalidate
15556 Content-Length: 246
15557 Content-Type: application/json
15558 Date: Wed, 19 Jun 2013 16:51:20 GMT
15559 Server: CouchDB (Erlang/OTP)
15560 {
15562 "committed_update_seq": 76215,
15563 "compact_running": true,
15564 "db_name": "my_db",
15565 "disk_format_version": 6,
15566 "doc_count": 5091,
15567 "doc_del_count": 0,
15568 "instance_start_time": "0",
15569 "purge_seq": 0,
15570 "sizes": {
15571 "active": 3787996,
15572 "disk": 17703025,
15573 "external": 4763321
15574 },
15575 "update_seq": 76215
15576 }
15577 Note that compaction_running field is true indicating that compaction
15579 is actually running. To track the compaction progress you may query the
15580 _active_tasks resource:
15581 curl http://localhost:5984/_active_tasks
15583 HTTP/1.1 200 OK
15585 Cache-Control: must-revalidate
15586 Content-Length: 175
15587 Content-Type: application/json
15588 Date: Wed, 19 Jun 2013 16:27:23 GMT
15589 Server: CouchDB (Erlang/OTP)
15590 [
15592 {
15593 "changes_done": 44461,
15594 "database": "my_db",
15595 "pid": "<0.218.0>",
15596 "progress": 58,
15597 "started_on": 1371659228,
15598 "total_changes": 76215,
15599 "type": "database_compaction",
15600 "updated_on": 1371659241
15601 }
15602 ]
15603 Views Compaction
15605 Views are also need compaction like databases, unlike databases views
15606 are compacted by groups per design document. To start their compaction
15607 there is need to send HTTP POST /{db}/_compact/{ddoc} request:
15608 curl -H "Content-Type: application/json" -X POST http://localhost:5984/dbname/_compact/designname
15610 {"ok":true}
15612 This compacts the view index from the current version of the specified
15614 design document. The HTTP response code is 202 Accepted (like
15615 compaction for databases) and a compaction background task will be cre‐
15616 ated.
15617 Views cleanup
15619 View indexes on disk are named after their MD5 hash of the view defini‐
15620 tion. When you change a view, old indexes remain on disk. To clean up
15621 all outdated view indexes (files named after the MD5 representation of
15622 views, that does not exist anymore) you can trigger a view cleanup:
15623 curl -H "Content-Type: application/json" -X POST http://localhost:5984/dbname/_view_cleanup
15625 {"ok":true}
15627 Automatic Compaction
15629 CouchDB’s automatic compaction daemon, internally known as “smoosh”,
15630 will trigger compaction jobs for both databases and views based on con‐
15631 figurable thresholds for the sparseness of a file and the total amount
15632 of space that can be recovered.
15633 Channels
15635 Smoosh works using the concept of channels. A channel is essentially a
15636 queue of pending compactions. There are separate sets of active chan‐
15637 nels for databases and views. Each channel is assigned a configuration
15638 which defines whether a compaction ends up in the channel’s queue and
15639 how compactions are prioritized within that queue.
15640 Smoosh takes each channel and works through the compactions queued in
15642 each in priority order. Each channel is processed concurrently, so the
15643 priority levels only matter within a given channel. Each channel has an
15644 assigned number of active compactions, which defines how many com‐
15645 pactions happen for that channel in parallel. For example, a cluster
15646 with a lot of database churn but few views might require more active
15647 compactions in the database channel(s).
15648 It’s important to remember that a channel is local to a CouchDB node;
15650 that is, each node maintains and processes an independent set of com‐
15651 pactions. Channels are defined as either “ratio” channels or “slack”
15652 channels, depending on the type of algorithm used for prioritization:
15653 · Ratio: uses the ratio of sizes.file / sizes.active as its driving
15655 calculation. The result X must be greater than some configurable
15656 value Y for a compaction to be added to the queue. Compactions are
15657 then prioritised for higher values of X.
15658 · Slack: uses the difference of sizes.file - sizes.active as its driv‐
15660 ing calculation. The result X must be greater than some configurable
15661 value Y for a compaction to be added to the queue. Compactions are
15662 prioritised for higher values of X.
15663 In both cases, Y is set using the min_priority configuration variable.
15665 CouchDB ships with four channels pre-configured: one channel of each
15666 type for databases, and another one for views.
15667 Channel Configuration
15669 Channels are defined using [smoosh.<channel_name>] configuration
15670 blocks, and activated by naming the channel in the db_channels or
15671 view_channels configuration setting in the [smoosh] block. The default
15672 configuration is
15673 [smoosh]
15675 db_channels = upgrade_dbs,ratio_dbs,slack_dbs
15676 view_channels = upgrade_views,ratio_views,slack_views
15677 [smoosh.ratio_dbs]
15679 priority = ratio
15680 min_priority = 2.0
15681 [smoosh.ratio_views]
15683 priority = ratio
15684 min_priority = 2.0
15685 [smoosh.slack_dbs]
15687 priority = slack
15688 min_priority = 16777216
15689 [smoosh.slack_views]
15691 priority = slack
15692 min_priority = 16777216
15693 The “upgrade” channels are a special pair of channels that only check
15695 whether the disk_format_version for the file matches the current ver‐
15696 sion, and enqueue the file for compaction (which has the side effect of
15697 upgrading the file format) if that’s not the case. There are several
15698 additional properties that can be configured for each channel; these
15699 are documented in the configuration API
15700 Scheduling Windows
15702 Each compaction channel can be configured to run only during certain
15703 hours of the day. The channel-specific from, to, and strict_window con‐
15704 figuration settings control this behavior. For example
15705 [smoosh.overnight_channel]
15707 from = 20:00
15708 to = 06:00
15709 strict_window = true
15710 where overnight_channel is the name of the channel you want to config‐
15712 ure.
15713 Note: CouchDB determines time via the UTC (GMT) timezone, so these set‐
15715 tings must be expressed as UTC (GMT).
15716 The strict_window setting will cause the compaction daemon to suspend
15718 all active compactions in this channel when exiting the window, and
15719 resume them when re-entering. If strict_window is left at its default
15720 of false, the active compactions will be allowed to complete but no new
15721 compactions will be started.
15722 Migration Guide
15724 Previous versions of CouchDB shipped with a simpler compaction daemon.
15725 The configuration system for the new daemon is not backwards-compatible
15726 with the old one, so users with customized compaction configurations
15727 will need to port them to the new setup. The old daemon’s compaction
15728 rules configuration looked like
15729 [compaction_daemon]
15731 min_file_size = 131072
15732 check_interval = 3600
15733 snooze_period_ms = 3000
15734 [compactions]
15736 mydb = [{db_fragmentation, "70%"}, {view_fragmentation, "60%"}, {parallel_view_compaction, true}]
15737 _default = [{db_fragmentation, "50%"}, {view_fragmentation, "55%"}, {from, "20:00"}, {to, "06:00"}, {strict_window, true}]
15738 Many of the elements of this configuration can be ported over to the
15740 new system. Examining each in detail:
15741 · min_file_size is now configured on a per-channel basis using the
15743 min_size config setting.
15744 · db_fragmentation is equivalent to configuring a priority = ratio
15746 channel with min_priority set to 1.0 / (1 - db_fragmentation/100) and
15747 then listing that channel in the [smoosh] db_channels config setting.
15748 · view_fragmention is likewise equivalent to configuring a priority =
15750 ratio channel with min_priority set to 1.0 / (1 - view_fragmenta‐
15751 tion/100) and then listing that channel in the [smoosh] view_channels
15752 config setting.
15753 · from / to / strict_window: each of these settings can be applied on a
15755 per-channel basis in the new daemon. The one behavior change is that
15756 the new daemon will suspend compactions upon exiting the allowed win‐
15757 dow instead of canceling them outright, and resume them when
15758 re-entering.
15759 · parallel_view_compaction: each compaction channel has a concurrency
15761 setting that controls how many compactions will execute in parallel
15762 in that channel. The total parallelism is the sum of the concurrency
15763 settings of all active channels. This is a departure from the previ‐
15764 ous behavior, in which the daemon would only focus on one database
15765 and/or its views (depending on the value of this flag) at a time.
15766 The check_interval and snooze_period_ms settings are obsolete in the
15768 event-driven design of the new daemon. The new daemon does not support
15769 setting database-specific thresholds as in the mydb setting above.
15770 Rather, channels can be configured to focus on specific classes of
15771 files: large databases, small view indexes, and so on. Most cases of
15772 named database compaction rules can be expressed using properties of
15773 those databases and/or their associated views.
15774 Performance
15776 With up to tens of thousands of documents you will generally find
15777 CouchDB to perform well no matter how you write your code. Once you
15778 start getting into the millions of documents you need to be a lot more
15779 careful.
15780 Disk I/O
15782 File Size
15783 The smaller your file size, the less I/O operations there will be, the
15784 more of the file can be cached by CouchDB and the operating system, the
15785 quicker it is to replicate, backup etc. Consequently you should care‐
15786 fully examine the data you are storing. For example it would be silly
15787 to use keys that are hundreds of characters long, but your program
15788 would be hard to maintain if you only used single character keys. Care‐
15789 fully consider data that is duplicated by putting it in views.
15790 Disk and File System Performance
15792 Using faster disks, striped RAID arrays and modern file systems can all
15793 speed up your CouchDB deployment. However, there is one option that can
15794 increase the responsiveness of your CouchDB server when disk perfor‐
15795 mance is a bottleneck. From the Erlang documentation for the file mod‐
15796 ule:
15797 On operating systems with thread support, it is possible to let file
15798 operations be performed in threads of their own, allowing other
15799 Erlang processes to continue executing in parallel with the file
15800 operations. See the command line flag +A in erl(1).
15801 Setting this argument to a number greater than zero can keep your
15803 CouchDB installation responsive even during periods of heavy disk uti‐
15804 lization. The easiest way to set this option is through the ERL_FLAGS
15805 environment variable. For example, to give Erlang four threads with
15806 which to perform I/O operations add the following to (pre‐
15807 fix)/etc/defaults/couchdb (or equivalent):
15808 export ERL_FLAGS="+A 4"
15810 System Resource Limits
15812 One of the problems that administrators run into as their deployments
15813 become large are resource limits imposed by the system and by the
15814 application configuration. Raising these limits can allow your deploy‐
15815 ment to grow beyond what the default configuration will support.
15816 CouchDB Configuration Options
15818 max_dbs_open
15819 In your configuration (local.ini or similar) familiarize yourself with
15820 the couchdb/max_dbs_open:
15821 [couchdb]
15823 max_dbs_open = 100
15824 This option places an upper bound on the number of databases that can
15826 be open at one time. CouchDB reference counts database accesses inter‐
15827 nally and will close idle databases when it must. Sometimes it is nec‐
15828 essary to keep more than the default open at once, such as in deploy‐
15829 ments where many databases will be continuously replicating.
15830 Erlang
15832 Even if you’ve increased the maximum connections CouchDB will allow,
15833 the Erlang runtime system will not allow more than 65536 connections by
15834 default. Adding the following directive to (prefix)/etc/vm.args (or
15835 equivalent) will increase this limit (in this case to 102400):
15836 +Q 102400
15838 Note that on Windows, Erlang will not actually increase the file
15840 descriptor limit past 8192 (i.e. the system header–defined value of
15841 FD_SETSIZE). On macOS, the limit may be as low as 1024. See this tip
15842 for a possible workaround and this thread for a deeper explanation.
15843 Maximum open file descriptors (ulimit)
15845 In general, modern UNIX-like systems can handle very large numbers of
15846 file handles per process (e.g. 100000) without problem. Don’t be afraid
15847 to increase this limit on your system.
15848 The method of increasing these limits varies, depending on your init
15850 system and particular OS release. The default value for many OSes is
15851 1024 or 4096. On a system with many databases or many views, CouchDB
15852 can very rapidly hit this limit.
15853 For systemd-based Linuxes (such as CentOS/RHEL 7, Ubuntu 16.04+, Debian
15855 8 or newer), assuming you are launching CouchDB from systemd, you must
15856 override the upper limit via editing the override file. The best prac‐
15857 tice for this is via the systemctl edit couchdb command. Add these
15858 lines to the file in the editor:
15859 [Service]
15861 LimitNOFILE=65536
15862 …or whatever value you like. To increase this value higher than 65536,
15864 you must also add the Erlang +Q parameter to your etc/vm.args file by
15865 adding the line:
15866 +Q 102400
15868 The old ERL_MAX_PORTS environment variable is ignored by the version of
15870 Erlang supplied with CouchDB.
15871 If your system is set up to use the Pluggable Authentication Modules (‐
15873 PAM), and you are not launching CouchDB from systemd, increasing this
15874 limit is straightforward. For example, creating a file named /etc/secu‐
15875 rity/limits.d/100-couchdb.conf with the following contents will ensure
15876 that CouchDB can open up to 65536 file descriptors at once:
15877 #<domain> <type> <item> <value>
15879 couchdb hard nofile 65536
15880 couchdb soft nofile 65536
15881 If you are using our Debian/Ubuntu sysvinit script
15883 (/etc/init.d/couchdb), you also need to raise the limits for the root
15884 user:
15885 #<domain> <type> <item> <value>
15887 root hard nofile 65536
15888 root soft nofile 65536
15889 You may also have to edit the /etc/pam.d/common-session and
15891 /etc/pam.d/common-session-noninteractive files to add the line:
15892 session required pam_limits.so
15894 if it is not already present.
15896 If your system does not use PAM, a ulimit command is usually available
15898 for use in a custom script to launch CouchDB with increased resource
15899 limits. Typical syntax would be something like ulimit -n 65536.
15900 Network
15902 There is latency overhead making and receiving each request/response.
15903 In general you should do your requests in batches. Most APIs have some
15904 mechanism to do batches, usually by supplying lists of documents or
15905 keys in the request body. Be careful what size you pick for the
15906 batches. The larger batch requires more time your client has to spend
15907 encoding the items into JSON and more time is spent decoding that num‐
15908 ber of responses. Do some benchmarking with your own configuration and
15909 typical data to find the sweet spot. It is likely to be between one
15910 and ten thousand documents.
15911 If you have a fast I/O system then you can also use concurrency - have
15913 multiple requests/responses at the same time. This mitigates the
15914 latency involved in assembling JSON, doing the networking and decoding
15915 JSON.
15916 As of CouchDB 1.1.0, users often report lower write performance of doc‐
15918 uments compared to older releases. The main reason is that this release
15919 ships with the more recent version of the HTTP server library MochiWeb,
15920 which by default sets the TCP socket option SO_NODELAY to false. This
15921 means that small data sent to the TCP socket, like the reply to a docu‐
15922 ment write request (or reading a very small document), will not be sent
15923 immediately to the network - TCP will buffer it for a while hoping that
15924 it will be asked to send more data through the same socket and then
15925 send all the data at once for increased performance. This TCP buffer‐
15926 ing behaviour can be disabled via httpd/socket_options:
15927 [httpd]
15929 socket_options = [{nodelay, true}]
15930 SEE ALSO:
15932 Bulk load and store API.
15933 Connection limit
15935 MochiWeb handles CouchDB requests. The default maximum number of con‐
15936 nections is 2048. To change this limit, use the server_options configu‐
15937 ration variable. max indicates maximum number of connections.
15938 [chttpd]
15940 server_options = [{backlog, 128}, {acceptor_pool_size, 16}, {max, 4096}]
15941 CouchDB
15943 DELETE operation
15944 When you DELETE a document the database will create a new revision
15945 which contains the _id and _rev fields as well as the _deleted flag.
15946 This revision will remain even after a database compaction so that the
15947 deletion can be replicated. Deleted documents, like non-deleted docu‐
15948 ments, can affect view build times, PUT and DELETE request times, and
15949 the size of the database since they increase the size of the B+Tree.
15950 You can see the number of deleted documents in database information. If
15951 your use case creates lots of deleted documents (for example, if you
15952 are storing short-term data like log entries, message queues, etc), you
15953 might want to periodically switch to a new database and delete the old
15954 one (once the entries in it have all expired).
15955 Document’s ID
15957 The db file size is derived from your document and view sizes but also
15958 on a multiple of your _id sizes. Not only is the _id present in the
15959 document, but it and parts of it are duplicated in the binary tree
15960 structure CouchDB uses to navigate the file to find the document in the
15961 first place. As a real world example for one user switching from 16
15962 byte ids to 4 byte ids made a database go from 21GB to 4GB with 10 mil‐
15963 lion documents (the raw JSON text when from 2.5GB to 2GB).
15964 Inserting with sequential (and at least sorted) ids is faster than ran‐
15966 dom ids. Consequently you should consider generating ids yourself,
15967 allocating them sequentially and using an encoding scheme that consumes
15968 fewer bytes. For example, something that takes 16 hex digits to repre‐
15969 sent can be done in 4 base 62 digits (10 numerals, 26 lower case, 26
15970 upper case).
15971 Views
15973 Views Generation
15974 Views with the JavaScript query server are extremely slow to generate
15975 when there are a non-trivial number of documents to process. The gener‐
15976 ation process won’t even saturate a single CPU let alone your I/O. The
15977 cause is the latency involved in the CouchDB server and separate
15978 couchjs query server, dramatically indicating how important it is to
15979 take latency out of your implementation.
15980 You can let view access be “stale” but it isn’t practical to determine
15982 when that will occur giving you a quick response and when views will be
15983 updated which will take a long time. (A 10 million document database
15984 took about 10 minutes to load into CouchDB but about 4 hours to do view
15985 generation).
15986 In a cluster, “stale” requests are serviced by a fixed set of shards in
15988 order to present users with consistent results between requests. This
15989 comes with an availability trade-off - the fixed set of shards might
15990 not be the most responsive / available within the cluster. If you don’t
15991 need this kind of consistency (e.g. your indexes are relatively
15992 static), you can tell CouchDB to use any available replica by specify‐
15993 ing stable=false&update=false instead of stale=ok, or sta‐
15994 ble=false&update=lazy instead of stale=update_after.
15995 View information isn’t replicated - it is rebuilt on each database so
15997 you can’t do the view generation on a separate sever.
15998 Built-In Reduce Functions
16000 If you’re using a very simple view function that only performs a sum or
16001 count reduction, you can call native Erlang implementations of them by
16002 simply writing _sum or _count in place of your function declaration.
16003 This will speed up things dramatically, as it cuts down on IO between
16004 CouchDB and the JavaScript query server. For example, as mentioned on
16005 the mailing list, the time for outputting an (already indexed and
16006 cached) view with about 78,000 items went down from 60 seconds to 4
16007 seconds.
16008 Before:
16010 {
16012 "_id": "_design/foo",
16013 "views": {
16014 "bar": {
16015 "map": "function (doc) { emit(doc.author, 1); }",
16016 "reduce": "function (keys, values, rereduce) { return sum(values); }"
16017 }
16018 }
16019 }
16020 After:
16022 {
16024 "_id": "_design/foo",
16025 "views": {
16026 "bar": {
16027 "map": "function (doc) { emit(doc.author, 1); }",
16028 "reduce": "_sum"
16029 }
16030 }
16031 }
16032 SEE ALSO:
16034 reducefun/builtin
16035 Backing up CouchDB
16037 CouchDB has three different types of files it can create during run‐
16038 time:
16039 · Database files (including secondary indexes)
16041 · Configuration files (*.ini)
16043 · Log files (if configured to log to disk)
16045 Below are strategies for ensuring consistent backups of all of these
16047 files.
16048 Database Backups
16050 The simplest and easiest approach for CouchDB backup is to use CouchDB
16051 replication to another CouchDB installation. You can choose between
16052 normal (one-shot) or continuous replications depending on your need.
16053 However, you can also copy the actual .couch files from the CouchDB
16055 data directory (by default, data/) at any time, without problem.
16056 CouchDB’s append-only storage format for both databases and secondary
16057 indexes ensures that this will work without issue.
16058 To ensure reliability of backups, it is recommended that you back up
16060 secondary indexes (stored under data/.shards) prior to backing up the
16061 main database files (stored under data/shards as well as the sys‐
16062 tem-level databases at the parent data/ directory). This is because
16063 CouchDB will automatically handle views/secondary indexes that are
16064 slightly out of date by updating them on the next read access, but
16065 views or secondary indexes that are newer than their associated data‐
16066 bases will trigger a full rebuild of the index. This can be a very
16067 costly and time-consuming operation, and can impact your ability to
16068 recover quickly in a disaster situation.
16069 On supported operating systems/storage environments, you can also make
16071 use of storage snapshots. These have the advantage of being
16072 near-instantaneous when working with block storage systems such as ZFS
16073 or LVM or Amazon EBS. When using snapshots at the block-storage level,
16074 be sure to quiesce the file system with an OS-level utility such as
16075 Linux’s fsfreeze if necessary. If unsure, consult your operating sys‐
16076 tem’s or cloud provider’s documentation for more detail.
16077 Configuration Backups
16079 CouchDB’s configuration system stores data in .ini files under the con‐
16080 figuration directory (by default, etc/). If changes are made to the
16081 configuration at runtime, the very last file in the configuration chain
16082 will be updated with the changes.
16083 Simple back up the entire etc/ directory to ensure a consistent config‐
16085 uration after restoring from backup.
16086 If no changes to the configuration are made at runtime through the HTTP
16088 API, and all configuration files are managed by a configuration manage‐
16089 ment system (such as Ansible or Chef), there is no need to backup the
16090 configuration directory.
16091 Log Backups
16093 If configured to log to a file, you may want to back up the log files
16094 written by CouchDB. Any backup solution for these files works.
16095 Under UNIX-like systems, if using log rotation software, a
16097 copy-then-truncate approach is necessary. This will truncate the origi‐
16098 nal log file to zero size in place after creating a copy. CouchDB does
16099 not recognize any signal to be told to close its log file and create a
16100 new one. Because of this, and because of differences in how file han‐
16101 dles function, there is no straightforward log rotation solution under
16102 Microsoft Windows other than periodic restarts of the CouchDB process.
16103
16105 Fauxton Setup
16106 Fauxton is included with CouchDB 2.0, so make sure CouchDB is running,
16107 then go to:
16108 http://127.0.0.1:5984/_utils/
16110 You can also upgrade to the latest version of Fauxton by using npm:
16112 $ npm install -g fauxton
16114 $ fauxton
16115 (Recent versions of node.js and npm are required.)
16117 Fauxton Visual Guide
16119 You can find the Visual Guide here:
16120 http://couchdb.apache.org/fauxton-visual-guide
16121 Development Server
16123 Recent versions of node.js and npm are required.
16124 Using the dev server is the easiest way to use Fauxton, specially when
16126 developing for it:
16127 $ git clone https://github.com/apache/couchdb-fauxton.git
16129 $ npm install && npm run dev
16130 Understanding Fauxton Code layout
16132 Each bit of functionality is its own separate module or addon.
16133 All core modules are stored under app/module and any addons that are
16135 optional are under app/addons.
16136 We use backbone.js and Backbone.layoutmanager quite heavily, so best to
16138 get an idea how they work. Its best at this point to read through a
16139 couple of the modules and addons to get an idea of how they work.
16140 Two good starting points are app/addon/config and app/modules/data‐
16142 bases.
16143 Each module must have a base.js file, this is read and compile when
16145 Fauxton is deployed.
16146 The resource.js file is usually for your Backbone.Models and Back‐
16148 bone.Collections, view.js for your Backbone.Views.
16149 The routes.js is used to register a url path for your view along with
16151 what layout, data, breadcrumbs and api point is required for the view.
16152 ToDo items
16154 Checkout JIRA or GitHub Issues for a list of items to do.
16155
16157 This is a list of experimental features in CouchDB. They are included
16158 in a release because the development team is requesting feedback from
16159 the larger developer community. As such, please play around with these
16160 features and send us feedback, thanks!
16161 Use at your own risk! Do not rely on these features for critical appli‐
16163 cations.
16164 Content-Security-Policy (CSP) Header Support for /_utils (Fauxton)
16166 This will just work with Fauxton. You can enable it in your config: you
16167 can enable the feature in general and change the default header that is
16168 sent for everything in /_utils.
16169 [csp]
16171 enable = true
16172 Then restart CouchDB.
16174 Have fun!
16176
16178 The components of the API URL path help determine the part of the
16179 CouchDB server that is being accessed. The result is the structure of
16180 the URL request both identifies and effectively describes the area of
16181 the database you are accessing.
16182 As with all URLs, the individual components are separated by a forward
16184 slash.
16185 As a general rule, URL components and JSON fields starting with the _
16187 (underscore) character represent a special component or entity within
16188 the server or returned object. For example, the URL fragment /_all_dbs
16189 gets a list of all of the databases in a CouchDB instance.
16190 This reference is structured according to the URL structure, as below.
16192 API Basics
16194 The CouchDB API is the primary method of interfacing to a CouchDB
16195 instance. Requests are made using HTTP and requests are used to
16196 request information from the database, store new data, and perform
16197 views and formatting of the information stored within the documents.
16198 Requests to the API can be categorised by the different areas of the
16200 CouchDB system that you are accessing, and the HTTP method used to send
16201 the request. Different methods imply different operations, for example
16202 retrieval of information from the database is typically handled by the
16203 GET operation, while updates are handled by either a POST or PUT
16204 request. There are some differences between the information that must
16205 be supplied for the different methods. For a guide to the basic HTTP
16206 methods and request structure, see Request Format and Responses.
16207 For nearly all operations, the submitted data, and the returned data
16209 structure, is defined within a JavaScript Object Notation (JSON)
16210 object. Basic information on the content and data types for JSON are
16211 provided in JSON Basics.
16212 Errors when accessing the CouchDB API are reported using standard HTTP
16214 Status Codes. A guide to the generic codes returned by CouchDB are pro‐
16215 vided in HTTP Status Codes.
16216 When accessing specific areas of the CouchDB API, specific information
16218 and examples on the HTTP methods and request, JSON structures, and
16219 error codes are provided.
16220 Request Format and Responses
16222 CouchDB supports the following HTTP request methods:
16223 · GET
16225 Request the specified item. As with normal HTTP requests, the format
16227 of the URL defines what is returned. With CouchDB this can include
16228 static items, database documents, and configuration and statistical
16229 information. In most cases the information is returned in the form of
16230 a JSON document.
16231 · HEAD
16233 The HEAD method is used to get the HTTP header of a GET request with‐
16235 out the body of the response.
16236 · POST
16238 Upload data. Within CouchDB POST is used to set values, including
16240 uploading documents, setting document values, and starting certain
16241 administration commands.
16242 · PUT
16244 Used to put a specified resource. In CouchDB PUT is used to create
16246 new objects, including databases, documents, views and design docu‐
16247 ments.
16248 · DELETE
16250 Deletes the specified resource, including documents, views, and
16252 design documents.
16253 · COPY
16255 A special method that can be used to copy documents and objects.
16257 If you use an unsupported HTTP request type with an URL that does not
16259 support the specified type then a 405 - Method Not Allowed will be
16260 returned, listing the supported HTTP methods. For example:
16261 {
16263 "error":"method_not_allowed",
16264 "reason":"Only GET,HEAD allowed"
16265 }
16266 HTTP Headers
16268 Because CouchDB uses HTTP for all communication, you need to ensure
16269 that the correct HTTP headers are supplied (and processed on retrieval)
16270 so that you get the right format and encoding. Different environments
16271 and clients will be more or less strict on the effect of these HTTP
16272 headers (especially when not present). Where possible you should be as
16273 specific as possible.
16274 Request Headers
16276 · Accept
16277 Specifies the list of accepted data types to be returned by the
16279 server (i.e. that are accepted/understandable by the client). The
16280 format should be a list of one or more MIME types, separated by
16281 colons.
16282 For the majority of requests the definition should be for JSON data
16284 (application/json). For attachments you can either specify the MIME
16285 type explicitly, or use */* to specify that all file types are sup‐
16286 ported. If the Accept header is not supplied, then the */* MIME type
16287 is assumed (i.e. client accepts all formats).
16288 The use of Accept in queries for CouchDB is not required, but is
16290 highly recommended as it helps to ensure that the data returned can
16291 be processed by the client.
16292 If you specify a data type using the Accept header, CouchDB will
16294 honor the specified type in the Content-type header field returned.
16295 For example, if you explicitly request application/json in the Accept
16296 of a request, the returned HTTP headers will use the value in the
16297 returned Content-type field.
16298 For example, when sending a request without an explicit Accept
16300 header, or when specifying */*:
16301 GET /recipes HTTP/1.1
16303 Host: couchdb:5984
16304 Accept: */*
16305 The returned headers are:
16307 HTTP/1.1 200 OK
16309 Server: CouchDB (Erlang/OTP)
16310 Date: Thu, 13 Jan 2011 13:39:34 GMT
16311 Content-Type: text/plain;charset=utf-8
16312 Content-Length: 227
16313 Cache-Control: must-revalidate
16314 NOTE:
16316 The returned content type is text/plain even though the informa‐
16317 tion returned by the request is in JSON format.
16318 Explicitly specifying the Accept header:
16320 GET /recipes HTTP/1.1
16322 Host: couchdb:5984
16323 Accept: application/json
16324 The headers returned include the application/json content type:
16326 HTTP/1.1 200 OK
16328 Server: CouchDB (Erlang/OTP)
16329 Date: Thu, 13 Jan 2013 13:40:11 GMT
16330 Content-Type: application/json
16331 Content-Length: 227
16332 Cache-Control: must-revalidate
16333 · Content-type
16335 Specifies the content type of the information being supplied within
16337 the request. The specification uses MIME type specifications. For the
16338 majority of requests this will be JSON (application/json). For some
16339 settings the MIME type will be plain text. When uploading attachments
16340 it should be the corresponding MIME type for the attachment or binary
16341 (application/octet-stream).
16342 The use of the Content-type on a request is highly recommended.
16344 Response Headers
16346 Response headers are returned by the server when sending back content
16347 and include a number of different header fields, many of which are
16348 standard HTTP response header and have no significance to CouchDB oper‐
16349 ation. The list of response headers important to CouchDB are listed
16350 below.
16351 · Cache-control
16353 The cache control HTTP response header provides a suggestion for
16355 client caching mechanisms on how to treat the returned information.
16356 CouchDB typically returns the must-revalidate, which indicates that
16357 the information should be revalidated if possible. This is used to
16358 ensure that the dynamic nature of the content is correctly updated.
16359 · Content-length
16361 The length (in bytes) of the returned content.
16363 · Content-type
16365 Specifies the MIME type of the returned data. For most request, the
16367 returned MIME type is text/plain. All text is encoded in Unicode
16368 (UTF-8), and this is explicitly stated in the returned Content-type,
16369 as text/plain;charset=utf-8.
16370 · Etag
16372 The Etag HTTP header field is used to show the revision for a docu‐
16374 ment, or a view.
16375 ETags have been assigned to a map/reduce group (the collection of
16377 views in a single design document). Any change to any of the indexes
16378 for those views would generate a new ETag for all view URLs in a sin‐
16379 gle design doc, even if that specific view’s results had not changed.
16380 Each _view URL has its own ETag which only gets updated when changes
16382 are made to the database that effect that index. If the index for
16383 that specific view does not change, that view keeps the original ETag
16384 head (therefore sending back 304 - Not Modified more often).
16385 · Transfer-Encoding
16387 If the response uses an encoding, then it is specified in this header
16389 field.
16390 Transfer-Encoding: chunked means that the response is sent in parts,
16392 a method known as chunked transfer encoding. This is used when
16393 CouchDB does not know beforehand the size of the data it will send
16394 (for example, the changes feed).
16395 JSON Basics
16397 The majority of requests and responses to CouchDB use the JavaScript
16398 Object Notation (JSON) for formatting the content and structure of the
16399 data and responses.
16400 JSON is used because it is the simplest and easiest solution for work‐
16402 ing with data within a web browser, as JSON structures can be evaluated
16403 and used as JavaScript objects within the web browser environment. JSON
16404 also integrates with the server-side JavaScript used within CouchDB.
16405 JSON supports the same basic types as supported by JavaScript, these
16407 are:
16408 · Array - a list of values enclosed in square brackets. For example:
16410 ["one", "two", "three"]
16412 · Boolean - a true or false value. You can use these strings directly.
16414 For example:
16415 { "value": true}
16417 · Number - an integer or floating-point number.
16419 · Object - a set of key/value pairs (i.e. an associative array, or
16421 hash). The key must be a string, but the value can be any of the sup‐
16422 ported JSON values. For example:
16423 {
16425 "servings" : 4,
16426 "subtitle" : "Easy to make in advance, and then cook when ready",
16427 "cooktime" : 60,
16428 "title" : "Chicken Coriander"
16429 }
16430 In CouchDB, the JSON object is used to represent a variety of struc‐
16432 tures, including the main CouchDB document.
16433 · String - this should be enclosed by double-quotes and supports Uni‐
16435 code characters and backslash escaping. For example:
16436 "A String"
16438 Parsing JSON into a JavaScript object is supported through the
16440 JSON.parse() function in JavaScript, or through various libraries that
16441 will perform the parsing of the content into a JavaScript object for
16442 you. Libraries for parsing and generating JSON are available in many
16443 languages, including Perl, Python, Ruby, Erlang and others.
16444 WARNING:
16446 Care should be taken to ensure that your JSON structures are valid,
16447 invalid structures will cause CouchDB to return an HTTP status code
16448 of 500 (server error).
16449 Number Handling
16451 Developers and users new to computer handling of numbers often
16452 encounter surprises when expecting that a number stored in JSON format
16453 does not necessarily return as the same number as compared character by
16454 character.
16455 Any numbers defined in JSON that contain a decimal point or exponent
16457 will be passed through the Erlang VM’s idea of the “double” data type.
16458 Any numbers that are used in views will pass through the view server’s
16459 idea of a number (the common JavaScript case means even integers pass
16460 through a double due to JavaScript’s definition of a number).
16461 Consider this document that we write to CouchDB:
16463 {
16465 "_id":"30b3b38cdbd9e3a587de9b8122000cff",
16466 "number": 1.1
16467 }
16468 Now let’s read that document back from CouchDB:
16470 {
16472 "_id":"30b3b38cdbd9e3a587de9b8122000cff",
16473 "_rev":"1-f065cee7c3fd93aa50f6c97acde93030",
16474 "number":1.1000000000000000888
16475 }
16476 What happens is CouchDB is changing the textual representation of the
16478 result of decoding what it was given into some numerical format. In
16479 most cases this is an IEEE 754 double precision floating point number
16480 which is exactly what almost all other languages use as well.
16481 What Erlang does a bit differently than other languages is that it does
16483 not attempt to pretty print the resulting output to use the shortest
16484 number of characters. For instance, this is why we have this relation‐
16485 ship:
16486 ejson:encode(ejson:decode(<<"1.1">>)).
16488 <<"1.1000000000000000888">>
16489 What can be confusing here is that internally those two formats decode
16491 into the same IEEE-754 representation. And more importantly, it will
16492 decode into a fairly close representation when passed through all major
16493 parsers that we know about.
16494 While we’ve only been discussing cases where the textual representation
16496 changes, another important case is when an input value contains more
16497 precision than can actually represented in a double. (You could argue
16498 that this case is actually “losing” data if you don’t accept that num‐
16499 bers are stored in doubles).
16500 Here’s a log for a couple of the more common JSON libraries that happen
16502 to be on the author’s machine:
16503 Ejson (CouchDB’s current parser) at CouchDB sha 168a663b:
16505 $ ./utils/run -i
16507 Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:2:2] [rq:2]
16508 [async-threads:4] [hipe] [kernel-poll:true]
16509 Eshell V5.8.5 (abort with ^G)
16511 1> ejson:encode(ejson:decode(<<"1.01234567890123456789012345678901234567890">>)).
16512 <<"1.0123456789012346135">>
16513 2> F = ejson:encode(ejson:decode(<<"1.01234567890123456789012345678901234567890">>)).
16514 <<"1.0123456789012346135">>
16515 3> ejson:encode(ejson:decode(F)).
16516 <<"1.0123456789012346135">>
16517 Node:
16519 $ node -v
16521 v0.6.15
16522 $ node
16523 JSON.stringify(JSON.parse("1.01234567890123456789012345678901234567890"))
16524 '1.0123456789012346'
16525 var f = JSON.stringify(JSON.parse("1.01234567890123456789012345678901234567890"))
16526 undefined
16527 JSON.stringify(JSON.parse(f))
16528 '1.0123456789012346'
16529 Python:
16531 $ python
16533 Python 2.7.2 (default, Jun 20 2012, 16:23:33)
16534 [GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)] on darwin
16535 Type "help", "copyright", "credits" or "license" for more information.
16536 import json
16537 json.dumps(json.loads("1.01234567890123456789012345678901234567890"))
16538 '1.0123456789012346'
16539 f = json.dumps(json.loads("1.01234567890123456789012345678901234567890"))
16540 json.dumps(json.loads(f))
16541 '1.0123456789012346'
16542 Ruby:
16544 $ irb --version
16546 irb 0.9.5(05/04/13)
16547 require 'JSON'
16548 => true
16549 JSON.dump(JSON.load("[1.01234567890123456789012345678901234567890]"))
16550 => "[1.01234567890123]"
16551 f = JSON.dump(JSON.load("[1.01234567890123456789012345678901234567890]"))
16552 => "[1.01234567890123]"
16553 JSON.dump(JSON.load(f))
16554 => "[1.01234567890123]"
16555 NOTE:
16557 A small aside on Ruby, it requires a top level object or array, so I
16558 just wrapped the value. Should be obvious it doesn’t affect the
16559 result of parsing the number though.
16560 Spidermonkey:
16562 $ js -h 2>&1 | head -n 1
16564 JavaScript-C 1.8.5 2011-03-31
16565 $ js
16566 js> JSON.stringify(JSON.parse("1.01234567890123456789012345678901234567890"))
16567 "1.0123456789012346"
16568 js> var f = JSON.stringify(JSON.parse("1.01234567890123456789012345678901234567890"))
16569 js> JSON.stringify(JSON.parse(f))
16570 "1.0123456789012346"
16571 As you can see they all pretty much behave the same except for Ruby
16573 actually does appear to be losing some precision over the other
16574 libraries.
16575 The astute observer will notice that ejson (the CouchDB JSON library)
16577 reported an extra three digits. While its tempting to think that this
16578 is due to some internal difference, its just a more specific case of
16579 the 1.1 input as described above.
16580 The important point to realize here is that a double can only hold a
16582 finite number of values. What we’re doing here is generating a string
16583 that when passed through the “standard” floating point parsing algo‐
16584 rithms (ie, strtod) will result in the same bit pattern in memory as we
16585 started with. Or, slightly different, the bytes in a JSON serialized
16586 number are chosen such that they refer to a single specific value that
16587 a double can represent.
16588 The important point to understand is that we’re mapping from one infi‐
16590 nite set onto a finite set. An easy way to see this is by reflecting on
16591 this:
16592 1.0 == 1.00 == 1.000 = 1.(infinite zeros)
16594 Obviously a computer can’t hold infinite bytes so we have to decimate
16596 our infinitely sized set to a finite set that can be represented con‐
16597 cisely.
16598 The game that other JSON libraries are playing is merely:
16600 “How few characters do I have to use to select this specific value for
16602 a double”
16603 And that game has lots and lots of subtle details that are difficult to
16605 duplicate in C without a significant amount of effort (it took Python
16606 over a year to get it sorted with their fancy build systems that auto‐
16607 matically run on a number of different architectures).
16608 Hopefully we’ve shown that CouchDB is not doing anything “funky” by
16610 changing input. Its behaving the same as any other common JSON library
16611 does, its just not pretty printing its output.
16612 On the other hand, if you actually are in a position where an IEEE-754
16614 double is not a satisfactory data type for your numbers, then the
16615 answer as has been stated is to not pass your numbers through this rep‐
16616 resentation. In JSON this is accomplished by encoding them as a string
16617 or by using integer types (although integer types can still bite you if
16618 you use a platform that has a different integer representation than
16619 normal, ie, JavaScript).
16620 Further information can be found easily, including the Floating Point
16622 Guide, and David Goldberg’s Reference.
16623 Also, if anyone is really interested in changing this behavior, we’re
16625 all ears for contributions to jiffy (which is theoretically going to
16626 replace ejson when we get around to updating the build system). The
16627 places we’ve looked for inspiration are TCL and Python. If you know a
16628 decent implementation of this float printing algorithm give us a
16629 holler.
16630 HTTP Status Codes
16632 With the interface to CouchDB working through HTTP, error codes and
16633 statuses are reported using a combination of the HTTP status code num‐
16634 ber, and corresponding data in the body of the response data.
16635 A list of the error codes returned by CouchDB, and generic descriptions
16637 of the related errors are provided below. The meaning of different sta‐
16638 tus codes for specific request types are provided in the corresponding
16639 API call reference.
16640 · 200 - OK
16642 Request completed successfully.
16644 · 201 - Created
16646 Document created successfully.
16648 · 202 - Accepted
16650 Request has been accepted, but the corresponding operation may not
16652 have completed. This is used for background operations, such as data‐
16653 base compaction.
16654 · 304 - Not Modified
16656 The additional content requested has not been modified. This is used
16658 with the ETag system to identify the version of information returned.
16659 · 400 - Bad Request
16661 Bad request structure. The error can indicate an error with the
16663 request URL, path or headers. Differences in the supplied MD5 hash
16664 and content also trigger this error, as this may indicate message
16665 corruption.
16666 · 401 - Unauthorized
16668 The item requested was not available using the supplied authoriza‐
16670 tion, or authorization was not supplied.
16671 · 403 - Forbidden
16673 The requested item or operation is forbidden.
16675 · 404 - Not Found
16677 The requested content could not be found. The content will include
16679 further information, as a JSON object, if available. The structure
16680 will contain two keys, error and reason. For example:
16681 {"error":"not_found","reason":"no_db_file"}
16683 · 405 - Method Not Allowed
16685 A request was made using an invalid HTTP request type for the URL
16687 requested. For example, you have requested a PUT when a POST is
16688 required. Errors of this type can also triggered by invalid URL
16689 strings.
16690 · 406 - Not Acceptable
16692 The requested content type is not supported by the server.
16694 · 409 - Conflict
16696 Request resulted in an update conflict.
16698 · 412 - Precondition Failed
16700 The request headers from the client and the capabilities of the
16702 server do not match.
16703 · 413 - Request Entity Too Large
16705 A document exceeds the configured couchdb/max_document_size value or
16707 the entire request exceeds the httpd/max_http_request_size value.
16708 · 415 - Unsupported Media Type
16710 The content types supported, and the content type of the information
16712 being requested or submitted indicate that the content type is not
16713 supported.
16714 · 416 - Requested Range Not Satisfiable
16716 The range specified in the request header cannot be satisfied by the
16718 server.
16719 · 417 - Expectation Failed
16721 When sending documents in bulk, the bulk load operation failed.
16723 · 500 - Internal Server Error
16725 The request was invalid, either because the supplied JSON was
16727 invalid, or invalid information was supplied as part of the request.
16728 Server
16730 The CouchDB server interface provides the basic interface to a CouchDB
16731 server for obtaining CouchDB information and getting and setting con‐
16732 figuration information.
16733 /
16735 GET / Accessing the root of a CouchDB instance returns meta informa‐
16736 tion about the instance. The response is a JSON structure con‐
16737 taining information about the server, including a welcome mes‐
16738 sage and the version of the server.
16739 Request Headers
16741 · Accept – .INDENT 2.0
16743 · application/json
16745 · text/plain
16747 Response Headers
16750 · Content-Type – .INDENT 2.0
16752 · application/json
16754 · text/plain; charset=utf-8
16756 Status Codes
16759 · 200 OK – Request completed successfully
16761 Request:
16763 GET / HTTP/1.1
16765 Accept: application/json
16766 Host: localhost:5984
16767 Response:
16769 HTTP/1.1 200 OK
16771 Cache-Control: must-revalidate
16772 Content-Length: 179
16773 Content-Type: application/json
16774 Date: Sat, 10 Aug 2013 06:33:33 GMT
16775 Server: CouchDB (Erlang/OTP)
16776 {
16778 "couchdb": "Welcome",
16779 "uuid": "85fb71bf700c17267fef77535820e371",
16780 "vendor": {
16781 "name": "The Apache Software Foundation",
16782 "version": "1.3.1"
16783 },
16784 "version": "1.3.1"
16785 }
16786 /_active_tasks
16788 Changed in version 2.1.0: Because of how the scheduling replicator
16789 works, continuous replication jobs could be periodically stopped and
16790 then started later. When they are not running they will not appear in
16791 the _active_tasks endpoint
16792 GET /_active_tasks
16795 List of running tasks, including the task type, name, status and
16796 process ID. The result is a JSON array of the currently running
16797 tasks, with each task being described with a single object.
16798 Depending on operation type set of response object fields might
16799 be different.
16800 Request Headers
16802 · Accept – .INDENT 2.0
16804 · application/json
16806 · text/plain
16808 Response Headers
16811 · Content-Type – .INDENT 2.0
16813 · application/json
16815 · text/plain; charset=utf-8
16817 Response JSON Object
16820 · changes_done (number) – Processed changes
16822 · database (string) – Source database
16824 · pid (string) – Process ID
16826 · progress (number) – Current percentage progress
16828 · started_on (number) – Task start time as unix timestamp
16830 · status (string) – Task status message
16832 · task (string) – Task name
16834 · total_changes (number) – Total changes to process
16836 · type (string) – Operation Type
16838 · updated_on (number) – Unix timestamp of last operation update
16840 Status Codes
16842 · 200 OK – Request completed successfully
16844 · 401 Unauthorized – CouchDB Server Administrator privileges
16846 required
16847 Request:
16849 GET /_active_tasks HTTP/1.1
16851 Accept: application/json
16852 Host: localhost:5984
16853 Response:
16855 HTTP/1.1 200 OK
16857 Cache-Control: must-revalidate
16858 Content-Length: 1690
16859 Content-Type: application/json
16860 Date: Sat, 10 Aug 2013 06:37:31 GMT
16861 Server: CouchDB (Erlang/OTP)
16862 [
16864 {
16865 "changes_done": 64438,
16866 "database": "mailbox",
16867 "pid": "<0.12986.1>",
16868 "progress": 84,
16869 "started_on": 1376116576,
16870 "total_changes": 76215,
16871 "type": "database_compaction",
16872 "updated_on": 1376116619
16873 },
16874 {
16875 "changes_done": 14443,
16876 "database": "mailbox",
16877 "design_document": "c9753817b3ba7c674d92361f24f59b9f",
16878 "pid": "<0.10461.3>",
16879 "progress": 18,
16880 "started_on": 1376116621,
16881 "total_changes": 76215,
16882 "type": "indexer",
16883 "updated_on": 1376116650
16884 },
16885 {
16886 "changes_done": 5454,
16887 "database": "mailbox",
16888 "design_document": "_design/meta",
16889 "pid": "<0.6838.4>",
16890 "progress": 7,
16891 "started_on": 1376116632,
16892 "total_changes": 76215,
16893 "type": "indexer",
16894 "updated_on": 1376116651
16895 },
16896 {
16897 "checkpointed_source_seq": 68585,
16898 "continuous": false,
16899 "doc_id": null,
16900 "doc_write_failures": 0,
16901 "docs_read": 4524,
16902 "docs_written": 4524,
16903 "missing_revisions_found": 4524,
16904 "pid": "<0.1538.5>",
16905 "progress": 44,
16906 "replication_id": "9bc1727d74d49d9e157e260bb8bbd1d5",
16907 "revisions_checked": 4524,
16908 "source": "mailbox",
16909 "source_seq": 154419,
16910 "started_on": 1376116644,
16911 "target": "http://mailsrv:5984/mailbox",
16912 "type": "replication",
16913 "updated_on": 1376116651
16914 }
16915 ]
16916 /_all_dbs
16918 GET /_all_dbs
16919 Returns a list of all the databases in the CouchDB instance.
16920 Request Headers
16922 · Accept – .INDENT 2.0
16924 · application/json
16926 · text/plain
16928 Query Parameters
16931 · descending (boolean) – Return the databases in descending
16933 order by key. Default is false.
16934 · endkey (json) – Stop returning databases when the specified
16936 key is reached.
16937 · end_key (json) – Alias for endkey param
16939 · limit (number) – Limit the number of the returned databases to
16941 the specified number.
16942 · skip (number) – Skip this number of databases before starting
16944 to return the results. Default is 0.
16945 · startkey (json) – Return databases starting with the specified
16947 key.
16948 · start_key (json) – Alias for startkey.
16950 Response Headers
16952 · Content-Type – .INDENT 2.0
16954 · application/json
16956 · text/plain; charset=utf-8
16958 Status Codes
16961 · 200 OK – Request completed successfully
16963 Request:
16965 GET /_all_dbs HTTP/1.1
16967 Accept: application/json
16968 Host: localhost:5984
16969 Response:
16971 HTTP/1.1 200 OK
16973 Cache-Control: must-revalidate
16974 Content-Length: 52
16975 Content-Type: application/json
16976 Date: Sat, 10 Aug 2013 06:57:48 GMT
16977 Server: CouchDB (Erlang/OTP)
16978 [
16980 "_users",
16981 "contacts",
16982 "docs",
16983 "invoices",
16984 "locations"
16985 ]
16986 /_dbs_info
16988 New in version 2.2.
16989 POST /_dbs_info
16992 Returns information of a list of the specified databases in the
16993 CouchDB instance. This enables you to request information about
16994 multiple databases in a single request, in place of multiple GET
16995 /{db} requests.
16996 Request Headers
16998 · Accept – .INDENT 2.0
17000 · application/json
17002 Response Headers
17005 · Content-Type – .INDENT 2.0
17007 · application/json
17009 Request JSON Object
17012 · keys (array) – Array of database names to be requested
17014 Status Codes
17016 · 200 OK – Request completed successfully
17018 · 400 Bad Request – Missing keys or exceeded keys in request
17020 Request:
17022 POST /_dbs_info HTTP/1.1
17024 Accept: application/json
17025 Host: localhost:5984
17026 Content-Type: application/json
17027 {
17029 "keys": [
17030 "animals",
17031 "plants"
17032 ]
17033 }
17034 Response:
17036 HTTP/1.1 200 OK
17038 Cache-Control: must-revalidate
17039 Content-Type: application/json
17040 Date: Sat, 20 Dec 2017 06:57:48 GMT
17041 Server: CouchDB (Erlang/OTP)
17042 [
17044 {
17045 "key": "animals",
17046 "info": {
17047 "db_name": "animals",
17048 "update_seq": "52232",
17049 "sizes": {
17050 "file": 1178613587,
17051 "external": 1713103872,
17052 "active": 1162451555
17053 },
17054 "purge_seq": 0,
17055 "doc_del_count": 0,
17056 "doc_count": 52224,
17057 "disk_format_version": 6,
17058 "compact_running": false,
17059 "cluster": {
17060 "q": 8,
17061 "n": 3,
17062 "w": 2,
17063 "r": 2
17064 },
17065 "instance_start_time": "0"
17066 }
17067 },
17068 {
17069 "key": "plants",
17070 "info": {
17071 "db_name": "plants",
17072 "update_seq": "303",
17073 "sizes": {
17074 "file": 3872387,
17075 "external": 2339,
17076 "active": 67475
17077 },
17078 "purge_seq": 0,
17079 "doc_del_count": 0,
17080 "doc_count": 11,
17081 "disk_format_version": 6,
17082 "compact_running": false,
17083 "cluster": {
17084 "q": 8,
17085 "n": 3,
17086 "w": 2,
17087 "r": 2
17088 },
17089 "instance_start_time": "0"
17090 }
17091 }
17092 ]
17093 NOTE:
17095 The supported number of the specified databases in the list can be
17096 limited by modifying the max_db_number_for_dbs_info_req entry in
17097 configuration file. The default limit is 100.
17098 /_cluster_setup
17100 New in version 2.0.
17101 GET /_cluster_setup
17104 Returns the status of the node or cluster, per the cluster setup
17105 wizard.
17106 Request Headers
17108 · Accept – .INDENT 2.0
17110 · application/json
17112 · text/plain
17114 Query Parameters
17117 · ensure_dbs_exist (array) – List of system databases to ensure
17119 exist on the node/cluster. Defaults to ["_users","_replica‐
17120 tor"].
17121 Response Headers
17123 · Content-Type – .INDENT 2.0
17125 · application/json
17127 · text/plain; charset=utf-8
17129 Response JSON Object
17132 · state (string) – Current state of the node and/or cluster (see
17134 below)
17135 Status Codes
17137 · 200 OK – Request completed successfully
17139 The state returned indicates the current node or cluster state, and is
17141 one of the following:
17142 · cluster_disabled: The current node is completely unconfigured.
17144 · single_node_disabled: The current node is configured as a sin‐
17146 gle (standalone) node ([cluster] n=1), but either does not
17147 have a server-level admin user defined, or does not have the
17148 standard system databases created. If the ensure_dbs_exist
17149 query parameter is specified, the list of databases provided
17150 overrides the default list of standard system databases.
17151 · single_node_enabled: The current node is configured as a sin‐
17153 gle (standalone) node, has a server-level admin user defined,
17154 and has the ensure_dbs_exist list (explicit or default) of
17155 databases created.
17156 · cluster_enabled: The current node has [cluster] n > 1, is not
17158 bound to 127.0.0.1 and has a server-level admin user defined.
17159 However, the full set of standard system databases have not
17160 been created yet. If the ensure_dbs_exist query parameter is
17161 specified, the list of databases provided overrides the
17162 default list of standard system databases.
17163 · cluster_finished: The current node has [cluster] n > 1, is not
17165 bound to 127.0.0.1, has a server-level admin user defined and
17166 has the ensure_dbs_exist list (explicit or default) of data‐
17167 bases created.
17168 Request:
17170 GET /_cluster_setup HTTP/1.1
17172 Accept: application/json
17173 Host: localhost:5984
17174 Response:
17176 HTTP/1.1 200 OK
17178 X-CouchDB-Body-Time: 0
17179 X-Couch-Request-ID: 5c058bdd37
17180 Server: CouchDB/2.1.0-7f17678 (Erlang OTP/17)
17181 Date: Sun, 30 Jul 2017 06:33:18 GMT
17182 Content-Type: application/json
17183 Content-Length: 29
17184 Cache-Control: must-revalidate
17185 {"state":"cluster_enabled"}
17187 POST /_cluster_setup
17189 Configure a node as a single (standalone) node, as part of a
17190 cluster, or finalise a cluster.
17191 Request Headers
17193 · Accept – .INDENT 2.0
17195 · application/json
17197 · text/plain
17199 · Content-Type – application/json
17202 Request JSON Object
17204 · action (string) – .INDENT 2.0
17206 · enable_single_node: Configure the current node as a single,
17208 standalone CouchDB server.
17209 · enable_cluster: Configure the local or remote node as one
17211 node, preparing it to be joined to a new CouchDB cluster.
17212 · add_node: Add the specified remote node to this cluster’s list
17214 of nodes, joining it to the cluster.
17215 · finish_cluster: Finalise the cluster by creating the standard
17217 system databases.
17218 · bind_address (string) – The IP address to which to bind the current
17221 node. The special value 0.0.0.0 may be specified to bind to all
17222 interfaces on the host. (enable_cluster and enable_single_node only)
17223 · username (string) – The username of the server-level administrator to
17225 create. (enable_cluster and enable_single_node only), or the remote
17226 server’s administrator username (add_node)
17227 · password (string) – The password for the server-level administrator
17229 to create. (enable_cluster and enable_single_node only), or the
17230 remote server’s administrator username (add_node)
17231 · port (number) – The TCP port to which to bind this node (enable_clus‐
17233 ter and enable_single_node only) or the TCP port to which to bind a
17234 remote node (add_node only).
17235 · node_count (number) – The total number of nodes to be joined into the
17237 cluster, including this one. Used to determine the value of the clus‐
17238 ter’s n, up to a maximum of 3. (enable_cluster only)
17239 · remote_node (string) – The IP address of the remote node to setup as
17241 part of this cluster’s list of nodes. (enable_cluster only)
17242 · remote_current_user (string) – The username of the server-level
17244 administrator authorized on the remote node. (enable_cluster only)
17245 · remote_current_password (string) – The password of the server-level
17247 administrator authorized on the remote node. (enable_cluster only)
17248 · host (string) – The remote node IP of the node to add to the cluster.
17250 (add_node only)
17251 · ensure_dbs_exist (array) – List of system databases to ensure exist
17253 on the node/cluster. Defaults to ["_users","_replicator"].
17254 No example request/response included here. For a worked example, please
17256 see cluster/setup/api.
17257 /_db_updates
17259 New in version 1.4.
17260 GET /_db_updates
17263 Returns a list of all database events in the CouchDB instance.
17264 The existence of the _global_changes database is required to use
17265 this endpoint.
17266 Request Headers
17268 · Accept – .INDENT 2.0
17270 · application/json
17272 · text/plain
17274 Query Parameters
17277 · feed (string) – .INDENT 2.0
17279 · normal: Returns all historical DB changes, then closes the
17281 connection. Default.
17282 · longpoll: Closes the connection after the first event.
17284 · continuous: Send a line of JSON per event. Keeps the socket
17286 open until timeout.
17287 · eventsource: Like, continuous, but sends the events in
17289 EventSource format.
17290 · timeout (number) – Number of seconds until CouchDB closes the connec‐
17293 tion. Default is 60.
17294 · heartbeat (number) – Period in milliseconds after which an empty line
17296 is sent in the results. Only applicable for longpoll, continuous, and
17297 eventsource feeds. Overrides any timeout to keep the feed alive
17298 indefinitely. Default is 60000. May be true to use default value.
17299 · since (string) – Return only updates since the specified sequence ID.
17301 If the sequence ID is specified but does not exist, all changes are
17302 returned. May be the string now to begin showing only new updates.
17303 Response Headers
17305 · Content-Type – .INDENT 2.0
17307 · application/json
17309 · text/plain; charset=utf-8
17311 · Transfer-Encoding – chunked
17314 Response JSON Object
17316 · results (array) – An array of database events. For longpoll
17318 and continuous modes, the entire response is the contents of
17319 the results array.
17320 · last_seq (string) – The last sequence ID reported.
17322 Status Codes
17324 · 200 OK – Request completed successfully
17326 · 401 Unauthorized – CouchDB Server Administrator privileges
17328 required
17329 The results field of database updates:
17331 JSON Object
17333 · db_name (string) – Database name.
17335 · type (string) – A database event is one of created,
17337 updated, deleted.
17338 · seq (json) – Update sequence of the event.
17340 Request:
17342 GET /_db_updates HTTP/1.1
17344 Accept: application/json
17345 Host: localhost:5984
17346 Response:
17348 HTTP/1.1 200 OK
17350 Cache-Control: must-revalidate
17351 Content-Type: application/json
17352 Date: Sat, 18 Mar 2017 19:01:35 GMT
17353 Etag: "C1KU98Y6H0LGM7EQQYL6VSL07"
17354 Server: CouchDB/2.0.0 (Erlang OTP/17)
17355 Transfer-Encoding: chunked
17356 X-Couch-Request-ID: ad87efc7ff
17357 X-CouchDB-Body-Time: 0
17358 {
17360 "results":[
17361 {"db_name":"mailbox","type":"created","seq":"1-g1AAAAFReJzLYWBg4MhgTmHgzcvPy09JdcjLz8gvLskBCjMlMiTJ____PyuDOZExFyjAnmJhkWaeaIquGIf2JAUgmWQPMiGRAZcaB5CaePxqEkBq6vGqyWMBkgwNQAqobD4h"},
17362 {"db_name":"mailbox","type":"deleted","seq":"2-g1AAAAFReJzLYWBg4MhgTmHgzcvPy09JdcjLz8gvLskBCjMlMiTJ____PyuDOZEpFyjAnmJhkWaeaIquGIf2JAUgmWQPMiGRAZcaB5CaePxqEkBq6vGqyWMBkgwNQAqobD4hdQsg6vYTUncAou4-IXUPIOpA7ssCAIFHa60"},
17363 ],
17364 "last_seq": "2-g1AAAAFReJzLYWBg4MhgTmHgzcvPy09JdcjLz8gvLskBCjMlMiTJ____PyuDOZEpFyjAnmJhkWaeaIquGIf2JAUgmWQPMiGRAZcaB5CaePxqEkBq6vGqyWMBkgwNQAqobD4hdQsg6vYTUncAou4-IXUPIOpA7ssCAIFHa60"
17365 }
17366 /_membership
17368 New in version 2.0.
17369 GET /_membership
17372 Displays the nodes that are part of the cluster as clus‐
17373 ter_nodes. The field all_nodes displays all nodes this node
17374 knows about, including the ones that are part of the cluster.
17375 The endpoint is useful when setting up a cluster, see clus‐
17376 ter/nodes
17377 Request Headers
17379 · Accept – .INDENT 2.0
17381 · application/json
17383 · text/plain
17385 Response Headers
17388 · Content-Type – .INDENT 2.0
17390 · application/json
17392 · text/plain; charset=utf-8
17394 Status Codes
17397 · 200 OK – Request completed successfully
17399 Request:
17401 GET /_membership HTTP/1.1
17403 Accept: application/json
17404 Host: localhost:5984
17405 Response:
17407 HTTP/1.1 200 OK
17409 Cache-Control: must-revalidate
17410 Content-Type: application/json
17411 Date: Sat, 11 Jul 2015 07:02:41 GMT
17412 Server: CouchDB (Erlang/OTP)
17413 Content-Length: 142
17414 {
17416 "all_nodes": [
17417 "node1@127.0.0.1",
17418 "node2@127.0.0.1",
17419 "node3@127.0.0.1"
17420 ],
17421 "cluster_nodes": [
17422 "node1@127.0.0.1",
17423 "node2@127.0.0.1",
17424 "node3@127.0.0.1"
17425 ]
17426 }
17427 /_replicate
17429 POST /_replicate
17430 Request, configure, or stop, a replication operation.
17431 Request Headers
17433 · Accept – .INDENT 2.0
17435 · application/json
17437 · text/plain
17439 · Content-Type – application/json
17442 Request JSON Object
17444 · cancel (boolean) – Cancels the replication
17446 · continuous (boolean) – Configure the replication to be contin‐
17448 uous
17449 · create_target (boolean) – Creates the target database.
17451 Required administrator’s privileges on target server.
17452 · doc_ids (array) – Array of document IDs to be synchronized
17454 · filter (string) – The name of a filter function.
17456 · source_proxy (string) – Address of a proxy server through
17458 which replication from the source should occur (protocol can
17459 be “http” or “socks5”)
17460 · target_proxy (string) – Address of a proxy server through
17462 which replication to the target should occur (protocol can be
17463 “http” or “socks5”)
17464 · source (string/object) – Fully qualified source database URL
17466 or an object which contains the full URL of the source data‐
17467 base with additional parameters like headers. Eg: ‘‐
17468 http://example.com/source_db_name’ or {“url”:”url in here”,
17469 “headers”: {“header1”:”value1”, …}} . For backwards compati‐
17470 bility, CouchDB 3.x will auto-convert bare database names by
17471 prepending the address and port CouchDB is listening on, to
17472 form a complete URL. This behaviour is deprecated in 3.x and
17473 will be removed in CouchDB 4.0.
17474 · target (string/object) – Fully qualified target database URL
17476 or an object which contains the full URL of the target data‐
17477 base with additional parameters like headers. Eg: ‘‐
17478 http://example.com/target_db_name’ or {“url”:”url in here”,
17479 “headers”: {“header1”:”value1”, …}} . For backwards compati‐
17480 bility, CouchDB 3.x will auto-convert bare database names by
17481 prepending the address and port CouchDB is listening on, to
17482 form a complete URL. This behaviour is deprecated in 3.x and
17483 will be removed in CouchDB 4.0.
17484 Response Headers
17486 · Content-Type – .INDENT 2.0
17488 · application/json
17490 · text/plain; charset=utf-8
17492 Response JSON Object
17495 · history (array) – Replication history (see below)
17497 · ok (boolean) – Replication status
17499 · replication_id_version (number) – Replication protocol version
17501 · session_id (string) – Unique session ID
17503 · source_last_seq (number) – Last sequence number read from
17505 source database
17506 Status Codes
17508 · 200 OK – Replication request successfully completed
17510 · 202 Accepted – Continuous replication request has been
17512 accepted
17513 · 400 Bad Request – Invalid JSON data
17515 · 401 Unauthorized – CouchDB Server Administrator privileges
17517 required
17518 · 404 Not Found – Either the source or target DB is not found or
17520 attempt to cancel unknown replication task
17521 · 500 Internal Server Error – JSON specification was invalid
17523The specification of the replication request is controlled through the JSON
17525content of the request. The JSON should be an object with the fields defining
17526the source, target and other options.
17527The Replication history is an array of objects with following structure:
17529 JSON Object
17531 · doc_write_failures (number) – Number of document write
17533 failures
17534 · docs_read (number) – Number of documents read
17536 · docs_written (number) – Number of documents written to
17538 target
17539 · end_last_seq (number) – Last sequence number in changes
17541 stream
17542 · end_time (string) – Date/Time replication operation
17544 completed in RFC 2822 format
17545 · missing_checked (number) – Number of missing documents
17547 checked
17548 · missing_found (number) – Number of missing documents
17550 found
17551 · recorded_seq (number) – Last recorded sequence number
17553 · session_id (string) – Session ID for this replication
17555 operation
17556 · start_last_seq (number) – First sequence number in
17558 changes stream
17559 · start_time (string) – Date/Time replication operation
17561 started in RFC 2822 format
17562
17564 As of CouchDB 2.0.0, fully qualified URLs are required for both the
17565 replication source and target parameters.
17566 Request
17568 POST /_replicate HTTP/1.1
17570 Accept: application/json
17571 Content-Length: 80
17572 Content-Type: application/json
17573 Host: localhost:5984
17574 {
17576 "source": "http://127.0.0.1:5984/db_a",
17577 "target": "http://127.0.0.1:5984/db_b"
17578 }
17579 Response
17581 HTTP/1.1 200 OK
17583 Cache-Control: must-revalidate
17584 Content-Length: 692
17585 Content-Type: application/json
17586 Date: Sun, 11 Aug 2013 20:38:50 GMT
17587 Server: CouchDB (Erlang/OTP)
17588 {
17590 "history": [
17591 {
17592 "doc_write_failures": 0,
17593 "docs_read": 10,
17594 "docs_written": 10,
17595 "end_last_seq": 28,
17596 "end_time": "Sun, 11 Aug 2013 20:38:50 GMT",
17597 "missing_checked": 10,
17598 "missing_found": 10,
17599 "recorded_seq": 28,
17600 "session_id": "142a35854a08e205c47174d91b1f9628",
17601 "start_last_seq": 1,
17602 "start_time": "Sun, 11 Aug 2013 20:38:50 GMT"
17603 },
17604 {
17605 "doc_write_failures": 0,
17606 "docs_read": 1,
17607 "docs_written": 1,
17608 "end_last_seq": 1,
17609 "end_time": "Sat, 10 Aug 2013 15:41:54 GMT",
17610 "missing_checked": 1,
17611 "missing_found": 1,
17612 "recorded_seq": 1,
17613 "session_id": "6314f35c51de3ac408af79d6ee0c1a09",
17614 "start_last_seq": 0,
17615 "start_time": "Sat, 10 Aug 2013 15:41:54 GMT"
17616 }
17617 ],
17618 "ok": true,
17619 "replication_id_version": 3,
17620 "session_id": "142a35854a08e205c47174d91b1f9628",
17621 "source_last_seq": 28
17622 }
17623 Replication Operation
17625 The aim of the replication is that at the end of the process, all
17626 active documents on the source database are also in the destination
17627 database and all documents that were deleted in the source databases
17628 are also deleted (if they exist) on the destination database.
17629 Replication can be described as either push or pull replication:
17631 · Pull replication is where the source is the remote CouchDB instance,
17633 and the target is the local database.
17634 Pull replication is the most useful solution to use if your source
17636 database has a permanent IP address, and your destination (local)
17637 database may have a dynamically assigned IP address (for example,
17638 through DHCP). This is particularly important if you are replicating
17639 to a mobile or other device from a central server.
17640 · Push replication is where the source is a local database, and target
17642 is a remote database.
17643 Specifying the Source and Target Database
17645 You must use the URL specification of the CouchDB database if you want
17646 to perform replication in either of the following two situations:
17647 · Replication with a remote database (i.e. another instance of CouchDB
17649 on the same host, or a different host)
17650 · Replication with a database that requires authentication
17652 For example, to request replication between a database local to the
17654 CouchDB instance to which you send the request, and a remote database
17655 you might use the following request:
17656 POST http://couchdb:5984/_replicate HTTP/1.1
17658 Content-Type: application/json
17659 Accept: application/json
17660 {
17662 "source" : "recipes",
17663 "target" : "http://coucdb-remote:5984/recipes",
17664 }
17665 In all cases, the requested databases in the source and target specifi‐
17667 cation must exist. If they do not, an error will be returned within the
17668 JSON object:
17669 {
17671 "error" : "db_not_found"
17672 "reason" : "could not open http://couchdb-remote:5984/ol1ka/",
17673 }
17674 You can create the target database (providing your user credentials
17676 allow it) by adding the create_target field to the request object:
17677 POST http://couchdb:5984/_replicate HTTP/1.1
17679 Content-Type: application/json
17680 Accept: application/json
17681 {
17683 "create_target" : true
17684 "source" : "recipes",
17685 "target" : "http://couchdb-remote:5984/recipes",
17686 }
17687 The create_target field is not destructive. If the database already
17689 exists, the replication proceeds as normal.
17690 Single Replication
17692 You can request replication of a database so that the two databases can
17693 be synchronized. By default, the replication process occurs one time
17694 and synchronizes the two databases together. For example, you can
17695 request a single synchronization between two databases by supplying the
17696 source and target fields within the request JSON content.
17697 POST http://couchdb:5984/_replicate HTTP/1.1
17699 Accept: application/json
17700 Content-Type: application/json
17701 {
17703 "source" : "recipes",
17704 "target" : "recipes-snapshot",
17705 }
17706 In the above example, the databases recipes and recipes-snapshot will
17708 be synchronized. These databases are local to the CouchDB instance
17709 where the request was made. The response will be a JSON structure con‐
17710 taining the success (or failure) of the synchronization process, and
17711 statistics about the process:
17712 {
17714 "ok" : true,
17715 "history" : [
17716 {
17717 "docs_read" : 1000,
17718 "session_id" : "52c2370f5027043d286daca4de247db0",
17719 "recorded_seq" : 1000,
17720 "end_last_seq" : 1000,
17721 "doc_write_failures" : 0,
17722 "start_time" : "Thu, 28 Oct 2010 10:24:13 GMT",
17723 "start_last_seq" : 0,
17724 "end_time" : "Thu, 28 Oct 2010 10:24:14 GMT",
17725 "missing_checked" : 0,
17726 "docs_written" : 1000,
17727 "missing_found" : 1000
17728 }
17729 ],
17730 "session_id" : "52c2370f5027043d286daca4de247db0",
17731 "source_last_seq" : 1000
17732 }
17733 Continuous Replication
17735 Synchronization of a database with the previously noted methods happens
17736 only once, at the time the replicate request is made. To have the tar‐
17737 get database permanently replicated from the source, you must set the
17738 continuous field of the JSON object within the request to true.
17739 With continuous replication changes in the source database are repli‐
17741 cated to the target database in perpetuity until you specifically
17742 request that replication ceases.
17743 POST http://couchdb:5984/_replicate HTTP/1.1
17745 Accept: application/json
17746 Content-Type: application/json
17747 {
17749 "continuous" : true
17750 "source" : "recipes",
17751 "target" : "http://couchdb-remote:5984/recipes",
17752 }
17753 Changes will be replicated between the two databases as long as a net‐
17755 work connection is available between the two instances.
17756 NOTE:
17758 Two keep two databases synchronized with each other, you need to set
17759 replication in both directions; that is, you must replicate from
17760 source to target, and separately from target to source.
17761 Canceling Continuous Replication
17763 You can cancel continuous replication by adding the cancel field to the
17764 JSON request object and setting the value to true. Note that the struc‐
17765 ture of the request must be identical to the original for the cancella‐
17766 tion request to be honoured. For example, if you requested continuous
17767 replication, the cancellation request must also contain the continuous
17768 field.
17769 For example, the replication request:
17771 POST http://couchdb:5984/_replicate HTTP/1.1
17773 Content-Type: application/json
17774 Accept: application/json
17775 {
17777 "source" : "recipes",
17778 "target" : "http://couchdb-remote:5984/recipes",
17779 "create_target" : true,
17780 "continuous" : true
17781 }
17782 Must be canceled using the request:
17784 POST http://couchdb:5984/_replicate HTTP/1.1
17786 Accept: application/json
17787 Content-Type: application/json
17788 {
17790 "cancel" : true,
17791 "continuous" : true
17792 "create_target" : true,
17793 "source" : "recipes",
17794 "target" : "http://couchdb-remote:5984/recipes",
17795 }
17796 Requesting cancellation of a replication that does not exist results in
17798 a 404 error.
17799 /_scheduler/jobs
17801 GET /_scheduler/jobs
17802 List of replication jobs. Includes replications created via
17803 /_replicate endpoint as well as those created from replication
17804 documents. Does not include replications which have completed or
17805 have failed to start because replication documents were mal‐
17806 formed. Each job description will include source and target
17807 information, replication id, a history of recent event, and a
17808 few other things.
17809 Request Headers
17811 · Accept – .INDENT 2.0
17813 · application/json
17815 Response Headers
17818 · Content-Type – .INDENT 2.0
17820 · application/json
17822 Query Parameters
17825 · limit (number) – How many results to return
17827 · skip (number) – How many result to skip starting at the begin‐
17829 ning, ordered by replication ID
17830 Response JSON Object
17832 · offset (number) – How many results were skipped
17834 · total_rows (number) – Total number of replication jobs
17836 · id (string) – Replication ID.
17838 · database (string) – Replication document database
17840 · doc_id (string) – Replication document ID
17842 · history (list) – Timestamped history of events as a list of
17844 objects
17845 · pid (string) – Replication process ID
17847 · node (string) – Cluster node where the job is running
17849 · source (string) – Replication source
17851 · target (string) – Replication target
17853 · start_time (string) – Timestamp of when the replication was
17855 started
17856 Status Codes
17858 · 200 OK – Request completed successfully
17860 · 401 Unauthorized – CouchDB Server Administrator privileges
17862 required
17863
17865 GET /_scheduler/jobs HTTP/1.1
17867 Accept: application/json
17868 Host: localhost:5984
17869 Response:
17871 HTTP/1.1 200 OK
17873 Cache-Control: must-revalidate
17874 Content-Length: 1690
17875 Content-Type: application/json
17876 Date: Sat, 29 Apr 2017 05:05:16 GMT
17877 Server: CouchDB (Erlang/OTP)
17878 {
17880 "jobs": [
17881 {
17882 "database": "_replicator",
17883 "doc_id": "cdyno-0000001-0000003",
17884 "history": [
17885 {
17886 "timestamp": "2017-04-29T05:01:37Z",
17887 "type": "started"
17888 },
17889 {
17890 "timestamp": "2017-04-29T05:01:37Z",
17891 "type": "added"
17892 }
17893 ],
17894 "id": "8f5b1bd0be6f9166ccfd36fc8be8fc22+continuous",
17895 "info": {
17896 "changes_pending": 0,
17897 "checkpointed_source_seq": "113-g1AAAACTeJzLYWBgYMpgTmHgz8tPSTV0MDQy1zMAQsMckEQiQ1L9____szKYE01ygQLsZsYGqcamiZjKcRqRxwIkGRqA1H-oSbZgk1KMLCzTDE0wdWUBAF6HJIQ",
17898 "doc_write_failures": 0,
17899 "docs_read": 113,
17900 "docs_written": 113,
17901 "missing_revisions_found": 113,
17902 "revisions_checked": 113,
17903 "source_seq": "113-g1AAAACTeJzLYWBgYMpgTmHgz8tPSTV0MDQy1zMAQsMckEQiQ1L9____szKYE01ygQLsZsYGqcamiZjKcRqRxwIkGRqA1H-oSbZgk1KMLCzTDE0wdWUBAF6HJIQ",
17904 "through_seq": "113-g1AAAACTeJzLYWBgYMpgTmHgz8tPSTV0MDQy1zMAQsMckEQiQ1L9____szKYE01ygQLsZsYGqcamiZjKcRqRxwIkGRqA1H-oSbZgk1KMLCzTDE0wdWUBAF6HJIQ"
17905 },
17906 "node": "node1@127.0.0.1",
17907 "pid": "<0.1850.0>",
17908 "source": "http://myserver.com/foo",
17909 "start_time": "2017-04-29T05:01:37Z",
17910 "target": "http://adm:*****@localhost:15984/cdyno-0000003/",
17911 "user": null
17912 },
17913 {
17914 "database": "_replicator",
17915 "doc_id": "cdyno-0000001-0000002",
17916 "history": [
17917 {
17918 "timestamp": "2017-04-29T05:01:37Z",
17919 "type": "started"
17920 },
17921 {
17922 "timestamp": "2017-04-29T05:01:37Z",
17923 "type": "added"
17924 }
17925 ],
17926 "id": "e327d79214831ca4c11550b4a453c9ba+continuous",
17927 "info": {
17928 "changes_pending": null,
17929 "checkpointed_source_seq": 0,
17930 "doc_write_failures": 0,
17931 "docs_read": 12,
17932 "docs_written": 12,
17933 "missing_revisions_found": 12,
17934 "revisions_checked": 12,
17935 "source_seq": "12-g1AAAACTeJzLYWBgYMpgTmHgz8tPSTV0MDQy1zMAQsMckEQiQ1L9____szKYE1lzgQLsBsZm5pZJJpjKcRqRxwIkGRqA1H-oSexgk4yMkhITjS0wdWUBADfEJBg",
17936 "through_seq": "12-g1AAAACTeJzLYWBgYMpgTmHgz8tPSTV0MDQy1zMAQsMckEQiQ1L9____szKYE1lzgQLsBsZm5pZJJpjKcRqRxwIkGRqA1H-oSexgk4yMkhITjS0wdWUBADfEJBg"
17937 },
17938 "node": "node2@127.0.0.1",
17939 "pid": "<0.1757.0>",
17940 "source": "http://myserver.com/foo",
17941 "start_time": "2017-04-29T05:01:37Z",
17942 "target": "http://adm:*****@localhost:15984/cdyno-0000002/",
17943 "user": null
17944 }
17945 ],
17946 "offset": 0,
17947 "total_rows": 2
17948 }
17949 /_scheduler/docs
17951 Changed in version 2.1.0: Use this endpoint to monitor the state of
17952 document-based replications. Previously needed to poll both documents
17953 and _active_tasks to get a complete state summary
17954 Changed in version 3.0.0: In error states the “info” field switched
17957 from being a string to being an object
17958 GET /_scheduler/docs
17961 List of replication document states. Includes information about
17962 all the documents, even in completed and failed states. For each
17963 document it returns the document ID, the database, the replica‐
17964 tion ID, source and target, and other information.
17965 Request Headers
17967 · Accept – .INDENT 2.0
17969 · application/json
17971 Response Headers
17974 · Content-Type – .INDENT 2.0
17976 · application/json
17978 Query Parameters
17981 · limit (number) – How many results to return
17983 · skip (number) – How many result to skip starting at the begin‐
17985 ning, if ordered by document ID
17986 Response JSON Object
17988 · offset (number) – How many results were skipped
17990 · total_rows (number) – Total number of replication documents.
17992 · id (string) – Replication ID, or null if state is completed or
17994 failed
17995 · state (string) – One of following states (see replica‐
17997 tor/states for descriptions): initializing, running, com‐
17998 pleted, pending, crashing, error, failed
17999 · database (string) – Database where replication document came
18001 from
18002 · doc_id (string) – Replication document ID
18004 · node (string) – Cluster node where the job is running
18006 · source (string) – Replication source
18008 · target (string) – Replication target
18010 · start_time (string) – Timestamp of when the replication was
18012 started
18013 · last_updated (string) – Timestamp of last state update
18015 · info (object) – Will contain additional information about the
18017 state. For errors, this will be an object with an “error”
18018 field and string value. For success states, see below.
18019 · error_count (number) – Consecutive errors count. Indicates how
18021 many times in a row this replication has crashed. Replication
18022 will be retried with an exponential backoff based on this num‐
18023 ber. As soon as the replication succeeds this count is reset
18024 to 0. To can be used to get an idea why a particular replica‐
18025 tion is not making progress.
18026 Status Codes
18028 · 200 OK – Request completed successfully
18030 · 401 Unauthorized – CouchDB Server Administrator privileges
18032 required
18033The info field of a scheduler doc:
18035 JSON Object
18037 · revisions_checked (number) – The count of revisions
18039 which have been checked since this replication began.
18040 · missing_revisions_found (number) – The count of revi‐
18042 sions which were found on the source, but missing from
18043 the target.
18044 · docs_read (number) – The count of docs which have been
18046 read from the source.
18047 · docs_written (number) – The count of docs which have
18049 been written to the target.
18050 · changes_pending (number) – The count of changes not yet
18052 replicated.
18053 · doc_write_failures (number) – The count of docs which
18055 failed to be written to the target.
18056 · checkpointed_source_seq (object) – The source sequence
18058 id which was last successfully replicated.
18059 Request:
18061 GET /_scheduler/docs HTTP/1.1
18063 Accept: application/json
18064 Host: localhost:5984
18065 Response:
18067 HTTP/1.1 200 OK
18069 Content-Type: application/json
18070 Date: Sat, 29 Apr 2017 05:10:08 GMT
18071 Server: Server: CouchDB (Erlang/OTP)
18072 Transfer-Encoding: chunked
18073 {
18075 "docs": [
18076 {
18077 "database": "_replicator",
18078 "doc_id": "cdyno-0000001-0000002",
18079 "error_count": 0,
18080 "id": "e327d79214831ca4c11550b4a453c9ba+continuous",
18081 "info": {
18082 "changes_pending": 15,
18083 "checkpointed_source_seq": "60-g1AAAACTeJzLYWBgYMpgTmHgz8tPSTV0MDQy1zMAQsMckEQiQ1L9____szKYEyVygQLsBsZm5pZJJpjKcRqRxwIkGRqA1H-oSSpgk4yMkhITjS0wdWUBAENCJEg",
18084 "doc_write_failures": 0,
18085 "docs_read": 67,
18086 "docs_written": 67,
18087 "missing_revisions_found": 67,
18088 "revisions_checked": 67,
18089 "source_seq": "67-g1AAAACTeJzLYWBgYMpgTmHgz8tPSTV0MDQy1zMAQsMckEQiQ1L9____szKYE2VygQLsBsZm5pZJJpjKcRqRxwIkGRqA1H-oSepgk4yMkhITjS0wdWUBAEVKJE8",
18090 "through_seq": "67-g1AAAACTeJzLYWBgYMpgTmHgz8tPSTV0MDQy1zMAQsMckEQiQ1L9____szKYE2VygQLsBsZm5pZJJpjKcRqRxwIkGRqA1H-oSepgk4yMkhITjS0wdWUBAEVKJE8"
18091 },
18092 "last_updated": "2017-04-29T05:01:37Z",
18093 "node": "node2@127.0.0.1",
18094 "source_proxy": null,
18095 "target_proxy": null,
18096 "source": "http://myserver.com/foo",
18097 "start_time": "2017-04-29T05:01:37Z",
18098 "state": "running",
18099 "target": "http://adm:*****@localhost:15984/cdyno-0000002/"
18100 },
18101 {
18102 "database": "_replicator",
18103 "doc_id": "cdyno-0000001-0000003",
18104 "error_count": 0,
18105 "id": "8f5b1bd0be6f9166ccfd36fc8be8fc22+continuous",
18106 "info": {
18107 "changes_pending": null,
18108 "checkpointed_source_seq": 0,
18109 "doc_write_failures": 0,
18110 "docs_read": 12,
18111 "docs_written": 12,
18112 "missing_revisions_found": 12,
18113 "revisions_checked": 12,
18114 "source_seq": "12-g1AAAACTeJzLYWBgYMpgTmHgz8tPSTV0MDQy1zMAQsMckEQiQ1L9____szKYE1lzgQLsBsZm5pZJJpjKcRqRxwIkGRqA1H-oSexgk4yMkhITjS0wdWUBADfEJBg",
18115 "through_seq": "12-g1AAAACTeJzLYWBgYMpgTmHgz8tPSTV0MDQy1zMAQsMckEQiQ1L9____szKYE1lzgQLsBsZm5pZJJpjKcRqRxwIkGRqA1H-oSexgk4yMkhITjS0wdWUBADfEJBg"
18116 },
18117 "last_updated": "2017-04-29T05:01:37Z",
18118 "node": "node1@127.0.0.1",
18119 "source_proxy": null,
18120 "target_proxy": null,
18121 "source": "http://myserver.com/foo",
18122 "start_time": "2017-04-29T05:01:37Z",
18123 "state": "running",
18124 "target": "http://adm:*****@localhost:15984/cdyno-0000003/"
18125 }
18126 ],
18127 "offset": 0,
18128 "total_rows": 2
18129 }
18130 GET /_scheduler/docs/{replicator_db}
18132 Get information about replication documents from a replicator
18133 database. The default replicator database is _replicator but
18134 other replicator databases can exist if their name ends with the
18135 suffix /_replicator.
18136 NOTE:
18138 As a convenience slashes (/) in replicator db names do not
18139 have to be escaped. So /_scheduler/docs/other/_replicator is
18140 valid and equivalent to /_scheduler/docs/other%2f_replicator
18141 Request Headers
18143 · Accept – .INDENT 2.0
18145 · application/json
18147 Response Headers
18150 · Content-Type – .INDENT 2.0
18152 · application/json
18154 Query Parameters
18157 · limit (number) – How many results to return
18159 · skip (number) – How many result to skip starting at the begin‐
18161 ning, if ordered by document ID
18162 Response JSON Object
18164 · offset (number) – How many results were skipped
18166 · total_rows (number) – Total number of replication documents.
18168 · id (string) – Replication ID, or null if state is completed or
18170 failed
18171 · state (string) – One of following states (see replica‐
18173 tor/states for descriptions): initializing, running, com‐
18174 pleted, pending, crashing, error, failed
18175 · database (string) – Database where replication document came
18177 from
18178 · doc_id (string) – Replication document ID
18180 · node (string) – Cluster node where the job is running
18182 · source (string) – Replication source
18184 · target (string) – Replication target
18186 · start_time (string) – Timestamp of when the replication was
18188 started
18189 · last_update (string) – Timestamp of last state update
18191 · info (object) – Will contain additional information about the
18193 state. For errors, this will be an object with an “error”
18194 field and string value. For success states, see below.
18195 · error_count (number) – Consecutive errors count. Indicates how
18197 many times in a row this replication has crashed. Replication
18198 will be retried with an exponential backoff based on this num‐
18199 ber. As soon as the replication succeeds this count is reset
18200 to 0. To can be used to get an idea why a particular replica‐
18201 tion is not making progress.
18202 Status Codes
18204 · 200 OK – Request completed successfully
18206 · 401 Unauthorized – CouchDB Server Administrator privileges
18208 required
18209The info field of a scheduler doc:
18211 JSON Object
18213 · revisions_checked (number) – The count of revisions
18215 which have been checked since this replication began.
18216 · missing_revisions_found (number) – The count of revi‐
18218 sions which were found on the source, but missing from
18219 the target.
18220 · docs_read (number) – The count of docs which have been
18222 read from the source.
18223 · docs_written (number) – The count of docs which have
18225 been written to the target.
18226 · changes_pending (number) – The count of changes not yet
18228 replicated.
18229 · doc_write_failures (number) – The count of docs which
18231 failed to be written to the target.
18232 · checkpointed_source_seq (object) – The source sequence
18234 id which was last successfully replicated.
18235 Request:
18237 GET /_scheduler/docs/other/_replicator HTTP/1.1
18239 Accept: application/json
18240 Host: localhost:5984
18241 Response:
18243 HTTP/1.1 200 OK
18245 Content-Type: application/json
18246 Date: Sat, 29 Apr 2017 05:10:08 GMT
18247 Server: Server: CouchDB (Erlang/OTP)
18248 Transfer-Encoding: chunked
18249 {
18251 "docs": [
18252 {
18253 "database": "other/_replicator",
18254 "doc_id": "cdyno-0000001-0000002",
18255 "error_count": 0,
18256 "id": "e327d79214831ca4c11550b4a453c9ba+continuous",
18257 "info": {
18258 "changes_pending": 0,
18259 "checkpointed_source_seq": "60-g1AAAACTeJzLYWBgYMpgTmHgz8tPSTV0MDQy1zMAQsMckEQiQ1L9____szKYEyVygQLsBsZm5pZJJpjKcRqRxwIkGRqA1H-oSSpgk4yMkhITjS0wdWUBAENCJEg",
18260 "doc_write_failures": 0,
18261 "docs_read": 67,
18262 "docs_written": 67,
18263 "missing_revisions_found": 67,
18264 "revisions_checked": 67,
18265 "source_seq": "67-g1AAAACTeJzLYWBgYMpgTmHgz8tPSTV0MDQy1zMAQsMckEQiQ1L9____szKYE2VygQLsBsZm5pZJJpjKcRqRxwIkGRqA1H-oSepgk4yMkhITjS0wdWUBAEVKJE8",
18266 "through_seq": "67-g1AAAACTeJzLYWBgYMpgTmHgz8tPSTV0MDQy1zMAQsMckEQiQ1L9____szKYE2VygQLsBsZm5pZJJpjKcRqRxwIkGRqA1H-oSepgk4yMkhITjS0wdWUBAEVKJE8"
18267 },
18268 "last_updated": "2017-04-29T05:01:37Z",
18269 "node": "node2@127.0.0.1",
18270 "source_proxy": null,
18271 "target_proxy": null,
18272 "source": "http://myserver.com/foo",
18273 "start_time": "2017-04-29T05:01:37Z",
18274 "state": "running",
18275 "target": "http://adm:*****@localhost:15984/cdyno-0000002/"
18276 }
18277 ],
18278 "offset": 0,
18279 "total_rows": 1
18280 }
18281 GET /_scheduler/docs/{replicator_db}/{docid}
18283 NOTE:
18285 As a convenience slashes (/) in replicator db names do not
18286 have to be escaped. So /_scheduler/docs/other/_replicator is
18287 valid and equivalent to /_scheduler/docs/other%2f_replicator
18288 Request Headers
18290 · Accept – .INDENT 2.0
18292 · application/json
18294 Response Headers
18297 · Content-Type – .INDENT 2.0
18299 · application/json
18301 Response JSON Object
18304 · id (string) – Replication ID, or null if state is completed or
18306 failed
18307 · state (string) – One of following states (see replica‐
18309 tor/states for descriptions): initializing, running, com‐
18310 pleted, pending, crashing, error, failed
18311 · database (string) – Database where replication document came
18313 from
18314 · doc_id (string) – Replication document ID
18316 · node (string) – Cluster node where the job is running
18318 · source (string) – Replication source
18320 · target (string) – Replication target
18322 · start_time (string) – Timestamp of when the replication was
18324 started
18325 · last_update (string) – Timestamp of last state update
18327 · info (object) – Will contain additional information about the
18329 state. For errors, this will be an object with an “error”
18330 field and string value. For success states, see below.
18331 · error_count (number) – Consecutive errors count. Indicates how
18333 many times in a row this replication has crashed. Replication
18334 will be retried with an exponential backoff based on this num‐
18335 ber. As soon as the replication succeeds this count is reset
18336 to 0. To can be used to get an idea why a particular replica‐
18337 tion is not making progress.
18338 Status Codes
18340 · 200 OK – Request completed successfully
18342 · 401 Unauthorized – CouchDB Server Administrator privileges
18344 required
18345The info field of a scheduler doc:
18347 JSON Object
18349 · revisions_checked (number) – The count of revisions
18351 which have been checked since this replication began.
18352 · missing_revisions_found (number) – The count of revi‐
18354 sions which were found on the source, but missing from
18355 the target.
18356 · docs_read (number) – The count of docs which have been
18358 read from the source.
18359 · docs_written (number) – The count of docs which have
18361 been written to the target.
18362 · changes_pending (number) – The count of changes not yet
18364 replicated.
18365 · doc_write_failures (number) – The count of docs which
18367 failed to be written to the target.
18368 · checkpointed_source_seq (object) – .INDENT 2.0
18370 The source sequence id which was last
18372 successfully replicated.
18373 Request:
18375 GET /_scheduler/docs/other/_replicator/cdyno-0000001-0000002 HTTP/1.1
18378 Accept: application/json
18379 Host: localhost:5984
18380 Response:
18382 HTTP/1.1 200 OK
18384 Content-Type: application/json
18385 Date: Sat, 29 Apr 2017 05:10:08 GMT
18386 Server: Server: CouchDB (Erlang/OTP)
18387 Transfer-Encoding: chunked
18388 {
18390 "database": "other/_replicator",
18391 "doc_id": "cdyno-0000001-0000002",
18392 "error_count": 0,
18393 "id": "e327d79214831ca4c11550b4a453c9ba+continuous",
18394 "info": {
18395 "changes_pending": 0,
18396 "checkpointed_source_seq": "60-g1AAAACTeJzLYWBgYMpgTmHgz8tPSTV0MDQy1zMAQsMckEQiQ1L9____szKYEyVygQLsBsZm5pZJJpjKcRqRxwIkGRqA1H-oSSpgk4yMkhITjS0wdWUBAENCJEg",
18397 "doc_write_failures": 0,
18398 "docs_read": 67,
18399 "docs_written": 67,
18400 "missing_revisions_found": 67,
18401 "revisions_checked": 67,
18402 "source_seq": "67-g1AAAACTeJzLYWBgYMpgTmHgz8tPSTV0MDQy1zMAQsMckEQiQ1L9____szKYE2VygQLsBsZm5pZJJpjKcRqRxwIkGRqA1H-oSepgk4yMkhITjS0wdWUBAEVKJE8",
18403 "through_seq": "67-g1AAAACTeJzLYWBgYMpgTmHgz8tPSTV0MDQy1zMAQsMckEQiQ1L9____szKYE2VygQLsBsZm5pZJJpjKcRqRxwIkGRqA1H-oSepgk4yMkhITjS0wdWUBAEVKJE8"
18404 },
18405 "last_updated": "2017-04-29T05:01:37Z",
18406 "node": "node2@127.0.0.1",
18407 "source_proxy": null,
18408 "target_proxy": null,
18409 "source": "http://myserver.com/foo",
18410 "start_time": "2017-04-29T05:01:37Z",
18411 "state": "running",
18412 "target": "http://adm:*****@localhost:15984/cdyno-0000002/"
18413 }
18414 /_node/{node-name}
18416 GET /_node/{node-name}
18417 The /_node/{node-name} endpoint can be used to confirm the
18418 Erlang node name of the server that processes the request. This
18419 is most useful when accessing /_node/_local to retrieve this
18420 information. Repeatedly retrieving this information for a
18421 CouchDB endpoint can be useful to determine if a CouchDB cluster
18422 is correctly proxied through a reverse load balancer.
18423 Request Headers
18425 · Accept – .INDENT 2.0
18427 · application/json
18429 · text/plain
18431 Response Headers
18434 · Content-Type – .INDENT 2.0
18436 · application/json
18438 · text/plain; charset=utf-8
18440 Status Codes
18443 · 200 OK – Request completed successfully
18445
18447 GET /_node/_local HTTP/1.1
18449 Accept: application/json
18450 Host: localhost:5984
18451 Response:
18453 HTTP/1.1 200 OK
18455 Cache-Control: must-revalidate
18456 Content-Length: 27
18457 Content-Type: application/json
18458 Date: Tue, 28 Jan 2020 19:25:51 GMT
18459 Server: CouchDB (Erlang OTP)
18460 X-Couch-Request-ID: 5b8db6c677
18461 X-CouchDB-Body-Time: 0
18462 {"name":"node1@127.0.0.1"}
18464 /_node/{node-name}/_stats
18466 GET /_node/{node-name}/_stats
18467 The _stats resource returns a JSON object containing the statis‐
18468 tics for the running server. The object is structured with
18469 top-level sections collating the statistics for a range of
18470 entries, with each individual statistic being easily identified,
18471 and the content of each statistic is self-describing.
18472 Statistics are sampled internally on a configurable interval.
18474 When monitoring the _stats endpoint, you need to use a polling
18475 frequency of at least twice this to observe accurate results.
18476 For example, if the interval is 10 seconds, poll _stats at least
18477 every 5 seconds.
18478 The literal string _local serves as an alias for the local node
18480 name, so for all stats URLs, {node-name} may be replaced with
18481 _local, to interact with the local node’s statistics.
18482 Request Headers
18484 · Accept – .INDENT 2.0
18486 · application/json
18488 · text/plain
18490 Response Headers
18493 · Content-Type – .INDENT 2.0
18495 · application/json
18497 · text/plain; charset=utf-8
18499 Status Codes
18502 · 200 OK – Request completed successfully
18504
18506 GET /_node/_local/_stats/couchdb/request_time HTTP/1.1
18508 Accept: application/json
18509 Host: localhost:5984
18510 Response:
18512 HTTP/1.1 200 OK
18514 Cache-Control: must-revalidate
18515 Content-Length: 187
18516 Content-Type: application/json
18517 Date: Sat, 10 Aug 2013 11:41:11 GMT
18518 Server: CouchDB (Erlang/OTP)
18519 {
18521 "value": {
18522 "min": 0,
18523 "max": 0,
18524 "arithmetic_mean": 0,
18525 "geometric_mean": 0,
18526 "harmonic_mean": 0,
18527 "median": 0,
18528 "variance": 0,
18529 "standard_deviation": 0,
18530 "skewness": 0,
18531 "kurtosis": 0,
18532 "percentile": [
18533 [
18534 50,
18535 0
18536 ],
18537 [
18538 75,
18539 0
18540 ],
18541 [
18542 90,
18543 0
18544 ],
18545 [
18546 95,
18547 0
18548 ],
18549 [
18550 99,
18551 0
18552 ],
18553 [
18554 999,
18555 0
18556 ]
18557 ],
18558 "histogram": [
18559 [
18560 0,
18561 0
18562 ]
18563 ],
18564 "n": 0
18565 },
18566 "type": "histogram",
18567 "desc": "length of a request inside CouchDB without MochiWeb"
18568 }
18569The fields provide the current, minimum and maximum, and a collection of sta‐
18571tistical means and quantities. The quantity in each case is not defined, but
18572the descriptions below provide sufficient detail to determine units.
18573Statistics are reported by ‘group’. The statistics are divided into the fol‐
18575lowing top-level sections:
18576 · couch_log: Logging subsystem
18578 · couch_replicator: Replication scheduler and subsystem
18580 · couchdb: Primary CouchDB database operations
18582 · fabric: Cluster-related operations
18584 · global_changes: Global changes feed
18586 · mem3: Node membership-related statistics
18588 · pread: CouchDB file-related exceptions
18590 · rexi: Cluster internal RPC-related statistics
18592 The type of the statistic is included in the type field, and is one of
18594 the following:
18595 · counter: Monotonically increasing counter, resets on restart
18597 · histogram: Binned set of values with meaningful subdivisions. Scoped
18599 to the current collection interval.
18600 · gauge: Single numerical value that can go up and down
18602 You can also access individual statistics by quoting the statistics
18604 sections and statistic ID as part of the URL path. For example, to get
18605 the request_time statistics within the couchdb section for the target
18606 node, you can use:
18607 GET /_node/_local/_stats/couchdb/request_time HTTP/1.1
18609 This returns an entire statistics object, as with the full request, but
18611 containing only the requested individual statistic.
18612 /_node/{node-name}/_system
18614 GET /_node/{node-name}/_system
18615 The _system resource returns a JSON object containing various
18616 system-level statistics for the running server. The object is
18617 structured with top-level sections collating the statistics for
18618 a range of entries, with each individual statistic being easily
18619 identified, and the content of each statistic is self-describ‐
18620 ing.
18621 The literal string _local serves as an alias for the local node
18623 name, so for all stats URLs, {node-name} may be replaced with
18624 _local, to interact with the local node’s statistics.
18625 Request Headers
18627 · Accept – .INDENT 2.0
18629 · application/json
18631 · text/plain
18633 Response Headers
18636 · Content-Type – .INDENT 2.0
18638 · application/json
18640 · text/plain; charset=utf-8
18642 Status Codes
18645 · 200 OK – Request completed successfully
18647
18649 GET /_node/_local/_system HTTP/1.1
18651 Accept: application/json
18652 Host: localhost:5984
18653 Response:
18655 HTTP/1.1 200 OK
18657 Cache-Control: must-revalidate
18658 Content-Length: 187
18659 Content-Type: application/json
18660 Date: Sat, 10 Aug 2013 11:41:11 GMT
18661 Server: CouchDB (Erlang/OTP)
18662 {
18664 "uptime": 259,
18665 "memory": {
18666 ...
18667 }
18668 These statistics are generally intended for CouchDB developers
18670 only.
18671 /_node/{node-name}/_restart
18673 POST /_node/{node-name}/_restart
18674 This API is to facilitate integration testing only it is not
18675 meant to be used in production
18676 Status Codes
18678 · 200 OK – Request completed successfully
18680 /_search_analyze
18682 WARNING:
18683 Search endpoints require a running search plugin connected to each
18684 cluster node. See Search Plugin Installation for details.
18685 New in version 3.0.
18687 POST /_search_analyze
18690 Tests the results of Lucene analyzer tokenization on sample
18691 text.
18692 Parameters
18694 · field – Type of analyzer
18696 · text – Analyzer token you want to test
18698 Status Codes
18700 · 200 OK – Request completed successfully
18702 · 400 Bad Request – Request body is wrong (malformed or
18704 missing one of the mandatory fields)
18705 · 500 Internal Server Error – A server error (or other
18707 kind of error) occurred
18708 Request:
18710 POST /_search_analyze HTTP/1.1
18712 Host: localhost:5984
18713 Content-Type: application/json
18714 {"analyzer":"english", "text":"running"}
18716 Response:
18718 {
18720 "tokens": [
18721 "run"
18722 ]
18723 }
18724 /_utils
18726 GET /_utils
18727 Accesses the built-in Fauxton administration interface for
18728 CouchDB.
18729 Response Headers
18731 · Location – New URI location
18733 Status Codes
18735 · 301 Moved Permanently – Redirects to GET /_utils/
18737 GET /_utils/
18739 Response Headers
18741 · Content-Type – text/html
18743 · Last-Modified – Static files modification timestamp
18745 Status Codes
18747 · 200 OK – Request completed successfully
18749 /_up
18751 New in version 2.0.
18752 GET /_up
18755 Confirms that the server is up, running, and ready to respond to
18756 requests. If maintenance_mode is true or nolb, the endpoint
18757 will return a 404 response.
18758 Response Headers
18760 · Content-Type – application/json
18762 Status Codes
18764 · 200 OK – Request completed successfully
18766 · 404 Not Found – The server is unavaialble for requests
18768 at this time.
18769 Response:
18771 HTTP/1.1 200 OK
18773 Cache-Control: must-revalidate
18774 Content-Length: 16
18775 Content-Type: application/json
18776 Date: Sat, 17 Mar 2018 04:46:26 GMT
18777 Server: CouchDB/2.2.0-f999071ec (Erlang OTP/19)
18778 X-Couch-Request-ID: c57a3b2787
18779 X-CouchDB-Body-Time: 0
18780 {"status":"ok"}
18782 /_uuids
18784 Changed in version 2.0.0.
18785 GET /_uuids
18788 Requests one or more Universally Unique Identifiers (UUIDs) from
18789 the CouchDB instance. The response is a JSON object providing a
18790 list of UUIDs.
18791 Request Headers
18793 · Accept – .INDENT 2.0
18795 · application/json
18797 · text/plain
18799 Query Parameters
18802 · count (number) – Number of UUIDs to return. Default is 1.
18804 Response Headers
18806 · Content-Type – .INDENT 2.0
18808 · application/json
18810 · text/plain; charset=utf-8
18812 · ETag – Response hash
18815 Status Codes
18817 · 200 OK – Request completed successfully
18819 · 400 Bad Request – Requested more UUIDs than is allowed to
18821 retrieve
18822
18824 GET /_uuids?count=10 HTTP/1.1
18826 Accept: application/json
18827 Host: localhost:5984
18828 Response:
18830 HTTP/1.1 200 OK
18832 Content-Length: 362
18833 Content-Type: application/json
18834 Date: Sat, 10 Aug 2013 11:46:25 GMT
18835 ETag: "DGRWWQFLUDWN5MRKSLKQ425XV"
18836 Expires: Fri, 01 Jan 1990 00:00:00 GMT
18837 Pragma: no-cache
18838 Server: CouchDB (Erlang/OTP)
18839 {
18841 "uuids": [
18842 "75480ca477454894678e22eec6002413",
18843 "75480ca477454894678e22eec600250b",
18844 "75480ca477454894678e22eec6002c41",
18845 "75480ca477454894678e22eec6003b90",
18846 "75480ca477454894678e22eec6003fca",
18847 "75480ca477454894678e22eec6004bef",
18848 "75480ca477454894678e22eec600528f",
18849 "75480ca477454894678e22eec6005e0b",
18850 "75480ca477454894678e22eec6006158",
18851 "75480ca477454894678e22eec6006161"
18852 ]
18853 }
18854The UUID type is determined by the UUID algorithm setting in the CouchDB con‐
18856figuration.
18857The UUID type may be changed at any time through the Configuration API. For
18859example, the UUID type could be changed to random by sending this HTTP
18860request:
18861 PUT http://couchdb:5984/_node/nonode@nohost/_config/uuids/algorithm HTTP/1.1
18863 Content-Type: application/json
18864 Accept: */*
18865 "random"
18867 You can verify the change by obtaining a list of UUIDs:
18869 {
18871 "uuids" : [
18872 "031aad7b469956cf2826fcb2a9260492",
18873 "6ec875e15e6b385120938df18ee8e496",
18874 "cff9e881516483911aa2f0e98949092d",
18875 "b89d37509d39dd712546f9510d4a9271",
18876 "2e0dbf7f6c4ad716f21938a016e4e59f"
18877 ]
18878 }
18879 /favicon.ico
18881 GET /favicon.ico
18882 Binary content for the favicon.ico site icon.
18883 Response Headers
18885 · Content-Type – image/x-icon
18887 Status Codes
18889 · 200 OK – Request completed successfully
18891 · 404 Not Found – The requested content could not be
18893 found
18894 /_reshard
18896 New in version 2.4.
18897 GET /_reshard
18900 Returns a count of completed, failed, running, stopped, and
18901 total jobs along with the state of resharding on the cluster.
18902 Request Headers
18904 · Accept – .INDENT 2.0
18906 · application/json
18908 Response Headers
18911 · Content-Type – .INDENT 2.0
18913 · application/json
18915 Response JSON Object
18918 · state (string) – stopped or running
18920 · state_reason (string) – null or string describing additional
18922 information or reason associated with the state
18923 · completed (number) – Count of completed resharding jobs
18925 · failed (number) – Count of failed resharding jobs
18927 · running (number) – Count of running resharding jobs
18929 · stopped (number) – Count of stopped resharding jobs
18931 · total (number) – Total count of resharding jobs
18933 Status Codes
18935 · 200 OK – Request completed successfully
18937 · 401 Unauthorized – CouchDB Server Administrator privileges
18939 required
18940
18942 GET /_reshard HTTP/1.1
18944 Accept: application/json
18945 Host: localhost:5984
18946 Response:
18948 HTTP/1.1 200 OK
18950 Content-Type: application/json
18951 {
18953 "completed": 21,
18954 "failed": 0,
18955 "running": 3,
18956 "state": "running",
18957 "state_reason": null,
18958 "stopped": 0,
18959 "total": 24
18960 }
18961 GET /_reshard/state
18963 Returns the resharding state and optional information about the
18964 state.
18965 Request Headers
18967 · Accept – .INDENT 2.0
18969 · application/json
18971 Response Headers
18974 · Content-Type – .INDENT 2.0
18976 · application/json
18978 Response JSON Object
18981 · state (string) – stopped or running
18983 · state_reason (string) – Additional information or reason
18985 associated with the state
18986 Status Codes
18988 · 200 OK – Request completed successfully
18990 · 401 Unauthorized – CouchDB Server Administrator privileges
18992 required
18993
18995 GET /_reshard/state HTTP/1.1
18997 Accept: application/json
18998 Host: localhost:5984
18999 Response:
19001 HTTP/1.1 200 OK
19003 Content-Type: application/json
19004 {
19006 "reason": null,
19007 "state": "running"
19008 }
19009 PUT /_reshard/state
19011 Change the resharding state on the cluster. The states are
19012 stopped or running. This starts and stops global resharding on
19013 all the nodes of the cluster. If there are any running jobs,
19014 they will be stopped when the state changes to stopped. When the
19015 state changes back to running those job will continue running.
19016 Request Headers
19018 · Accept – .INDENT 2.0
19020 · application/json
19022 Response Headers
19025 · Content-Type – .INDENT 2.0
19027 · application/json
19029 Request JSON Object
19032 · state (string) – stopped or running
19034 · state_reason (string) – Optional string describing additional
19036 information or reason associated with the state
19037 Response JSON Object
19039 · ok (boolean) – true
19041 Status Codes
19043 · 200 OK – Request completed successfully
19045 · 400 Bad Request – Invalid request. Could be a bad or missing
19047 state name.
19048 · 401 Unauthorized – CouchDB Server Administrator privileges
19050 required
19051
19053 PUT /_reshard/state HTTP/1.1
19055 Accept: application/json
19056 Host: localhost:5984
19057 {
19059 "state": "stopped",
19060 "reason": "Rebalancing in progress"
19061 }
19062 Response:
19064 HTTP/1.1 200 OK
19066 Content-Type: application/json
19067 {
19069 "ok": true
19070 }
19071 GET /_reshard/jobs
19073 NOTE:
19075 The shape of the response and the total_rows and offset field
19076 in particular are meant to be consistent with the _sched‐
19077 uler/jobs endpoint.
19078 Request Headers
19080 · Accept – .INDENT 2.0
19082 · application/json
19084 Response Headers
19087 · Content-Type – .INDENT 2.0
19089 · application/json
19091 Response JSON Object
19094 · jobs (list) – Array of json objects, one for each resharding
19096 job. For the fields of each job see the /_reshard/job/{jobid}
19097 endpoint.
19098 · offset (number) – Offset in the list of jobs object. Currently
19100 hard-coded at 0.
19101 · total_rows (number) – Total number of resharding jobs on the
19103 cluster.
19104 Status Codes
19106 · 200 OK – Request completed successfully
19108 · 401 Unauthorized – CouchDB Server Administrator privileges
19110 required
19111
19113 GET /_reshard/jobs HTTP/1.1
19115 Accept: application/json
19116 Response:
19118 HTTP/1.1 200 OK
19120 Content-Type: application/json
19121 {
19123 "jobs": [
19124 {
19125 "history": [
19126 {
19127 "detail": null,
19128 "timestamp": "2019-03-28T15:28:02Z",
19129 "type": "new"
19130 },
19131 {
19132 "detail": "initial_copy",
19133 "timestamp": "2019-03-28T15:28:02Z",
19134 "type": "running"
19135 },
19136 ...
19137 ],
19138 "id": "001-171d1211418996ff47bd610b1d1257fc4ca2628868def4a05e63e8f8fe50694a",
19139 "job_state": "completed",
19140 "node": "node1@127.0.0.1",
19141 "source": "shards/00000000-1fffffff/d1.1553786862",
19142 "split_state": "completed",
19143 "start_time": "2019-03-28T15:28:02Z",
19144 "state_info": {},
19145 "target": [
19146 "shards/00000000-0fffffff/d1.1553786862",
19147 "shards/10000000-1fffffff/d1.1553786862"
19148 ],
19149 "type": "split",
19150 "update_time": "2019-03-28T15:28:08Z"
19151 },
19152 ...
19153 ],
19154 "offset": 0,
19155 "total_rows": 24
19156 }
19157 GET /_reshard/jobs/{jobid}
19159 Get information about the resharding job identified by jobid.
19160 Request Headers
19162 · Accept – .INDENT 2.0
19164 · application/json
19166 Response Headers
19169 · Content-Type – .INDENT 2.0
19171 · application/json
19173 Response JSON Object
19176 · id (string) – Job ID.
19178 · type (string) – Currently only split is implemented.
19180 · job_state (string) – The running state of the job. Could be
19182 one of new, running, stopped, completed or failed.
19183 · split_state (string) – State detail specific to shard split‐
19185 ting. It indicates how far has shard splitting progressed, and
19186 can be one of new, initial_copy, topoff1, build_indices,
19187 topoff2, copy_local_docs, update_shardmap, wait_source_close,
19188 topoff3, source_delete or completed.
19189 · state_info (object) – Optional additional info associated with
19191 the current state.
19192 · source (string) – For split jobs this will be the source
19194 shard.
19195 · target (list) – For split jobs this will be a list of two or
19197 more target shards.
19198 · history (list) – List of json objects recording a job’s state
19200 transition history.
19201 Status Codes
19203 · 200 OK – Request completed successfully
19205 · 401 Unauthorized – CouchDB Server Administrator privileges
19207 required
19208
19210 GET /_reshard/jobs/001-171d1211418996ff47bd610b1d1257fc4ca2628868def4a05e63e8f8fe50694a HTTP/1.1
19212 Accept: application/json
19213 Response:
19215 HTTP/1.1 200 OK
19217 Content-Type: application/json
19218 {
19220 "id": "001-171d1211418996ff47bd610b1d1257fc4ca2628868def4a05e63e8f8fe50694a",
19222 "job_state": "completed",
19223 "node": "node1@127.0.0.1",
19224 "source": "shards/00000000-1fffffff/d1.1553786862",
19225 "split_state": "completed",
19226 "start_time": "2019-03-28T15:28:02Z",
19227 "state_info": {},
19228 "target": [
19229 "shards/00000000-0fffffff/d1.1553786862",
19230 "shards/10000000-1fffffff/d1.1553786862"
19231 ],
19232 "type": "split",
19233 "update_time": "2019-03-28T15:28:08Z",
19234 "history": [
19235 {
19236 "detail": null,
19237 "timestamp": "2019-03-28T15:28:02Z",
19238 "type": "new"
19239 },
19240 {
19241 "detail": "initial_copy",
19242 "timestamp": "2019-03-28T15:28:02Z",
19243 "type": "running"
19244 },
19245 ...
19246 ]
19247 }
19248 POST /_reshard/jobs
19250 Depending on what fields are specified in the request, one or
19251 more resharding jobs will be created. The response is a json
19252 array of results. Each result object represents a single
19253 resharding job for a particular node and range. Some of the
19254 responses could be successful and some could fail. Successful
19255 results will have the "ok": true key and and value, and failed
19256 jobs will have the "error": "{error_message}" key and value.
19257 Request Headers
19259 · Accept – .INDENT 2.0
19261 · application/json
19263 Response Headers
19266 · Content-Type – .INDENT 2.0
19268 · application/json
19270 Request JSON Object
19273 · type (string) – Type of job. Currently only "split" is
19275 accepted.
19276 · db (string) – Database to split. This is mutually exclusive
19278 with the "shard” field.
19279 · node (string) – Split shards on a particular node. This is an
19281 optional parameter. The value should be one of the nodes
19282 returned from the _membership endpoint.
19283 · range (string) – Split shards copies in the given range. The
19285 range format is hhhhhhhh-hhhhhhhh where h is a hexadecimal
19286 digit. This format is used since this is how the ranges are
19287 represented in the file system. This is parameter is optional
19288 and is mutually exclusive with the "shard" field.
19289 · shard (string) – Split a particular shard. The shard should be
19291 specified as "shards/{range}/{db}.{suffix}". Where range has
19292 the hhhhhhhh-hhhhhhhh format, db is the database name, and
19293 suffix is the shard (timestamp) creation suffix.
19294 · error (string) – Error message if a job could be not be cre‐
19296 ated.
19297 · node – Cluster node where the job was created and is running.
19299 Response JSON Object
19301 · ok (boolean) – true if job created successfully.
19303 Status Codes
19305 · 201 Created – One or more jobs were successfully created
19307 · 400 Bad Request – Invalid request. Parameter validation might
19309 have failed.
19310 · 401 Unauthorized – CouchDB Server Administrator privileges
19312 required
19313 · 404 Not Found – Db, node, range or shard was not found
19315
19317 POST /_reshard/jobs HTTP/1.1
19319 Accept: application/json
19320 Content-Type: application/json
19321 {
19323 "db": "db3",
19324 "range": "80000000-ffffffff",
19325 "type": "split"
19326 }
19327 Response:
19329 HTTP/1.1 201 Created
19331 Content-Type: application/json
19332 [
19334 {
19335 "id": "001-30d7848a6feeb826d5e3ea5bb7773d672af226fd34fd84a8fb1ca736285df557",
19336 "node": "node1@127.0.0.1",
19337 "ok": true,
19338 "shard": "shards/80000000-ffffffff/db3.1554148353"
19339 },
19340 {
19341 "id": "001-c2d734360b4cb3ff8b3feaccb2d787bf81ce2e773489eddd985ddd01d9de8e01",
19342 "node": "node2@127.0.0.1",
19343 "ok": true,
19344 "shard": "shards/80000000-ffffffff/db3.1554148353"
19345 }
19346 ]
19347 DELETE /_reshard/jobs/{jobid}
19349 If the job is running, stop the job and then remove it.
19350 Response JSON Object
19352 · ok (boolean) – true if the job was removed success‐
19354 fully.
19355 Status Codes
19357 · 200 OK – The job was removed successfully
19359 · 401 Unauthorized – CouchDB Server Administrator privi‐
19361 leges required
19362 · 404 Not Found – The job was not found
19364 Request:
19366 DELETE /_reshard/jobs/001-171d1211418996ff47bd610b1d1257fc4ca2628868def4a05e63e8f8fe50694a HTTP/1.1
19368 Response:
19370 HTTP/1.1 200 OK
19372 Content-Type: application/json
19373 {
19375 "ok": true
19376 }
19377 GET /_reshard/jobs/{jobid}/state
19379 Returns the running state of a resharding job identified by
19380 jobid.
19381 Request Headers
19383 · Accept – .INDENT 2.0
19385 · application/json
19387 Response Headers
19390 · Content-Type – .INDENT 2.0
19392 · application/json
19394 Request JSON Object
19397 · state (string) – One of new, running, stopped, completed or
19399 failed.
19400 · state_reason (string) – Additional information associated with
19402 the state
19403 Status Codes
19405 · 200 OK – Request completed successfully
19407 · 401 Unauthorized – CouchDB Server Administrator privileges
19409 required
19410 · 404 Not Found – The job was not found
19412
19414 GET /_reshard/jobs/001-b3da04f969bbd682faaab5a6c373705cbcca23f732c386bb1a608cfbcfe9faff/state HTTP/1.1
19416 Accept: application/json
19417 Host: localhost:5984
19418 Response:
19420 HTTP/1.1 200 OK
19422 Content-Type: application/json
19423 {
19425 "reason": null,
19426 "state": "running"
19427 }
19428 PUT /_reshard/jobs/{jobid}/state
19430 Change the state of a particular resharding job identified by
19431 jobid. The state can be changed from stopped to running or from
19432 running to stopped. If an individual job is stopped via this API
19433 it will stay stopped even after the global resharding state is
19434 toggled from stopped to running. If the job is already completed
19435 its state will stay completed.
19436 Request Headers
19438 · Accept – .INDENT 2.0
19440 · application/json
19442 Response Headers
19445 · Content-Type – .INDENT 2.0
19447 · application/json
19449 Request JSON Object
19452 · state (string) – stopped or running
19454 · state_reason (string) – Optional string describing additional
19456 information or reason associated with the state
19457 Response JSON Object
19459 · ok (boolean) – true
19461 Status Codes
19463 · 200 OK – Request completed successfully
19465 · 400 Bad Request – Invalid request. Could be a bad state name,
19467 for example.
19468 · 401 Unauthorized – CouchDB Server Administrator privileges
19470 required
19471 · 404 Not Found – The job was not found
19473
19475 PUT /_reshard/state/001-b3da04f969bbd682faaab5a6c373705cbcca23f732c386bb1a608cfbcfe9faff/state HTTP/1.1
19477 Accept: application/json
19478 Host: localhost:5984
19479 {
19481 "state": "stopped",
19482 "reason": "Rebalancing in progress"
19483 }
19484 Response:
19486 HTTP/1.1 200 OK
19488 Content-Type: application/json
19489 {
19491 "ok": true
19492 }
19493 Authentication
19495 Interfaces for obtaining session and authorization data.
19496 NOTE:
19498 We also strongly recommend you set up SSL to improve all authentica‐
19499 tion methods’ security.
19500 Basic Authentication
19502 Basic authentication (RFC 2617) is a quick and simple way to authenti‐
19503 cate with CouchDB. The main drawback is the need to send user creden‐
19504 tials with each request which may be insecure and could hurt operation
19505 performance (since CouchDB must compute the password hash with every
19506 request):
19507 Request:
19509 GET / HTTP/1.1
19511 Accept: application/json
19512 Authorization: Basic cm9vdDpyZWxheA==
19513 Host: localhost:5984
19514 Response:
19516 HTTP/1.1 200 OK
19518 Cache-Control: must-revalidate
19519 Content-Length: 177
19520 Content-Type: application/json
19521 Date: Mon, 03 Dec 2012 00:44:47 GMT
19522 Server: CouchDB (Erlang/OTP)
19523 {
19525 "couchdb":"Welcome",
19526 "uuid":"0a959b9b8227188afc2ac26ccdf345a6",
19527 "version":"1.3.0",
19528 "vendor": {
19529 "version":"1.3.0",
19530 "name":"The Apache Software Foundation"
19531 }
19532 }
19533 Cookie Authentication
19535 For cookie authentication (RFC 2109) CouchDB generates a token that the
19536 client can use for the next few requests to CouchDB. Tokens are valid
19537 until a timeout. When CouchDB sees a valid token in a subsequent
19538 request, it will authenticate the user by this token without requesting
19539 the password again. By default, cookies are valid for 10 minutes, but
19540 it’s adjustable. Also it’s possible to make cookies persistent.
19541 To obtain the first token and thus authenticate a user for the first
19543 time, the username and password must be sent to the _session API.
19544 /_session
19546 POST /_session
19547 Initiates new session for specified user credentials by provid‐
19548 ing Cookie value.
19549 Request Headers
19551 · Content-Type – .INDENT 2.0
19553 · application/x-www-form-urlencoded
19555 · application/json
19557 Query Parameters
19560 · next (string) – Enforces redirect after successful login to
19562 the specified location. This location is relative from server
19563 root. Optional.
19564 Form Parameters
19566 · name – User name
19568 · password – Password
19570 Response Headers
19572 · Set-Cookie – Authorization token
19574 Response JSON Object
19576 · ok (boolean) – Operation status
19578 · name (string) – Username
19580 · roles (array) – List of user roles
19582 Status Codes
19584 · 200 OK – Successfully authenticated
19586 · 302 Found – Redirect after successful authentication
19588 · 401 Unauthorized – Username or password wasn’t recognized
19590 Request:
19592 POST /_session HTTP/1.1
19594 Accept: application/json
19595 Content-Length: 24
19596 Content-Type: application/x-www-form-urlencoded
19597 Host: localhost:5984
19598 name=root&password=relax
19600 It’s also possible to send data as JSON:
19602 POST /_session HTTP/1.1
19604 Accept: application/json
19605 Content-Length: 37
19606 Content-Type: application/json
19607 Host: localhost:5984
19608 {
19610 "name": "root",
19611 "password": "relax"
19612 }
19613 Response:
19615 HTTP/1.1 200 OK
19617 Cache-Control: must-revalidate
19618 Content-Length: 43
19619 Content-Type: application/json
19620 Date: Mon, 03 Dec 2012 01:23:14 GMT
19621 Server: CouchDB (Erlang/OTP)
19622 Set-Cookie: AuthSession=cm9vdDo1MEJCRkYwMjq0LO0ylOIwShrgt8y-UkhI-c6BGw; Version=1; Path=/; HttpOnly
19623 {"ok":true,"name":"root","roles":["_admin"]}
19625 If next query parameter was provided the response will trigger
19627 redirection to the specified location in case of successful
19628 authentication:
19629 Request:
19631 POST /_session?next=/blog/_design/sofa/_rewrite/recent-posts HTTP/1.1
19633 Accept: application/json
19634 Content-Type: application/x-www-form-urlencoded
19635 Host: localhost:5984
19636 name=root&password=relax
19638 Response:
19640 HTTP/1.1 302 Moved Temporarily
19642 Cache-Control: must-revalidate
19643 Content-Length: 43
19644 Content-Type: application/json
19645 Date: Mon, 03 Dec 2012 01:32:46 GMT
19646 Location: http://localhost:5984/blog/_design/sofa/_rewrite/recent-posts
19647 Server: CouchDB (Erlang/OTP)
19648 Set-Cookie: AuthSession=cm9vdDo1MEJDMDEzRTp7Vu5GKCkTxTVxwXbpXsBARQWnhQ; Version=1; Path=/; HttpOnly
19649 {"ok":true,"name":null,"roles":["_admin"]}
19651 GET /_session
19653 Returns information about the authenticated user, including a
19654 userctx_object, the authentication method and database that were
19655 used, and a list of configured authentication handlers on the
19656 server.
19657 Query Parameters
19659 · basic (boolean) – Accept Basic Auth by requesting this
19661 resource. Optional.
19662 Response JSON Object
19664 · ok (boolean) – Operation status
19666 · userCtx (object) – User context for the current user
19668 · info (object) – Server authentication configuration
19670 Status Codes
19672 · 200 OK – Successfully authenticated.
19674 · 401 Unauthorized – Username or password wasn’t recog‐
19676 nized.
19677 Request:
19679 GET /_session HTTP/1.1
19681 Host: localhost:5984
19682 Accept: application/json
19683 Cookie: AuthSession=cm9vdDo1MEJDMDQxRDpqb-Ta9QfP9hpdPjHLxNTKg_Hf9w
19684 Response:
19686 HTTP/1.1 200 OK
19688 Cache-Control: must-revalidate
19689 Content-Length: 175
19690 Content-Type: application/json
19691 Date: Fri, 09 Aug 2013 20:27:45 GMT
19692 Server: CouchDB (Erlang/OTP)
19693 Set-Cookie: AuthSession=cm9vdDo1MjA1NTBDMTqmX2qKt1KDR--GUC80DQ6-Ew_XIw; Version=1; Path=/; HttpOnly
19694 {
19696 "info": {
19697 "authenticated": "cookie",
19698 "authentication_db": "_users",
19699 "authentication_handlers": [
19700 "cookie",
19701 "default"
19702 ]
19703 },
19704 "ok": true,
19705 "userCtx": {
19706 "name": "root",
19707 "roles": [
19708 "_admin"
19709 ]
19710 }
19711 }
19712 DELETE /_session
19714 Closes user’s session by instructing the browser to clear the
19715 cookie. This does not invalidate the session from the server’s
19716 perspective, as there is no way to do this because CouchDB cook‐
19717 ies are stateless. This means calling this endpoint is purely
19718 optional from a client perspective, and it does not protect
19719 against theft of a session cookie.
19720 Status Codes
19722 · 200 OK – Successfully close session.
19724 Request:
19726 DELETE /_session HTTP/1.1
19728 Accept: application/json
19729 Cookie: AuthSession=cm9vdDo1MjA1NEVGMDo1QXNQkqC_0Qmgrk8Fw61_AzDeXw
19730 Host: localhost:5984
19731 Response:
19733 HTTP/1.1 200 OK
19735 Cache-Control: must-revalidate
19736 Content-Length: 12
19737 Content-Type: application/json
19738 Date: Fri, 09 Aug 2013 20:30:12 GMT
19739 Server: CouchDB (Erlang/OTP)
19740 Set-Cookie: AuthSession=; Version=1; Path=/; HttpOnly
19741 {
19743 "ok": true
19744 }
19745 Proxy Authentication
19747 NOTE:
19748 To use this authentication method make sure that the {chttpd_auth,
19749 proxy_authentication_handler} value is added to the list of the
19750 active chttpd/authentication_handlers:
19751 [chttpd]
19753 authentication_handlers = {chttpd_auth, cookie_authentication_handler}, {chttpd_auth, proxy_authentication_handler}, {chttpd_auth, default_authentication_handler}
19754 Proxy authentication is very useful in case your application already
19756 uses some external authentication service and you don’t want to dupli‐
19757 cate users and their roles in CouchDB.
19758 This authentication method allows creation of a userctx_object for
19760 remotely authenticated user. By default, the client just needs to pass
19761 specific headers to CouchDB with related requests:
19762 · X-Auth-CouchDB-UserName: username;
19764 · X-Auth-CouchDB-Roles: comma-separated (,) list of user roles;
19766 · X-Auth-CouchDB-Token: authentication token. When proxy_use_secret is
19768 set (which is strongly recommended!), this header provides an HMAC of
19769 the username to authenticate and the secret token to prevent requests
19770 from untrusted sources.
19771 Request:
19773 GET /_session HTTP/1.1
19775 Host: localhost:5984
19776 Accept: application/json
19777 Content-Type: application/json; charset=utf-8
19778 X-Auth-CouchDB-Roles: users,blogger
19779 X-Auth-CouchDB-UserName: foo
19780 Response:
19782 HTTP/1.1 200 OK
19784 Cache-Control: must-revalidate
19785 Content-Length: 190
19786 Content-Type: application/json
19787 Date: Fri, 14 Jun 2013 10:16:03 GMT
19788 Server: CouchDB (Erlang/OTP)
19789 {
19791 "info": {
19792 "authenticated": "proxy",
19793 "authentication_db": "_users",
19794 "authentication_handlers": [
19795 "cookie",
19796 "proxy",
19797 "default"
19798 ]
19799 },
19800 "ok": true,
19801 "userCtx": {
19802 "name": "foo",
19803 "roles": [
19804 "users",
19805 "blogger"
19806 ]
19807 }
19808 }
19809 Note that you don’t need to request session to be authenticated by this
19811 method if all required HTTP headers are provided.
19812 JWT Authentication
19814 NOTE:
19815 To use this authentication method, make sure that the {chttpd_auth,
19816 jwt_authentication_handler} value is added to the list of the active
19817 chttpd/authentication_handlers:
19818 [chttpd]
19820 authentication_handlers = {chttpd_auth, cookie_authentication_handler}, {chttpd_auth, jwt_authentication_handler}, {chttpd_auth, default_authentication_handler}
19821 JWT authentication enables CouchDB to use externally-generated JWT
19823 tokens instead of defining users or roles in the _users database.
19824 The JWT authentication handler requires that all JWT tokens are signed
19826 by a key that CouchDB has been configured to trust (there is no support
19827 for JWT’s “NONE” algorithm).
19828 Additionally, CouchDB can be configured to reject JWT tokens that are
19830 missing a configurable set of claims (e.g, a CouchDB administrator
19831 could insist on the exp claim).
19832 All claims presented in a JWT token are validated if presented, regard‐
19834 less of whether they are required.
19835 Two sections of config exist to configure JWT authentication;
19837 The required_claims config setting is a comma-separated list of addi‐
19839 tional mandatory JWT claims that must be present in any presented JWT
19840 token. A :code 400:Bad Request is sent if any are missing.
19841 The alg claim is mandatory as it used to lookup the correct key for
19843 verifying the signature.
19844 The sub claim is mandatory and is used as the CouchDB user’s name if
19846 the JWT token is valid.
19847 A private claim called _couchdb.roles is optional. If presented, as a
19849 JSON array of strings, it is used as the CouchDB user’s roles list as
19850 long as the JWT token is valid.
19851 ; [jwt_keys]
19853 ; Configure at least one key here if using the JWT auth handler.
19854 ; If your JWT tokens do not include a "kid" attribute, use "_default"
19855 ; as the config key, otherwise use the kid as the config key.
19856 ; Examples
19857 ; hmac:_default = aGVsbG8=
19858 ; hmac:foo = aGVsbG8=
19859 ; The config values can represent symmetric and asymmetrics keys.
19860 ; For symmetrics keys, the value is base64 encoded;
19861 ; hmac:_default = aGVsbG8= # base64-encoded form of "hello"
19862 ; For asymmetric keys, the value is the PEM encoding of the public
19863 ; key with newlines replaced with the escape sequence \n.
19864 ; rsa:foo = -----BEGIN PUBLIC KEY-----\nMIIBIjAN...IDAQAB\n-----END PUBLIC KEY-----\n
19865 ; ec:bar = -----BEGIN PUBLIC KEY-----\nMHYwEAYHK...AzztRs\n-----END PUBLIC KEY-----\n
19866 The jwt_key section lists all the keys that this CouchDB server trusts.
19868 You should ensure that all nodes of your cluster have the same list.
19869 JWT tokens that do not include a kid claim will be validated against
19871 the $alg:_default key.
19872 It is mandatory to specify the algorithm associated with every key for
19874 security reasons (notably presenting a HMAC-signed token using an RSA
19875 or EC public key that the server trusts:
19876 https://auth0.com/blog/critical-vulnerabilities-in-json-web-token-libraries/).
19877 Request:
19879 GET /_session HTTP/1.1
19881 Host: localhost:5984
19882 Accept: application/json
19883 Content-Type: application/json; charset=utf-8
19884 Authorization: Bearer <JWT token>
19885 Response:
19887 HTTP/1.1 200 OK
19889 Cache-Control: must-revalidate
19890 Content-Length: 188
19891 Content-Type: application/json
19892 Date: Sun, 19 Apr 2020 08:29:15 GMT
19893 Server: CouchDB (Erlang/OTP)
19894 {
19896 "info": {
19897 "authenticated": "jwt",
19898 "authentication_db": "_users",
19899 "authentication_handlers": [
19900 "cookie",
19901 "proxy",
19902 "default"
19903 ]
19904 },
19905 "ok": true,
19906 "userCtx": {
19907 "name": "foo",
19908 "roles": [
19909 "users",
19910 "blogger"
19911 ]
19912 }
19913 }
19914 Note that you don’t need to request session to be authenticated by this
19916 method if the required HTTP header is provided.
19917 Configuration
19919 The CouchDB Server Configuration API provide an interface to query and
19920 update the various configuration values within a running CouchDB
19921 instance.
19922 Accessing the local node’s configuration
19924 The literal string _local serves as an alias for the local node name,
19925 so for all configuration URLs, {node-name} may be replaced with _local,
19926 to interact with the local node’s configuration.
19927 /_node/{node-name}/_config
19929 GET /_node/{node-name}/_config
19930 Returns the entire CouchDB server configuration as a JSON struc‐
19931 ture. The structure is organized by different configuration sec‐
19932 tions, with individual values.
19933 Request Headers
19935 · Accept – .INDENT 2.0
19937 · application/json
19939 · text/plain
19941 Response Headers
19944 · Content-Type – .INDENT 2.0
19946 · application/json
19948 · text/plain; charset=utf-8
19950 Status Codes
19953 · 200 OK – Request completed successfully
19955 · 401 Unauthorized – CouchDB Server Administrator privileges
19957 required
19958
19960 GET /_node/nonode@nohost/_config HTTP/1.1
19962 Accept: application/json
19963 Host: localhost:5984
19964 Response:
19966 HTTP/1.1 200 OK
19968 Cache-Control: must-revalidate
19969 Content-Length: 4148
19970 Content-Type: application/json
19971 Date: Sat, 10 Aug 2013 12:01:42 GMT
19972 Server: CouchDB (Erlang/OTP)
19973 {
19975 "attachments": {
19976 "compressible_types": "text/*, application/javascript, application/json, application/xml",
19977 "compression_level": "8"
19978 },
19979 "couchdb": {
19980 "users_db_suffix": "_users",
19981 "database_dir": "/var/lib/couchdb",
19982 "max_attachment_chunk_size": "4294967296",
19983 "max_dbs_open": "100",
19984 "os_process_timeout": "5000",
19985 "uri_file": "/var/lib/couchdb/couch.uri",
19986 "util_driver_dir": "/usr/lib64/couchdb/erlang/lib/couch-1.5.0/priv/lib",
19987 "view_index_dir": "/var/lib/couchdb"
19988 },
19989 "chttpd": {
19990 "backlog": "512",
19991 "bind_address": "0.0.0.0",
19992 "port": "5984",
19993 "require_valid_user": "false",
19994 "socket_options": "[{sndbuf, 262144}, {nodelay, true}]",
19995 "server_options": "[{recbuf, undefined}]"
19996 },
19997 "httpd": {
19998 "allow_jsonp": "false",
19999 "authentication_handlers": "{couch_httpd_auth, cookie_authentication_handler}, {couch_httpd_auth, default_authentication_handler}",
20000 "bind_address": "192.168.0.2",
20001 "max_connections": "2048",
20002 "port": "5984",
20003 "secure_rewrites": "true"
20004 },
20005 "log": {
20006 "writer": "file",
20007 "file": "/var/log/couchdb/couch.log",
20008 "include_sasl": "true",
20009 "level": "info"
20010 },
20011 "query_server_config": {
20012 "reduce_limit": "true"
20013 },
20014 "replicator": {
20015 "max_http_pipeline_size": "10",
20016 "max_http_sessions": "10"
20017 },
20018 "stats": {
20019 "interval": "10"
20020 },
20021 "uuids": {
20022 "algorithm": "utc_random"
20023 }
20024 }
20025 /_node/{node-name}/_config/{section}
20027 GET /_node/{node-name}/_config/{section}
20028 Gets the configuration structure for a single section.
20029 Parameters
20031 · section – Configuration section name
20033 Request Headers
20035 · Accept – .INDENT 2.0
20037 · application/json
20039 · text/plain
20041 Response Headers
20044 · Content-Type – .INDENT 2.0
20046 · application/json
20048 · text/plain; charset=utf-8
20050 Status Codes
20053 · 200 OK – Request completed successfully
20055 · 401 Unauthorized – CouchDB Server Administrator privileges
20057 required
20058
20060 GET /_node/nonode@nohost/_config/httpd HTTP/1.1
20062 Accept: application/json
20063 Host: localhost:5984
20064 Response:
20066 HTTP/1.1 200 OK
20068 Cache-Control: must-revalidate
20069 Content-Length: 444
20070 Content-Type: application/json
20071 Date: Sat, 10 Aug 2013 12:10:40 GMT
20072 Server: CouchDB (Erlang/OTP)
20073 {
20075 "allow_jsonp": "false",
20076 "authentication_handlers": "{couch_httpd_auth, cookie_authentication_handler}, {couch_httpd_auth, default_authentication_handler}",
20077 "bind_address": "127.0.0.1",
20078 "default_handler": "{couch_httpd_db, handle_request}",
20079 "enable_cors": "false",
20080 "port": "5984",
20081 "secure_rewrites": "true"
20082 }
20083 /_node/{node-name}/_config/{section}/{key}
20085 GET /_node/{node-name}/_config/{section}/{key}
20086 Gets a single configuration value from within a specific config‐
20087 uration section.
20088 Parameters
20090 · section – Configuration section name
20092 · key – Configuration option name
20094 Request Headers
20096 · Accept – .INDENT 2.0
20098 · application/json
20100 · text/plain
20102 Response Headers
20105 · Content-Type – .INDENT 2.0
20107 · application/json
20109 · text/plain; charset=utf-8
20111 Status Codes
20114 · 200 OK – Request completed successfully
20116 · 401 Unauthorized – CouchDB Server Administrator privileges
20118 required
20119
20121 GET /_node/nonode@nohost/_config/log/level HTTP/1.1
20123 Accept: application/json
20124 Host: localhost:5984
20125 Response:
20127 HTTP/1.1 200 OK
20129 Cache-Control: must-revalidate
20130 Content-Length: 8
20131 Content-Type: application/json
20132 Date: Sat, 10 Aug 2013 12:12:59 GMT
20133 Server: CouchDB (Erlang/OTP)
20134 "debug"
20136 NOTE:
20138 The returned value will be the JSON of the value, which may
20139 be a string or numeric value, or an array or object. Some
20140 client environments may not parse simple strings or numeric
20141 values as valid JSON.
20142 PUT /_node/{node-name}/_config/{section}/{key}
20144 Updates a configuration value. The new value should be supplied
20145 in the request body in the corresponding JSON format. If you are
20146 setting a string value, you must supply a valid JSON string. In
20147 response CouchDB sends old value for target section key.
20148 Parameters
20150 · section – Configuration section name
20152 · key – Configuration option name
20154 Request Headers
20156 · Accept – .INDENT 2.0
20158 · application/json
20160 · text/plain
20162 · Content-Type – application/json
20165 Response Headers
20167 · Content-Type – .INDENT 2.0
20169 · application/json
20171 · text/plain; charset=utf-8
20173 Status Codes
20176 · 200 OK – Request completed successfully
20178 · 400 Bad Request – Invalid JSON request body
20180 · 401 Unauthorized – CouchDB Server Administrator privileges
20182 required
20183 · 500 Internal Server Error – Error setting configuration
20185
20187 PUT /_node/nonode@nohost/_config/log/level HTTP/1.1
20189 Accept: application/json
20190 Content-Length: 7
20191 Content-Type: application/json
20192 Host: localhost:5984
20193 "info"
20195 Response:
20197 HTTP/1.1 200 OK
20199 Cache-Control: must-revalidate
20200 Content-Length: 8
20201 Content-Type: application/json
20202 Date: Sat, 10 Aug 2013 12:12:59 GMT
20203 Server: CouchDB (Erlang/OTP)
20204 "debug"
20206 DELETE /_node/{node-name}/_config/{section}/{key}
20208 Deletes a configuration value. The returned JSON will be the
20209 value of the configuration parameter before it was deleted.
20210 Parameters
20212 · section – Configuration section name
20214 · key – Configuration option name
20216 Request Headers
20218 · Accept – .INDENT 2.0
20220 · application/json
20222 · text/plain
20224 Response Headers
20227 · Content-Type – .INDENT 2.0
20229 · application/json
20231 · text/plain; charset=utf-8
20233 Status Codes
20236 · 200 OK – Request completed successfully
20238 · 401 Unauthorized – CouchDB Server Administrator privileges
20240 required
20241 · 404 Not Found – Specified configuration option not found
20243
20245 DELETE /_node/nonode@nohost/_config/log/level HTTP/1.1
20247 Accept: application/json
20248 Host: localhost:5984
20249 Response:
20251 HTTP/1.1 200 OK
20253 Cache-Control: must-revalidate
20254 Content-Length: 7
20255 Content-Type: application/json
20256 Date: Sat, 10 Aug 2013 12:29:03 GMT
20257 Server: CouchDB (Erlang/OTP)
20258 "info"
20260 /_node/{node-name}/_config/_reload
20262 New in version 3.0.
20263 POST /_node/{node-name}/_config/_reload
20266 Reloads the configuration from disk. This has a side effect of
20267 flushing any in-memory configuration changes that have not been
20268 committed to disk.
20269 Request:
20271 POST /_node/nonode@nohost/_config/_reload HTTP/1.1
20273 Host: localhost:5984
20274 Response:
20276 HTTP/1.1 200 OK
20278 Cache-Control: must-revalidate
20279 Content-Length: 12
20280 Content-Type: application/json
20281 Date: Tues, 21 Jan 2020 11:09:35
20282 Server: CouchDB/3.0.0 (Erlang OTP)
20283 {"ok":true}
20285 Databases
20287 The Database endpoint provides an interface to an entire database with
20288 in CouchDB. These are database-level, rather than document-level
20289 requests.
20290 For all these requests, the database name within the URL path should be
20292 the database name that you wish to perform the operation on. For exam‐
20293 ple, to obtain the meta information for the database recipes, you would
20294 use the HTTP request:
20295 GET /recipes
20297 For clarity, the form below is used in the URL paths:
20299 GET /db
20301 Where db is the name of any database.
20303 /db
20305 HEAD /{db}
20306 Returns the HTTP Headers containing a minimal amount of informa‐
20307 tion about the specified database. Since the response body is
20308 empty, using the HEAD method is a lightweight way to check if
20309 the database exists already or not.
20310 Parameters
20312 · db – Database name
20314 Status Codes
20316 · 200 OK – Database exists
20318 · 404 Not Found – Requested database not found
20320 Request:
20322 HEAD /test HTTP/1.1
20324 Host: localhost:5984
20325 Response:
20327 HTTP/1.1 200 OK
20329 Cache-Control: must-revalidate
20330 Content-Type: application/json
20331 Date: Mon, 12 Aug 2013 01:27:41 GMT
20332 Server: CouchDB (Erlang/OTP)
20333 GET /{db}
20335 Gets information about the specified database.
20336 Parameters
20338 · db – Database name
20340 Request Headers
20342 · Accept – .INDENT 2.0
20344 · application/json
20346 · text/plain
20348 Response Headers
20351 · Content-Type – .INDENT 2.0
20353 · application/json
20355 · text/plain; charset=utf-8
20357 Response JSON Object
20360 · cluster.n (number) – Replicas. The number of copies of every
20362 document.
20363 · cluster.q (number) – Shards. The number of range partitions.
20365 · cluster.r (number) – Read quorum. The number of consistent
20367 copies of a document that need to be read before a successful
20368 reply.
20369 · cluster.w (number) – Write quorum. The number of copies of a
20371 document that need to be written before a successful reply.
20372 · compact_running (boolean) – Set to true if the database com‐
20374 paction routine is operating on this database.
20375 · db_name (string) – The name of the database.
20377 · disk_format_version (number) – The version of the physical
20379 format used for the data when it is stored on disk.
20380 · doc_count (number) – A count of the documents in the specified
20382 database.
20383 · doc_del_count (number) – Number of deleted documents
20385 · instance_start_time (string) – Always "0". (Returned for
20387 legacy reasons.)
20388 · purge_seq (string) – An opaque string that describes the purge
20390 state of the database. Do not rely on this string for counting
20391 the number of purge operations.
20392 · sizes.active (number) – The size of live data inside the data‐
20394 base, in bytes.
20395 · sizes.external (number) – The uncompressed size of database
20397 contents in bytes.
20398 · sizes.file (number) – The size of the database file on disk in
20400 bytes. Views indexes are not included in the calculation.
20401 · update_seq (string) – An opaque string that describes the
20403 state of the database. Do not rely on this string for counting
20404 the number of updates.
20405 · props.partitioned (boolean) – (optional) If present and true,
20407 this indicates that the database is partitioned.
20408 Status Codes
20410 · 200 OK – Request completed successfully
20412 · 404 Not Found – Requested database not found
20414
20416 GET /receipts HTTP/1.1
20418 Accept: application/json
20419 Host: localhost:5984
20420 Response:
20422 HTTP/1.1 200 OK
20424 Cache-Control: must-revalidate
20425 Content-Length: 258
20426 Content-Type: application/json
20427 Date: Mon, 12 Aug 2013 01:38:57 GMT
20428 Server: CouchDB (Erlang/OTP)
20429 {
20431 "cluster": {
20432 "n": 3,
20433 "q": 8,
20434 "r": 2,
20435 "w": 2
20436 },
20437 "compact_running": false,
20438 "db_name": "receipts",
20439 "disk_format_version": 6,
20440 "doc_count": 6146,
20441 "doc_del_count": 64637,
20442 "instance_start_time": "0",
20443 "props": {},
20444 "purge_seq": 0,
20445 "sizes": {
20446 "active": 65031503,
20447 "external": 66982448,
20448 "file": 137433211
20449 },
20450 "update_seq": "292786-g1AAAAF..."
20451 }
20452 PUT /{db}
20454 Creates a new database. The database name {db} must be composed
20455 by following next rules:
20456 · Name must begin with a lowercase letter (a-z)
20458 · Lowercase characters (a-z)
20460 · Digits (0-9)
20462 · Any of the characters _, $, (, ), +, -, and /.
20464 If you’re familiar with Regular Expressions, the rules above
20466 could be written as ^[a-z][a-z0-9_$()+/-]*$.
20467 Parameters
20469 · db – Database name
20471 Query Parameters
20473 · q (integer) – Shards, aka the number of range parti‐
20475 tions. Default is 8, unless overridden in the cluster
20476 config.
20477 · n (integer) – Replicas. The number of copies of the
20479 database in the cluster. The default is 3, unless over‐
20480 ridden in the cluster config .
20481 · partitioned (boolean) – Whether to create a partitioned
20483 database. Default is false.
20484 Request Headers
20486 · Accept – .INDENT 2.0
20488 · application/json
20490 · text/plain
20492 Response Headers
20495 · Content-Type – .INDENT 2.0
20497 · application/json
20499 · text/plain; charset=utf-8
20501 · Location – Database URI location
20504 Response JSON Object
20506 · ok (boolean) – Operation status. Available in case of success
20508 · error (string) – Error type. Available if response code is 4xx
20510 · reason (string) – Error description. Available if response
20512 code is 4xx
20513 Status Codes
20515 · 201 Created – Database created successfully (quorum is met)
20517 · 202 Accepted – Accepted (at least by one node)
20519 · 400 Bad Request – Invalid database name
20521 · 401 Unauthorized – CouchDB Server Administrator privileges
20523 required
20524 · 412 Precondition Failed – Database already exists
20526
20528 PUT /db HTTP/1.1
20530 Accept: application/json
20531 Host: localhost:5984
20532 Response:
20534 HTTP/1.1 201 Created
20536 Cache-Control: must-revalidate
20537 Content-Length: 12
20538 Content-Type: application/json
20539 Date: Mon, 12 Aug 2013 08:01:45 GMT
20540 Location: http://localhost:5984/db
20541 Server: CouchDB (Erlang/OTP)
20542 {
20544 "ok": true
20545 }
20546 If we repeat the same request to CouchDB, it will response with
20548 412 since the database already exists:
20549 Request:
20551 PUT /db HTTP/1.1
20553 Accept: application/json
20554 Host: localhost:5984
20555 Response:
20557 HTTP/1.1 412 Precondition Failed
20559 Cache-Control: must-revalidate
20560 Content-Length: 95
20561 Content-Type: application/json
20562 Date: Mon, 12 Aug 2013 08:01:16 GMT
20563 Server: CouchDB (Erlang/OTP)
20564 {
20566 "error": "file_exists",
20567 "reason": "The database could not be created, the file already exists."
20568 }
20569 If an invalid database name is supplied, CouchDB returns
20571 response with 400:
20572 Request:
20574 PUT /_db HTTP/1.1
20576 Accept: application/json
20577 Host: localhost:5984
20578 Request:
20580 HTTP/1.1 400 Bad Request
20582 Cache-Control: must-revalidate
20583 Content-Length: 194
20584 Content-Type: application/json
20585 Date: Mon, 12 Aug 2013 08:02:10 GMT
20586 Server: CouchDB (Erlang/OTP)
20587 {
20589 "error": "illegal_database_name",
20590 "reason": "Name: '_db'. Only lowercase characters (a-z), digits (0-9), and any of the characters _, $, (, ), +, -, and / are allowed. Must begin with a letter."
20591 }
20592 DELETE /{db}
20594 Deletes the specified database, and all the documents and
20595 attachments contained within it.
20596 NOTE:
20598 To avoid deleting a database, CouchDB will respond with the
20599 HTTP status code 400 when the request URL includes a ?rev=
20600 parameter. This suggests that one wants to delete a document
20601 but forgot to add the document id to the URL.
20602 Parameters
20604 · db – Database name
20606 Request Headers
20608 · Accept – .INDENT 2.0
20610 · application/json
20612 · text/plain
20614 Response Headers
20617 · Content-Type – .INDENT 2.0
20619 · application/json
20621 · text/plain; charset=utf-8
20623 Response JSON Object
20626 · ok (boolean) – Operation status
20628 Status Codes
20630 · 200 OK – Database removed successfully (quorum is met and
20632 database is deleted by at least one node)
20633 · 202 Accepted – Accepted (deleted by at least one of the nodes,
20635 quorum is not met yet)
20636 · 400 Bad Request – Invalid database name or forgotten document
20638 id by accident
20639 · 401 Unauthorized – CouchDB Server Administrator privileges
20641 required
20642 · 404 Not Found – Database doesn’t exist or invalid database
20644 name
20645
20647 DELETE /db HTTP/1.1
20649 Accept: application/json
20650 Host: localhost:5984
20651 Response:
20653 HTTP/1.1 200 OK
20655 Cache-Control: must-revalidate
20656 Content-Length: 12
20657 Content-Type: application/json
20658 Date: Mon, 12 Aug 2013 08:54:00 GMT
20659 Server: CouchDB (Erlang/OTP)
20660 {
20662 "ok": true
20663 }
20664 POST /{db}
20666 Creates a new document in the specified database, using the sup‐
20667 plied JSON document structure.
20668 If the JSON structure includes the _id field, then the document
20670 will be created with the specified document ID.
20671 If the _id field is not specified, a new unique ID will be gen‐
20673 erated, following whatever UUID algorithm is configured for that
20674 server.
20675 Parameters
20677 · db – Database name
20679 Request Headers
20681 · Accept – .INDENT 2.0
20683 · application/json
20685 · text/plain
20687 · Content-Type – application/json
20690 Query Parameters
20692 · batch (string) – Stores document in batch mode Possible val‐
20694 ues: ok. Optional
20695 Response Headers
20697 · Content-Type – .INDENT 2.0
20699 · application/json
20701 · text/plain; charset=utf-8
20703 · Location – Document’s URI
20706 Response JSON Object
20708 · id (string) – Document ID
20710 · ok (boolean) – Operation status
20712 · rev (string) – Revision info
20714 Status Codes
20716 · 201 Created – Document created and stored on disk
20718 · 202 Accepted – Document data accepted, but not yet stored on
20720 disk
20721 · 400 Bad Request – Invalid database name
20723 · 401 Unauthorized – Write privileges required
20725 · 404 Not Found – Database doesn’t exist
20727 · 409 Conflict – A Conflicting Document with same ID already
20729 exists
20730
20732 POST /db HTTP/1.1
20734 Accept: application/json
20735 Content-Length: 81
20736 Content-Type: application/json
20737 {
20739 "servings": 4,
20740 "subtitle": "Delicious with fresh bread",
20741 "title": "Fish Stew"
20742 }
20743 Response:
20745 HTTP/1.1 201 Created
20747 Cache-Control: must-revalidate
20748 Content-Length: 95
20749 Content-Type: application/json
20750 Date: Tue, 13 Aug 2013 15:19:25 GMT
20751 Location: http://localhost:5984/db/ab39fe0993049b84cfa81acd6ebad09d
20752 Server: CouchDB (Erlang/OTP)
20753 {
20755 "id": "ab39fe0993049b84cfa81acd6ebad09d",
20756 "ok": true,
20757 "rev": "1-9c65296036141e575d32ba9c034dd3ee"
20758 }
20759 Specifying the Document ID
20761 The document ID can be specified by including the _id field in the JSON
20762 of the submitted record. The following request will create the same
20763 document with the ID FishStew.
20764 Request:
20765 POST /db HTTP/1.1
20767 Accept: application/json
20768 Content-Length: 98
20769 Content-Type: application/json
20770 {
20772 "_id": "FishStew",
20773 "servings": 4,
20774 "subtitle": "Delicious with fresh bread",
20775 "title": "Fish Stew"
20776 }
20777 Response:
20779 HTTP/1.1 201 Created
20781 Cache-Control: must-revalidate
20782 Content-Length: 71
20783 Content-Type: application/json
20784 Date: Tue, 13 Aug 2013 15:19:25 GMT
20785 ETag: "1-9c65296036141e575d32ba9c034dd3ee"
20786 Location: http://localhost:5984/db/FishStew
20787 Server: CouchDB (Erlang/OTP)
20788 {
20790 "id": "FishStew",
20791 "ok": true,
20792 "rev": "1-9c65296036141e575d32ba9c034dd3ee"
20793 }
20794 Batch Mode Writes
20796 You can write documents to the database at a higher rate by using the
20797 batch option. This collects document writes together in memory (on a
20798 per-user basis) before they are committed to disk. This increases the
20799 risk of the documents not being stored in the event of a failure, since
20800 the documents are not written to disk immediately.
20801 Batch mode is not suitable for critical data, but may be ideal for
20803 applications such as log data, when the risk of some data loss due to a
20804 crash is acceptable.
20805 To use batch mode, append the batch=ok query argument to the URL of a
20807 POST /{db}, PUT /{db}/{docid}, or DELETE /{db}/{docid} request. The
20808 CouchDB server will respond with an HTTP 202 Accepted response code
20809 immediately.
20810 NOTE:
20812 Creating or updating documents with batch mode doesn’t guarantee
20813 that all documents will be successfully stored on disk. For example,
20814 individual documents may not be saved due to conflicts, rejection by
20815 validation function or by other reasons, even if overall the batch
20816 was successfully submitted.
20817 Request:
20819 POST /db?batch=ok HTTP/1.1
20821 Accept: application/json
20822 Content-Length: 98
20823 Content-Type: application/json
20824 {
20826 "_id": "FishStew",
20827 "servings": 4,
20828 "subtitle": "Delicious with fresh bread",
20829 "title": "Fish Stew"
20830 }
20831 Response:
20833 HTTP/1.1 202 Accepted
20835 Cache-Control: must-revalidate
20836 Content-Length: 28
20837 Content-Type: application/json
20838 Date: Tue, 13 Aug 2013 15:19:25 GMT
20839 Location: http://localhost:5984/db/FishStew
20840 Server: CouchDB (Erlang/OTP)
20841 {
20843 "id": "FishStew",
20844 "ok": true
20845 }
20846 /{db}/_all_docs
20848 GET /{db}/_all_docs
20849 Executes the built-in _all_docs view, returning all of the docu‐
20850 ments in the database. With the exception of the URL parameters
20851 (described below), this endpoint works identically to any other
20852 view. Refer to the view endpoint documentation for a complete
20853 description of the available query parameters and the format of
20854 the returned data.
20855 Parameters
20857 · db – Database name
20859 Request Headers
20861 · Content-Type – application/json
20863 Response Headers
20865 · Content-Type – .INDENT 2.0
20867 · application/json
20869 Status Codes
20872 · 200 OK – Request completed successfully
20874 · 404 Not Found – Requested database not found
20876 Request:
20878 GET /db/_all_docs HTTP/1.1
20880 Accept: application/json
20881 Host: localhost:5984
20882 Response:
20884 HTTP/1.1 200 OK
20886 Cache-Control: must-revalidate
20887 Content-Type: application/json
20888 Date: Sat, 10 Aug 2013 16:22:56 GMT
20889 ETag: "1W2DJUZFZSZD9K78UFA3GZWB4"
20890 Server: CouchDB (Erlang/OTP)
20891 Transfer-Encoding: chunked
20892 {
20894 "offset": 0,
20895 "rows": [
20896 {
20897 "id": "16e458537602f5ef2a710089dffd9453",
20898 "key": "16e458537602f5ef2a710089dffd9453",
20899 "value": {
20900 "rev": "1-967a00dff5e02add41819138abb3284d"
20901 }
20902 },
20903 {
20904 "id": "a4c51cdfa2069f3e905c431114001aff",
20905 "key": "a4c51cdfa2069f3e905c431114001aff",
20906 "value": {
20907 "rev": "1-967a00dff5e02add41819138abb3284d"
20908 }
20909 },
20910 {
20911 "id": "a4c51cdfa2069f3e905c4311140034aa",
20912 "key": "a4c51cdfa2069f3e905c4311140034aa",
20913 "value": {
20914 "rev": "5-6182c9c954200ab5e3c6bd5e76a1549f"
20915 }
20916 },
20917 {
20918 "id": "a4c51cdfa2069f3e905c431114003597",
20919 "key": "a4c51cdfa2069f3e905c431114003597",
20920 "value": {
20921 "rev": "2-7051cbe5c8faecd085a3fa619e6e6337"
20922 }
20923 },
20924 {
20925 "id": "f4ca7773ddea715afebc4b4b15d4f0b3",
20926 "key": "f4ca7773ddea715afebc4b4b15d4f0b3",
20927 "value": {
20928 "rev": "2-7051cbe5c8faecd085a3fa619e6e6337"
20929 }
20930 }
20931 ],
20932 "total_rows": 5
20933 }
20934 POST /{db}/_all_docs
20936 POST _all_docs functionality supports identical parameters and
20937 behavior as specified in the GET /{db}/_all_docs API but allows
20938 for the query string parameters to be supplied as keys in a JSON
20939 object in the body of the POST request.
20940 Request:
20942 POST /db/_all_docs HTTP/1.1
20944 Accept: application/json
20945 Content-Length: 70
20946 Content-Type: application/json
20947 Host: localhost:5984
20948 {
20950 "keys" : [
20951 "Zingylemontart",
20952 "Yogurtraita"
20953 ]
20954 }
20955 Response:
20957 {
20959 "total_rows" : 2666,
20960 "rows" : [
20961 {
20962 "value" : {
20963 "rev" : "1-a3544d296de19e6f5b932ea77d886942"
20964 },
20965 "id" : "Zingylemontart",
20966 "key" : "Zingylemontart"
20967 },
20968 {
20969 "value" : {
20970 "rev" : "1-91635098bfe7d40197a1b98d7ee085fc"
20971 },
20972 "id" : "Yogurtraita",
20973 "key" : "Yogurtraita"
20974 }
20975 ],
20976 "offset" : 0
20977 }
20978 /{db}/_design_docs
20980 New in version 2.2.
20981 GET /{db}/_design_docs
20984 Returns a JSON structure of all of the design documents in a
20985 given database. The information is returned as a JSON structure
20986 containing meta information about the return structure, includ‐
20987 ing a list of all design documents and basic contents, consist‐
20988 ing the ID, revision and key. The key is the design document’s
20989 _id.
20990 Parameters
20992 · db – Database name
20994 Request Headers
20996 · Accept – .INDENT 2.0
20998 · application/json
21000 · text/plain
21002 Query Parameters
21005 · conflicts (boolean) – Includes conflicts information in
21007 response. Ignored if include_docs isn’t true. Default is
21008 false.
21009 · descending (boolean) – Return the design documents in descend‐
21011 ing by key order. Default is false.
21012 · endkey (string) – Stop returning records when the specified
21014 key is reached. Optional.
21015 · end_key (string) – Alias for endkey param.
21017 · endkey_docid (string) – Stop returning records when the speci‐
21019 fied design document ID is reached. Optional.
21020 · end_key_doc_id (string) – Alias for endkey_docid param.
21022 · include_docs (boolean) – Include the full content of the
21024 design documents in the return. Default is false.
21025 · inclusive_end (boolean) – Specifies whether the specified end
21027 key should be included in the result. Default is true.
21028 · key (string) – Return only design documents that match the
21030 specified key. Optional.
21031 · keys (string) – Return only design documents that match the
21033 specified keys. Optional.
21034 · limit (number) – Limit the number of the returned design docu‐
21036 ments to the specified number. Optional.
21037 · skip (number) – Skip this number of records before starting to
21039 return the results. Default is 0.
21040 · startkey (string) – Return records starting with the specified
21042 key. Optional.
21043 · start_key (string) – Alias for startkey param.
21045 · startkey_docid (string) – Return records starting with the
21047 specified design document ID. Optional.
21048 · start_key_doc_id (string) – Alias for startkey_docid param.
21050 · update_seq (boolean) – Response includes an update_seq value
21052 indicating which sequence id of the underlying database the
21053 view reflects. Default is false.
21054 Response Headers
21056 · Content-Type – .INDENT 2.0
21058 · application/json
21060 · text/plain; charset=utf-8
21062 · ETag – Response signature
21065 Response JSON Object
21067 · offset (number) – Offset where the design document list
21069 started
21070 · rows (array) – Array of view row objects. By default the
21072 information returned contains only the design document ID and
21073 revision.
21074 · total_rows (number) – Number of design documents in the data‐
21076 base. Note that this is not the number of rows returned in the
21077 actual query.
21078 · update_seq (number) – Current update sequence for the database
21080 Status Codes
21082 · 200 OK – Request completed successfully
21084 · 404 Not Found – Requested database not found
21086
21088 GET /db/_design_docs HTTP/1.1
21090 Accept: application/json
21091 Host: localhost:5984
21092 Response:
21094 HTTP/1.1 200 OK
21096 Cache-Control: must-revalidate
21097 Content-Type: application/json
21098 Date: Sat, 23 Dec 2017 16:22:56 GMT
21099 ETag: "1W2DJUZFZSZD9K78UFA3GZWB4"
21100 Server: CouchDB (Erlang/OTP)
21101 Transfer-Encoding: chunked
21102 {
21104 "offset": 0,
21105 "rows": [
21106 {
21107 "id": "_design/ddoc01",
21108 "key": "_design/ddoc01",
21109 "value": {
21110 "rev": "1-7407569d54af5bc94c266e70cbf8a180"
21111 }
21112 },
21113 {
21114 "id": "_design/ddoc02",
21115 "key": "_design/ddoc02",
21116 "value": {
21117 "rev": "1-d942f0ce01647aa0f46518b213b5628e"
21118 }
21119 },
21120 {
21121 "id": "_design/ddoc03",
21122 "key": "_design/ddoc03",
21123 "value": {
21124 "rev": "1-721fead6e6c8d811a225d5a62d08dfd0"
21125 }
21126 },
21127 {
21128 "id": "_design/ddoc04",
21129 "key": "_design/ddoc04",
21130 "value": {
21131 "rev": "1-32c76b46ca61351c75a84fbcbceece2f"
21132 }
21133 },
21134 {
21135 "id": "_design/ddoc05",
21136 "key": "_design/ddoc05",
21137 "value": {
21138 "rev": "1-af856babf9cf746b48ae999645f9541e"
21139 }
21140 }
21141 ],
21142 "total_rows": 5
21143 }
21144 POST /{db}/_design_docs
21146 POST _design_docs functionality supports identical parameters
21147 and behavior as specified in the GET /{db}/_design_docs API but
21148 allows for the query string parameters to be supplied as keys in
21149 a JSON object in the body of the POST request.
21150 Request:
21152 POST /db/_design_docs HTTP/1.1
21154 Accept: application/json
21155 Content-Length: 70
21156 Content-Type: application/json
21157 Host: localhost:5984
21158 {
21160 "keys" : [
21161 "_design/ddoc02",
21162 "_design/ddoc05"
21163 ]
21164 }
21165 The returned JSON is the all documents structure, but with only
21167 the selected keys in the output:
21168 {
21170 "total_rows" : 5,
21171 "rows" : [
21172 {
21173 "value" : {
21174 "rev" : "1-d942f0ce01647aa0f46518b213b5628e"
21175 },
21176 "id" : "_design/ddoc02",
21177 "key" : "_design/ddoc02"
21178 },
21179 {
21180 "value" : {
21181 "rev" : "1-af856babf9cf746b48ae999645f9541e"
21182 },
21183 "id" : "_design/ddoc05",
21184 "key" : "_design/ddoc05"
21185 }
21186 ],
21187 "offset" : 0
21188 }
21189 Sending multiple queries to a database
21191 New in version 2.2.
21192 POST /{db}/_all_docs/queries
21195 Executes multiple specified built-in view queries of all docu‐
21196 ments in this database. This enables you to request multiple
21197 queries in a single request, in place of multiple POST
21198 /{db}/_all_docs requests.
21199 Parameters
21201 · db – Database name
21203 Request Headers
21205 · Content-Type – .INDENT 2.0
21207 · application/json
21209 · Accept – .INDENT 2.0
21212 · application/json
21214 Request JSON Object
21217 · queries – An array of query objects with fields for the param‐
21219 eters of each individual view query to be executed. The field
21220 names and their meaning are the same as the query parameters
21221 of a regular _all_docs request.
21222 Response Headers
21224 · Content-Type – .INDENT 2.0
21226 · application/json
21228 · text/plain; charset=utf-8
21230 · ETag – Response signature
21233 · Transfer-Encoding – chunked
21235 Response JSON Object
21237 · results (array) – An array of result objects - one for each
21239 query. Each result object contains the same fields as the
21240 response to a regular _all_docs request.
21241 Status Codes
21243 · 200 OK – Request completed successfully
21245 · 400 Bad Request – Invalid request
21247 · 401 Unauthorized – Read permission required
21249 · 404 Not Found – Specified database is missing
21251 · 500 Internal Server Error – Query execution error
21253
21255 POST /db/_all_docs/queries HTTP/1.1
21257 Content-Type: application/json
21258 Accept: application/json
21259 Host: localhost:5984
21260 {
21262 "queries": [
21263 {
21264 "keys": [
21265 "meatballs",
21266 "spaghetti"
21267 ]
21268 },
21269 {
21270 "limit": 3,
21271 "skip": 2
21272 }
21273 ]
21274 }
21275 Response:
21277 HTTP/1.1 200 OK
21279 Cache-Control: must-revalidate
21280 Content-Type: application/json
21281 Date: Wed, 20 Dec 2017 11:17:07 GMT
21282 ETag: "1H8RGBCK3ABY6ACDM7ZSC30QK"
21283 Server: CouchDB (Erlang/OTP)
21284 Transfer-Encoding: chunked
21285 {
21287 "results" : [
21288 {
21289 "rows": [
21290 {
21291 "id": "meatballs",
21292 "key": "meatballs",
21293 "value": 1
21294 },
21295 {
21296 "id": "spaghetti",
21297 "key": "spaghetti",
21298 "value": 1
21299 }
21300 ],
21301 "total_rows": 3
21302 },
21303 {
21304 "offset" : 2,
21305 "rows" : [
21306 {
21307 "id" : "Adukiandorangecasserole-microwave",
21308 "key" : "Aduki and orange casserole - microwave",
21309 "value" : [
21310 null,
21311 "Aduki and orange casserole - microwave"
21312 ]
21313 },
21314 {
21315 "id" : "Aioli-garlicmayonnaise",
21316 "key" : "Aioli - garlic mayonnaise",
21317 "value" : [
21318 null,
21319 "Aioli - garlic mayonnaise"
21320 ]
21321 },
21322 {
21323 "id" : "Alabamapeanutchicken",
21324 "key" : "Alabama peanut chicken",
21325 "value" : [
21326 null,
21327 "Alabama peanut chicken"
21328 ]
21329 }
21330 ],
21331 "total_rows" : 2667
21332 }
21333 ]
21334 }
21335 NOTE:
21337 The multiple queries are also supported in /db/_local_docs/queries
21338 and /db/_design_docs/queries (similar to /db/_all_docs/queries).
21339 /{db}/_bulk_get
21341 POST /{db}/_bulk_get
21342 This method can be called to query several documents in bulk. It
21343 is well suited for fetching a specific revision of documents, as
21344 replicators do for example, or for getting revision history.
21345 Parameters
21347 · db – Database name
21349 Request Headers
21351 · Accept – .INDENT 2.0
21353 · application/json
21355 · multipart/related
21357 · multipart/mixed
21359 · Content-Type – application/json
21362 Query Parameters
21364 · revs (boolean) – Give the revisions history
21366 Request JSON Object
21368 · docs (array) – List of document objects, with id, and option‐
21370 ally rev and atts_since
21371 Response Headers
21373 · Content-Type – .INDENT 2.0
21375 · application/json
21377 Response JSON Object
21380 · results (object) – an array of results for each requested doc‐
21382 ument/rev pair. id key lists the requested document ID, docs
21383 contains a single-item array of objects, each of which has
21384 either an error key and value describing the error, or ok key
21385 and associated value of the requested document, with the addi‐
21386 tional _revisions property that lists the parent revisions if
21387 revs=true.
21388 Status Codes
21390 · 200 OK – Request completed successfully
21392 · 400 Bad Request – The request provided invalid JSON data or
21394 invalid query parameter
21395 · 401 Unauthorized – Read permission required
21397 · 404 Not Found – Invalid database name
21399 · 415 Unsupported Media Type – Bad Content-Type value
21401
21403 POST /db/_bulk_get HTTP/1.1
21405 Accept: application/json
21406 Content-Type:application/json
21407 Host: localhost:5984
21408 {
21410 "docs": [
21411 {
21412 "id": "foo"
21413 "rev": "4-753875d51501a6b1883a9d62b4d33f91",
21414 },
21415 {
21416 "id": "foo"
21417 "rev": "1-4a7e4ae49c4366eaed8edeaea8f784ad",
21418 },
21419 {
21420 "id": "bar",
21421 }
21422 {
21423 "id": "baz",
21424 }
21425 ]
21426 }
21427 Response:
21429 HTTP/1.1 200 OK
21431 Cache-Control: must-revalidate
21432 Content-Type: application/json
21433 Date: Mon, 19 Mar 2018 15:27:34 GMT
21434 Server: CouchDB (Erlang/OTP)
21435 {
21437 "results": [
21438 {
21439 "id": "foo",
21440 "docs": [
21441 {
21442 "ok": {
21443 "_id": "foo",
21444 "_rev": "4-753875d51501a6b1883a9d62b4d33f91",
21445 "value": "this is foo",
21446 "_revisions": {
21447 "start": 4,
21448 "ids": [
21449 "753875d51501a6b1883a9d62b4d33f91",
21450 "efc54218773c6acd910e2e97fea2a608",
21451 "2ee767305024673cfb3f5af037cd2729",
21452 "4a7e4ae49c4366eaed8edeaea8f784ad"
21453 ]
21454 }
21455 }
21456 }
21457 ]
21458 },
21459 {
21460 "id": "foo",
21461 "docs": [
21462 {
21463 "ok": {
21464 "_id": "foo",
21465 "_rev": "1-4a7e4ae49c4366eaed8edeaea8f784ad",
21466 "value": "this is the first revision of foo",
21467 "_revisions": {
21468 "start": 1,
21469 "ids": [
21470 "4a7e4ae49c4366eaed8edeaea8f784ad"
21471 ]
21472 }
21473 }
21474 }
21475 ]
21476 },
21477 {
21478 "id": "bar",
21479 "docs": [
21480 {
21481 "ok": {
21482 "_id": "bar",
21483 "_rev": "2-9b71d36dfdd9b4815388eb91cc8fb61d",
21484 "baz": true,
21485 "_revisions": {
21486 "start": 2,
21487 "ids": [
21488 "9b71d36dfdd9b4815388eb91cc8fb61d",
21489 "309651b95df56d52658650fb64257b97"
21490 ]
21491 }
21492 }
21493 }
21494 ]
21495 },
21496 {
21497 "id": "baz",
21498 "docs": [
21499 {
21500 "error": {
21501 "id": "baz",
21502 "rev": "undefined",
21503 "error": "not_found",
21504 "reason": "missing"
21505 }
21506 }
21507 ]
21508 }
21509 ]
21510 }
21511 Example response with a conflicted document:
21513 Request:
21515 POST /db/_bulk_get HTTP/1.1
21517 Accept: application/json
21518 Content-Type:application/json
21519 Host: localhost:5984
21520 {
21522 "docs": [
21523 {
21524 "id": "a"
21525 }
21526 ]
21527 }
21528 Response:
21530 HTTP/1.1 200 OK
21532 Cache-Control: must-revalidate
21533 Content-Type: application/json
21534 Date: Mon, 19 Mar 2018 15:27:34 GMT
21535 Server: CouchDB (Erlang/OTP)
21536 {
21538 "results": [
21539 {
21540 "id": "a",
21541 "docs": [
21542 {
21543 "ok": {
21544 "_id": "a",
21545 "_rev": "1-23202479633c2b380f79507a776743d5",
21546 "a": 1
21547 }
21548 },
21549 {
21550 "ok": {
21551 "_id": "a",
21552 "_rev": "1-967a00dff5e02add41819138abb3284d"
21553 }
21554 }
21555 ]
21556 }
21557 ]
21558 }
21559 /{db}/_bulk_docs
21561 POST /{db}/_bulk_docs
21562 The bulk document API allows you to create and update multiple
21563 documents at the same time within a single request. The basic
21564 operation is similar to creating or updating a single document,
21565 except that you batch the document structure and information.
21566 When creating new documents the document ID (_id) is optional.
21568 For updating existing documents, you must provide the document
21570 ID, revision information (_rev), and new document values.
21571 In case of batch deleting documents all fields as document ID,
21573 revision information and deletion status (_deleted) are
21574 required.
21575 Parameters
21577 · db – Database name
21579 Request Headers
21581 · Accept – .INDENT 2.0
21583 · application/json
21585 · text/plain
21587 · Content-Type – application/json
21590 Request JSON Object
21592 · docs (array) – List of documents objects
21594 · new_edits (boolean) – If false, prevents the database from
21596 assigning them new revision IDs. Default is true. Optional
21597 Response Headers
21599 · Content-Type – .INDENT 2.0
21601 · application/json
21603 · text/plain; charset=utf-8
21605 Response JSON Array of Objects
21608 · id (string) – Document ID
21610 · rev (string) – New document revision token. Available if docu‐
21612 ment has saved without errors. Optional
21613 · error (string) – Error type. Optional
21615 · reason (string) – Error reason. Optional
21617 Status Codes
21619 · 201 Created – Document(s) have been created or updated
21621 · 400 Bad Request – The request provided invalid JSON data
21623 · 404 Not Found – Requested database not found
21625
21627 POST /db/_bulk_docs HTTP/1.1
21629 Accept: application/json
21630 Content-Length: 109
21631 Content-Type:application/json
21632 Host: localhost:5984
21633 {
21635 "docs": [
21636 {
21637 "_id": "FishStew"
21638 },
21639 {
21640 "_id": "LambStew",
21641 "_rev": "2-0786321986194c92dd3b57dfbfc741ce",
21642 "_deleted": true
21643 }
21644 ]
21645 }
21646 Response:
21648 HTTP/1.1 201 Created
21650 Cache-Control: must-revalidate
21651 Content-Length: 144
21652 Content-Type: application/json
21653 Date: Mon, 12 Aug 2013 00:15:05 GMT
21654 Server: CouchDB (Erlang/OTP)
21655 [
21657 {
21658 "ok": true,
21659 "id": "FishStew",
21660 "rev":" 1-967a00dff5e02add41819138abb3284d"
21661 },
21662 {
21663 "ok": true,
21664 "id": "LambStew",
21665 "rev": "3-f9c62b2169d0999103e9f41949090807"
21666 }
21667 ]
21668 Inserting Documents in Bulk
21670 Each time a document is stored or updated in CouchDB, the internal
21671 B-tree is updated. Bulk insertion provides efficiency gains in both
21672 storage space, and time, by consolidating many of the updates to inter‐
21673 mediate B-tree nodes.
21674 It is not intended as a way to perform ACID-like transactions in
21676 CouchDB, the only transaction boundary within CouchDB is a single
21677 update to a single database. The constraints are detailed in Bulk Docu‐
21678 ments Transaction Semantics.
21679 To insert documents in bulk into a database you need to supply a JSON
21681 structure with the array of documents that you want to add to the data‐
21682 base. You can either include a document ID, or allow the document ID
21683 to be automatically generated.
21684 For example, the following update inserts three new documents, two with
21686 the supplied document IDs, and one which will have a document ID gener‐
21687 ated:
21688 POST /source/_bulk_docs HTTP/1.1
21690 Accept: application/json
21691 Content-Length: 323
21692 Content-Type: application/json
21693 Host: localhost:5984
21694 {
21696 "docs": [
21697 {
21698 "_id": "FishStew",
21699 "servings": 4,
21700 "subtitle": "Delicious with freshly baked bread",
21701 "title": "FishStew"
21702 },
21703 {
21704 "_id": "LambStew",
21705 "servings": 6,
21706 "subtitle": "Serve with a whole meal scone topping",
21707 "title": "LambStew"
21708 },
21709 {
21710 "servings": 8,
21711 "subtitle": "Hand-made dumplings make a great accompaniment",
21712 "title": "BeefStew"
21713 }
21714 ]
21715 }
21716 The return type from a bulk insertion will be 201 Created, with the
21718 content of the returned structure indicating specific success or other‐
21719 wise messages on a per-document basis.
21720 The return structure from the example above contains a list of the doc‐
21722 uments created, here with the combination and their revision IDs:
21723 HTTP/1.1 201 Created
21725 Cache-Control: must-revalidate
21726 Content-Length: 215
21727 Content-Type: application/json
21728 Date: Sat, 26 Oct 2013 00:10:39 GMT
21729 Server: CouchDB (Erlang OTP)
21730 [
21732 {
21733 "id": "FishStew",
21734 "ok": true,
21735 "rev": "1-6a466d5dfda05e613ba97bd737829d67"
21736 },
21737 {
21738 "id": "LambStew",
21739 "ok": true,
21740 "rev": "1-648f1b989d52b8e43f05aa877092cc7c"
21741 },
21742 {
21743 "id": "00a271787f89c0ef2e10e88a0c0003f0",
21744 "ok": true,
21745 "rev": "1-e4602845fc4c99674f50b1d5a804fdfa"
21746 }
21747 ]
21748 For details of the semantic content and structure of the returned JSON
21750 see Bulk Documents Transaction Semantics. Conflicts and validation
21751 errors when updating documents in bulk must be handled separately; see
21752 Bulk Document Validation and Conflict Errors.
21753 Updating Documents in Bulk
21755 The bulk document update procedure is similar to the insertion proce‐
21756 dure, except that you must specify the document ID and current revision
21757 for every document in the bulk update JSON string.
21758 For example, you could send the following request:
21760 POST /recipes/_bulk_docs HTTP/1.1
21762 Accept: application/json
21763 Content-Length: 464
21764 Content-Type: application/json
21765 Host: localhost:5984
21766 {
21768 "docs": [
21769 {
21770 "_id": "FishStew",
21771 "_rev": "1-6a466d5dfda05e613ba97bd737829d67",
21772 "servings": 4,
21773 "subtitle": "Delicious with freshly baked bread",
21774 "title": "FishStew"
21775 },
21776 {
21777 "_id": "LambStew",
21778 "_rev": "1-648f1b989d52b8e43f05aa877092cc7c",
21779 "servings": 6,
21780 "subtitle": "Serve with a whole meal scone topping",
21781 "title": "LambStew"
21782 },
21783 {
21784 "_id": "BeefStew",
21785 "_rev": "1-e4602845fc4c99674f50b1d5a804fdfa",
21786 "servings": 8,
21787 "subtitle": "Hand-made dumplings make a great accompaniment",
21788 "title": "BeefStew"
21789 }
21790 ]
21791 }
21792 The return structure is the JSON of the updated documents, with the new
21794 revision and ID information:
21795 HTTP/1.1 201 Created
21797 Cache-Control: must-revalidate
21798 Content-Length: 215
21799 Content-Type: application/json
21800 Date: Sat, 26 Oct 2013 00:10:39 GMT
21801 Server: CouchDB (Erlang OTP)
21802 [
21804 {
21805 "id": "FishStew",
21806 "ok": true,
21807 "rev": "2-2bff94179917f1dec7cd7f0209066fb8"
21808 },
21809 {
21810 "id": "LambStew",
21811 "ok": true,
21812 "rev": "2-6a7aae7ac481aa98a2042718d09843c4"
21813 },
21814 {
21815 "id": "BeefStew",
21816 "ok": true,
21817 "rev": "2-9801936a42f06a16f16c30027980d96f"
21818 }
21819 ]
21820 You can optionally delete documents during a bulk update by adding the
21822 _deleted field with a value of true to each document ID/revision combi‐
21823 nation within the submitted JSON structure.
21824 The return type from a bulk insertion will be 201 Created, with the
21826 content of the returned structure indicating specific success or other‐
21827 wise messages on a per-document basis.
21828 The content and structure of the returned JSON will depend on the
21830 transaction semantics being used for the bulk update; see Bulk Docu‐
21831 ments Transaction Semantics for more information. Conflicts and valida‐
21832 tion errors when updating documents in bulk must be handled separately;
21833 see Bulk Document Validation and Conflict Errors.
21834 Bulk Documents Transaction Semantics
21836 Bulk document operations are non-atomic. This means that CouchDB does
21837 not guarantee that any individual document included in the bulk update
21838 (or insert) will be saved when you send the request. The response will
21839 contain the list of documents successfully inserted or updated during
21840 the process. In the event of a crash, some of the documents may have
21841 been successfully saved, while others lost.
21842 The response structure will indicate whether the document was updated
21844 by supplying the new _rev parameter indicating a new document revision
21845 was created. If the update failed, you will get an error of type con‐
21846 flict. For example:
21847 [
21849 {
21850 "id" : "FishStew",
21851 "error" : "conflict",
21852 "reason" : "Document update conflict."
21853 },
21854 {
21855 "id" : "LambStew",
21856 "error" : "conflict",
21857 "reason" : "Document update conflict."
21858 },
21859 {
21860 "id" : "BeefStew",
21861 "error" : "conflict",
21862 "reason" : "Document update conflict."
21863 }
21864 ]
21865 In this case no new revision has been created and you will need to sub‐
21867 mit the document update, with the correct revision tag, to update the
21868 document.
21869 Replication of documents is independent of the type of insert or
21871 update. The documents and revisions created during a bulk insert or
21872 update are replicated in the same way as any other document.
21873 Bulk Document Validation and Conflict Errors
21875 The JSON returned by the _bulk_docs operation consists of an array of
21876 JSON structures, one for each document in the original submission. The
21877 returned JSON structure should be examined to ensure that all of the
21878 documents submitted in the original request were successfully added to
21879 the database.
21880 When a document (or document revision) is not correctly committed to
21882 the database because of an error, you should check the error field to
21883 determine error type and course of action. Errors will be one of the
21884 following type:
21885 · conflict
21887 The document as submitted is in conflict. The new revision will not
21889 have been created and you will need to re-submit the document to the
21890 database.
21891 Conflict resolution of documents added using the bulk docs interface
21893 is identical to the resolution procedures used when resolving con‐
21894 flict errors during replication.
21895 · forbidden
21897 Entries with this error type indicate that the validation routine
21899 applied to the document during submission has returned an error.
21900 For example, if your validation routine includes the following:
21902 throw({forbidden: 'invalid recipe ingredient'});
21904 The error response returned will be:
21906 HTTP/1.1 201 Created
21908 Cache-Control: must-revalidate
21909 Content-Length: 80
21910 Content-Type: application/json
21911 Date: Sat, 26 Oct 2013 00:05:17 GMT
21912 Server: CouchDB (Erlang OTP)
21913 [
21915 {
21916 "id": "LambStew",
21917 "error": "forbidden",
21918 "reason": "invalid recipe ingredient"
21919 }
21920 ]
21921 /db/_find
21923 POST /{db}/_find
21924 Find documents using a declarative JSON querying syntax.
21925 Queries can use the built-in _all_docs index or custom indexes,
21926 specified using the _index endpoint.
21927 Parameters
21929 · db – Database name
21931 Request Headers
21933 · Content-Type – .INDENT 2.0
21935 · application/json
21937 Request JSON Object
21940 · selector (json) – JSON object describing criteria used to
21942 select documents. More information provided in the section on
21943 selector syntax. Required
21944 · limit (number) – Maximum number of results returned. Default
21946 is 25. Optional
21947 · skip (number) – Skip the first ‘n’ results, where ‘n’ is the
21949 value specified. Optional
21950 · sort (json) – JSON array following sort syntax. Optional
21952 · fields (array) – JSON array specifying which fields of each
21954 object should be returned. If it is omitted, the entire object
21955 is returned. More information provided in the section on
21956 filtering fields. Optional
21957 · use_index (string|array) – Instruct a query to use a specific
21959 index. Specified either as "<design_document>" or
21960 ["<design_document>", "<index_name>"]. Optional
21961 · r (number) – Read quorum needed for the result. This defaults
21963 to 1, in which case the document found in the index is
21964 returned. If set to a higher value, each document is read from
21965 at least that many replicas before it is returned in the
21966 results. This is likely to take more time than using only the
21967 document stored locally with the index. Optional, default: 1
21968 · bookmark (string) – A string that enables you to specify which
21970 page of results you require. Used for paging through result
21971 sets. Every query returns an opaque string under the bookmark
21972 key that can then be passed back in a query to get the next
21973 page of results. If any part of the selector query changes
21974 between requests, the results are undefined. Optional,
21975 default: null
21976 · update (boolean) – Whether to update the index prior to
21978 returning the result. Default is true. Optional
21979 · stable (boolean) – Whether or not the view results should be
21981 returned from a “stable” set of shards. Optional
21982 · stale (string) – Combination of update=false and stable=true
21984 options. Possible options: "ok", false (default). Optional
21985 Note that this parameter is deprecated. Use stable and update
21986 instead. See views/generation for more details.
21987 · execution_stats (boolean) – Include execution statistics in
21989 the query response. Optional, default: ``false``
21990 Response Headers
21992 · Content-Type – application/json
21994 · Transfer-Encoding – chunked
21996 Response JSON Object
21998 · docs (object) – Array of documents matching the search. In
22000 each matching document, the fields specified in the fields
22001 part of the request body are listed, along with their values.
22002 · warning (string) – Execution warnings
22004 · execution_stats (object) – Execution statistics
22006 · bookmark (string) – An opaque string used for paging. See the
22008 bookmark field in the request (above) for usage details.
22009 Status Codes
22011 · 200 OK – Request completed successfully
22013 · 400 Bad Request – Invalid request
22015 · 401 Unauthorized – Read permission required
22017 · 404 Not Found – Requested database not found
22019 · 500 Internal Server Error – Query execution error
22021The limit and skip values are exactly as you would expect. While skip exists,
22023it is not intended to be used for paging. The reason is that the bookmark fea‐
22024ture is more efficient.
22025 Request:
22026 Example request body for finding documents using an index:
22028 POST /movies/_find HTTP/1.1
22030 Accept: application/json
22031 Content-Type: application/json
22032 Content-Length: 168
22033 Host: localhost:5984
22034 {
22036 "selector": {
22037 "year": {"$gt": 2010}
22038 },
22039 "fields": ["_id", "_rev", "year", "title"],
22040 "sort": [{"year": "asc"}],
22041 "limit": 2,
22042 "skip": 0,
22043 "execution_stats": true
22044 }
22045 Response:
22047 Example response when finding documents using an index:
22049 HTTP/1.1 200 OK
22051 Cache-Control: must-revalidate
22052 Content-Type: application/json
22053 Date: Thu, 01 Sep 2016 15:41:53 GMT
22054 Server: CouchDB (Erlang OTP)
22055 Transfer-Encoding: chunked
22056 {
22058 "docs": [
22059 {
22060 "_id": "176694",
22061 "_rev": "1-54f8e950cc338d2385d9b0cda2fd918e",
22062 "year": 2011,
22063 "title": "The Tragedy of Man"
22064 },
22065 {
22066 "_id": "780504",
22067 "_rev": "1-5f14bab1a1e9ac3ebdf85905f47fb084",
22068 "year": 2011,
22069 "title": "Drive"
22070 }
22071 ],
22072 "execution_stats": {
22073 "total_keys_examined": 0,
22074 "total_docs_examined": 200,
22075 "total_quorum_docs_examined": 0,
22076 "results_returned": 2,
22077 "execution_time_ms": 5.52
22078 }
22079 }
22080 Selector Syntax
22082 Selectors are expressed as a JSON object describing documents of inter‐
22083 est. Within this structure, you can apply conditional logic using spe‐
22084 cially named fields.
22085 Whilst selectors have some similarities with MongoDB query documents,
22087 these arise from a similarity of purpose and do not necessarily extend
22088 to commonality of function or result.
22089 Selector Basics
22091 Elementary selector syntax requires you to specify one or more fields,
22092 and the corresponding values required for those fields. This selector
22093 matches all documents whose “director” field has the value “Lars von
22094 Trier”.
22095 {
22097 "director": "Lars von Trier"
22098 }
22099 A simple selector, inspecting specific fields
22101 "selector": {
22103 "$title": "Live And Let Die"
22104 },
22105 "fields": [
22106 "title",
22107 "cast"
22108 ]
22109 You can create more complex selector expressions by combining opera‐
22111 tors. For best performance, it is best to combine ‘combination’ or
22112 ‘array logical’ operators, such as $regex, with an equality operators
22113 such as $eq, $gt, $gte, $lt, and $lte (but not $ne). For more informa‐
22114 tion about creating complex selector expressions, see creating selector
22115 expressions.
22116 Selector with 2 fields
22118 This selector matches any document with a name field containing "Paul",
22119 and that also has a location field with the value "Boston".
22120 {
22122 "name": "Paul",
22123 "location": "Boston"
22124 }
22125 Subfields
22127 A more complex selector enables you to specify the values for field of
22128 nested objects, or subfields. For example, you might use a standard
22129 JSON structure for specifying a field and subfield.
22130 Example of a field and subfield selector, using a standard JSON struc‐
22132 ture:
22133 {
22135 "imdb": {
22136 "rating": 8
22137 }
22138 }
22139 An abbreviated equivalent uses a dot notation to combine the field and
22141 subfield names into a single name.
22142 {
22144 "imdb.rating": 8
22145 }
22146 Operators
22148 Operators are identified by the use of a dollar sign ($) prefix in the
22149 name field.
22150 There are two core types of operators in the selector syntax:
22152 · Combination operators
22154 · Condition operators
22156 In general, combination operators are applied at the topmost level of
22158 selection. They are used to combine conditions, or to create combina‐
22159 tions of conditions, into one selector.
22160 Every explicit operator has the form:
22162 {"$operator": argument}
22164 A selector without an explicit operator is considered to have an
22166 implicit operator. The exact implicit operator is determined by the
22167 structure of the selector expression.
22168 Implicit Operators
22170 There are two implicit operators:
22171 · Equality
22173 · And
22175 In a selector, any field containing a JSON value, but that has no oper‐
22177 ators in it, is considered to be an equality condition. The implicit
22178 equality test applies also for fields and subfields.
22179 Any JSON object that is not the argument to a condition operator is an
22181 implicit $and operator on each field.
22182 In the below example, we use an operator to match any document, where
22184 the "year" field has a value greater than 2010:
22185 {
22187 "year": {
22188 "$gt": 2010
22189 }
22190 }
22191 In this next example, there must be a field "director" in a matching
22193 document, and the field must have a value exactly equal to "Lars von
22194 Trier".
22195 {
22197 "director": "Lars von Trier"
22198 }
22199 You can also make the equality operator explicit.
22201 {
22203 "director": {
22204 "$eq": "Lars von Trier"
22205 }
22206 }
22207 In the next example using subfields, the required field "imdb" in a
22209 matching document must also have a subfield "rating" and the subfield
22210 must have a value equal to 8.
22211 Example of implicit operator applied to a subfield test
22213 {
22215 "imdb": {
22216 "rating": 8
22217 }
22218 }
22219 Again, you can make the equality operator explicit.
22221 {
22223 "imdb": {
22224 "rating": { "$eq": 8 }
22225 }
22226 }
22227 An example of the $eq operator used with full text indexing
22229 {
22231 "selector": {
22232 "year": {
22233 "$eq": 2001
22234 }
22235 },
22236 "sort": [
22237 "title:string"
22238 ],
22239 "fields": [
22240 "title"
22241 ]
22242 }
22243 An example of the $eq operator used with database indexed on the field
22245 "year"
22246 {
22248 "selector": {
22249 "year": {
22250 "$eq": 2001
22251 }
22252 },
22253 "sort": [
22254 "year"
22255 ],
22256 "fields": [
22257 "year"
22258 ]
22259 }
22260 In this example, the field "director" must be present and contain the
22262 value "Lars von Trier" and the field "year" must exist and have the
22263 value 2003.
22264 {
22266 "director": "Lars von Trier",
22267 "year": 2003
22268 }
22269 You can make both the $and operator and the equality operator explicit.
22271 Example of using explicit $and and $eq operators
22272 {
22274 "$and": [
22275 {
22276 "director": {
22277 "$eq": "Lars von Trier"
22278 }
22279 },
22280 {
22281 "year": {
22282 "$eq": 2003
22283 }
22284 }
22285 ]
22286 }
22287 Explicit Operators
22289 All operators, apart from ‘Equality’ and ‘And’, must be stated explic‐
22290 itly.
22291 Combination Operators
22293 Combination operators are used to combine selectors. In addition to the
22294 common boolean operators found in most programming languages, there are
22295 three combination operators ($all, $elemMatch, and $allMatch) that help
22296 you work with JSON arrays and one that works with JSON maps ($keyMap‐
22297 Match).
22298 A combination operator takes a single argument. The argument is either
22300 another selector, or an array of selectors.
22301 The list of combination operators:
22303 ┌─────────────┬──────────┬─────────────────────┐
22305 │Operator │ Argument │ Purpose │
22306 ├─────────────┼──────────┼─────────────────────┤
22307 │$and │ Array │ Matches if all the │
22308 │ │ │ selectors in the │
22309 │ │ │ array match. │
22310 ├─────────────┼──────────┼─────────────────────┤
22311 │$or │ Array │ Matches if any of │
22312 │ │ │ the selectors in │
22313 │ │ │ the array match. │
22314 │ │ │ All selectors must │
22315 │ │ │ use the same index. │
22316 ├─────────────┼──────────┼─────────────────────┤
22317 │$not │ Selector │ Matches if the │
22318 │ │ │ given selector does │
22319 │ │ │ not match. │
22320 ├─────────────┼──────────┼─────────────────────┤
22321 │$nor │ Array │ Matches if none of │
22322 │ │ │ the selectors in │
22323 │ │ │ the array match. │
22324 ├─────────────┼──────────┼─────────────────────┤
22325 │$all │ Array │ Matches an array │
22326 │ │ │ value if it con‐ │
22327 │ │ │ tains all the ele‐ │
22328 │ │ │ ments of the argu‐ │
22329 │ │ │ ment array. │
22330 ├─────────────┼──────────┼─────────────────────┤
22331 │$elemMatch │ Selector │ Matches and returns │
22332 │ │ │ all documents that │
22333 │ │ │ contain an array │
22334 │ │ │ field with at least │
22335 │ │ │ one element that │
22336 │ │ │ matches all the │
22337 │ │ │ specified query │
22338 │ │ │ criteria. │
22339 ├─────────────┼──────────┼─────────────────────┤
22340 │$allMatch │ Selector │ Matches and returns │
22341 │ │ │ all documents that │
22342 │ │ │ contain an array │
22343 │ │ │ field with all its │
22344 │ │ │ elements matching │
22345 │ │ │ all the specified │
22346 │ │ │ query criteria. │
22347 └─────────────┴──────────┴─────────────────────┘
22348 │$keyMapMatch │ Selector │ Matches and returns │
22356 │ │ │ all documents that │
22357 │ │ │ contain a map that │
22358 │ │ │ contains at least │
22359 │ │ │ one key that │
22360 │ │ │ matches all the │
22361 │ │ │ specified query │
22362 │ │ │ criteria. │
22363 └─────────────┴──────────┴─────────────────────┘
22364 The $and operator
22366 $and operator used with two fields
22367 {
22369 "selector": {
22370 "$and": [
22371 {
22372 "$title": "Total Recall"
22373 },
22374 {
22375 "year": {
22376 "$in": [1984, 1991]
22377 }
22378 }
22379 ]
22380 },
22381 "fields": [
22382 "year",
22383 "title",
22384 "cast"
22385 ]
22386 }
22387 The $and operator matches if all the selectors in the array match.
22389 Below is an example using the primary index (`_all_docs`):
22390 {
22392 "$and": [
22393 {
22394 "_id": { "$gt": null }
22395 },
22396 {
22397 "year": {
22398 "$in": [2014, 2015]
22399 }
22400 }
22401 ]
22402 }
22403 The $or operator
22405 The $or operator matches if any of the selectors in the array match.
22407 Below is an example used with an index on the field "year":
22408 {
22410 "year": 1977,
22411 "$or": [
22412 { "director": "George Lucas" },
22413 { "director": "Steven Spielberg" }
22414 ]
22415 }
22416 The $not operator
22418 The $not operator matches if the given selector does not match. Below
22420 is an example used with an index on the field "year":
22421 {
22423 "year": {
22424 "$gte": 1900
22425 },
22426 "year": {
22427 "$lte": 1903
22428 },
22429 "$not": {
22430 "year": 1901
22431 }
22432 }
22433 The $nor operator
22435 The $nor operator matches if the given selector does not match. Below
22437 is an example used with an index on the field "year":
22438 {
22440 "year": {
22441 "$gte": 1900
22442 },
22443 "year": {
22444 "$lte": 1910
22445 },
22446 "$nor": [
22447 { "year": 1901 },
22448 { "year": 1905 },
22449 { "year": 1907 }
22450 ]
22451 }
22452 The $all operator
22454 The $all operator matches an array value if it contains all the ele‐
22456 ments of the argument array. Below is an example used with the primary
22457 index (_all_docs):
22458 {
22460 "_id": {
22461 "$gt": null
22462 },
22463 "genre": {
22464 "$all": ["Comedy","Short"]
22465 }
22466 }
22467 The $elemMatch operator
22469 The $elemMatch operator matches and returns all documents that contain
22471 an array field with at least one element matching the supplied query
22472 criteria. Below is an example used with the primary index (_all_docs):
22473 {
22475 "_id": { "$gt": null },
22476 "genre": {
22477 "$elemMatch": {
22478 "$eq": "Horror"
22479 }
22480 }
22481 }
22482 The $allMatch operator
22484 The $allMatch operator matches and returns all documents that contain
22486 an array field with all its elements matching the supplied query crite‐
22487 ria. Below is an example used with the primary index (_all_docs):
22488 {
22490 "_id": { "$gt": null },
22491 "genre": {
22492 "$allMatch": {
22493 "$eq": "Horror"
22494 }
22495 }
22496 }
22497 The $keyMapMatch operator
22499 The $keyMapMatch operator matches and returns all documents that con‐
22501 tain a map that contains at least one key that matches all the speci‐
22502 fied query criteria. Below is an example used with the primary index
22503 (_all_docs):
22504 {
22506 "_id": { "$gt": null },
22507 "cameras": {
22508 "$keyMapMatch": {
22509 "$eq": "secondary"
22510 }
22511 }
22512 }
22513 Condition Operators
22515 Condition operators are specific to a field, and are used to evaluate
22516 the value stored in that field. For instance, the basic $eq operator
22517 matches when the specified field contains a value that is equal to the
22518 supplied argument.
22519 The basic equality and inequality operators common to most programming
22521 languages are supported. In addition, some ‘meta’ condition operators
22522 are available. Some condition operators accept any valid JSON content
22523 as the argument. Other condition operators require the argument to be
22524 in a specific JSON format.
22525 ┌──────────────┬──────────┬──────────────────┬──────────────────┐
22527 │Operator type │ Operator │ Argument │ Purpose │
22528 ├──────────────┼──────────┼──────────────────┼──────────────────┤
22529 │(In)equality │ $lt │ Any JSON │ The field is │
22530 │ │ │ │ less than the │
22531 │ │ │ │ argument │
22532 ├──────────────┼──────────┼──────────────────┼──────────────────┤
22533 │ │ $lte │ Any JSON │ The field is │
22534 │ │ │ │ less than or │
22535 │ │ │ │ equal to the │
22536 │ │ │ │ argument. │
22537 ├──────────────┼──────────┼──────────────────┼──────────────────┤
22538 │ │ $eq │ Any JSON │ The field is │
22539 │ │ │ │ equal to the │
22540 │ │ │ │ argument │
22541 ├──────────────┼──────────┼──────────────────┼──────────────────┤
22542 │ │ $ne │ Any JSON │ The field is not │
22543 │ │ │ │ equal to the │
22544 │ │ │ │ argument. │
22545 ├──────────────┼──────────┼──────────────────┼──────────────────┤
22546 │ │ $gte │ Any JSON │ The field is │
22547 │ │ │ │ greater than or │
22548 │ │ │ │ equal to the │
22549 │ │ │ │ argument. │
22550 ├──────────────┼──────────┼──────────────────┼──────────────────┤
22551 │ │ $gt │ Any JSON │ The field is │
22552 │ │ │ │ greater than the │
22553 │ │ │ │ to the argument. │
22554 ├──────────────┼──────────┼──────────────────┼──────────────────┤
22555 │Object │ $exists │ Boolean │ Check whether │
22556 │ │ │ │ the field exists │
22557 │ │ │ │ or not, regard‐ │
22558 │ │ │ │ less of its │
22559 │ │ │ │ value. │
22560 ├──────────────┼──────────┼──────────────────┼──────────────────┤
22561 │ │ $type │ String │ Check the docu‐ │
22562 │ │ │ │ ment field’s │
22563 │ │ │ │ type. Valid │
22564 │ │ │ │ values are │
22565 │ │ │ │ "null", "bool‐ │
22566 │ │ │ │ ean", "number", │
22567 │ │ │ │ "string", │
22568 │ │ │ │ "array", and │
22569 │ │ │ │ "object". │
22570 ├──────────────┼──────────┼──────────────────┼──────────────────┤
22571 │Array │ $in │ Array of JSON │ The document │
22572 │ │ │ values │ field must exist │
22573 │ │ │ │ in the list pro‐ │
22574 │ │ │ │ vided. │
22575 └──────────────┴──────────┴──────────────────┴──────────────────┘
22576 │ │ $nin │ Array of JSON │ The document │
22582 │ │ │ values │ field not must │
22583 │ │ │ │ exist in the │
22584 │ │ │ │ list provided. │
22585 ├──────────────┼──────────┼──────────────────┼──────────────────┤
22586 │ │ $size │ Integer │ Special condi‐ │
22587 │ │ │ │ tion to match │
22588 │ │ │ │ the length of an │
22589 │ │ │ │ array field in a │
22590 │ │ │ │ document. │
22591 │ │ │ │ Non-array fields │
22592 │ │ │ │ cannot match │
22593 │ │ │ │ this condition. │
22594 ├──────────────┼──────────┼──────────────────┼──────────────────┤
22595 │Miscellaneous │ $mod │ [Divisor, │ Divisor and │
22596 │ │ │ Remainder] │ Remainder are │
22597 │ │ │ │ both positive or │
22598 │ │ │ │ negative inte‐ │
22599 │ │ │ │ gers. Non-inte‐ │
22600 │ │ │ │ ger values │
22601 │ │ │ │ result in a 404. │
22602 │ │ │ │ Matches docu‐ │
22603 │ │ │ │ ments where │
22604 │ │ │ │ field % Divisor │
22605 │ │ │ │ == Remainder is │
22606 │ │ │ │ true, and only │
22607 │ │ │ │ when the docu‐ │
22608 │ │ │ │ ment field is an │
22609 │ │ │ │ integer. │
22610 ├──────────────┼──────────┼──────────────────┼──────────────────┤
22611 │ │ $regex │ String │ A regular │
22612 │ │ │ │ expression pat‐ │
22613 │ │ │ │ tern to match │
22614 │ │ │ │ against the doc‐ │
22615 │ │ │ │ ument field. │
22616 │ │ │ │ Only matches │
22617 │ │ │ │ when the field │
22618 │ │ │ │ is a string │
22619 │ │ │ │ value and │
22620 │ │ │ │ matches the sup‐ │
22621 │ │ │ │ plied regular │
22622 │ │ │ │ expression. The │
22623 │ │ │ │ matching algo‐ │
22624 │ │ │ │ rithms are based │
22625 │ │ │ │ on the Perl Com‐ │
22626 │ │ │ │ patible Regular │
22627 │ │ │ │ Expression │
22628 │ │ │ │ (PCRE) library. │
22629 │ │ │ │ For more infor‐ │
22630 │ │ │ │ mation about │
22631 │ │ │ │ what is imple‐ │
22632 │ │ │ │ mented, see the │
22633 │ │ │ │ see the Erlang │
22634 │ │ │ │ Regular Expres‐ │
22635 │ │ │ │ sion │
22636 └──────────────┴──────────┴──────────────────┴──────────────────┘
22637 WARNING:
22639 Regular expressions do not work with indexes, so they should not be
22640 used to filter large data sets. They can, however, be used to
22641 restrict a partial index.
22642 Creating Selector Expressions
22644 We have seen examples of combining selector expressions, such as using
22645 explicit $and and $eq operators.
22646 In general, whenever you have an operator that takes an argument, that
22648 argument can itself be another operator with arguments of its own. This
22649 enables us to build up more complex selector expressions.
22650 However, only equality operators such as $eq, $gt, $gte, $lt, and $lte
22652 (but not $ne) can be used as the basis of a query. You should include
22653 at least one of these in a selector.
22654 For example, if you try to perform a query that attempts to match all
22656 documents that have a field called afieldname containing a value that
22657 begins with the letter A, this will trigger a warning because no index
22658 could be used and the database performs a full scan of the primary
22659 index:
22660 Request
22661 POST /movies/_find HTTP/1.1
22663 Accept: application/json
22664 Content-Type: application/json
22665 Content-Length: 112
22666 Host: localhost:5984
22667 {
22669 "selector": {
22670 "afieldname": {"$regex": "^A"}
22671 }
22672 }
22673 Response:
22675 HTTP/1.1 200 OK
22677 Cache-Control: must-revalidate
22678 Content-Type: application/json
22679 Date: Thu, 01 Sep 2016 17:25:51 GMT
22680 Server: CouchDB (Erlang OTP)
22681 Transfer-Encoding: chunked
22682 {
22684 "warning":"no matching index found, create an index to optimize
22685 query time",
22686 "docs":[
22687 ]
22688 }
22689 WARNING:
22691 It’s always recommended that you create an appropriate index when
22692 deploying in production.
22693 Most selector expressions work exactly as you would expect for the
22695 given operator. But it is not always the case: for example, comparison
22696 of strings is done with ICU and can can give surprising results if you
22697 were expecting ASCII ordering. See views/collation for more details.
22698 Sort Syntax
22700 The sort field contains a list of field name and direction pairs,
22701 expressed as a basic array. The first field name and direction pair is
22702 the topmost level of sort. The second pair, if provided, is the next
22703 level of sort.
22704 The field can be any field, using dotted notation if desired for
22706 sub-document fields.
22707 The direction value is "asc" for ascending, and "desc" for descending.
22709 If you omit the direction value, the default "asc" is used.
22710 Example, sorting by 2 fields:
22712 [{"fieldName1": "desc"}, {"fieldName2": "desc" }]
22714 Example, sorting by 2 fields, assuming default direction for both :
22716 ["fieldNameA", "fieldNameB"]
22718 A typical requirement is to search for some content using a selector,
22720 then to sort the results according to the specified field, in the
22721 required direction.
22722 To use sorting, ensure that:
22724 · At least one of the sort fields is included in the selector.
22726 ·
22728 There is an index already defined, with all the sort fields in the
22730 same
22731 order.
22732 · Each object in the sort array has a single key.
22734 If an object in the sort array does not have a single key, the result‐
22736 ing sort order is implementation specific and might change.
22737 Find does not support multiple fields with different sort orders, so
22739 the directions must be either all ascending or all descending.
22740 For field names in text search sorts, it is sometimes necessary for a
22742 field type to be specified, for example:
22743 { "<fieldname>:string": "asc"}
22745 If possible, an attempt is made to discover the field type based on the
22747 selector. In ambiguous cases the field type must be provided explic‐
22748 itly.
22749 The sorting order is undefined when fields contain different data
22751 types. This is an important difference between text and view indexes.
22752 Sorting behavior for fields with different data types might change in
22753 future versions.
22754 A simple query, using sorting:
22755 {
22757 "selector": {"Actor_name": "Robert De Niro"},
22758 "sort": [{"Actor_name": "asc"}, {"Movie_runtime": "asc"}]
22759 }
22760 Filtering Fields
22762 It is possible to specify exactly which fields are returned for a docu‐
22763 ment when selecting from a database. The two advantages are:
22764 ·
22766 Your results are limited to only those parts of the document that are
22768 required for your application.
22769 · A reduction in the size of the response.
22771 The fields returned are specified as an array.
22773 Only the specified filter fields are included, in the response. There
22775 is no automatic inclusion of the _id or other metadata fields when a
22776 field list is included.
22777 Example of selective retrieval of fields from matching documents:
22779 {
22781 "selector": { "Actor_name": "Robert De Niro" },
22782 "fields": ["Actor_name", "Movie_year", "_id", "_rev"]
22783 }
22784 Pagination
22786 Mango queries support pagination via the bookmark field. Every _find
22787 response contains a bookmark - a token that CouchDB uses to determine
22788 where to resume from when subsequent queries are made. To get the next
22789 set of query results, add the bookmark that was received in the previ‐
22790 ous response to your next request. Remember to keep the selector the
22791 same, otherwise you will receive unexpected results. To paginate back‐
22792 wards, you can use a previous bookmark to return the previous set of
22793 results.
22794 Note that the presence of a bookmark doesn’t guarantee that there are
22796 more results. You can to test whether you have reached the end of the
22797 result set by comparing the number of results returned with the page
22798 size requested - if results returned < limit, there are no more.
22799 Execution Statistics
22801 Find can return basic execution statistics for a specific request. Com‐
22802 bined with the _explain endpoint, this should provide some insight as
22803 to whether indexes are being used effectively.
22804 The execution statistics currently include:
22806 ┌───────────────────────────┬────────────────────────────┐
22808 │Field │ Description │
22809 ├───────────────────────────┼────────────────────────────┤
22810 │total_keys_examined │ Number of index keys exam‐ │
22811 │ │ ined. Currently always 0. │
22812 ├───────────────────────────┼────────────────────────────┤
22813 │total_docs_examined │ Number of documents │
22814 │ │ fetched from the database │
22815 │ │ / index, equivalent to │
22816 │ │ using include_docs=true in │
22817 │ │ a view. These may then be │
22818 │ │ filtered in-memory to fur‐ │
22819 │ │ ther narrow down the │
22820 │ │ result set based on the │
22821 │ │ selector. │
22822 ├───────────────────────────┼────────────────────────────┤
22823 │total_quorum_docs_examined │ Number of documents │
22824 │ │ fetched from the database │
22825 │ │ using an out-of-band docu‐ │
22826 │ │ ment fetch. This is only │
22827 │ │ non-zero when read quorum │
22828 │ │ > 1 is specified in the │
22829 │ │ query parameters. │
22830 ├───────────────────────────┼────────────────────────────┤
22831 │results_returned │ Number of results returned │
22832 │ │ from the query. Ideally │
22833 │ │ this should not be signif‐ │
22834 │ │ icantly lower than the │
22835 │ │ total documents / keys │
22836 │ │ examined. │
22837 ├───────────────────────────┼────────────────────────────┤
22838 │execution_time_ms │ Total execution time in │
22839 │ │ milliseconds as measured │
22840 │ │ by the database. │
22841 └───────────────────────────┴────────────────────────────┘
22842 /db/_index
22844 Mango is a declarative JSON querying language for CouchDB databases.
22845 Mango wraps several index types, starting with the Primary Index
22846 out-of-the-box. Mango indexes, with index type json, are built using
22847 MapReduce Views.
22848 POST /{db}/_index
22850 Create a new index on a database
22851 Parameters
22853 · db – Database name
22855 Request Headers
22857 · Content-Type – .INDENT 2.0
22859 · application/json
22861 Query Parameters
22864 · index (json) – JSON object describing the index to create.
22866 · ddoc (string) – Name of the design document in which the index
22868 will be created. By default, each index will be created in its
22869 own design document. Indexes can be grouped into design docu‐
22870 ments for efficiency. However, a change to one index in a
22871 design document will invalidate all other indexes in the same
22872 document (similar to views). Optional
22873 · name (string) – Name of the index. If no name is provided, a
22875 name will be generated automatically. Optional
22876 · type (string) – Can be "json" or "text". Defaults to json.
22878 Geospatial indexes will be supported in the future. Optional
22879 Text indexes are supported via a third party library Optional
22880 · partial_filter_selector (json) – A selector to apply to docu‐
22882 ments at indexing time, creating a partial index. Optional
22883 · partitioned (boolean) – Determines whether a JSON index is
22885 partitioned or global. The default value of partitioned is the
22886 partitioned property of the database. To create a global index
22887 on a partitioned database, specify false for the "partitioned"
22888 field. If you specify true for the "partitioned" field on an
22889 unpartitioned database, an error occurs.
22890 Response Headers
22892 · Content-Type – application/json
22894 · Transfer-Encoding – chunked
22896 Response JSON Object
22898 · result (string) – Flag to show whether the index was created
22900 or one already exists. Can be “created” or “exists”.
22901 · id (string) – Id of the design document the index was created
22903 in.
22904 · name (string) – Name of the index created.
22906 Status Codes
22908 · 200 OK – Index created successfully or already exists
22910 · 400 Bad Request – Invalid request
22912 · 401 Unauthorized – Admin permission required
22914 · 500 Internal Server Error – Execution error
22916
22918The index object is a JSON array of field names following the sort syntax.
22920Nested fields are also allowed, e.g. “person.name”.
22921Example of creating a new index for the field called foo:
22923 Request:
22924 POST /db/_index HTTP/1.1
22926 Content-Type: application/json
22927 Content-Length: 116
22928 Host: localhost:5984
22929 {
22931 "index": {
22932 "fields": ["foo"]
22933 },
22934 "name" : "foo-index",
22935 "type" : "json"
22936 }
22937 The returned JSON confirms the index has been created:
22939 Response:
22940 HTTP/1.1 200 OK
22942 Cache-Control: must-revalidate
22943 Content-Length: 96
22944 Content-Type: application/json
22945 Date: Thu, 01 Sep 2016 18:17:48 GMT
22946 Server: CouchDB (Erlang OTP/18)
22947 {
22949 "result":"created",
22950 "id":"_design/a5f4711fc9448864a13c81dc71e660b524d7410c",
22951 "name":"foo-index"
22952 }
22953 Example index creation using all available query parameters
22955 {
22957 "selector": {
22958 "year": {
22959 "$gt": 2010
22960 }
22961 },
22962 "fields": ["_id", "_rev", "year", "title"],
22963 "sort": [{"year": "asc"}],
22964 "limit": 10,
22965 "skip": 0
22966 }
22967 By default, a JSON index will include all documents that have the
22969 indexed fields present, including those which have null values.
22970 Partial Indexes
22972 Partial indexes allow documents to be filtered at indexing time, poten‐
22973 tially offering significant performance improvements for query selec‐
22974 tors that don’t map cleanly to a range query on an index.
22975 Let’s look at an example query:
22977 {
22979 "selector": {
22980 "status": {
22981 "$ne": "archived"
22982 },
22983 "type": "user"
22984 }
22985 }
22986 Without a partial index, this requires a full index scan to find all
22988 the documents of "type":"user" that do not have a status of "archived".
22989 This is because a normal index can only be used to match contiguous
22990 rows, and the "$ne" operator cannot guarantee that.
22991 To improve response times, we can create an index which excludes docu‐
22993 ments where "status": { "$ne": "archived" } at index time using the
22994 "partial_filter_selector" field:
22995 POST /db/_index HTTP/1.1
22997 Content-Type: application/json
22998 Content-Length: 144
22999 Host: localhost:5984
23000 {
23002 "index": {
23003 "partial_filter_selector": {
23004 "status": {
23005 "$ne": "archived"
23006 }
23007 },
23008 "fields": ["type"]
23009 },
23010 "ddoc" : "type-not-archived",
23011 "type" : "json"
23012 }
23013 Partial indexes are not currently used by the query planner unless
23015 specified by a "use_index" field, so we need to modify the original
23016 query:
23017 {
23019 "selector": {
23020 "status": {
23021 "$ne": "archived"
23022 },
23023 "type": "user"
23024 },
23025 "use_index": "type-not-archived"
23026 }
23027 Technically, we don’t need to include the filter on the "status" field
23029 in the query selector - the partial index ensures this is always true -
23030 but including it makes the intent of the selector clearer and will make
23031 it easier to take advantage of future improvements to query planning
23032 (e.g. automatic selection of partial indexes).
23033 GET /{db}/_index
23035 When you make a GET request to /db/_index, you get a list of all
23036 indexes in the database. In addition to the information avail‐
23037 able through this API, indexes are also stored in design docu‐
23038 ments <index-functions>. Design documents are regular documents
23039 that have an ID starting with _design/. Design documents can be
23040 retrieved and modified in the same way as any other document,
23041 although this is not necessary when using Mango.
23042 Parameters
23044 · db – Database name.
23046 Response Headers
23048 · Content-Type – application/json
23050 · Transfer-Encoding – chunked
23052 Response JSON Object
23054 · total_rows (number) – Number of indexes
23056 · indexes (object) – Array of index definitions
23058 Status Codes
23060 · 200 OK – Success
23062 · 400 Bad Request – Invalid request
23064 · 401 Unauthorized – Read permission required
23066 · 500 Internal Server Error – Execution error
23068 Format of index objects:
23070 ·
23072 ddoc: ID of the design document the index belongs to.
23074 This ID
23075 can be used to retrieve the design document con‐
23076 taining the index, by making a GET request to
23077 /db/ddoc, where ddoc is the value of this field.
23078 · name: Name of the index.
23080 ·
23082 type: Type of the index. Currently “json” is the only
23084 supported type.
23085 ·
23087 def: Definition of the index, containing the indexed
23089 fields
23090 and the sort order: ascending or descending.
23091 Request:
23093 GET /db/_index HTTP/1.1
23095 Accept: application/json
23096 Host: localhost:5984
23097 Response:
23099 HTTP/1.1 200 OK
23101 Cache-Control: must-revalidate
23102 Content-Length: 238
23103 Content-Type: application/json
23104 Date: Thu, 01 Sep 2016 18:17:48 GMT
23105 Server: CouchDB (Erlang OTP/18)
23106 {
23108 "total_rows": 2,
23109 "indexes": [
23110 {
23111 "ddoc": null,
23112 "name": "_all_docs",
23113 "type": "special",
23114 "def": {
23115 "fields": [
23116 {
23117 "_id": "asc"
23118 }
23119 ]
23120 }
23121 },
23122 {
23123 "ddoc": "_design/a5f4711fc9448864a13c81dc71e660b524d7410c",
23124 "name": "foo-index",
23125 "type": "json",
23126 "def": {
23127 "fields": [
23128 {
23129 "foo": "asc"
23130 }
23131 ]
23132 }
23133 }
23134 ]
23135 }
23136 DELETE /{db}/_index/{designdoc}/json/{name}
23138 Parameters
23140 · db – Database name.
23142 · designdoc – Design document name.
23144 · name – Index name.
23146 Response Headers
23148 · Content-Type – application/json
23150 Response JSON Object
23152 · ok (string) – “true” if successful.
23154 Status Codes
23156 · 200 OK – Success
23158 · 400 Bad Request – Invalid request
23160 · 401 Unauthorized – Writer permission required
23162 · 404 Not Found – Index not found
23164 · 500 Internal Server Error – Execution error
23166 Request:
23168 DELETE /db/_index/_design/a5f4711fc9448864a13c81dc71e660b524d7410c/json/foo-index HTTP/1.1
23170 Accept: */*
23171 Host: localhost:5984
23172 Response:
23174 HTTP/1.1 200 OK
23176 Cache-Control: must-revalidate
23177 Content-Length: 12
23178 Content-Type: application/json
23179 Date: Thu, 01 Sep 2016 19:21:40 GMT
23180 Server: CouchDB (Erlang OTP/18)
23181 {
23183 "ok": true
23184 }
23185 /db/_explain
23187 POST /{db}/_explain
23188 Shows which index is being used by the query. Parameters are
23189 the same as _find
23190 Parameters
23192 · db – Database name
23194 Request Headers
23196 · Content-Type – application/json
23198 Response Headers
23200 · Content-Type – application/json
23202 · Transfer-Encoding – chunked
23204 Response JSON Object
23206 · dbname (string) – Name of database
23208 · index (object) – Index used to fulfill the query
23210 · selector (object) – Query selector used
23212 · opts (object) – Query options used
23214 · limit (number) – Limit parameter used
23216 · skip (number) – Skip parameter used
23218 · fields (array) – Fields to be returned by the query
23220 · range (object) – Range parameters passed to the under‐
23222 lying view
23223 Status Codes
23225 · 200 OK – Request completed successfully
23227 · 400 Bad Request – Invalid request
23229 · 401 Unauthorized – Read permission required
23231 · 500 Internal Server Error – Execution error
23233 Request:
23235 POST /movies/_explain HTTP/1.1
23237 Accept: application/json
23238 Content-Type: application/json
23239 Content-Length: 168
23240 Host: localhost:5984
23241 {
23243 "selector": {
23244 "year": {"$gt": 2010}
23245 },
23246 "fields": ["_id", "_rev", "year", "title"],
23247 "sort": [{"year": "asc"}],
23248 "limit": 2,
23249 "skip": 0
23250 }
23251 Response:
23253 HTTP/1.1 200 OK
23255 Cache-Control: must-revalidate
23256 Content-Type: application/json
23257 Date: Thu, 01 Sep 2016 15:41:53 GMT
23258 Server: CouchDB (Erlang OTP)
23259 Transfer-Encoding: chunked
23260 {
23262 "dbname": "movies",
23263 "index": {
23264 "ddoc": "_design/0d61d9177426b1e2aa8d0fe732ec6e506f5d443c",
23265 "name": "0d61d9177426b1e2aa8d0fe732ec6e506f5d443c",
23266 "type": "json",
23267 "def": {
23268 "fields": [
23269 {
23270 "year": "asc"
23271 }
23272 ]
23273 }
23274 },
23275 "selector": {
23276 "year": {
23277 "$gt": 2010
23278 }
23279 },
23280 "opts": {
23281 "use_index": [],
23282 "bookmark": "nil",
23283 "limit": 2,
23284 "skip": 0,
23285 "sort": {},
23286 "fields": [
23287 "_id",
23288 "_rev",
23289 "year",
23290 "title"
23291 ],
23292 "r": [
23293 49
23294 ],
23295 "conflicts": false
23296 },
23297 "limit": 2,
23298 "skip": 0,
23299 "fields": [
23300 "_id",
23301 "_rev",
23302 "year",
23303 "title"
23304 ],
23305 "range": {
23306 "start_key": [
23307 2010
23308 ],
23309 "end_key": [
23310 {}
23311 ]
23312 }
23313 }
23314 Index selection
23316 _find chooses which index to use for responding to a query, unless you
23317 specify an index at query time.
23318 The query planner looks at the selector section and finds the index
23320 with the closest match to operators and fields used in the query. If
23321 there are two or more json type indexes that match, the index with the
23322 smallest number of fields in the index is preferred. If there are
23323 still two or more candidate indexes, the index with the first alphabet‐
23324 ical name is chosen.
23325 NOTE:
23327 It’s good practice to specify indexes explicitly in your queries.
23328 This prevents existing queries being affected by new indexes that
23329 might get added in a production environment.
23330 /db/_shards
23332 New in version 2.0.
23333 GET /{db}/_shards
23336 The response will contain a list of database shards. Each shard
23337 will have its internal database range, and the nodes on which
23338 replicas of those shards are stored.
23339 Parameters
23341 · db – Database name
23343 Request Headers
23345 · Accept – .INDENT 2.0
23347 · application/json
23349 · text/plain
23351 Response Headers
23354 · Content-Type – .INDENT 2.0
23356 · application/json
23358 · text/plain; charset=utf-8
23360 Response JSON Object
23363 · shards (object) – Mapping of shard ranges to individual shard
23365 replicas on each node in the cluster
23366 Status Codes
23368 · 200 OK – Request completed successfully
23370 · 400 Bad Request – Invalid database name
23372 · 401 Unauthorized – Read privilege required
23374 · 415 Unsupported Media Type – Bad Content-Type value
23376 · 500 Internal Server Error – Internal server error or timeout
23378
23380 GET /db/_shards HTTP/1.1
23382 Accept: */*
23383 Host: localhost:5984
23384 Response:
23386 HTTP/1.1 200 OK
23388 Cache-Control: must-revalidate
23389 Content-Length: 621
23390 Content-Type: application/json
23391 Date: Fri, 18 Jan 2019 19:55:14 GMT
23392 Server: CouchDB/2.4.0 (Erlang OTP/19)
23393 {
23395 "shards": {
23396 "00000000-1fffffff": [
23397 "couchdb@node1.example.com",
23398 "couchdb@node2.example.com",
23399 "couchdb@node3.example.com"
23400 ],
23401 "20000000-3fffffff": [
23402 "couchdb@node1.example.com",
23403 "couchdb@node2.example.com",
23404 "couchdb@node3.example.com"
23405 ],
23406 "40000000-5fffffff": [
23407 "couchdb@node1.example.com",
23408 "couchdb@node2.example.com",
23409 "couchdb@node3.example.com"
23410 ],
23411 "60000000-7fffffff": [
23412 "couchdb@node1.example.com",
23413 "couchdb@node2.example.com",
23414 "couchdb@node3.example.com"
23415 ],
23416 "80000000-9fffffff": [
23417 "couchdb@node1.example.com",
23418 "couchdb@node2.example.com",
23419 "couchdb@node3.example.com"
23420 ],
23421 "a0000000-bfffffff": [
23422 "couchdb@node1.example.com",
23423 "couchdb@node2.example.com",
23424 "couchdb@node3.example.com"
23425 ],
23426 "c0000000-dfffffff": [
23427 "couchdb@node1.example.com",
23428 "couchdb@node2.example.com",
23429 "couchdb@node3.example.com"
23430 ],
23431 "e0000000-ffffffff": [
23432 "couchdb@node1.example.com",
23433 "couchdb@node2.example.com",
23434 "couchdb@node3.example.com"
23435 ]
23436 }
23437 }
23438 /db/_shards/doc
23440 GET /{db}/_shards/{docid}
23441 Returns information about the specific shard into which a given
23442 document has been stored, along with information about the nodes
23443 on which that shard has a replica.
23444 Parameters
23446 · db – Database name
23448 · docid – Document ID
23450 Request Headers
23452 · Accept – .INDENT 2.0
23454 · application/json
23456 · text/plain
23458 Response Headers
23461 · Content-Type – .INDENT 2.0
23463 · application/json
23465 · text/plain; charset=utf-8
23467 Response JSON Object
23470 · range (string) – The shard range in which the document is
23472 stored
23473 · nodes (array) – List of nodes serving a replica of the shard
23475 Status Codes
23477 · 200 OK – Request completed successfully
23479 · 401 Unauthorized – Read privilege required
23481 · 404 Not Found – Database or document not found
23483 · 500 Internal Server Error – Internal server error or timeout
23485
23487 HTTP/1.1 200 OK
23489 Cache-Control: must-revalidate
23490 Content-Length: 94
23491 Content-Type: application/json
23492 Date: Fri, 18 Jan 2019 20:08:07 GMT
23493 Server: CouchDB/2.3.0-9d4cb03c2 (Erlang OTP/19)
23494 Response:
23496 HTTP/1.1 200 OK
23498 Cache-Control: must-revalidate
23499 Content-Length: 94
23500 Content-Type: application/json
23501 Date: Fri, 18 Jan 2019 20:26:33 GMT
23502 Server: CouchDB/2.3.0-9d4cb03c2 (Erlang OTP/19)
23503 {
23505 "range": "e0000000-ffffffff",
23506 "nodes": [
23507 "node1@127.0.0.1",
23508 "node2@127.0.0.1",
23509 "node3@127.0.0.1"
23510 ]
23511 }
23512 /db/_sync_shards
23514 New in version 2.3.1.
23515 POST /{db}/_sync_shards
23518 For the given database, force-starts internal shard synchroniza‐
23519 tion for all replicas of all database shards.
23520 This is typically only used when performing cluster maintenance,
23522 such as moving a shard.
23523 Parameters
23525 · db – Database name
23527 Request Headers
23529 · Accept – .INDENT 2.0
23531 · application/json
23533 · text/plain
23535 Response Headers
23538 · Content-Type – .INDENT 2.0
23540 · application/json
23542 · text/plain; charset=utf-8
23544 Response JSON Object
23547 · ok (boolean) – Operation status. Available in case of success
23549 · error (string) – Error type. Available if response code is 4xx
23551 · reason (string) – Error description. Available if response
23553 code is 4xx
23554 Status Codes
23556 · 202 Accepted – Request accepted
23558 · 400 Bad Request – Invalid database name
23560 · 401 Unauthorized – CouchDB Server Administrator privileges
23562 required
23563 · 404 Not Found – Database not found
23565 · 500 Internal Server Error – Internal server error or timeout
23567
23569 POST /db/_sync_shards HTTP/1.1
23571 Host: localhost:5984
23572 Accept: */*
23573 Response:
23575 HTTP/1.1 202 Accepted
23577 Cache-Control: must-revalidate
23578 Content-Length: 12
23579 Content-Type: application/json
23580 Date: Fri, 18 Jan 2019 20:19:23 GMT
23581 Server: CouchDB/2.3.0-9d4cb03c2 (Erlang OTP/19)
23582 X-Couch-Request-ID: 14f0b8d252
23583 X-CouchDB-Body-Time: 0
23584 {
23586 "ok": true
23587 }
23588
23590 Admins may want to bump their [mem3] sync_concurrency value to a
23591 larger figure for the duration of the shards sync.
23592 /db/_changes
23594 GET /{db}/_changes
23595 Returns a sorted list of changes made to documents in the data‐
23596 base, in time order of application, can be obtained from the
23597 database’s _changes resource. Only the most recent change for a
23598 given document is guaranteed to be provided, for example if a
23599 document has had fields added, and then deleted, an API client
23600 checking for changes will not necessarily receive the intermedi‐
23601 ate state of added documents.
23602 This can be used to listen for update and modifications to the
23604 database for post processing or synchronization, and for practi‐
23605 cal purposes, a continuously connected _changes feed is a rea‐
23606 sonable approach for generating a real-time log for most appli‐
23607 cations.
23608 Parameters
23610 · db – Database name
23612 Request Headers
23614 · Accept – .INDENT 2.0
23616 · application/json
23618 · text/event-stream
23620 · text/plain
23622 · Last-Event-ID – ID of the last events received by the server
23625 on a previous connection. Overrides since query parameter.
23626 Query Parameters
23628 · doc_ids (array) – List of document IDs to filter the changes
23630 feed as valid JSON array. Used with _doc_ids filter. Since
23631 length of URL is limited, it is better to use POST
23632 /{db}/_changes instead.
23633 · conflicts (boolean) – Includes conflicts information in
23635 response. Ignored if include_docs isn’t true. Default is
23636 false.
23637 · descending (boolean) – Return the change results in descending
23639 sequence order (most recent change first). Default is false.
23640 · feed (string) – .INDENT 2.0
23642 · normal Specifies Normal Polling Mode. All past changes are
23644 returned immediately. Default.
23645 · longpoll Specifies Long Polling Mode. Waits until at least one
23647 change has occurred, sends the change, then closes the connec‐
23648 tion. Most commonly used in conjunction with since=now, to
23649 wait for the next change.
23650 · continuous Sets Continuous Mode. Sends a line of JSON per
23652 event. Keeps the socket open until timeout.
23653 · eventsource Sets Event Source Mode. Works the same as Continu‐
23655 ous Mode, but sends the events in EventSource format.
23656 · filter (string) – Reference to a filter function from a design docu‐
23659 ment that will filter whole stream emitting only filtered events. See
23660 the section Change Notifications in the book CouchDB The Definitive
23661 Guide for more information.
23662 · heartbeat (number) – Period in milliseconds after which an empty line
23664 is sent in the results. Only applicable for longpoll, continuous, and
23665 eventsource feeds. Overrides any timeout to keep the feed alive
23666 indefinitely. Default is 60000. May be true to use default value.
23667 · include_docs (boolean) – Include the associated document with each
23669 result. If there are conflicts, only the winning revision is
23670 returned. Default is false.
23671 · attachments (boolean) – Include the Base64-encoded content of attach‐
23673 ments in the documents that are included if include_docs is true.
23674 Ignored if include_docs isn’t true. Default is false.
23675 · att_encoding_info (boolean) – Include encoding information in attach‐
23677 ment stubs if include_docs is true and the particular attachment is
23678 compressed. Ignored if include_docs isn’t true. Default is false.
23679 · last-event-id (number) – Alias of Last-Event-ID header.
23681 · limit (number) – Limit number of result rows to the specified value
23683 (note that using 0 here has the same effect as 1).
23684 · since – Start the results from the change immediately after the given
23686 update sequence. Can be valid update sequence or now value. Default
23687 is 0.
23688 · style (string) – Specifies how many revisions are returned in the
23690 changes array. The default, main_only, will only return the current
23691 “winning” revision; all_docs will return all leaf revisions (includ‐
23692 ing conflicts and deleted former conflicts).
23693 · timeout (number) – Maximum period in milliseconds to wait for a
23695 change before the response is sent, even if there are no results.
23696 Only applicable for longpoll or continuous feeds. Default value is
23697 specified by httpd/changes_timeout configuration option. Note that
23698 60000 value is also the default maximum timeout to prevent undetected
23699 dead connections.
23700 · view (string) – Allows to use view functions as filters. Documents
23702 counted as “passed” for view filter in case if map function emits at
23703 least one record for them. See _view for more info.
23704 · seq_interval (number) – When fetching changes in a batch, setting the
23706 seq_interval parameter tells CouchDB to only calculate the update seq
23707 with every Nth result returned. By setting seq_interval=<batch size>
23708 , where <batch size> is the number of results requested per batch,
23709 load can be reduced on the source CouchDB database; computing the seq
23710 value across many shards (esp. in highly-sharded databases) is expen‐
23711 sive in a heavily loaded CouchDB cluster.
23712 Response Headers
23714 · Cache-Control – no-cache if changes feed is eventsource
23716 · Content-Type – .INDENT 2.0
23718 · application/json
23720 · text/event-stream
23722 · text/plain; charset=utf-8
23724 · ETag – Response hash if changes feed is normal
23727 · Transfer-Encoding – chunked
23729 Response JSON Object
23731 · last_seq (json) – Last change update sequence
23733 · pending (number) – Count of remaining items in the feed
23735 · results (array) – Changes made to a database
23737 Status Codes
23739 · 200 OK – Request completed successfully
23741 · 400 Bad Request – Bad request
23743The results field of database changes:
23745 JSON Object
23747 · changes (array) – List of document’s leaves with single
23749 field rev.
23750 · id (string) – Document ID.
23752 · seq (json) – Update sequence.
23754 · deleted (bool) – true if the document is deleted.
23756 Request:
23758 GET /db/_changes?style=all_docs HTTP/1.1
23760 Accept: application/json
23761 Host: localhost:5984
23762 Response:
23764 HTTP/1.1 200 OK
23766 Cache-Control: must-revalidate
23767 Content-Type: application/json
23768 Date: Mon, 12 Aug 2013 00:54:58 GMT
23769 ETag: "6ASLEKEMSRABT0O5XY9UPO9Z"
23770 Server: CouchDB (Erlang/OTP)
23771 Transfer-Encoding: chunked
23772 {
23774 "last_seq": "5-g1AAAAIreJyVkEsKwjAURZ-toI5cgq5A0sQ0OrI70XyppcaRY92J7kR3ojupaSPUUgotgRd4yTlwbw4A0zRUMLdnpaMkwmyF3Ily9xBwEIuiKLI05KOTW0wkV4rruP29UyGWbordzwKVxWBNOGMKZhertDlarbr5pOT3DV4gudUC9-MPJX9tpEAYx4TQASns2E24ucuJ7rXJSL1BbEgf3vTwpmedCZkYa7Pulck7Xt7x_usFU2aIHOD4eEfVTVA5KMGUkqhNZV-8_o5i",
23775 "pending": 0,
23776 "results": [
23777 {
23778 "changes": [
23779 {
23780 "rev": "2-7051cbe5c8faecd085a3fa619e6e6337"
23781 }
23782 ],
23783 "id": "6478c2ae800dfc387396d14e1fc39626",
23784 "seq": "3-g1AAAAG3eJzLYWBg4MhgTmHgz8tPSTV0MDQy1zMAQsMcoARTIkOS_P___7MSGXAqSVIAkkn2IFUZzIkMuUAee5pRqnGiuXkKA2dpXkpqWmZeagpu_Q4g_fGEbEkAqaqH2sIItsXAyMjM2NgUUwdOU_JYgCRDA5ACGjQfn30QlQsgKvcjfGaQZmaUmmZClM8gZhyAmHGfsG0PICrBPmQC22ZqbGRqamyIqSsLAAArcXo"
23785 },
23786 {
23787 "changes": [
23788 {
23789 "rev": "3-7379b9e515b161226c6559d90c4dc49f"
23790 }
23791 ],
23792 "deleted": true,
23793 "id": "5bbc9ca465f1b0fcd62362168a7c8831",
23794 "seq": "4-g1AAAAHXeJzLYWBg4MhgTmHgz8tPSTV0MDQy1zMAQsMcoARTIkOS_P___7MymBMZc4EC7MmJKSmJqWaYynEakaQAJJPsoaYwgE1JM0o1TjQ3T2HgLM1LSU3LzEtNwa3fAaQ_HqQ_kQG3qgSQqnoUtxoYGZkZG5uS4NY8FiDJ0ACkgAbNx2cfROUCiMr9CJ8ZpJkZpaaZEOUziBkHIGbcJ2zbA4hKsA-ZwLaZGhuZmhobYurKAgCz33kh"
23795 },
23796 {
23797 "changes": [
23798 {
23799 "rev": "6-460637e73a6288cb24d532bf91f32969"
23800 },
23801 {
23802 "rev": "5-eeaa298781f60b7bcae0c91bdedd1b87"
23803 }
23804 ],
23805 "id": "729eb57437745e506b333068fff665ae",
23806 "seq": "5-g1AAAAIReJyVkE0OgjAQRkcwUVceQU9g-mOpruQm2tI2SLCuXOtN9CZ6E70JFmpCCCFCmkyTdt6bfJMDwDQNFcztWWkcY8JXyB2cu49AgFwURZGloRid3MMkEUoJHbXbOxVy6arc_SxQWQzRVHCuYHaxSpuj1aqbj0t-3-AlSrZakn78oeSvjRSIkIhSNiCFHbsKN3c50b02mURvEB-yD296eNOzzoRMRLRZ98rkHS_veGcC_nR-fGe1gaCaxihhjOI2lX0BhniHaA"
23807 }
23808 ]
23809 }
23810Changed in version 0.11.0: added include_docs parameter
23812Changed in version 1.2.0: added view parameter and special value _view for
23816Changed in version 1.3.0: since parameter could take now value to start listen
23819changes since current seq number.
23820Changed in version 1.3.0: eventsource feed type added.
23823Changed in version 1.4.0: Support Last-Event-ID header.
23826Changed in version 1.6.0: added attachments and att_encoding_info parameters
23829Changed in version 2.0.0: update sequences can be any valid json object, added
23833
23836 If the specified replicas of the shards in any given since value are
23837 unavailable, alternative replicas are selected, and the last known
23838 checkpoint between them is used. If this happens, you might see
23839 changes again that you have previously seen. Therefore, an applica‐
23840 tion making use of the _changes feed should be ‘idempotent’, that
23841 is, able to receive the same data multiple times, safely.
23842 NOTE:
23844 Cloudant Sync and PouchDB already optimize the replication process
23845 by setting seq_interval parameter to the number of results expected
23846 per batch. This parameter increases throughput by reducing latency
23847 between sequential requests in bulk document transfers. This has
23848 resulted in up to a 20% replication performance improvement in
23849 highly-sharded databases.
23850 WARNING:
23852 Using the attachments parameter to include attachments in the
23853 changes feed is not recommended for large attachment sizes. Also
23854 note that the Base64-encoding that is used leads to a 33% overhead
23855 (i.e. one third) in transfer size for attachments.
23856 WARNING:
23858 The results returned by _changes are partially ordered. In other
23859 words, the order is not guaranteed to be preserved for multiple
23860 calls.
23861 POST /{db}/_changes
23863 Requests the database changes feed in the same way as GET
23864 /{db}/_changes does, but is widely used with ?filter=_doc_ids
23865 query parameter and allows one to pass a larger list of document
23866 IDs to filter.
23867 Request:
23869 POST /recipes/_changes?filter=_doc_ids HTTP/1.1
23871 Accept: application/json
23872 Content-Length: 40
23873 Content-Type: application/json
23874 Host: localhost:5984
23875 {
23877 "doc_ids": [
23878 "SpaghettiWithMeatballs"
23879 ]
23880 }
23881 Response:
23883 HTTP/1.1 200 OK
23885 Cache-Control: must-revalidate
23886 Content-Type: application/json
23887 Date: Sat, 28 Sep 2013 07:23:09 GMT
23888 ETag: "ARIHFWL3I7PIS0SPVTFU6TLR2"
23889 Server: CouchDB (Erlang OTP)
23890 Transfer-Encoding: chunked
23891 {
23893 "last_seq": "5-g1AAAAIreJyVkEsKwjAURZ-toI5cgq5A0sQ0OrI70XyppcaRY92J7kR3ojupaSPUUgotgRd4yTlwbw4A0zRUMLdnpaMkwmyF3Ily9xBwEIuiKLI05KOTW0wkV4rruP29UyGWbordzwKVxWBNOGMKZhertDlarbr5pOT3DV4gudUC9-MPJX9tpEAYx4TQASns2E24ucuJ7rXJSL1BbEgf3vTwpmedCZkYa7Pulck7Xt7x_usFU2aIHOD4eEfVTVA5KMGUkqhNZV8_o5i",
23894 "pending": 0,
23895 "results": [
23896 {
23897 "changes": [
23898 {
23899 "rev": "13-bcb9d6388b60fd1e960d9ec4e8e3f29e"
23900 }
23901 ],
23902 "id": "SpaghettiWithMeatballs",
23903 "seq": "5-g1AAAAIReJyVkE0OgjAQRkcwUVceQU9g-mOpruQm2tI2SLCuXOtN9CZ6E70JFmpCCCFCmkyTdt6bfJMDwDQNFcztWWkcY8JXyB2cu49AgFwURZGloRid3MMkEUoJHbXbOxVy6arc_SxQWQzRVHCuYHaxSpuj1aqbj0t-3-AlSrZakn78oeSvjRSIkIhSNiCFHbsKN3c50b02mURvEB-yD296eNOzzoRMRLRZ98rkHS_veGcC_nR-fGe1gaCaxihhjOI2lX0BhniHaA"
23904 }
23905 ]
23906 }
23907 Changes Feeds
23909 Polling
23910 By default all changes are immediately returned within the JSON body:
23911 GET /somedatabase/_changes HTTP/1.1
23913 {"results":[
23915 {"seq":"1-g1AAAAF9eJzLYWBg4MhgTmHgz8tPSTV0MDQy1zMAQsMcoARTIkOS_P__7MSGXAqSVIAkkn2IFUZzIkMuUAee5pRqnGiuXkKA2dpXkpqWmZeagpu_Q4g_fGEbEkAqaqH2sIItsXAyMjM2NgUUwdOU_JYgCRDA5ACGjQfn30QlQsgKvcTVnkAovI-YZUPICpBvs0CAN1eY_c","id":"fresh","changes":[{"rev":"1-967a00dff5e02add41819138abb3284d"}]},
23916 {"seq":"3-g1AAAAG3eJzLYWBg4MhgTmHgz8tPSTV0MDQy1zMAQsMcoARTIkOS_P___7MSGXAqSVIAkkn2IFUZzIkMuUAee5pRqnGiuXkKA2dpXkpqWmZeagpu_Q4g_fGEbEkAqaqH2sIItsXAyMjM2NgUUwdOU_JYgCRDA5ACGjQfn30QlQsgKvcjfGaQZmaUmmZClM8gZhyAmHGfsG0PICrBPmQC22ZqbGRqamyIqSsLAAArcXo","id":"updated","changes":[{"rev":"2-7051cbe5c8faecd085a3fa619e6e6337CFCmkyTdt6bfJMDwDQNFcztWWkcY8JXyB2cu49AgFwURZGloRid3MMkEUoJHbXbOxVy6arc_SxQWQzRVHCuYHaxSpuj1aqbj0t-3-AlSrZakn78oeSvjRSIkIhSNiCFHbsKN3c50b02mURvEB-yD296eNOzzoRMRLRZ98rkHS_veGcC_nR-fGe1gaCaxihhjOI2lX0BhniHaA","id":"deleted","changes":[{"rev":"2-eec205a9d413992850a6e32678485900"}],"deleted":true}
23917 ],
23918 "last_seq":"5-g1AAAAIreJyVkEsKwjAURZ-toI5cgq5A0sQ0OrI70XyppcaRY92J7kR3ojupaSPUUgotgRd4yTlwbw4A0zRUMLdnpaMkwmyF3Ily9xBwEIuiKLI05KOTW0wkV4rruP29UyGWbordzwKVxWBNOGMKZhertDlarbr5pOT3DV4gudUC9-MPJX9tpEAYx4TQASns2E24ucuJ7rXJSL1BbEgf3vTwpmedCZkYa7Pulck7Xt7x_usFU2aIHOD4eEfVTVA5KMGUkqhNZV-8_o5i",
23919 "pending": 0}
23920 results is the list of changes in sequential order. New and changed
23922 documents only differ in the value of the rev; deleted documents
23923 include the "deleted": true attribute. (In the style=all_docs mode,
23924 deleted applies only to the current/winning revision. The other revi‐
23925 sions listed might be deleted even if there is no deleted property; you
23926 have to GET them individually to make sure.)
23927 last_seq is the update sequence of the last update returned (Equivalent
23929 to the last item in the results).
23930 Sending a since param in the query string skips all changes up to and
23932 including the given update sequence:
23933 GET /somedatabase/_changes?since=4-g1AAAAHXeJzLYWBg4MhgTmHgz8tPSTV0MDQy1zMAQsMcoARTIkOS_P___7MymBMZc4EC7MmJKSmJqWaYynEakaQAJJPsoaYwgE1JM0o1TjQ3T2HgLM1LSU3LzEtNwa3fAaQ_HqQ_kQG3qgSQqnoUtxoYGZkZG5uS4NY8FiDJ0ACkgAbNx2cfROUCiMr9CJ8ZpJkZpaaZEOUziBkHIGbcJ2zbA4hKsA-ZwLaZGhuZmhobYurKAgCz33kh HTTP/1.1
23935 The return structure for normal and longpoll modes is a JSON array of
23937 changes objects, and the last update sequence.
23938 In the return format for continuous mode, the server sends a CRLF (car‐
23940 riage-return, linefeed) delimited line for each change. Each line con‐
23941 tains the JSON object described above.
23942 You can also request the full contents of each document change (instead
23944 of just the change notification) by using the include_docs parameter.
23945 {
23947 "last_seq": "5-g1AAAAIreJyVkEsKwjAURZ-toI5cgq5A0sQ0OrI70XyppcaRY92J7kR3ojupaSPUUgotgRd4yTlwbw4A0zRUMLdnpaMkwmyF3Ily9xBwEIuiKLI05KOTW0wkV4rruP29UyGWbordzwKVxWBNOGMKZhertDlarbr5pOT3DV4gudUC9-MPJX9tpEAYx4TQASns2E24ucuJ7rXJSL1BbEgf3vTwpmedCZkYa7Pulck7Xt7x_usFU2aIHOD4eEfVTVA5KMGUkqhNZV-8_o5i",
23948 "pending": 0,
23949 "results": [
23950 {
23951 "changes": [
23952 {
23953 "rev": "2-eec205a9d413992850a6e32678485900"
23954 }
23955 ],
23956 "deleted": true,
23957 "id": "deleted",
23958 "seq": "5-g1AAAAIReJyVkE0OgjAQRkcwUVceQU9g-mOpruQm2tI2SLCuXOtN9CZ6E70JFmpCCCFCmkyTdt6bfJMDwDQNFcztWWkcY8JXyB2cu49AgFwURZGloRid3MMkEUoJHbXbOxVy6arc_SxQWQzRVHCuYHaxSpuj1aqbj0t-3-AlSrZakn78oeSvjRSIkIhSNiCFHbsKN3c50b02mURvEByD296eNOzzoRMRLRZ98rkHS_veGcC_nR-fGe1gaCaxihhjOI2lX0BhniHaA",
23959 }
23960 ]
23961 }
23962 Long Polling
23964 The longpoll feed, probably most applicable for a browser, is a more
23965 efficient form of polling that waits for a change to occur before the
23966 response is sent. longpoll avoids the need to frequently poll CouchDB
23967 to discover nothing has changed!
23968 The request to the server will remain open until a change is made on
23970 the database and is subsequently transferred, and then the connection
23971 will close. This is low load for both server and client.
23972 The response is basically the same JSON as is sent for the normal feed.
23974 Because the wait for a change can be significant you can set a timeout
23976 before the connection is automatically closed (the timeout argument).
23977 You can also set a heartbeat interval (using the heartbeat query argu‐
23978 ment), which sends a newline to keep the connection active.
23979 Continuous
23981 Continually polling the CouchDB server is not ideal - setting up new
23982 HTTP connections just to tell the client that nothing happened puts
23983 unnecessary strain on CouchDB.
23984 A continuous feed stays open and connected to the database until
23986 explicitly closed and changes are sent to the client as they happen,
23987 i.e. in near real-time.
23988 As with the longpoll feed type you can set both the timeout and heart‐
23990 beat intervals to ensure that the connection is kept open for new
23991 changes and updates.
23992 The continuous feed’s response is a little different than the other
23994 feed types to simplify the job of the client - each line of the
23995 response is either empty or a JSON object representing a single change,
23996 as found in the normal feed’s results.
23997 If limit has been specified the feed will end with a { last_seq }
23999 object.
24000 GET /somedatabase/_changes?feed=continuous HTTP/1.1
24002 {"seq":"1-g1AAAAF9eJzLYWBg4MhgTmHgz8tPSTV0MDQy1zMAQsMcoARTIkOS_P___7MSGXAqSVIAkkn2IFUZzIkMuUAee5pRqnGiuXkKA2dpXkpqWmZeagpu_Q4g_fGEbEkAqaqH2sIItsXAyMjM2NgUUwdOU_JYgCRDA5ACGjQfn30QlQsgKvcTVnkAovI-YZUPICpBvs0CAN1eY_c","id":"fresh","changes":[{"rev":"5-g1AAAAHxeJzLYWBg4MhgTmHgz8tPSTV0MDQy1zMAQsMcoARTIkOS_P___7MymBOZcoEC7MmJKSmJqWaYynEakaQAJJPsoaYwgE1JM0o1TjQ3T2HgLM1LSU3LzEtNwa3fAaQ_HkV_kkGyZWqSEXH6E0D666H6GcH6DYyMzIyNTUnwRR4LkGRoAFJAg-YjwiMtOdXCwJyU8ICYtABi0n6EnwzSzIxS00yI8hPEjAMQM-5nJTIQUPkAovI_UGUWAA0SgOI","id":"updated","changes":[{"rev":"2-7051cbe5c8faecd085a3fa619e6e6337"}]}
24004 {"seq":"3-g1AAAAHReJzLYWBg4MhgTmHgz8tPSTV0MDQy1zMAQsMcoARTIkOS_P___7MymBOZcoEC7MmJKSmJqWaYynEakaQAJJPsoaYwgE1JM0o1TjQ3T2HgLM1LSU3LzEtNwa3fAaQ_HkV_kkGyZWqSEXH6E0D660H6ExlwqspjAZIMDUAKqHA-yCZGiEuTUy0MzEnxL8SkBRCT9iPcbJBmZpSaZkKUmyFmHICYcZ-wux9AVIJ8mAUABgp6XQ","id":"deleted","changes":[{"rev":"2-eec205a9d413992850a6e32678485900"}],"deleted":true}
24005 ... tum tee tum ...
24006 {"seq":"6-g1AAAAIreJyVkEsKwjAURWMrqCOXoCuQ9MU0OrI70XyppcaRY92J7kR3ojupaVNopRQsgRd4yTlwb44QmqahQnN7VjpKImAr7E6Uu4eAI7EoiiJLQx6c3GIiuVJcx93vvQqxdFPsaguqLAY04YwpNLtYpc3RatXPJyW__-EFllst4D_-UPLXmh9VPAaICaEDUtixm-jmLie6N30YqTeYDenDmx7e9GwyYRODNuu_MnnHyzverV6AMkPkAMfHO1rdUAKUkqhLZV-_0o5j","id":"updated","changes":[{"rev":"3-825cb35de44c433bfb2df415563a19de"}]}
24007 Obviously, … tum tee tum … does not appear in the actual response, but
24009 represents a long pause before the change with seq 6 occurred.
24010 Event Source
24012 The eventsource feed provides push notifications that can be consumed
24013 in the form of DOM events in the browser. Refer to the W3C eventsource
24014 specification for further details. CouchDB also honours the
24015 Last-Event-ID parameter.
24016 GET /somedatabase/_changes?feed=eventsource HTTP/1.1
24018 // define the event handling function
24020 if (window.EventSource) {
24021 var source = new EventSource("/somedatabase/_changes?feed=eventsource");
24023 source.onerror = function(e) {
24024 alert('EventSource failed.');
24025 };
24026 var results = [];
24028 var sourceListener = function(e) {
24029 var data = JSON.parse(e.data);
24030 results.push(data);
24031 };
24032 // start listening for events
24034 source.addEventListener('message', sourceListener, false);
24035 // stop listening for events
24037 source.removeEventListener('message', sourceListener, false);
24038 }
24040 If you set a heartbeat interval (using the heartbeat query argument),
24042 CouchDB will send a hearbeat event that you can subscribe to with:
24043 source.addEventListener('heartbeat', function () {}, false);
24045 This can be monitored by the client application to restart the
24047 EventSource connection if needed (i.e. if the TCP connection gets stuck
24048 in a half-open state).
24049 NOTE:
24051 EventSource connections are subject to cross-origin resource sharing
24052 restrictions. You might need to configure CORS support to get the
24053 EventSource to work in your application.
24054 Filtering
24056 You can filter the contents of the changes feed in a number of ways.
24057 The most basic way is to specify one or more document IDs to the query.
24058 This causes the returned structure value to only contain changes for
24059 the specified IDs. Note that the value of this query argument should be
24060 a JSON formatted array.
24061 You can also filter the _changes feed by defining a filter function
24063 within a design document. The specification for the filter is the same
24064 as for replication filters. You specify the name of the filter function
24065 to the filter parameter, specifying the design document name and filter
24066 name. For example:
24067 GET /db/_changes?filter=design_doc/filtername HTTP/1.1
24069 Additionally, a couple of built-in filters are available and described
24071 below.
24072 _doc_ids
24074 This filter accepts only changes for documents which ID in specified in
24075 doc_ids query parameter or payload’s object array. See POST
24076 /{db}/_changes for an example.
24077 _selector
24079 New in version 2.0.
24080 This filter accepts only changes for documents which match a specified
24083 selector, defined using the same selector syntax used for _find.
24084 This is significantly more efficient than using a JavaScript filter
24086 function and is the recommended option if filtering on document
24087 attributes only.
24088 Note that, unlike JavaScript filters, selectors do not have access to
24090 the request object.
24091 Request:
24093 POST /recipes/_changes?filter=_selector HTTP/1.1
24095 Content-Type: application/json
24096 Host: localhost:5984
24097 {
24099 "selector": { "_id": { "$regex": "^_design/" } }
24100 }
24101 Response:
24103 HTTP/1.1 200 OK
24105 Cache-Control: must-revalidate
24106 Content-Type: application/json
24107 Date: Tue, 06 Sep 2016 20:03:23 GMT
24108 Etag: "1H8RGBCK3ABY6ACDM7ZSC30QK"
24109 Server: CouchDB (Erlang OTP/18)
24110 Transfer-Encoding: chunked
24111 {
24113 "last_seq": "11-g1AAAAIreJyVkEEKwjAQRUOrqCuPoCeQZGIaXdmbaNIk1FLjyrXeRG-iN9Gb1LQRaimFlsAEJnkP_s8RQtM0VGhuz0qTmABfYXdI7h4CgeSiKIosDUVwcotJIpQSOmp_71TIpZty97OgymJAU8G5QrOLVdocrVbdfFzy-wYvcbLVEvrxh5K_NlJggIhSNiCFHbmJbu5yonttMoneYD6kD296eNOzzoRNBNqse2Xyjpd3vP96AcYNTQY4Pt5RdTOuHIwCY5S0qewLwY6OaA",
24114 "pending": 0,
24115 "results": [
24116 {
24117 "changes": [
24118 {
24119 "rev": "10-304cae84fd862832ea9814f02920d4b2"
24120 }
24121 ],
24122 "id": "_design/ingredients",
24123 "seq": "8-g1AAAAHxeJzLYWBg4MhgTmHgz8tPSTV0MDQy1zMAQsMcoARTIkOS_P___7MymBOZcoEC7MmJKSmJqWaYynEakaQAJJPsoaYwgE1JM0o1TjQ3T2HgLM1LSU3LzEtNwa3fAaQ_HkV_kkGyZWqSEXH6E0D666H6GcH6DYyMzIyNTUnwRR4LkGRoAFJAg-ZnJTIQULkAonI_ws0GaWZGqWkmRLkZYsYBiBn3Cdv2AKIS7ENWsG2mxkampsaGmLqyAOYpgEo"
24124 },
24125 {
24126 "changes": [
24127 {
24128 "rev": "123-6f7c1b7c97a9e4f0d22bdf130e8fd817"
24129 }
24130 ],
24131 "deleted": true,
24132 "id": "_design/cookbook",
24133 "seq": "9-g1AAAAHxeJzLYWBg4MhgTmHgz8tPSTV0MDQy1zMAQsMcoARTIkOS_P___7MymBOZcoEC7MmJKSmJqWaYynEakaQAJJPsoaYwgE1JM0o1TjQ3T2HgLM1LSU3LzEtNwa3fAaQ_HkV_kkGyZWqSEXH6E0D661F8YWBkZGZsbEqCL_JYgCRDA5ACGjQ_K5GBgMoFEJX7EW42SDMzSk0zIcrNEDMOQMy4T9i2BxCVYB-ygm0zNTYyNTU2xNSVBQDnK4BL"
24134 },
24135 {
24136 "changes": [
24137 {
24138 "rev": "6-5b8a52c22580e922e792047cff3618f3"
24139 }
24140 ],
24141 "deleted": true,
24142 "id": "_design/meta",
24143 "seq": "11-g1AAAAIReJyVkE0OgjAQRiegUVceQU9g-mOpruQm2tI2SLCuXOtN9CZ6E70JFmpCCCFCmkyTdt6bfJMDwDQNFcztWWkcY8JXyB2cu49AgFwURZGloQhO7mGSCKWEjtrtnQq5dFXufhaoLIZoKjhXMLtYpc3RatXNxyW_b_ASJVstST_-UPLXRgpESEQpG5DCjlyFm7uc6F6bTKI3iA_Zhzc9vOlZZ0ImItqse2Xyjpd3vDMBfzo_vrPawLiaxihhjOI2lX0BirqHbg"
24144 }
24145 ]
24146 }
24147 Missing selector
24149 If the selector object is missing from the request body, the error mes‐
24150 sage is similar to the following example:
24151 {
24153 "error": "bad request",
24154 "reason": "Selector must be specified in POST payload"
24155 }
24156 Not a valid JSON object
24158 If the selector object is not a well-formed JSON object, the error mes‐
24159 sage is similar to the following example:
24160 {
24162 "error": "bad request",
24163 "reason": "Selector error: expected a JSON object"
24164 }
24165 Not a valid selector
24167 If the selector object does not contain a valid selection expression,
24168 the error message is similar to the following example:
24169 {
24171 "error": "bad request",
24172 "reason": "Selector error: expected a JSON object"
24173 }
24174 _design
24176 The _design filter accepts only changes for any design document within
24177 the requested database.
24178 Request:
24180 GET /recipes/_changes?filter=_design HTTP/1.1
24182 Accept: application/json
24183 Host: localhost:5984
24184 Response:
24186 HTTP/1.1 200 OK
24188 Cache-Control: must-revalidate
24189 Content-Type: application/json
24190 Date: Tue, 06 Sep 2016 12:55:12 GMT
24191 ETag: "ARIHFWL3I7PIS0SPVTFU6TLR2"
24192 Server: CouchDB (Erlang OTP)
24193 Transfer-Encoding: chunked
24194 {
24196 "last_seq": "11-g1AAAAIreJyVkEEKwjAQRUOrqCuPoCeQZGIaXdmbaNIk1FLjyrXeRG-iN9Gb1LQRaimFlsAEJnkP_s8RQtM0VGhuz0qTmABfYXdI7h4CgeSiKIosDUVwcotJIpQSOmp_71TIpZty97OgymJAU8G5QrOLVdocrVbdfFzy-wYvcbLVEvrxh5K_NlJggIhSNiCFHbmJbu5yonttMoneYD6kD296eNOzzoRNBNqse2Xyjpd3vP96AcYNTQY4Pt5RdTOuHIwCY5S0qewLwY6OaA",
24197 "pending": 0,
24198 "results": [
24199 {
24200 "changes": [
24201 {
24202 "rev": "10-304cae84fd862832ea9814f02920d4b2"
24203 }
24204 ],
24205 "id": "_design/ingredients",
24206 "seq": "8-g1AAAAHxeJzLYWBg4MhgTmHgz8tPSTV0MDQy1zMAQsMcoARTIkOS_P___7MymBOZcoEC7MmJKSmJqWaYynEakaQAJJPsoaYwgE1JM0o1TjQ3T2HgLM1LSU3LzEtNwa3fAaQ_HkV_kkGyZWqSEXH6E0D666H6GcH6DYyMzIyNTUnwRR4LkGRoAFJAg-ZnJTIQULkAonI_ws0GaWZGqWkmRLkZYsYBiBn3Cdv2AKIS7ENWsG2mxkampsaGmLqyAOYpgEo"
24207 },
24208 {
24209 "changes": [
24210 {
24211 "rev": "123-6f7c1b7c97a9e4f0d22bdf130e8fd817"
24212 }
24213 ],
24214 "deleted": true,
24215 "id": "_design/cookbook",
24216 "seq": "9-g1AAAAHxeJzLYWBg4MhgTmHgz8tPSTV0MDQy1zMAQsMcoARTIkOS_P___7MymBOZcoEC7MmJKSmJqWaYynEakaQAJJPsoaYwgE1JM0o1TjQ3T2HgLM1LSU3LzEtNwa3fAaQ_HkV_kkGyZWqSEXH6E0D661F8YWBkZGZsbEqCL_JYgCRDA5ACGjQ_K5GBgMoFEJX7EW42SDMzSk0zIcrNEDMOQMy4T9i2BxCVYB-ygm0zNTYyNTU2xNSVBQDnK4BL"
24217 },
24218 {
24219 "changes": [
24220 {
24221 "rev": "6-5b8a52c22580e922e792047cff3618f3"
24222 }
24223 ],
24224 "deleted": true,
24225 "id": "_design/meta",
24226 "seq": "11-g1AAAAIReJyVkE0OgjAQRiegUVceQU9g-mOpruQm2tI2SLCuXOtN9CZ6E70JFmpCCCFCmkyTdt6bfJMDwDQNFcztWWkcY8JXyB2cu49AgFwURZGloQhO7mGSCKWEjtrtnQq5dFXufhaoLIZoKjhXMLtYpc3RatXNxyW_b_ASJVstST_-UPLXRgpESEQpG5DCjlyFm7uc6F6bTKI3iA_Zhzc9vOlZZ0ImItqse2Xyjpd3vDMBfzo_vrPawLiaxihhjOI2lX0BirqHbg"
24227 }
24228 ]
24229 }
24230 _view
24232 New in version 1.2.
24233 The special filter _view allows to use existing map function as the
24236 filter. If the map function emits anything for the processed document
24237 it counts as accepted and the changes event emits to the feed. For most
24238 use-practice cases filter functions are very similar to map ones, so
24239 this feature helps to reduce amount of duplicated code.
24240 WARNING:
24242 While map functions doesn’t process the design documents, using
24243 _view filter forces them to do this. You need to be sure, that they
24244 are ready to handle documents with alien structure without panic.
24245 NOTE:
24247 Using _view filter doesn’t queries the view index files, so you can‐
24248 not use common view query parameters to additionally filter the
24249 changes feed by index key. Also, CouchDB doesn’t returns the result
24250 instantly as it does for views - it really uses the specified map
24251 function as filter.
24252 Moreover, you cannot make such filters dynamic e.g. process the
24254 request query parameters or handle the userctx_object - the map
24255 function is only operates with the document.
24256 Request:
24258 GET /recipes/_changes?filter=_view&view=ingredients/by_recipe HTTP/1.1
24260 Accept: application/json
24261 Host: localhost:5984
24262 Response:
24264 HTTP/1.1 200 OK
24266 Cache-Control: must-revalidate
24267 Content-Type: application/json
24268 Date: Tue, 06 Sep 2016 12:57:56 GMT
24269 ETag: "ARIHFWL3I7PIS0SPVTFU6TLR2"
24270 Server: CouchDB (Erlang OTP)
24271 Transfer-Encoding: chunked
24272 {
24274 "last_seq": "11-g1AAAAIreJyVkEEKwjAQRUOrqCuPoCeQZGIaXdmbaNIk1FLjyrXeRG-iN9Gb1LQRaimFlsAEJnkP_s8RQtM0VGhuz0qTmABfYXdI7h4CgeSiKIosDUVwcotJIpQSOmp_71TIpZty97OgymJAU8G5QrOLVdocrVbdfFzy-wYvcbLVEvrxh5K_NlJggIhSNiCFHbmJbu5yonttMoneYD6kD296eNOzzoRNBNqse2Xyjpd3vP96AcYNTQY4Pt5RdTOuHIwCY5S0qewLwY6OaA",
24275 "results": [
24276 {
24277 "changes": [
24278 {
24279 "rev": "13-bcb9d6388b60fd1e960d9ec4e8e3f29e"
24280 }
24281 ],
24282 "id": "SpaghettiWithMeatballs",
24283 "seq": "11-g1AAAAIReJyVkE0OgjAQRiegUVceQU9g-mOpruQm2tI2SLCuXOtN9CZ6E70JFmpCCCFCmkyTdt6bfJMDwDQNFcztWWkcY8JXyB2cu49AgFwURZGloQhO7mGSCKWEjtrtnQq5dFXufhaoLIZoKjhXMLtYpc3RatXNxyW_b_ASJVstST_-UPLXRgpESEQpG5DCjlyFm7uc6F6bTKI3iA_Zhzc9vOlZZ0ImItqse2Xyjpd3vDMBfzo_vrPawLiaxihhjOI2lX0BirqHbg"
24284 }
24285 ]
24286 }
24287 /db/_compact
24289 POST /{db}/_compact
24290 Request compaction of the specified database. Compaction com‐
24291 presses the disk database file by performing the following oper‐
24292 ations:
24293 · Writes a new, optimised, version of the database file, remov‐
24295 ing any unused sections from the new version during write.
24296 Because a new file is temporarily created for this purpose,
24297 you may require up to twice the current storage space of the
24298 specified database in order for the compaction routine to com‐
24299 plete.
24300 · Removes the bodies of any non-leaf revisions of documents from
24302 the database.
24303 · Removes old revision history beyond the limit specified by the
24305 _revs_limit database parameter.
24306 Compaction can only be requested on an individual database; you
24308 cannot compact all the databases for a CouchDB instance. The
24309 compaction process runs as a background process.
24310 You can determine if the compaction process is operating on a
24312 database by obtaining the database meta information, the com‐
24313 pact_running value of the returned database structure will be
24314 set to true. See GET /{db}.
24315 You can also obtain a list of running processes to determine
24317 whether compaction is currently running. See
24318 api/server/active_tasks.
24319 Parameters
24321 · db – Database name
24323 Request Headers
24325 · Accept – .INDENT 2.0
24327 · application/json
24329 · text/plain
24331 · Content-Type – application/json
24334 Response Headers
24336 · Content-Type – .INDENT 2.0
24338 · application/json
24340 · text/plain; charset=utf-8
24342 Response JSON Object
24345 · ok (boolean) – Operation status
24347 Status Codes
24349 · 202 Accepted – Compaction request has been accepted
24351 · 400 Bad Request – Invalid database name
24353 · 401 Unauthorized – CouchDB Server Administrator privileges
24355 required
24356 · 415 Unsupported Media Type – Bad Content-Type value
24358
24360 POST /db/_compact HTTP/1.1
24362 Accept: application/json
24363 Content-Type: application/json
24364 Host: localhost:5984
24365 Response:
24367 HTTP/1.1 202 Accepted
24369 Cache-Control: must-revalidate
24370 Content-Length: 12
24371 Content-Type: application/json
24372 Date: Mon, 12 Aug 2013 09:27:43 GMT
24373 Server: CouchDB (Erlang/OTP)
24374 {
24376 "ok": true
24377 }
24378 /db/_compact/design-doc
24380 POST /{db}/_compact/{ddoc}
24381 Compacts the view indexes associated with the specified design
24382 document. It may be that compacting a large view can return
24383 more storage than compacting the actual db. Thus, you can use
24384 this in place of the full database compaction if you know a spe‐
24385 cific set of view indexes have been affected by a recent data‐
24386 base change.
24387 Parameters
24389 · db – Database name
24391 · ddoc – Design document name
24393 Request Headers
24395 · Accept – .INDENT 2.0
24397 · application/json
24399 · text/plain
24401 · Content-Type – application/json
24404 Response Headers
24406 · Content-Type – .INDENT 2.0
24408 · application/json
24410 · text/plain; charset=utf-8
24412 Response JSON Object
24415 · ok (boolean) – Operation status
24417 Status Codes
24419 · 202 Accepted – Compaction request has been accepted
24421 · 400 Bad Request – Invalid database name
24423 · 401 Unauthorized – CouchDB Server Administrator privileges
24425 required
24426 · 404 Not Found – Design document not found
24428 · 415 Unsupported Media Type – Bad Content-Type value
24430
24432 POST /db/_compact/posts HTTP/1.1
24434 Accept: application/json
24435 Content-Type: application/json
24436 Host: localhost:5984
24437 Response:
24439 HTTP/1.1 202 Accepted
24441 Cache-Control: must-revalidate
24442 Content-Length: 12
24443 Content-Type: application/json
24444 Date: Mon, 12 Aug 2013 09:36:44 GMT
24445 Server: CouchDB (Erlang/OTP)
24446 {
24448 "ok": true
24449 }
24450 NOTE:
24452 View indexes are stored in a separate .couch file based on a
24453 hash of the design document’s relevant functions, in a sub
24454 directory of where the main .couch database files are
24455 located.
24456 /db/_ensure_full_commit
24458 POST /{db}/_ensure_full_commit
24459 Changed in version 3.0.0: Deprecated; endpoint is a no-op.
24460 Before 3.0 this was used to commit recent changes to the data‐
24463 base in case the delayed_commits=true option was set. That
24464 option is always false now, so commits are never delayed. How‐
24465 ever, this endpoint is kept for compatibility with older repli‐
24466 cators.
24467 Parameters
24469 · db – Database name
24471 Request Headers
24473 · Accept – .INDENT 2.0
24475 · application/json
24477 · text/plain
24479 · Content-Type – application/json
24482 Response Headers
24484 · Content-Type – .INDENT 2.0
24486 · application/json
24488 · text/plain; charset=utf-8
24490 Response JSON Object
24493 · instance_start_time (string) – Always "0". (Returned for
24495 legacy reasons.)
24496 · ok (boolean) – Operation status
24498 Status Codes
24500 · 201 Created – Commit completed successfully
24502 · 400 Bad Request – Invalid database name
24504 · 415 Unsupported Media Type – Bad Content-Type value
24506
24508 POST /db/_ensure_full_commit HTTP/1.1
24510 Accept: application/json
24511 Content-Type: application/json
24512 Host: localhost:5984
24513 Response:
24515 HTTP/1.1 201 Created
24517 Cache-Control: must-revalidate
24518 Content-Length: 53
24519 Content-Type: application/json
24520 Date: Mon, 12 Aug 2013 10:22:19 GMT
24521 Server: CouchDB (Erlang/OTP)
24522 {
24524 "instance_start_time": "0",
24525 "ok": true
24526 }
24527 /db/_view_cleanup
24529 POST /{db}/_view_cleanup
24530 Removes view index files that are no longer required by CouchDB
24531 as a result of changed views within design documents. As the
24532 view filename is based on a hash of the view functions, over
24533 time old views will remain, consuming storage. This call cleans
24534 up the cached view output on disk for a given view.
24535 Parameters
24537 · db – Database name
24539 Request Headers
24541 · Accept – .INDENT 2.0
24543 · application/json
24545 · text/plain
24547 · Content-Type – application/json
24550 Response Headers
24552 · Content-Type – .INDENT 2.0
24554 · application/json
24556 · text/plain; charset=utf-8
24558 Response JSON Object
24561 · ok (boolean) – Operation status
24563 Status Codes
24565 · 202 Accepted – Compaction request has been accepted
24567 · 400 Bad Request – Invalid database name
24569 · 401 Unauthorized – CouchDB Server Administrator privileges
24571 required
24572 · 415 Unsupported Media Type – Bad Content-Type value
24574
24576 POST /db/_view_cleanup HTTP/1.1
24578 Accept: application/json
24579 Content-Type: application/json
24580 Host: localhost:5984
24581 Response:
24583 HTTP/1.1 202 Accepted
24585 Cache-Control: must-revalidate
24586 Content-Length: 12
24587 Content-Type: application/json
24588 Date: Mon, 12 Aug 2013 09:27:43 GMT
24589 Server: CouchDB (Erlang/OTP)
24590 {
24592 "ok": true
24593 }
24594 /db/_security
24596 GET /{db}/_security
24597 Returns the current security object from the specified database.
24598 The security object consists of two compulsory elements, admins
24600 and members, which are used to specify the list of users and/or
24601 roles that have admin and members rights to the database respec‐
24602 tively:
24603 · members: they can read all types of documents from the DB, and
24605 they can write (and edit) documents to the DB except for
24606 design documents.
24607 · admins: they have all the privileges of members plus the priv‐
24609 ileges: write (and edit) design documents, add/remove database
24610 admins and members and set the database revisions limit. They
24611 can not create a database nor delete a database.
24612 Both members and admins objects contain two array-typed fields:
24614 · names: List of CouchDB user names
24616 · roles: List of users roles
24618 Any additional fields in the security object are optional. The
24620 entire security object is made available to validation and other
24621 internal functions so that the database can control and limit
24622 functionality.
24623 If both the names and roles fields of either the admins or mem‐
24625 bers properties are empty arrays, or are not existent, it means
24626 the database has no admins or members.
24627 Having no admins, only server admins (with the reserved _admin
24629 role) are able to update design document and make other admin
24630 level changes.
24631 Having no members, any user can write regular documents (any
24633 non-design document) and read documents from the database.
24634 If there are any member names or roles defined for a database,
24636 then only authenticated users having a matching name or role are
24637 allowed to read documents from the database (or do a GET /{db}
24638 call).
24639 NOTE:
24641 If the security object for a database has never been set,
24642 then the value returned will be empty.
24643 Also note, that security objects are not regular versioned
24645 documents (that is, they are not under MVCC rules). This is a
24646 design choice to speed up authorization checks (avoids
24647 traversing a database’s documents B-Tree).
24648 Parameters
24650 · db – Database name
24652 Request Headers
24654 · Accept – .INDENT 2.0
24656 · application/json
24658 · text/plain
24660 Response Headers
24663 · Content-Type – .INDENT 2.0
24665 · application/json
24667 · text/plain; charset=utf-8
24669 Response JSON Object
24672 · admins (object) – Object with two fields as names and roles.
24674 See description above for more info.
24675 · members (object) – Object with two fields as names and roles.
24677 See description above for more info.
24678 Status Codes
24680 · 200 OK – Request completed successfully
24682
24684 GET /db/_security HTTP/1.1
24686 Accept: application/json
24687 Host: localhost:5984
24688 Response:
24690 HTTP/1.1 200 OK
24692 Cache-Control: must-revalidate
24693 Content-Length: 109
24694 Content-Type: application/json
24695 Date: Mon, 12 Aug 2013 19:05:29 GMT
24696 Server: CouchDB (Erlang/OTP)
24697 {
24699 "admins": {
24700 "names": [
24701 "superuser"
24702 ],
24703 "roles": [
24704 "admins"
24705 ]
24706 },
24707 "members": {
24708 "names": [
24709 "user1",
24710 "user2"
24711 ],
24712 "roles": [
24713 "developers"
24714 ]
24715 }
24716 }
24717 PUT /{db}/_security
24719 Sets the security object for the given database.
24720 Parameters
24722 · db – Database name
24724 Request Headers
24726 · Accept – .INDENT 2.0
24728 · application/json
24730 · text/plain
24732 · Content-Type – application/json
24735 Request JSON Object
24737 · admins (object) – Object with two fields as names and roles.
24739 See description above for more info.
24740 · members (object) – Object with two fields as names and roles.
24742 See description above for more info.
24743 Response Headers
24745 · Content-Type – .INDENT 2.0
24747 · application/json
24749 · text/plain; charset=utf-8
24751 Response JSON Object
24754 · ok (boolean) – Operation status
24756 Status Codes
24758 · 200 OK – Request completed successfully
24760 · 401 Unauthorized – CouchDB Server Administrator privileges
24762 required
24763
24765 shell> curl http://localhost:5984/pineapple/_security -X PUT -H 'content-type: application/json' -H 'accept: application/json' -d '{"admins":{"names":["superuser"],"roles":["admins"]},"members":{"names": ["user1","user2"],"roles": ["developers"]}}'
24767 PUT /db/_security HTTP/1.1
24769 Accept: application/json
24770 Content-Length: 121
24771 Content-Type: application/json
24772 Host: localhost:5984
24773 {
24775 "admins": {
24776 "names": [
24777 "superuser"
24778 ],
24779 "roles": [
24780 "admins"
24781 ]
24782 },
24783 "members": {
24784 "names": [
24785 "user1",
24786 "user2"
24787 ],
24788 "roles": [
24789 "developers"
24790 ]
24791 }
24792 }
24793 Response:
24795 HTTP/1.1 200 OK
24797 Cache-Control: must-revalidate
24798 Content-Length: 12
24799 Content-Type: application/json
24800 Date: Tue, 13 Aug 2013 11:26:28 GMT
24801 Server: CouchDB (Erlang/OTP)
24802 {
24804 "ok": true
24805 }
24806 /db/_purge
24808 POST /{db}/_purge
24809 A database purge permanently removes the references to documents
24810 in the database. Normal deletion of a document within CouchDB
24811 does not remove the document from the database, instead, the
24812 document is marked as _deleted=true (and a new revision is cre‐
24813 ated). This is to ensure that deleted documents can be repli‐
24814 cated to other databases as having been deleted. This also means
24815 that you can check the status of a document and identify that
24816 the document has been deleted by its absence.
24817 The purge request must include the document IDs, and for each
24819 document ID, one or more revisions that must be purged. Docu‐
24820 ments can be previously deleted, but it is not necessary. Revi‐
24821 sions must be leaf revisions.
24822 The response will contain a list of the document IDs and revi‐
24824 sions successfully purged.
24825 Parameters
24827 · db – Database name
24829 Request Headers
24831 · Accept – .INDENT 2.0
24833 · application/json
24835 · text/plain
24837 · Content-Type – application/json
24840 Request JSON Object
24842 · object – Mapping of document ID to list of revisions to purge
24844 Response Headers
24846 · Content-Type – .INDENT 2.0
24848 · application/json
24850 · text/plain; charset=utf-8
24852 Response JSON Object
24855 · purge_seq (string) – Purge sequence string
24857 · purged (object) – Mapping of document ID to list of purged
24859 revisions
24860 Status Codes
24862 · 201 Created – Request completed successfully
24864 · 202 Accepted – Request was accepted, and was completed suc‐
24866 cessfully on at least one replica, but quorum was not reached.
24867 · 400 Bad Request – Invalid database name or JSON payload
24869 · 415 Unsupported Media Type – Bad Content-Type value
24871 · 500 Internal Server Error – Internal server error or timeout
24873
24875 POST /db/_purge HTTP/1.1
24877 Accept: application/json
24878 Content-Length: 76
24879 Content-Type: application/json
24880 Host: localhost:5984
24881 {
24883 "c6114c65e295552ab1019e2b046b10e": [
24884 "3-b06fcd1c1c9e0ec7c480ee8aa467bf3b",
24885 "3-c50a32451890a3f1c3e423334cc92745"
24886 ]
24887 }
24888 Response:
24890 HTTP/1.1 201 Created
24892 Cache-Control: must-revalidate
24893 Content-Length: 107
24894 Content-Type: application/json
24895 Date: Fri, 02 Jun 2017 18:55:54 GMT
24896 Server: CouchDB/2.0.0-2ccd4bf (Erlang OTP/18)
24897 {
24899 "purge_seq": null,
24900 "purged": {
24901 "c6114c65e295552ab1019e2b046b10e": [
24902 "3-c50a32451890a3f1c3e423334cc92745"
24903 ]
24904 }
24905 }
24906 [image: Document Revision Tree 1] [image] Document Revision Tree
24907 1.UNINDENT
24908 For example, given the above purge tree and issuing the above purge
24910 request, the whole document will be purged, as it contains only a
24911 single branch with a leaf revision
24912 3-c50a32451890a3f1c3e423334cc92745 that will be purged. As a result
24913 of this purge operation, a document with
24914 _id:c6114c65e295552ab1019e2b046b10e will be completely removed from
24915 the database’s document b+tree, and sequence b+tree. It will not be
24916 available through _all_docs or _changes endpoints, as though this
24917 document never existed. Also as a result of purge operation, the
24918 database’s purge_seq and update_seq will be increased.
24919 Notice, how revision 3-b06fcd1c1c9e0ec7c480ee8aa467bf3b was ignored.
24921 Revisions that have already been purged and non-leaf revisions are
24922 ignored in a purge request.
24923 If a document has two conflict revisions with the following revision
24925 history:
24926 [image: Document Revision Tree 1] [image] Document Revision Tree
24927 2.UNINDENT
24928 the above purge request will purge only one branch, leaving the docu‐
24930 ment’s revision tree with only a single branch:
24931 [image: Document Revision Tree 3] [image] Document Revision Tree
24932 3.UNINDENT
24933 As a result of this purge operation, a new updated version of the
24935 document will be available in _all_docs and _changes, creating a new
24936 record in _changes. The database’s purge_seq and update_seq will be
24937 increased.
24938 Internal Replication
24940 Purges are automatically replicated between replicas of the same data‐
24941 base. Each database has an internal purge tree that stores a certain
24942 number of the most recent purges. This allows internal synchonization
24943 between replicas of the same database.
24944 External Replication
24946 Purge operations are not replicated to other external databases. Exter‐
24947 nal replication works by identifying a source’s document revisions that
24948 are missing on target, and copying these revisions from source to tar‐
24949 get. A purge operation completely purges revisions from a document’s
24950 purge tree making external replication of purges impossible.
24951 NOTE:
24953 If you need a purge to be effective across multiple effective
24954 databases, you must run the purge separately on each of the
24955 databases.
24956 Updating Indexes
24958 The number of purges on a database is tracked using a purge sequence.
24959 This is used by the view indexer to optimize the updating of views that
24960 contain the purged documents.
24961 Each internal database indexer, including the view indexer, keeps its
24963 own purge sequence. The purge sequence stored in the index can be much
24964 smaller than the database’s purge sequence up to the number of purge
24965 requests allowed to be stored in the purge trees of the database. Mul‐
24966 tiple purge requests can be processed by the indexer without incurring
24967 a rebuild of the index. The index will be updated according to these
24968 purge requests.
24969 The index of documents is based on the winner of the revision tree.
24971 Depending on which revision is specified in the purge request, the
24972 index update observes the following behavior:
24973 · If the winner of the revision tree is not specified in the purge
24975 request, there is no change to the index record of this document.
24976 · If the winner of the revision tree is specified in the purge request,
24978 and there is still a revision left after purging, the index record of
24979 the document will be built according to the new winner of the revi‐
24980 sion tree.
24981 · If all revisions of the document are specified in the purge request,
24983 the index record of the document will be deleted. The document will
24984 no longer be found in searches.
24985 /db/_purged_infos_limit
24987 GET /{db}/_purged_infos_limit
24988 Gets the current purged_infos_limit (purged documents limit)
24989 setting, the maximum number of historical purges (purged docu‐
24990 ment Ids with their revisions) that can be stored in the data‐
24991 base.
24992 Parameters
24994 · db – Database name
24996 Request Headers
24998 · Accept – .INDENT 2.0
25000 · application/json
25002 · text/plain
25004 Response Headers
25007 · Content-Type – .INDENT 2.0
25009 · application/json
25011 · text/plain; charset=utf-8
25013 Status Codes
25016 · 200 OK – Request completed successfully
25018 Request:
25020 GET /db/_purged_infos_limit HTTP/1.1
25022 Accept: application/json
25023 Host: localhost:5984
25024 Response:
25026 HTTP/1.1 200 OK
25028 Cache-Control: must-revalidate
25029 Content-Length: 5
25030 Content-Type: application/json
25031 Date: Wed, 14 Jun 2017 14:43:42 GMT
25032 Server: CouchDB (Erlang/OTP)
25033 1000
25035 PUT /{db}/_purged_infos_limit
25037 Sets the maximum number of purges (requested purged Ids with
25038 their revisions) that will be tracked in the database, even
25039 after compaction has occurred. You can set the purged documents
25040 limit on a database with a scalar integer of the limit that you
25041 want to set as the request body.
25042 The default value of historical stored purges is 1000. This
25044 means up to 1000 purges can be synchronized between replicas of
25045 the same databases in case of one of the replicas was down when
25046 purges occurred.
25047 This request sets the soft limit for stored purges. During the
25049 compaction CouchDB will try to keep only _purged_infos_limit of
25050 purges in the database, but occasionally the number of stored
25051 purges can exceed this value. If a database has not completed
25052 purge synchronization with active indexes or active internal
25053 replications, it may temporarily store a higher number of his‐
25054 torical purges.
25055 Parameters
25057 · db – Database name
25059 Request Headers
25061 · Accept – .INDENT 2.0
25063 · application/json
25065 · text/plain
25067 · Content-Type – application/json
25070 Response Headers
25072 · Content-Type – .INDENT 2.0
25074 · application/json
25076 · text/plain; charset=utf-8
25078 Response JSON Object
25081 · ok (boolean) – Operation status
25083 Status Codes
25085 · 200 OK – Request completed successfully
25087 · 400 Bad Request – Invalid JSON data
25089 Request:
25091 PUT /db/_purged_infos_limit HTTP/1.1
25093 Accept: application/json
25094 Content-Length: 4
25095 Content-Type: application/json
25096 Host: localhost:5984
25097 1500
25099 Response:
25101 HTTP/1.1 200 OK
25103 Cache-Control: must-revalidate
25104 Content-Length: 12
25105 Content-Type: application/json
25106 Date: Wed, 14 Jun 2017 14:45:34 GMT
25107 Server: CouchDB (Erlang/OTP)
25108 {
25110 "ok": true
25111 }
25112 /db/_missing_revs
25114 POST /{db}/_missing_revs
25115 With given a list of document revisions, returns the document
25116 revisions that do not exist in the database.
25117 Parameters
25119 · db – Database name
25121 Request Headers
25123 · Accept – .INDENT 2.0
25125 · application/json
25127 · text/plain
25129 · Content-Type – application/json
25132 Request JSON Object
25134 · object – Mapping of document ID to list of revisions to lookup
25136 Response Headers
25138 · Content-Type – .INDENT 2.0
25140 · application/json
25142 · text/plain; charset=utf-8
25144 Response JSON Object
25147 · missing_revs (object) – Mapping of document ID to list of
25149 missed revisions
25150 Status Codes
25152 · 200 OK – Request completed successfully
25154 · 400 Bad Request – Invalid database name or JSON payload
25156
25158 POST /db/_missing_revs HTTP/1.1
25160 Accept: application/json
25161 Content-Length: 76
25162 Content-Type: application/json
25163 Host: localhost:5984
25164 {
25166 "c6114c65e295552ab1019e2b046b10e": [
25167 "3-b06fcd1c1c9e0ec7c480ee8aa467bf3b",
25168 "3-0e871ef78849b0c206091f1a7af6ec41"
25169 ]
25170 }
25171 Response:
25173 HTTP/1.1 200 OK
25175 Cache-Control: must-revalidate
25176 Content-Length: 64
25177 Content-Type: application/json
25178 Date: Mon, 12 Aug 2013 10:53:24 GMT
25179 Server: CouchDB (Erlang/OTP)
25180 {
25182 "missing_revs":{
25183 "c6114c65e295552ab1019e2b046b10e": [
25184 "3-b06fcd1c1c9e0ec7c480ee8aa467bf3b"
25185 ]
25186 }
25187 }
25188 /db/_revs_diff
25190 POST /{db}/_revs_diff
25191 Given a set of document/revision IDs, returns the subset of
25192 those that do not correspond to revisions stored in the data‐
25193 base.
25194 Its primary use is by the replicator, as an important optimiza‐
25196 tion: after receiving a set of new revision IDs from the source
25197 database, the replicator sends this set to the destination data‐
25198 base’s _revs_diff to find out which of them already exist there.
25199 It can then avoid fetching and sending already-known document
25200 bodies.
25201 Both the request and response bodies are JSON objects whose keys
25203 are document IDs; but the values are structured differently:
25204 · In the request, a value is an array of revision IDs for that
25206 document.
25207 · In the response, a value is an object with a missing: key,
25209 whose value is a list of revision IDs for that document (the
25210 ones that are not stored in the database) and optionally a
25211 possible_ancestors key, whose value is an array of revision
25212 IDs that are known that might be ancestors of the missing
25213 revisions.
25214 Parameters
25216 · db – Database name
25218 Request Headers
25220 · Accept – .INDENT 2.0
25222 · application/json
25224 · text/plain
25226 · Content-Type – application/json
25229 Request JSON Object
25231 · object – Mapping of document ID to list of revisions to lookup
25233 Response Headers
25235 · Content-Type – .INDENT 2.0
25237 · application/json
25239 · text/plain; charset=utf-8
25241 Response JSON Object
25244 · missing (array) – List of missed revisions for specified docu‐
25246 ment
25247 · possible_ancestors (array) – List of revisions that may be
25249 ancestors for specified document and its current revision in
25250 requested database
25251 Status Codes
25253 · 200 OK – Request completed successfully
25255 · 400 Bad Request – Invalid database name or JSON payload
25257
25259 POST /db/_revs_diff HTTP/1.1
25261 Accept: application/json
25262 Content-Length: 113
25263 Content-Type: application/json
25264 Host: localhost:5984
25265 {
25267 "190f721ca3411be7aa9477db5f948bbb": [
25268 "3-bb72a7682290f94a985f7afac8b27137",
25269 "4-10265e5a26d807a3cfa459cf1a82ef2e",
25270 "5-067a00dff5e02add41819138abb3284d"
25271 ]
25272 }
25273 Response:
25275 HTTP/1.1 200 OK
25277 Cache-Control: must-revalidate
25278 Content-Length: 88
25279 Content-Type: application/json
25280 Date: Mon, 12 Aug 2013 16:56:02 GMT
25281 Server: CouchDB (Erlang/OTP)
25282 {
25284 "190f721ca3411be7aa9477db5f948bbb": {
25285 "missing": [
25286 "3-bb72a7682290f94a985f7afac8b27137",
25287 "5-067a00dff5e02add41819138abb3284d"
25288 ],
25289 "possible_ancestors": [
25290 "4-10265e5a26d807a3cfa459cf1a82ef2e"
25291 ]
25292 }
25293 }
25294 /db/_revs_limit
25296 GET /{db}/_revs_limit
25297 Gets the current revs_limit (revision limit) setting.
25298 Parameters
25300 · db – Database name
25302 Request Headers
25304 · Accept – .INDENT 2.0
25306 · application/json
25308 · text/plain
25310 Response Headers
25313 · Content-Type – .INDENT 2.0
25315 · application/json
25317 · text/plain; charset=utf-8
25319 Status Codes
25322 · 200 OK – Request completed successfully
25324
25326 GET /db/_revs_limit HTTP/1.1
25328 Accept: application/json
25329 Host: localhost:5984
25330 Response:
25332 HTTP/1.1 200 OK
25334 Cache-Control: must-revalidate
25335 Content-Length: 5
25336 Content-Type: application/json
25337 Date: Mon, 12 Aug 2013 17:27:30 GMT
25338 Server: CouchDB (Erlang/OTP)
25339 1000
25341 PUT /{db}/_revs_limit
25343 Sets the maximum number of document revisions that will be
25344 tracked by CouchDB, even after compaction has occurred. You can
25345 set the revision limit on a database with a scalar integer of
25346 the limit that you want to set as the request body.
25347 Parameters
25349 · db – Database name
25351 Request Headers
25353 · Accept – .INDENT 2.0
25355 · application/json
25357 · text/plain
25359 · Content-Type – application/json
25362 Response Headers
25364 · Content-Type – .INDENT 2.0
25366 · application/json
25368 · text/plain; charset=utf-8
25370 Response JSON Object
25373 · ok (boolean) – Operation status
25375 Status Codes
25377 · 200 OK – Request completed successfully
25379 · 400 Bad Request – Invalid JSON data
25381
25383 PUT /db/_revs_limit HTTP/1.1
25385 Accept: application/json
25386 Content-Length: 5
25387 Content-Type: application/json
25388 Host: localhost:5984
25389 1000
25391 Response:
25393 HTTP/1.1 200 OK
25395 Cache-Control: must-revalidate
25396 Content-Length: 12
25397 Content-Type: application/json
25398 Date: Mon, 12 Aug 2013 17:47:52 GMT
25399 Server: CouchDB (Erlang/OTP)
25400 {
25402 "ok": true
25403 }
25404 Documents
25406 Details on how to create, read, update and delete documents within a
25407 database.
25408 /db/doc
25410 HEAD /{db}/{docid}
25411 Returns the HTTP Headers containing a minimal amount of informa‐
25412 tion about the specified document. The method supports the same
25413 query arguments as the GET /{db}/{docid} method, but only the
25414 header information (including document size, and the revision as
25415 an ETag), is returned.
25416 The ETag header shows the current revision for the requested
25418 document, and the Content-Length specifies the length of the
25419 data, if the document were requested in full.
25420 Adding any of the query arguments (see GET /{db}/{docid}), then
25422 the resulting HTTP Headers will correspond to what would be
25423 returned.
25424 Parameters
25426 · db – Database name
25428 · docid – Document ID
25430 Request Headers
25432 · If-None-Match – Double quoted document’s revision token
25434 Response Headers
25436 · Content-Length – Document size
25438 · ETag – Double quoted document’s revision token
25440 Status Codes
25442 · 200 OK – Document exists
25444 · 304 Not Modified – Document wasn’t modified since spec‐
25446 ified revision
25447 · 401 Unauthorized – Read privilege required
25449 · 404 Not Found – Document not found
25451 Request:
25453 HEAD /db/SpaghettiWithMeatballs HTTP/1.1
25455 Accept: application/json
25456 Host: localhost:5984
25457 Response:
25459 HTTP/1.1 200 OK
25461 Cache-Control: must-revalidate
25462 Content-Length: 660
25463 Content-Type: application/json
25464 Date: Tue, 13 Aug 2013 21:35:37 GMT
25465 ETag: "12-151bb8678d45aaa949ec3698ef1c7e78"
25466 Server: CouchDB (Erlang/OTP)
25467 GET /{db}/{docid}
25469 Returns document by the specified docid from the specified db.
25470 Unless you request a specific revision, the latest revision of
25471 the document will always be returned.
25472 Parameters
25474 · db – Database name
25476 · docid – Document ID
25478 Request Headers
25480 · Accept – .INDENT 2.0
25482 · application/json
25484 · multipart/related
25486 · multipart/mixed
25488 · text/plain
25490 · If-None-Match – Double quoted document’s revision token
25493 Query Parameters
25495 · attachments (boolean) – Includes attachments bodies in
25497 response. Default is false
25498 · att_encoding_info (boolean) – Includes encoding information in
25500 attachment stubs if the particular attachment is compressed.
25501 Default is false.
25502 · atts_since (array) – Includes attachments only since specified
25504 revisions. Doesn’t includes attachments for specified revi‐
25505 sions. Optional
25506 · conflicts (boolean) – Includes information about conflicts in
25508 document. Default is false
25509 · deleted_conflicts (boolean) – Includes information about
25511 deleted conflicted revisions. Default is false
25512 · latest (boolean) – Forces retrieving latest “leaf” revision,
25514 no matter what rev was requested. Default is false
25515 · local_seq (boolean) – Includes last update sequence for the
25517 document. Default is false
25518 · meta (boolean) – Acts same as specifying all conflicts,
25520 deleted_conflicts and revs_info query parameters. Default is
25521 false
25522 · open_revs (array) – Retrieves documents of specified leaf
25524 revisions. Additionally, it accepts value as all to return
25525 all leaf revisions. Optional
25526 · rev (string) – Retrieves document of specified revision.
25528 Optional
25529 · revs (boolean) – Includes list of all known document revi‐
25531 sions. Default is false
25532 · revs_info (boolean) – Includes detailed information for all
25534 known document revisions. Default is false
25535 Response Headers
25537 · Content-Type – .INDENT 2.0
25539 · application/json
25541 · multipart/related
25543 · multipart/mixed
25545 · text/plain; charset=utf-8
25547 · ETag – Double quoted document’s revision token. Not available when
25550 retrieving conflicts-related information
25551 · Transfer-Encoding – chunked. Available if requested with query param‐
25553 eter open_revs
25554 Response JSON Object
25556 · _id (string) – Document ID
25558 · _rev (string) – Revision MVCC token
25560 · _deleted (boolean) – Deletion flag. Available if document was
25562 removed
25563 · _attachments (object) – Attachment’s stubs. Available if docu‐
25565 ment has any attachments
25566 · _conflicts (array) – List of conflicted revisions. Available
25568 if requested with conflicts=true query parameter
25569 · _deleted_conflicts (array) – List of deleted conflicted revi‐
25571 sions. Available if requested with deleted_conflicts=true
25572 query parameter
25573 · _local_seq (string) – Document’s update sequence in current
25575 database. Available if requested with local_seq=true query
25576 parameter
25577 · _revs_info (array) – List of objects with information about
25579 local revisions and their status. Available if requested with
25580 open_revs query parameter
25581 · _revisions (object) – List of local revision tokens without.
25583 Available if requested with revs=true query parameter
25584 Status Codes
25586 · 200 OK – Request completed successfully
25588 · 304 Not Modified – Document wasn’t modified since specified
25590 revision
25591 · 400 Bad Request – The format of the request or revision was
25593 invalid
25594 · 401 Unauthorized – Read privilege required
25596 · 404 Not Found – Document not found
25598
25600 GET /recipes/SpaghettiWithMeatballs HTTP/1.1
25602 Accept: application/json
25603 Host: localhost:5984
25604 Response:
25606 HTTP/1.1 200 OK
25608 Cache-Control: must-revalidate
25609 Content-Length: 660
25610 Content-Type: application/json
25611 Date: Tue, 13 Aug 2013 21:35:37 GMT
25612 ETag: "1-917fa2381192822767f010b95b45325b"
25613 Server: CouchDB (Erlang/OTP)
25614 {
25616 "_id": "SpaghettiWithMeatballs",
25617 "_rev": "1-917fa2381192822767f010b95b45325b",
25618 "description": "An Italian-American dish that usually consists of spaghetti, tomato sauce and meatballs.",
25619 "ingredients": [
25620 "spaghetti",
25621 "tomato sauce",
25622 "meatballs"
25623 ],
25624 "name": "Spaghetti with meatballs"
25625 }
25626 PUT /{db}/{docid}
25628 The PUT method creates a new named document, or creates a new
25629 revision of the existing document. Unlike the POST /{db}, you
25630 must specify the document ID in the request URL.
25631 When updating an existing document, the current document revi‐
25633 sion must be included in the document (i.e. the request body),
25634 as the rev query parameter, or in the If-Match request header.
25635 Parameters
25637 · db – Database name
25639 · docid – Document ID
25641 Request Headers
25643 · Accept – .INDENT 2.0
25645 · application/json
25647 · text/plain
25649 · Content-Type – .INDENT 2.0
25652 · application/json
25654 · multipart/related
25656 · If-Match – Document’s revision. Alternative to rev query parameter or
25659 document key. Optional
25660 Query Parameters
25662 · rev (string) – Document’s revision if updating an existing
25664 document. Alternative to If-Match header or document key.
25665 Optional
25666 · batch (string) – Stores document in batch mode. Possible val‐
25668 ues: ok. Optional
25669 · new_edits (boolean) – Prevents insertion of a conflicting doc‐
25671 ument. Possible values: true (default) and false. If false, a
25672 well-formed _rev must be included in the document.
25673 new_edits=false is used by the replicator to insert documents
25674 into the target database even if that leads to the creation of
25675 conflicts. Optional
25676 Response Headers
25678 · Content-Type – .INDENT 2.0
25680 · application/json
25682 · text/plain; charset=utf-8
25684 · multipart/related
25686 · ETag – Quoted document’s new revision
25689 · Location – Document URI
25691 Response JSON Object
25693 · id (string) – Document ID
25695 · ok (boolean) – Operation status
25697 · rev (string) – Revision MVCC token
25699 Status Codes
25701 · 201 Created – Document created and stored on disk
25703 · 202 Accepted – Document data accepted, but not yet stored on
25705 disk
25706 · 400 Bad Request – Invalid request body or parameters
25708 · 401 Unauthorized – Write privileges required
25710 · 404 Not Found – Specified database or document ID doesn’t
25712 exists
25713 · 409 Conflict – Document with the specified ID already exists
25715 or specified revision is not latest for target document
25716
25718 PUT /recipes/SpaghettiWithMeatballs HTTP/1.1
25720 Accept: application/json
25721 Content-Length: 196
25722 Content-Type: application/json
25723 Host: localhost:5984
25724 {
25726 "description": "An Italian-American dish that usually consists of spaghetti, tomato sauce and meatballs.",
25727 "ingredients": [
25728 "spaghetti",
25729 "tomato sauce",
25730 "meatballs"
25731 ],
25732 "name": "Spaghetti with meatballs"
25733 }
25734 Response:
25736 HTTP/1.1 201 Created
25738 Cache-Control: must-revalidate
25739 Content-Length: 85
25740 Content-Type: application/json
25741 Date: Wed, 14 Aug 2013 20:31:39 GMT
25742 ETag: "1-917fa2381192822767f010b95b45325b"
25743 Location: http://localhost:5984/recipes/SpaghettiWithMeatballs
25744 Server: CouchDB (Erlang/OTP)
25745 {
25747 "id": "SpaghettiWithMeatballs",
25748 "ok": true,
25749 "rev": "1-917fa2381192822767f010b95b45325b"
25750 }
25751 DELETE /{db}/{docid}
25753 Marks the specified document as deleted by adding a field
25754 _deleted with the value true. Documents with this field will not
25755 be returned within requests anymore, but stay in the database.
25756 You must supply the current (latest) revision, either by using
25757 the rev parameter or by using the If-Match header to specify the
25758 revision.
25759 NOTE:
25761 CouchDB doesn’t completely delete the specified document.
25762 Instead, it leaves a tombstone with very basic information
25763 about the document. The tombstone is required so that the
25764 delete action can be replicated across databases.
25765 SEE ALSO:
25767 Retrieving Deleted Documents
25768 Parameters
25770 · db – Database name
25772 · docid – Document ID
25774 Request Headers
25776 · Accept – .INDENT 2.0
25778 · application/json
25780 · text/plain
25782 · If-Match – Document’s revision. Alternative to rev query
25785 parameter
25786 Query Parameters
25788 · rev (string) – Actual document’s revision
25790 · batch (string) – Stores document in batch mode Possible val‐
25792 ues: ok. Optional
25793 Response Headers
25795 · Content-Type – .INDENT 2.0
25797 · application/json
25799 · text/plain; charset=utf-8
25801 · ETag – Double quoted document’s new revision
25804 Response JSON Object
25806 · id (string) – Document ID
25808 · ok (boolean) – Operation status
25810 · rev (string) – Revision MVCC token
25812 Status Codes
25814 · 200 OK – Document successfully removed
25816 · 202 Accepted – Request was accepted, but changes are not yet
25818 stored on disk
25819 · 400 Bad Request – Invalid request body or parameters
25821 · 401 Unauthorized – Write privileges required
25823 · 404 Not Found – Specified database or document ID doesn’t
25825 exists
25826 · 409 Conflict – Specified revision is not the latest for target
25828 document
25829
25831 DELETE /recipes/FishStew?rev=1-9c65296036141e575d32ba9c034dd3ee HTTP/1.1
25833 Accept: application/json
25834 Host: localhost:5984
25835 Alternatively, instead of rev query parameter you may use
25837 If-Match header:
25838 DELETE /recipes/FishStew HTTP/1.1
25840 Accept: application/json
25841 If-Match: 1-9c65296036141e575d32ba9c034dd3ee
25842 Host: localhost:5984
25843 Response:
25845 HTTP/1.1 200 OK
25847 Cache-Control: must-revalidate
25848 Content-Length: 71
25849 Content-Type: application/json
25850 Date: Wed, 14 Aug 2013 12:23:13 GMT
25851 ETag: "2-056f5f44046ecafc08a2bc2b9c229e20"
25852 Server: CouchDB (Erlang/OTP)
25853 {
25855 "id": "FishStew",
25856 "ok": true,
25857 "rev": "2-056f5f44046ecafc08a2bc2b9c229e20"
25858 }
25859 COPY /{db}/{docid}
25861 The COPY (which is non-standard HTTP) copies an existing docu‐
25862 ment to a new or existing document. Copying a document is only
25863 possible within the same database.
25864 The source document is specified on the request line, with the
25866 Destination header of the request specifying the target docu‐
25867 ment.
25868 Parameters
25870 · db – Database name
25872 · docid – Document ID
25874 Request Headers
25876 · Accept – .INDENT 2.0
25878 · application/json
25880 · text/plain
25882 · Destination – Destination document. Must contain the target
25885 document ID, and optionally the target document revision, if
25886 copying to an existing document. See Copying to an Existing
25887 Document.
25888 · If-Match – Source document’s revision. Alternative to rev
25890 query parameter
25891 Query Parameters
25893 · rev (string) – Revision to copy from. Optional
25895 · batch (string) – Stores document in batch mode Possible val‐
25897 ues: ok. Optional
25898 Response Headers
25900 · Content-Type – .INDENT 2.0
25902 · application/json
25904 · text/plain; charset=utf-8
25906 · ETag – Double quoted document’s new revision
25909 · Location – Document URI
25911 Response JSON Object
25913 · id (string) – Document document ID
25915 · ok (boolean) – Operation status
25917 · rev (string) – Revision MVCC token
25919 Status Codes
25921 · 201 Created – Document successfully created
25923 · 202 Accepted – Request was accepted, but changes are not yet
25925 stored on disk
25926 · 400 Bad Request – Invalid request body or parameters
25928 · 401 Unauthorized – Read or write privileges required
25930 · 404 Not Found – Specified database, document ID or revision
25932 doesn’t exists
25933 · 409 Conflict – Document with the specified ID already exists
25935 or specified revision is not latest for target document
25936
25938 COPY /recipes/SpaghettiWithMeatballs HTTP/1.1
25940 Accept: application/json
25941 Destination: SpaghettiWithMeatballs_Italian
25942 Host: localhost:5984
25943 Response:
25945 HTTP/1.1 201 Created
25947 Cache-Control: must-revalidate
25948 Content-Length: 93
25949 Content-Type: application/json
25950 Date: Wed, 14 Aug 2013 14:21:00 GMT
25951 ETag: "1-e86fdf912560c2321a5fcefc6264e6d9"
25952 Location: http://localhost:5984/recipes/SpaghettiWithMeatballs_Italian
25953 Server: CouchDB (Erlang/OTP)
25954 {
25956 "id": "SpaghettiWithMeatballs_Italian",
25957 "ok": true,
25958 "rev": "1-e86fdf912560c2321a5fcefc6264e6d9"
25959 }
25960 Attachments
25962 If the document includes attachments, then the returned structure will
25963 contain a summary of the attachments associated with the document, but
25964 not the attachment data itself.
25965 The JSON for the returned document will include the _attachments field,
25967 with one or more attachment definitions.
25968 The _attachments object keys are attachments names while values are
25970 information objects with next structure:
25971 · content_type (string): Attachment MIME type
25973 · data (string): Base64-encoded content. Available if attachment con‐
25975 tent is requested by using the following query parameters:
25976 · attachments=true when querying a document
25978 · attachments=true&include_docs=true when querying a changes feed
25980 or a view
25981 · atts_since.
25983 · digest (string): Content hash digest. It starts with prefix which
25985 announce hash type (md5-) and continues with Base64-encoded hash
25986 digest
25987 · encoded_length (number): Compressed attachment size in bytes. Avail‐
25989 able if content_type is in list of compressible types when the
25990 attachment was added and the following query parameters are speci‐
25991 fied:
25992 · att_encoding_info=true when querying a document
25994 · att_encoding_info=true&include_docs=true when querying a changes
25996 feed or a view
25997 · encoding (string): Compression codec. Available if content_type is in
25999 list of compressible types when the attachment was added and the fol‐
26000 lowing query parameters are specified:
26001 · att_encoding_info=true when querying a document
26003 · att_encoding_info=true&include_docs=true when querying a changes
26005 feed or a view
26006 · length (number): Real attachment size in bytes. Not available if
26008 attachment content requested
26009 · revpos (number): Revision number when attachment was added
26011 · stub (boolean): Has true value if object contains stub info and no
26013 content. Otherwise omitted in response
26014 Basic Attachments Info
26016 Request:
26017 GET /recipes/SpaghettiWithMeatballs HTTP/1.1
26019 Accept: application/json
26020 Host: localhost:5984
26021 Response:
26023 HTTP/1.1 200 OK
26025 Cache-Control: must-revalidate
26026 Content-Length: 660
26027 Content-Type: application/json
26028 Date: Tue, 13 Aug 2013 21:35:37 GMT
26029 ETag: "5-fd96acb3256302bf0dd2f32713161f2a"
26030 Server: CouchDB (Erlang/OTP)
26031 {
26033 "_attachments": {
26034 "grandma_recipe.txt": {
26035 "content_type": "text/plain",
26036 "digest": "md5-Ids41vtv725jyrN7iUvMcQ==",
26037 "length": 1872,
26038 "revpos": 4,
26039 "stub": true
26040 },
26041 "my_recipe.txt": {
26042 "content_type": "text/plain",
26043 "digest": "md5-198BPPNiT5fqlLxoYYbjBA==",
26044 "length": 85,
26045 "revpos": 5,
26046 "stub": true
26047 },
26048 "photo.jpg": {
26049 "content_type": "image/jpeg",
26050 "digest": "md5-7Pv4HW2822WY1r/3WDbPug==",
26051 "length": 165504,
26052 "revpos": 2,
26053 "stub": true
26054 }
26055 },
26056 "_id": "SpaghettiWithMeatballs",
26057 "_rev": "5-fd96acb3256302bf0dd2f32713161f2a",
26058 "description": "An Italian-American dish that usually consists of spaghetti, tomato sauce and meatballs.",
26059 "ingredients": [
26060 "spaghetti",
26061 "tomato sauce",
26062 "meatballs"
26063 ],
26064 "name": "Spaghetti with meatballs"
26065 }
26066 Retrieving Attachments Content
26068 It’s possible to retrieve document with all attached files content by
26069 using attachments=true query parameter:
26070 Request:
26072 GET /db/pixel?attachments=true HTTP/1.1
26074 Accept: application/json
26075 Host: localhost:5984
26076 Response:
26078 HTTP/1.1 200 OK
26080 Cache-Control: must-revalidate
26081 Content-Length: 553
26082 Content-Type: application/json
26083 Date: Wed, 14 Aug 2013 11:32:40 GMT
26084 ETag: "4-f1bcae4bf7bbb92310079e632abfe3f4"
26085 Server: CouchDB (Erlang/OTP)
26086 {
26088 "_attachments": {
26089 "pixel.gif": {
26090 "content_type": "image/gif",
26091 "data": "R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7",
26092 "digest": "md5-2JdGiI2i2VELZKnwMers1Q==",
26093 "revpos": 2
26094 },
26095 "pixel.png": {
26096 "content_type": "image/png",
26097 "data": "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABAQMAAAAl21bKAAAAAXNSR0IArs4c6QAAAANQTFRFAAAAp3o92gAAAAF0Uk5TAEDm2GYAAAABYktHRACIBR1IAAAACXBIWXMAAAsTAAALEwEAmpwYAAAAB3RJTUUH3QgOCx8VHgmcNwAAAApJREFUCNdjYAAAAAIAAeIhvDMAAAAASUVORK5CYII=",
26098 "digest": "md5-Dgf5zxgGuchWrve73evvGQ==",
26099 "revpos": 3
26100 }
26101 },
26102 "_id": "pixel",
26103 "_rev": "4-f1bcae4bf7bbb92310079e632abfe3f4"
26104 }
26105 Or retrieve attached files content since specific revision using
26107 atts_since query parameter:
26108 Request:
26110 GET /recipes/SpaghettiWithMeatballs?atts_since=[%224-874985bc28906155ba0e2e0538f67b05%22] HTTP/1.1
26112 Accept: application/json
26113 Host: localhost:5984
26114 Response:
26116 HTTP/1.1 200 OK
26118 Cache-Control: must-revalidate
26119 Content-Length: 760
26120 Content-Type: application/json
26121 Date: Tue, 13 Aug 2013 21:35:37 GMT
26122 ETag: "5-fd96acb3256302bf0dd2f32713161f2a"
26123 Server: CouchDB (Erlang/OTP)
26124 {
26126 "_attachments": {
26127 "grandma_recipe.txt": {
26128 "content_type": "text/plain",
26129 "digest": "md5-Ids41vtv725jyrN7iUvMcQ==",
26130 "length": 1872,
26131 "revpos": 4,
26132 "stub": true
26133 },
26134 "my_recipe.txt": {
26135 "content_type": "text/plain",
26136 "data": "MS4gQ29vayBzcGFnaGV0dGkKMi4gQ29vayBtZWV0YmFsbHMKMy4gTWl4IHRoZW0KNC4gQWRkIHRvbWF0byBzYXVjZQo1LiAuLi4KNi4gUFJPRklUIQ==",
26137 "digest": "md5-198BPPNiT5fqlLxoYYbjBA==",
26138 "revpos": 5
26139 },
26140 "photo.jpg": {
26141 "content_type": "image/jpeg",
26142 "digest": "md5-7Pv4HW2822WY1r/3WDbPug==",
26143 "length": 165504,
26144 "revpos": 2,
26145 "stub": true
26146 }
26147 },
26148 "_id": "SpaghettiWithMeatballs",
26149 "_rev": "5-fd96acb3256302bf0dd2f32713161f2a",
26150 "description": "An Italian-American dish that usually consists of spaghetti, tomato sauce and meatballs.",
26151 "ingredients": [
26152 "spaghetti",
26153 "tomato sauce",
26154 "meatballs"
26155 ],
26156 "name": "Spaghetti with meatballs"
26157 }
26158 Efficient Multiple Attachments Retrieving
26160 As noted above, retrieving document with attachments=true returns a
26161 large JSON object with all attachments included. When your document
26162 and files are smaller it’s ok, but if you have attached something big‐
26163 ger like media files (audio/video), parsing such response might be very
26164 expensive.
26165 To solve this problem, CouchDB allows to get documents in multi‐
26167 part/related format:
26168 Request:
26170 GET /recipes/secret?attachments=true HTTP/1.1
26172 Accept: multipart/related
26173 Host: localhost:5984
26174 Response:
26176 HTTP/1.1 200 OK
26178 Content-Length: 538
26179 Content-Type: multipart/related; boundary="e89b3e29388aef23453450d10e5aaed0"
26180 Date: Sat, 28 Sep 2013 08:08:22 GMT
26181 ETag: "2-c1c6c44c4bc3c9344b037c8690468605"
26182 Server: CouchDB (Erlang OTP)
26183 --e89b3e29388aef23453450d10e5aaed0
26185 Content-Type: application/json
26186 {"_id":"secret","_rev":"2-c1c6c44c4bc3c9344b037c8690468605","_attachments":{"recipe.txt":{"content_type":"text/plain","revpos":2,"digest":"md5-HV9aXJdEnu0xnMQYTKgOFA==","length":86,"follows":true}}}
26188 --e89b3e29388aef23453450d10e5aaed0
26189 Content-Disposition: attachment; filename="recipe.txt"
26190 Content-Type: text/plain
26191 Content-Length: 86
26192 1. Take R
26194 2. Take E
26195 3. Mix with L
26196 4. Add some A
26197 5. Serve with X
26198 --e89b3e29388aef23453450d10e5aaed0--
26200 In this response the document contains only attachments stub informa‐
26202 tion and quite short while all attachments goes as separate entities
26203 which reduces memory footprint and processing overhead (you’d noticed,
26204 that attachment content goes as raw data, not in base64 encoding,
26205 right?).
26206 Retrieving Attachments Encoding Info
26208 By using att_encoding_info=true query parameter you may retrieve infor‐
26209 mation about compressed attachments size and used codec.
26210 Request:
26212 GET /recipes/SpaghettiWithMeatballs?att_encoding_info=true HTTP/1.1
26214 Accept: application/json
26215 Host: localhost:5984
26216 Response:
26218 HTTP/1.1 200 OK
26220 Cache-Control: must-revalidate
26221 Content-Length: 736
26222 Content-Type: application/json
26223 Date: Tue, 13 Aug 2013 21:35:37 GMT
26224 ETag: "5-fd96acb3256302bf0dd2f32713161f2a"
26225 Server: CouchDB (Erlang/OTP)
26226 {
26228 "_attachments": {
26229 "grandma_recipe.txt": {
26230 "content_type": "text/plain",
26231 "digest": "md5-Ids41vtv725jyrN7iUvMcQ==",
26232 "encoded_length": 693,
26233 "encoding": "gzip",
26234 "length": 1872,
26235 "revpos": 4,
26236 "stub": true
26237 },
26238 "my_recipe.txt": {
26239 "content_type": "text/plain",
26240 "digest": "md5-198BPPNiT5fqlLxoYYbjBA==",
26241 "encoded_length": 100,
26242 "encoding": "gzip",
26243 "length": 85,
26244 "revpos": 5,
26245 "stub": true
26246 },
26247 "photo.jpg": {
26248 "content_type": "image/jpeg",
26249 "digest": "md5-7Pv4HW2822WY1r/3WDbPug==",
26250 "length": 165504,
26251 "revpos": 2,
26252 "stub": true
26253 }
26254 },
26255 "_id": "SpaghettiWithMeatballs",
26256 "_rev": "5-fd96acb3256302bf0dd2f32713161f2a",
26257 "description": "An Italian-American dish that usually consists of spaghetti, tomato sauce and meatballs.",
26258 "ingredients": [
26259 "spaghetti",
26260 "tomato sauce",
26261 "meatballs"
26262 ],
26263 "name": "Spaghetti with meatballs"
26264 }
26265 Creating Multiple Attachments
26267 To create a document with multiple attachments with single request you
26268 need just inline base64 encoded attachments data into the document
26269 body:
26270 {
26272 "_id":"multiple_attachments",
26273 "_attachments":
26274 {
26275 "foo.txt":
26276 {
26277 "content_type":"text\/plain",
26278 "data": "VGhpcyBpcyBhIGJhc2U2NCBlbmNvZGVkIHRleHQ="
26279 },
26280 "bar.txt":
26282 {
26283 "content_type":"text\/plain",
26284 "data": "VGhpcyBpcyBhIGJhc2U2NCBlbmNvZGVkIHRleHQ="
26285 }
26286 }
26287 }
26288 Alternatively, you can upload a document with attachments more effi‐
26290 ciently in multipart/related format. This avoids having to
26291 Base64-encode the attachments, saving CPU and bandwidth. To do this,
26292 set the Content-Type header of the PUT /{db}/{docid} request to multi‐
26293 part/related.
26294 The first MIME body is the document itself, which should have its own
26296 Content-Type of application/json". It also should include an _attach‐
26297 ments metadata object in which each attachment object has a key follows
26298 with value true.
26299 The subsequent MIME bodies are the attachments.
26301 Request:
26303 PUT /temp/somedoc HTTP/1.1
26305 Accept: application/json
26306 Content-Length: 372
26307 Content-Type: multipart/related;boundary="abc123"
26308 Host: localhost:5984
26309 User-Agent: HTTPie/0.6.0
26310 --abc123
26312 Content-Type: application/json
26313 {
26315 "body": "This is a body.",
26316 "_attachments": {
26317 "foo.txt": {
26318 "follows": true,
26319 "content_type": "text/plain",
26320 "length": 21
26321 },
26322 "bar.txt": {
26323 "follows": true,
26324 "content_type": "text/plain",
26325 "length": 20
26326 }
26327 }
26328 }
26329 --abc123
26331 this is 21 chars long
26333 --abc123
26334 this is 20 chars lon
26336 --abc123--
26337 Response:
26339 HTTP/1.1 201 Created
26341 Cache-Control: must-revalidate
26342 Content-Length: 72
26343 Content-Type: application/json
26344 Date: Sat, 28 Sep 2013 09:13:24 GMT
26345 ETag: "1-5575e26acdeb1df561bb5b70b26ba151"
26346 Location: http://localhost:5984/temp/somedoc
26347 Server: CouchDB (Erlang OTP)
26348 {
26350 "id": "somedoc",
26351 "ok": true,
26352 "rev": "1-5575e26acdeb1df561bb5b70b26ba151"
26353 }
26354 Getting a List of Revisions
26356 You can obtain a list of the revisions for a given document by adding
26357 the revs=true parameter to the request URL:
26358 Request:
26360 GET /recipes/SpaghettiWithMeatballs?revs=true HTTP/1.1
26362 Accept: application/json
26363 Host: localhost:5984
26364 Response:
26366 HTTP/1.1 200 OK
26368 Cache-Control: must-revalidate
26369 Content-Length: 584
26370 Content-Type: application/json
26371 Date: Wed, 14 Aug 2013 11:38:26 GMT
26372 ETag: "5-fd96acb3256302bf0dd2f32713161f2a"
26373 Server: CouchDB (Erlang/OTP)
26374 {
26376 "_id": "SpaghettiWithMeatballs",
26377 "_rev": "8-6f5ad8db0f34af24a6e0984cd1a6cfb9",
26378 "_revisions": {
26379 "ids": [
26380 "6f5ad8db0f34af24a6e0984cd1a6cfb9",
26381 "77fba3a059497f51ec99b9b478b569d2",
26382 "136813b440a00a24834f5cb1ddf5b1f1",
26383 "fd96acb3256302bf0dd2f32713161f2a",
26384 "874985bc28906155ba0e2e0538f67b05",
26385 "0de77a37463bf391d14283e626831f2e",
26386 "d795d1b924777732fdea76538c558b62",
26387 "917fa2381192822767f010b95b45325b"
26388 ],
26389 "start": 8
26390 },
26391 "description": "An Italian-American dish that usually consists of spaghetti, tomato sauce and meatballs.",
26392 "ingredients": [
26393 "spaghetti",
26394 "tomato sauce",
26395 "meatballs"
26396 ],
26397 "name": "Spaghetti with meatballs"
26398 }
26399 The returned JSON structure includes the original document, including a
26401 _revisions structure that includes the revision information in next
26402 form:
26403 · ids (array): Array of valid revision IDs, in reverse order (latest
26405 first)
26406 · start (number): Prefix number for the latest revision
26408 Obtaining an Extended Revision History
26410 You can get additional information about the revisions for a given doc‐
26411 ument by supplying the revs_info argument to the query:
26412 Request:
26414 GET /recipes/SpaghettiWithMeatballs?revs_info=true HTTP/1.1
26416 Accept: application/json
26417 Host: localhost:5984
26418 Response:
26420 HTTP/1.1 200 OK
26422 Cache-Control: must-revalidate
26423 Content-Length: 802
26424 Content-Type: application/json
26425 Date: Wed, 14 Aug 2013 11:40:55 GMT
26426 Server: CouchDB (Erlang/OTP)
26427 {
26429 "_id": "SpaghettiWithMeatballs",
26430 "_rev": "8-6f5ad8db0f34af24a6e0984cd1a6cfb9",
26431 "_revs_info": [
26432 {
26433 "rev": "8-6f5ad8db0f34af24a6e0984cd1a6cfb9",
26434 "status": "available"
26435 },
26436 {
26437 "rev": "7-77fba3a059497f51ec99b9b478b569d2",
26438 "status": "deleted"
26439 },
26440 {
26441 "rev": "6-136813b440a00a24834f5cb1ddf5b1f1",
26442 "status": "available"
26443 },
26444 {
26445 "rev": "5-fd96acb3256302bf0dd2f32713161f2a",
26446 "status": "missing"
26447 },
26448 {
26449 "rev": "4-874985bc28906155ba0e2e0538f67b05",
26450 "status": "missing"
26451 },
26452 {
26453 "rev": "3-0de77a37463bf391d14283e626831f2e",
26454 "status": "missing"
26455 },
26456 {
26457 "rev": "2-d795d1b924777732fdea76538c558b62",
26458 "status": "missing"
26459 },
26460 {
26461 "rev": "1-917fa2381192822767f010b95b45325b",
26462 "status": "missing"
26463 }
26464 ],
26465 "description": "An Italian-American dish that usually consists of spaghetti, tomato sauce and meatballs.",
26466 "ingredients": [
26467 "spaghetti",
26468 "tomato sauce",
26469 "meatballs"
26470 ],
26471 "name": "Spaghetti with meatballs"
26472 }
26473 The returned document contains _revs_info field with extended revision
26475 information, including the availability and status of each revision.
26476 This array field contains objects with following structure:
26477 · rev (string): Full revision string
26479 · status (string): Status of the revision. Maybe one of:
26481 · available: Revision is available for retrieving with rev query
26483 parameter
26484 · missing: Revision is not available
26486 · deleted: Revision belongs to deleted document
26488 Obtaining a Specific Revision
26490 To get a specific revision, use the rev argument to the request, and
26491 specify the full revision number. The specified revision of the docu‐
26492 ment will be returned, including a _rev field specifying the revision
26493 that was requested.
26494 Request:
26496 GET /recipes/SpaghettiWithMeatballs?rev=6-136813b440a00a24834f5cb1ddf5b1f1 HTTP/1.1
26498 Accept: application/json
26499 Host: localhost:5984
26500 Response:
26502 HTTP/1.1 200 OK
26504 Cache-Control: must-revalidate
26505 Content-Length: 271
26506 Content-Type: application/json
26507 Date: Wed, 14 Aug 2013 11:40:55 GMT
26508 Server: CouchDB (Erlang/OTP)
26509 {
26511 "_id": "SpaghettiWithMeatballs",
26512 "_rev": "6-136813b440a00a24834f5cb1ddf5b1f1",
26513 "description": "An Italian-American dish that usually consists of spaghetti, tomato sauce and meatballs.",
26514 "ingredients": [
26515 "spaghetti",
26516 "tomato sauce",
26517 "meatballs"
26518 ],
26519 "name": "Spaghetti with meatballs"
26520 }
26521 Retrieving Deleted Documents
26523 CouchDB doesn’t actually delete documents via DELETE /{db}/{docid}.
26524 Instead, it leaves tombstone with very basic information about the doc‐
26525 ument. If you just GET /{db}/{docid} CouchDB returns 404 Not Found
26526 response:
26527 Request:
26529 GET /recipes/FishStew HTTP/1.1
26531 Accept: application/json
26532 Host: localhost:5984
26533 Response:
26535 HTTP/1.1 404 Object Not Found
26537 Cache-Control: must-revalidate
26538 Content-Length: 41
26539 Content-Type: application/json
26540 Date: Wed, 14 Aug 2013 12:23:27 GMT
26541 Server: CouchDB (Erlang/OTP)
26542 {
26544 "error": "not_found",
26545 "reason": "deleted"
26546 }
26547 However, you may retrieve document’s tombstone by using rev query
26549 parameter with GET /{db}/{docid} request:
26550 Request:
26552 GET /recipes/FishStew?rev=2-056f5f44046ecafc08a2bc2b9c229e20 HTTP/1.1
26554 Accept: application/json
26555 Host: localhost:5984
26556 Response:
26558 HTTP/1.1 200 OK
26560 Cache-Control: must-revalidate
26561 Content-Length: 79
26562 Content-Type: application/json
26563 Date: Wed, 14 Aug 2013 12:30:22 GMT
26564 ETag: "2-056f5f44046ecafc08a2bc2b9c229e20"
26565 Server: CouchDB (Erlang/OTP)
26566 {
26568 "_deleted": true,
26569 "_id": "FishStew",
26570 "_rev": "2-056f5f44046ecafc08a2bc2b9c229e20"
26571 }
26572 Updating an Existing Document
26574 To update an existing document you must specify the current revision
26575 number within the _rev parameter.
26576 Request:
26578 PUT /recipes/SpaghettiWithMeatballs HTTP/1.1
26580 Accept: application/json
26581 Content-Length: 258
26582 Content-Type: application/json
26583 Host: localhost:5984
26584 {
26586 "_rev": "1-917fa2381192822767f010b95b45325b",
26587 "description": "An Italian-American dish that usually consists of spaghetti, tomato sauce and meatballs.",
26588 "ingredients": [
26589 "spaghetti",
26590 "tomato sauce",
26591 "meatballs"
26592 ],
26593 "name": "Spaghetti with meatballs",
26594 "serving": "hot"
26595 }
26596 Alternatively, you can supply the current revision number in the
26598 If-Match HTTP header of the request:
26599 PUT /recipes/SpaghettiWithMeatballs HTTP/1.1
26601 Accept: application/json
26602 Content-Length: 258
26603 Content-Type: application/json
26604 If-Match: 1-917fa2381192822767f010b95b45325b
26605 Host: localhost:5984
26606 {
26608 "description": "An Italian-American dish that usually consists of spaghetti, tomato sauce and meatballs.",
26609 "ingredients": [
26610 "spaghetti",
26611 "tomato sauce",
26612 "meatballs"
26613 ],
26614 "name": "Spaghetti with meatballs",
26615 "serving": "hot"
26616 }
26617 Response:
26619 HTTP/1.1 201 Created
26621 Cache-Control: must-revalidate
26622 Content-Length: 85
26623 Content-Type: application/json
26624 Date: Wed, 14 Aug 2013 20:33:56 GMT
26625 ETag: "2-790895a73b63fb91dd863388398483dd"
26626 Location: http://localhost:5984/recipes/SpaghettiWithMeatballs
26627 Server: CouchDB (Erlang/OTP)
26628 {
26630 "id": "SpaghettiWithMeatballs",
26631 "ok": true,
26632 "rev": "2-790895a73b63fb91dd863388398483dd"
26633 }
26634 Copying from a Specific Revision
26636 To copy from a specific version, use the rev argument to the query
26637 string or If-Match:
26638 Request:
26640 COPY /recipes/SpaghettiWithMeatballs HTTP/1.1
26642 Accept: application/json
26643 Destination: SpaghettiWithMeatballs_Original
26644 If-Match: 1-917fa2381192822767f010b95b45325b
26645 Host: localhost:5984
26646 Response:
26648 HTTP/1.1 201 Created
26650 Cache-Control: must-revalidate
26651 Content-Length: 93
26652 Content-Type: application/json
26653 Date: Wed, 14 Aug 2013 14:21:00 GMT
26654 ETag: "1-917fa2381192822767f010b95b45325b"
26655 Location: http://localhost:5984/recipes/SpaghettiWithMeatballs_Original
26656 Server: CouchDB (Erlang/OTP)
26657 {
26659 "id": "SpaghettiWithMeatballs_Original",
26660 "ok": true,
26661 "rev": "1-917fa2381192822767f010b95b45325b"
26662 }
26663 Copying to an Existing Document
26665 To copy to an existing document, you must specify the current revision
26666 string for the target document by appending the rev parameter to the
26667 Destination header string.
26668 Request:
26670 COPY /recipes/SpaghettiWithMeatballs?rev=8-6f5ad8db0f34af24a6e0984cd1a6cfb9 HTTP/1.1
26672 Accept: application/json
26673 Destination: SpaghettiWithMeatballs_Original?rev=1-917fa2381192822767f010b95b45325b
26674 Host: localhost:5984
26675 Response:
26677 HTTP/1.1 201 Created
26679 Cache-Control: must-revalidate
26680 Content-Length: 93
26681 Content-Type: application/json
26682 Date: Wed, 14 Aug 2013 14:21:00 GMT
26683 ETag: "2-62e778c9ec09214dd685a981dcc24074""
26684 Location: http://localhost:5984/recipes/SpaghettiWithMeatballs_Original
26685 Server: CouchDB (Erlang/OTP)
26686 {
26688 "id": "SpaghettiWithMeatballs_Original",
26689 "ok": true,
26690 "rev": "2-62e778c9ec09214dd685a981dcc24074"
26691 }
26692 /db/doc/attachment
26694 HEAD /{db}/{docid}/{attname}
26695 Returns the HTTP headers containing a minimal amount of informa‐
26696 tion about the specified attachment. The method supports the
26697 same query arguments as the GET /{db}/{docid}/{attname} method,
26698 but only the header information (including attachment size,
26699 encoding and the MD5 hash as an ETag), is returned.
26700 Parameters
26702 · db – Database name
26704 · docid – Document ID
26706 · attname – Attachment name
26708 Request Headers
26710 · If-Match – Document’s revision. Alternative to rev
26712 query parameter
26713 · If-None-Match – Attachment’s base64 encoded MD5 binary
26715 digest. Optional
26716 Query Parameters
26718 · rev (string) – Document’s revision. Optional
26720 Response Headers
26722 · Accept-Ranges – Range request aware. Used for attach‐
26724 ments with application/octet-stream content type
26725 · Content-Encoding – Used compression codec. Available if
26727 attachment’s content_type is in list of compressible
26728 types
26729 · Content-Length – Attachment size. If compression codec
26731 was used, this value is about compressed size, not
26732 actual
26733 · ETag – Double quoted base64 encoded MD5 binary digest
26735 Status Codes
26737 · 200 OK – Attachment exists
26739 · 401 Unauthorized – Read privilege required
26741 · 404 Not Found – Specified database, document or attach‐
26743 ment was not found
26744 Request:
26746 HEAD /recipes/SpaghettiWithMeatballs/recipe.txt HTTP/1.1
26748 Host: localhost:5984
26749 Response:
26751 HTTP/1.1 200 OK
26753 Accept-Ranges: none
26754 Cache-Control: must-revalidate
26755 Content-Encoding: gzip
26756 Content-Length: 100
26757 Content-Type: text/plain
26758 Date: Thu, 15 Aug 2013 12:42:42 GMT
26759 ETag: "vVa/YgiE1+Gh0WfoFJAcSg=="
26760 Server: CouchDB (Erlang/OTP)
26761 GET /{db}/{docid}/{attname}
26763 Returns the file attachment associated with the document. The
26764 raw data of the associated attachment is returned (just as if
26765 you were accessing a static file. The returned Content-Type will
26766 be the same as the content type set when the document attachment
26767 was submitted into the database.
26768 Parameters
26770 · db – Database name
26772 · docid – Document ID
26774 · attname – Attachment name
26776 Request Headers
26778 · If-Match – Document’s revision. Alternative to rev
26780 query parameter
26781 · If-None-Match – Attachment’s base64 encoded MD5 binary
26783 digest. Optional
26784 Query Parameters
26786 · rev (string) – Document’s revision. Optional
26788 Response Headers
26790 · Accept-Ranges – Range request aware. Used for attach‐
26792 ments with application/octet-stream
26793 · Content-Encoding – Used compression codec. Available if
26795 attachment’s content_type is in list of compressible
26796 types
26797 · Content-Length – Attachment size. If compression codec
26799 is used, this value is about compressed size, not
26800 actual
26801 · ETag – Double quoted base64 encoded MD5 binary digest
26803 Response
26805 Stored content
26806 Status Codes
26808 · 200 OK – Attachment exists
26810 · 401 Unauthorized – Read privilege required
26812 · 404 Not Found – Specified database, document or attach‐
26814 ment was not found
26815 PUT /{db}/{docid}/{attname}
26817 Uploads the supplied content as an attachment to the specified
26818 document. The attachment name provided must be a URL encoded
26819 string. You must supply the Content-Type header, and for an
26820 existing document you must also supply either the rev query
26821 argument or the If-Match HTTP header. If the revision is omit‐
26822 ted, a new, otherwise empty document will be created with the
26823 provided attachment, or a conflict will occur.
26824 If case when uploading an attachment using an existing attach‐
26826 ment name, CouchDB will update the corresponding stored content
26827 of the database. Since you must supply the revision information
26828 to add an attachment to the document, this serves as validation
26829 to update the existing attachment.
26830 NOTE:
26832 Uploading an attachment updates the corresponding document
26833 revision. Revisions are tracked for the parent document, not
26834 individual attachments.
26835 Parameters
26837 · db – Database name
26839 · docid – Document ID
26841 · attname – Attachment name
26843 Request Headers
26845 · Content-Type – Attachment MIME type. Default: applica‐
26847 tion/octet-stream Optional
26848 · If-Match – Document revision. Alternative to rev query
26850 parameter
26851 Query Parameters
26853 · rev (string) – Document revision. Optional
26855 Response JSON Object
26857 · id (string) – Document ID
26859 · ok (boolean) – Operation status
26861 · rev (string) – Revision MVCC token
26863 Status Codes
26865 · 201 Created – Attachment created and stored on disk
26867 · 202 Accepted – Request was accepted, but changes are
26869 not yet stored on disk
26870 · 400 Bad Request – Invalid request body or parameters
26872 · 401 Unauthorized – Write privileges required
26874 · 404 Not Found – Specified database, document or attach‐
26876 ment was not found
26877 · 409 Conflict – Document’s revision wasn’t specified or
26879 it’s not the latest
26880 Request:
26882 PUT /recipes/SpaghettiWithMeatballs/recipe.txt HTTP/1.1
26884 Accept: application/json
26885 Content-Length: 86
26886 Content-Type: text/plain
26887 Host: localhost:5984
26888 If-Match: 1-917fa2381192822767f010b95b45325b
26889 1. Cook spaghetti
26891 2. Cook meatballs
26892 3. Mix them
26893 4. Add tomato sauce
26894 5. ...
26895 6. PROFIT!
26896 Response:
26898 HTTP/1.1 201 Created
26900 Cache-Control: must-revalidate
26901 Content-Length: 85
26902 Content-Type: application/json
26903 Date: Thu, 15 Aug 2013 12:38:04 GMT
26904 ETag: "2-ce91aed0129be8f9b0f650a2edcfd0a4"
26905 Location: http://localhost:5984/recipes/SpaghettiWithMeatballs/recipe.txt
26906 Server: CouchDB (Erlang/OTP)
26907 {
26909 "id": "SpaghettiWithMeatballs",
26910 "ok": true,
26911 "rev": "2-ce91aed0129be8f9b0f650a2edcfd0a4"
26912 }
26913 DELETE /{db}/{docid}/{attname}
26915 Deletes the attachment with filename {attname} of the specified
26916 doc. You must supply the rev query parameter or If-Match with
26917 the current revision to delete the attachment.
26918 NOTE:
26920 Deleting an attachment updates the corresponding document
26921 revision. Revisions are tracked for the parent document, not
26922 individual attachments.
26923 Parameters
26925 · db – Database name
26927 · docid – Document ID
26929 Request Headers
26931 · Accept – .INDENT 2.0
26933 · application/json
26935 · text/plain
26937 · If-Match – Document revision. Alternative to rev query parame‐
26940 ter
26941 Query Parameters
26943 · rev (string) – Document revision. Required
26945 · batch (string) – Store changes in batch mode Possible values:
26947 ok. Optional
26948 Response Headers
26950 · Content-Type – .INDENT 2.0
26952 · application/json
26954 · text/plain; charset=utf-8
26956 · ETag – Double quoted document’s new revision
26959 Response JSON Object
26961 · id (string) – Document ID
26963 · ok (boolean) – Operation status
26965 · rev (string) – Revision MVCC token
26967 Status Codes
26969 · 200 OK – Attachment successfully removed
26971 · 202 Accepted – Request was accepted, but changes are not yet
26973 stored on disk
26974 · 400 Bad Request – Invalid request body or parameters
26976 · 401 Unauthorized – Write privileges required
26978 · 404 Not Found – Specified database, document or attachment was
26980 not found
26981 · 409 Conflict – Document’s revision wasn’t specified or it’s
26983 not the latest
26984
26986 DELETE /recipes/SpaghettiWithMeatballs?rev=6-440b2dd39c20413045748b42c6aba6e2 HTTP/1.1
26988 Accept: application/json
26989 Host: localhost:5984
26990 Alternatively, instead of rev query parameter you may use
26992 If-Match header:
26993 DELETE /recipes/SpaghettiWithMeatballs HTTP/1.1
26995 Accept: application/json
26996 If-Match: 6-440b2dd39c20413045748b42c6aba6e2
26997 Host: localhost:5984
26998 Response:
27000 HTTP/1.1 200 OK
27002 Cache-Control: must-revalidate
27003 Content-Length: 85
27004 Content-Type: application/json
27005 Date: Wed, 14 Aug 2013 12:23:13 GMT
27006 ETag: "7-05185cf5fcdf4b6da360af939431d466"
27007 Server: CouchDB (Erlang/OTP)
27008 {
27010 "id": "SpaghettiWithMeatballs",
27011 "ok": true,
27012 "rev": "7-05185cf5fcdf4b6da360af939431d466"
27013 }
27014 HTTP Range Requests
27016 HTTP allows you to specify byte ranges for requests. This allows the
27017 implementation of resumable downloads and skippable audio and video
27018 streams alike. This is available for all attachments inside CouchDB.
27019 This is just a real quick run through how this looks under the hood.
27021 Usually, you will have larger binary files to serve from CouchDB, like
27022 MP3s and videos, but to make things a little more obvious, I use a text
27023 file here (Note that I use the application/octet-stream :header`Con‐
27024 tent-Type` instead of text/plain).
27025 shell> cat file.txt
27027 My hovercraft is full of eels!
27028 Now let’s store this text file as an attachment in CouchDB. First, we
27030 create a database:
27031 shell> curl -X PUT http://127.0.0.1:5984/test
27033 {"ok":true}
27034 Then we create a new document and the file attachment in one go:
27036 shell> curl -X PUT http://127.0.0.1:5984/test/doc/file.txt \
27038 -H "Content-Type: application/octet-stream" -d@file.txt
27039 {"ok":true,"id":"doc","rev":"1-287a28fa680ae0c7fb4729bf0c6e0cf2"}
27040 Now we can request the whole file easily:
27042 shell> curl -X GET http://127.0.0.1:5984/test/doc/file.txt
27044 My hovercraft is full of eels!
27045 But say we only want the first 13 bytes:
27047 shell> curl -X GET http://127.0.0.1:5984/test/doc/file.txt \
27049 -H "Range: bytes=0-12"
27050 My hovercraft
27051 HTTP supports many ways to specify single and even multiple byte
27053 ranges. Read all about it in RFC 2616#section-14.27.
27054 NOTE:
27056 Databases that have been created with CouchDB 1.0.2 or earlier will
27057 support range requests in 3.1, but they are using a less-optimal
27058 algorithm. If you plan to make heavy use of this feature, make sure
27059 to compact your database with CouchDB 3.1 to take advantage of a
27060 better algorithm to find byte ranges.
27061 Design Documents
27063 In CouchDB, design documents provide the main interface for building a
27064 CouchDB application. The design document defines the views used to
27065 extract information from CouchDB through one or more views. Design doc‐
27066 uments are created within your CouchDB instance in the same way as you
27067 create database documents, but the content and definition of the docu‐
27068 ments is different. Design Documents are named using an ID defined with
27069 the design document URL path, and this URL can then be used to access
27070 the database contents.
27071 Views and lists operate together to provide automated (and formatted)
27073 output from your database.
27074 /db/_design/design-doc
27076 HEAD /{db}/_design/{ddoc}
27077 Returns the HTTP Headers containing a minimal amount of informa‐
27078 tion about the specified design document.
27079 SEE ALSO:
27081 HEAD /{db}/{docid}
27082 GET /{db}/_design/{ddoc}
27084 Returns the contents of the design document specified with the
27085 name of the design document and from the specified database from
27086 the URL. Unless you request a specific revision, the latest
27087 revision of the document will always be returned.
27088 SEE ALSO:
27090 GET /{db}/{docid}
27091 PUT /{db}/_design/{ddoc}
27093 The PUT method creates a new named design document, or creates a
27094 new revision of the existing design document.
27095 The design documents have some agreement upon their fields and
27097 structure. Currently it is the following:
27098 · language (string): Defines Query Server to process design doc‐
27100 ument functions
27101 · options (object): View’s default options
27103 · filters (object): Filter functions definition
27105 · lists (object): List functions definition. Deprecated.
27107 · rewrites (array or string): Rewrite rules definition. Depre‐
27109 cated.
27110 · shows (object): Show functions definition. Deprecated.
27112 · updates (object): Update functions definition
27114 · validate_doc_update (string): Validate document update func‐
27116 tion source
27117 · views (object): View functions definition.
27119 · autoupdate (boolean): Indicates whether to automatically build
27121 indexes defined in this design document. Default is true.
27122 Note, that for filters, lists, shows and updates fields objects
27124 are mapping of function name to string function source code. For
27125 views mapping is the same except that values are objects with
27126 map and reduce (optional) keys which also contains functions
27127 source code.
27128 SEE ALSO:
27130 PUT /{db}/{docid}
27131 DELETE /{db}/_design/{ddoc}
27133 Deletes the specified document from the database. You must sup‐
27134 ply the current (latest) revision, either by using the rev
27135 parameter to specify the revision.
27136 SEE ALSO:
27138 DELETE /{db}/{docid}
27139 COPY /{db}/_design/{ddoc}
27141 The COPY (which is non-standard HTTP) copies an existing design
27142 document to a new or existing one.
27143 Given that view indexes on disk are named after their MD5 hash
27145 of the view definition, and that a COPY operation won’t actually
27146 change that definition, the copied views won’t have to be recon‐
27147 structed. Both views will be served from the same index on
27148 disk.
27149 SEE ALSO:
27151 COPY /{db}/{docid}
27152 /db/_design/design-doc/attachment
27154 HEAD /{db}/_design/{ddoc}/{attname}
27155 Returns the HTTP headers containing a minimal amount of informa‐
27156 tion about the specified attachment.
27157 SEE ALSO:
27159 HEAD /{db}/{docid}/{attname}
27160 GET /{db}/_design/{ddoc}/{attname}
27162 Returns the file attachment associated with the design document.
27163 The raw data of the associated attachment is returned (just as
27164 if you were accessing a static file.
27165 SEE ALSO:
27167 GET /{db}/{docid}/{attname}
27168 PUT /{db}/_design/{ddoc}/{attname}
27170 Uploads the supplied content as an attachment to the specified
27171 design document. The attachment name provided must be a URL
27172 encoded string.
27173 SEE ALSO:
27175 PUT /{db}/{docid}/{attname}
27176 DELETE /{db}/_design/{ddoc}/{attname}
27178 Deletes the attachment of the specified design document.
27179 SEE ALSO:
27181 DELETE /{db}/{docid}/{attname}
27182 /db/_design/design-doc/_info
27184 GET /{db}/_design/{ddoc}/_info
27185 Obtains information about the specified design document, includ‐
27186 ing the index, index size and current status of the design docu‐
27187 ment and associated index information.
27188 Parameters
27190 · db – Database name
27192 · ddoc – Design document name
27194 Request Headers
27196 · Accept – .INDENT 2.0
27198 · application/json
27200 · text/plain
27202 Response Headers
27205 · Content-Type – .INDENT 2.0
27207 · application/json
27209 · text/plain; charset=utf-8
27211 Response JSON Object
27214 · name (string) – Design document name
27216 · view_index (object) – View Index Information
27218 Status Codes
27220 · 200 OK – Request completed successfully
27222
27224 GET /recipes/_design/recipe/_info HTTP/1.1
27226 Accept: application/json
27227 Host: localhost:5984
27228 Response:
27230 HTTP/1.1 200 OK
27232 Cache-Control: must-revalidate
27233 Content-Length: 263
27234 Content-Type: application/json
27235 Date: Sat, 17 Aug 2013 12:54:17 GMT
27236 Server: CouchDB (Erlang/OTP)
27237 {
27239 "name": "recipe",
27240 "view_index": {
27241 "compact_running": false,
27242 "language": "python",
27243 "purge_seq": 0,
27244 "signature": "a59a1bb13fdf8a8a584bc477919c97ac",
27245 "sizes": {
27246 "active": 926691,
27247 "disk": 1982704,
27248 "external": 1535701
27249 },
27250 "update_seq": 12397,
27251 "updater_running": false,
27252 "waiting_clients": 0,
27253 "waiting_commit": false
27254 }
27255 }
27256 View Index Information
27258 The response from GET /{db}/_design/{ddoc}/_info contains view_index
27259 (object) field with the next structure:
27260 · compact_running (boolean): Indicates whether a compaction routine is
27262 currently running on the view
27263 · sizes.active (number): The size of live data inside the view, in
27265 bytes
27266 · sizes.external (number): The uncompressed size of view contents in
27268 bytes
27269 · sizes.file (number): Size in bytes of the view as stored on disk
27271 · language (string): Language for the defined views
27273 · purge_seq (number): The purge sequence that has been processed
27275 · signature (string): MD5 signature of the views for the design docu‐
27277 ment
27278 · update_seq (number / string): The update sequence of the correspond‐
27280 ing database that has been indexed
27281 · updater_running (boolean): Indicates if the view is currently being
27283 updated
27284 · waiting_clients (number): Number of clients waiting on views from
27286 this design document
27287 · waiting_commit (boolean): Indicates if there are outstanding commits
27289 to the underlying database that need to processed
27290 /db/_design/design-doc/_view/view-name
27292 GET /{db}/_design/{ddoc}/_view/{view}
27293 Executes the specified view function from the specified design
27294 document.
27295 Parameters
27297 · db – Database name
27299 · ddoc – Design document name
27301 · view – View function name
27303 Request Headers
27305 · Accept – .INDENT 2.0
27307 · application/json
27309 · text/plain
27311 Query Parameters
27314 · conflicts (boolean) – Include conflicts information in
27316 response. Ignored if include_docs isn’t true. Default is
27317 false.
27318 · descending (boolean) – Return the documents in descending
27320 order by key. Default is false.
27321 · endkey (json) – Stop returning records when the specified key
27323 is reached.
27324 · end_key (json) – Alias for endkey param
27326 · endkey_docid (string) – Stop returning records when the speci‐
27328 fied document ID is reached. Ignored if endkey is not set.
27329 · end_key_doc_id (string) – Alias for endkey_docid.
27331 · group (boolean) – Group the results using the reduce function
27333 to a group or single row. Implies reduce is true and the maxi‐
27334 mum group_level. Default is false.
27335 · group_level (number) – Specify the group level to be used.
27337 Implies group is true.
27338 · include_docs (boolean) – Include the associated document with
27340 each row. Default is false.
27341 · attachments (boolean) – Include the Base64-encoded content of
27343 attachments in the documents that are included if include_docs
27344 is true. Ignored if include_docs isn’t true. Default is false.
27345 · att_encoding_info (boolean) – Include encoding information in
27347 attachment stubs if include_docs is true and the particular
27348 attachment is compressed. Ignored if include_docs isn’t true.
27349 Default is false.
27350 · inclusive_end (boolean) – Specifies whether the specified end
27352 key should be included in the result. Default is true.
27353 · key (json) – Return only documents that match the specified
27355 key.
27356 · keys (json-array) – Return only documents where the key
27358 matches one of the keys specified in the array.
27359 · limit (number) – Limit the number of the returned documents to
27361 the specified number.
27362 · reduce (boolean) – Use the reduction function. Default is true
27364 when a reduce function is defined.
27365 · skip (number) – Skip this number of records before starting to
27367 return the results. Default is 0.
27368 · sorted (boolean) – Sort returned rows (see Sorting Returned
27370 Rows). Setting this to false offers a performance boost. The
27371 total_rows and offset fields are not available when this is
27372 set to false. Default is true.
27373 · stable (boolean) – Whether or not the view results should be
27375 returned from a stable set of shards. Default is false.
27376 · stale (string) – Allow the results from a stale view to be
27378 used. Supported values: ok and update_after. ok is equiva‐
27379 lent to stable=true&update=false. update_after is equivalent
27380 to stable=true&update=lazy. The default behavior is equiva‐
27381 lent to stable=false&update=true. Note that this parameter is
27382 deprecated. Use stable and update instead. See views/genera‐
27383 tion for more details.
27384 · startkey (json) – Return records starting with the specified
27386 key.
27387 · start_key (json) – Alias for startkey.
27389 · startkey_docid (string) – Return records starting with the
27391 specified document ID. Ignored if startkey is not set.
27392 · start_key_doc_id (string) – Alias for startkey_docid param
27394 · update (string) – Whether or not the view in question should
27396 be updated prior to responding to the user. Supported values:
27397 true, false, lazy. Default is true.
27398 · update_seq (boolean) – Whether to include in the response an
27400 update_seq value indicating the sequence id of the database
27401 the view reflects. Default is false.
27402 Response Headers
27404 · Content-Type – .INDENT 2.0
27406 · application/json
27408 · text/plain; charset=utf-8
27410 · ETag – Response signature
27413 · Transfer-Encoding – chunked
27415 Response JSON Object
27417 · offset (number) – Offset where the document list started.
27419 · rows (array) – Array of view row objects. By default the
27421 information returned contains only the document ID and revi‐
27422 sion.
27423 · total_rows (number) – Number of documents in the data‐
27425 base/view.
27426 · update_seq (object) – Current update sequence for the data‐
27428 base.
27429 Status Codes
27431 · 200 OK – Request completed successfully
27433 · 400 Bad Request – Invalid request
27435 · 401 Unauthorized – Read permission required
27437 · 404 Not Found – Specified database, design document or view is
27439 missed
27440
27442 GET /recipes/_design/ingredients/_view/by_name HTTP/1.1
27444 Accept: application/json
27445 Host: localhost:5984
27446 Response:
27448 HTTP/1.1 200 OK
27450 Cache-Control: must-revalidate
27451 Content-Type: application/json
27452 Date: Wed, 21 Aug 2013 09:12:06 GMT
27453 ETag: "2FOLSBSW4O6WB798XU4AQYA9B"
27454 Server: CouchDB (Erlang/OTP)
27455 Transfer-Encoding: chunked
27456 {
27458 "offset": 0,
27459 "rows": [
27460 {
27461 "id": "SpaghettiWithMeatballs",
27462 "key": "meatballs",
27463 "value": 1
27464 },
27465 {
27466 "id": "SpaghettiWithMeatballs",
27467 "key": "spaghetti",
27468 "value": 1
27469 },
27470 {
27471 "id": "SpaghettiWithMeatballs",
27472 "key": "tomato sauce",
27473 "value": 1
27474 }
27475 ],
27476 "total_rows": 3
27477 }
27478Changed in version 1.6.0: added attachments and att_encoding_info parameters
27480Changed in version 2.0.0: added sorted parameter
27483Changed in version 2.1.0: added stable and update parameters
27486
27489 Using the attachments parameter to include attachments in view
27490 results is not recommended for large attachment sizes. Also note
27491 that the Base64-encoding that is used leads to a 33% overhead (i.e.
27492 one third) in transfer size for attachments.
27493 POST /{db}/_design/{ddoc}/_view/{view}
27495 Executes the specified view function from the specified design
27496 document. POST view functionality supports identical parameters
27497 and behavior as specified in the GET
27498 /{db}/_design/{ddoc}/_view/{view} API but allows for the query
27499 string parameters to be supplied as keys in a JSON object in the
27500 body of the POST request.
27501 Request:
27503 POST /recipes/_design/ingredients/_view/by_name HTTP/1.1
27505 Accept: application/json
27506 Content-Length: 37
27507 Host: localhost:5984
27508 {
27510 "keys": [
27511 "meatballs",
27512 "spaghetti"
27513 ]
27514 }
27515 Response:
27517 HTTP/1.1 200 OK
27519 Cache-Control: must-revalidate
27520 Content-Type: application/json
27521 Date: Wed, 21 Aug 2013 09:14:13 GMT
27522 ETag: "6R5NM8E872JIJF796VF7WI3FZ"
27523 Server: CouchDB (Erlang/OTP)
27524 Transfer-Encoding: chunked
27525 {
27527 "offset": 0,
27528 "rows": [
27529 {
27530 "id": "SpaghettiWithMeatballs",
27531 "key": "meatballs",
27532 "value": 1
27533 },
27534 {
27535 "id": "SpaghettiWithMeatballs",
27536 "key": "spaghetti",
27537 "value": 1
27538 }
27539 ],
27540 "total_rows": 3
27541 }
27542 View Options
27544 There are two view indexing options that can be defined in a design
27545 document as boolean properties of an options object. Unlike the others
27546 querying options, these aren’t URL parameters because they take effect
27547 when the view index is generated, not when it’s accessed:
27548 · local_seq (boolean): Makes documents’ local sequence numbers avail‐
27550 able to map functions (as a _local_seq document property)
27551 · include_design (boolean): Allows map functions to be called on design
27553 documents as well as regular documents
27554 Querying Views and Indexes
27556 The definition of a view within a design document also creates an index
27557 based on the key information defined within each view. The production
27558 and use of the index significantly increases the speed of access and
27559 searching or selecting documents from the view.
27560 However, the index is not updated when new documents are added or modi‐
27562 fied in the database. Instead, the index is generated or updated,
27563 either when the view is first accessed, or when the view is accessed
27564 after a document has been updated. In each case, the index is updated
27565 before the view query is executed against the database.
27566 View indexes are updated incrementally in the following situations:
27568 · A new document has been added to the database.
27570 · A document has been deleted from the database.
27572 · A document in the database has been updated.
27574 View indexes are rebuilt entirely when the view definition changes. To
27576 achieve this, a ‘fingerprint’ of the view definition is created when
27577 the design document is updated. If the fingerprint changes, then the
27578 view indexes are entirely rebuilt. This ensures that changes to the
27579 view definitions are reflected in the view indexes.
27580 NOTE:
27582 View index rebuilds occur when one view from the same the view group
27583 (i.e. all the views defined within a single a design document) has
27584 been determined as needing a rebuild. For example, if if you have a
27585 design document with different views, and you update the database,
27586 all three view indexes within the design document will be updated.
27587 Because the view is updated when it has been queried, it can result in
27589 a delay in returned information when the view is accessed, especially
27590 if there are a large number of documents in the database and the view
27591 index does not exist. There are a number of ways to mitigate, but not
27592 completely eliminate, these issues. These include:
27593 · Create the view definition (and associated design documents) on your
27595 database before allowing insertion or updates to the documents. If
27596 this is allowed while the view is being accessed, the index can be
27597 updated incrementally.
27598 · Manually force a view request from the database. You can do this
27600 either before users are allowed to use the view, or you can access
27601 the view manually after documents are added or updated.
27602 · Use the changes feed to monitor for changes to the database and then
27604 access the view to force the corresponding view index to be updated.
27605 None of these can completely eliminate the need for the indexes to be
27607 rebuilt or updated when the view is accessed, but they may lessen the
27608 effects on end-users of the index update affecting the user experience.
27609 Another alternative is to allow users to access a ‘stale’ version of
27611 the view index, rather than forcing the index to be updated and dis‐
27612 playing the updated results. Using a stale view may not return the lat‐
27613 est information, but will return the results of the view query using an
27614 existing version of the index.
27615 For example, to access the existing stale view by_recipe in the recipes
27617 design document:
27618 http://localhost:5984/recipes/_design/recipes/_view/by_recipe?stale=ok
27620 Accessing a stale view:
27622 · Does not trigger a rebuild of the view indexes, even if there have
27624 been changes since the last access.
27625 · Returns the current version of the view index, if a current version
27627 exists.
27628 · Returns an empty result set if the given view index does exist.
27630 As an alternative, you use the update_after value to the stale parame‐
27632 ter. This causes the view to be returned as a stale view, but for the
27633 update process to be triggered after the view information has been
27634 returned to the client.
27635 In addition to using stale views, you can also make use of the
27637 update_seq query argument. Using this query argument generates the view
27638 information including the update sequence of the database from which
27639 the view was generated. The returned value can be compared this to the
27640 current update sequence exposed in the database information (returned
27641 by GET /{db}).
27642 Sorting Returned Rows
27644 Each element within the returned array is sorted using native UTF-8
27645 sorting according to the contents of the key portion of the emitted
27646 content. The basic order of output is as follows:
27647 · null
27649 · false
27651 · true
27653 · Numbers
27655 · Text (case sensitive, lowercase first)
27657 · Arrays (according to the values of each element, in order)
27659 · Objects (according to the values of keys, in key order)
27661 Request:
27663 GET /db/_design/test/_view/sorting HTTP/1.1
27665 Accept: application/json
27666 Host: localhost:5984
27667 Response:
27669 HTTP/1.1 200 OK
27671 Cache-Control: must-revalidate
27672 Content-Type: application/json
27673 Date: Wed, 21 Aug 2013 10:09:25 GMT
27674 ETag: "8LA1LZPQ37B6R9U8BK9BGQH27"
27675 Server: CouchDB (Erlang/OTP)
27676 Transfer-Encoding: chunked
27677 {
27679 "offset": 0,
27680 "rows": [
27681 {
27682 "id": "dummy-doc",
27683 "key": null,
27684 "value": null
27685 },
27686 {
27687 "id": "dummy-doc",
27688 "key": false,
27689 "value": null
27690 },
27691 {
27692 "id": "dummy-doc",
27693 "key": true,
27694 "value": null
27695 },
27696 {
27697 "id": "dummy-doc",
27698 "key": 0,
27699 "value": null
27700 },
27701 {
27702 "id": "dummy-doc",
27703 "key": 1,
27704 "value": null
27705 },
27706 {
27707 "id": "dummy-doc",
27708 "key": 10,
27709 "value": null
27710 },
27711 {
27712 "id": "dummy-doc",
27713 "key": 42,
27714 "value": null
27715 },
27716 {
27717 "id": "dummy-doc",
27718 "key": "10",
27719 "value": null
27720 },
27721 {
27722 "id": "dummy-doc",
27723 "key": "hello",
27724 "value": null
27725 },
27726 {
27727 "id": "dummy-doc",
27728 "key": "Hello",
27729 "value": null
27730 },
27731 {
27732 "id": "dummy-doc",
27733 "key": "\u043f\u0440\u0438\u0432\u0435\u0442",
27734 "value": null
27735 },
27736 {
27737 "id": "dummy-doc",
27738 "key": [],
27739 "value": null
27740 },
27741 {
27742 "id": "dummy-doc",
27743 "key": [
27744 1,
27745 2,
27746 3
27747 ],
27748 "value": null
27749 },
27750 {
27751 "id": "dummy-doc",
27752 "key": [
27753 2,
27754 3
27755 ],
27756 "value": null
27757 },
27758 {
27759 "id": "dummy-doc",
27760 "key": [
27761 3
27762 ],
27763 "value": null
27764 },
27765 {
27766 "id": "dummy-doc",
27767 "key": {},
27768 "value": null
27769 },
27770 {
27771 "id": "dummy-doc",
27772 "key": {
27773 "foo": "bar"
27774 },
27775 "value": null
27776 }
27777 ],
27778 "total_rows": 17
27779 }
27780 You can reverse the order of the returned view information by using the
27782 descending query value set to true:
27783 Request:
27785 GET /db/_design/test/_view/sorting?descending=true HTTP/1.1
27787 Accept: application/json
27788 Host: localhost:5984
27789 Response:
27791 HTTP/1.1 200 OK
27793 Cache-Control: must-revalidate
27794 Content-Type: application/json
27795 Date: Wed, 21 Aug 2013 10:09:25 GMT
27796 ETag: "Z4N468R15JBT98OM0AMNSR8U"
27797 Server: CouchDB (Erlang/OTP)
27798 Transfer-Encoding: chunked
27799 {
27801 "offset": 0,
27802 "rows": [
27803 {
27804 "id": "dummy-doc",
27805 "key": {
27806 "foo": "bar"
27807 },
27808 "value": null
27809 },
27810 {
27811 "id": "dummy-doc",
27812 "key": {},
27813 "value": null
27814 },
27815 {
27816 "id": "dummy-doc",
27817 "key": [
27818 3
27819 ],
27820 "value": null
27821 },
27822 {
27823 "id": "dummy-doc",
27824 "key": [
27825 2,
27826 3
27827 ],
27828 "value": null
27829 },
27830 {
27831 "id": "dummy-doc",
27832 "key": [
27833 1,
27834 2,
27835 3
27836 ],
27837 "value": null
27838 },
27839 {
27840 "id": "dummy-doc",
27841 "key": [],
27842 "value": null
27843 },
27844 {
27845 "id": "dummy-doc",
27846 "key": "\u043f\u0440\u0438\u0432\u0435\u0442",
27847 "value": null
27848 },
27849 {
27850 "id": "dummy-doc",
27851 "key": "Hello",
27852 "value": null
27853 },
27854 {
27855 "id": "dummy-doc",
27856 "key": "hello",
27857 "value": null
27858 },
27859 {
27860 "id": "dummy-doc",
27861 "key": "10",
27862 "value": null
27863 },
27864 {
27865 "id": "dummy-doc",
27866 "key": 42,
27867 "value": null
27868 },
27869 {
27870 "id": "dummy-doc",
27871 "key": 10,
27872 "value": null
27873 },
27874 {
27875 "id": "dummy-doc",
27876 "key": 1,
27877 "value": null
27878 },
27879 {
27880 "id": "dummy-doc",
27881 "key": 0,
27882 "value": null
27883 },
27884 {
27885 "id": "dummy-doc",
27886 "key": true,
27887 "value": null
27888 },
27889 {
27890 "id": "dummy-doc",
27891 "key": false,
27892 "value": null
27893 },
27894 {
27895 "id": "dummy-doc",
27896 "key": null,
27897 "value": null
27898 }
27899 ],
27900 "total_rows": 17
27901 }
27902 Sorting order and startkey/endkey
27904 The sorting direction is applied before the filtering applied using the
27905 startkey and endkey query arguments. For example the following query:
27906 GET http://couchdb:5984/recipes/_design/recipes/_view/by_ingredient?startkey=%22carrots%22&endkey=%22egg%22 HTTP/1.1
27908 Accept: application/json
27909 will operate correctly when listing all the matching entries between
27911 carrots and egg. If the order of output is reversed with the descending
27912 query argument, the view request will return no entries:
27913 GET /recipes/_design/recipes/_view/by_ingredient?descending=true&startkey=%22carrots%22&endkey=%22egg%22 HTTP/1.1
27915 Accept: application/json
27916 Host: localhost:5984
27917 {
27919 "total_rows" : 26453,
27920 "rows" : [],
27921 "offset" : 21882
27922 }
27923 The results will be empty because the entries in the view are reversed
27925 before the key filter is applied, and therefore the endkey of “egg”
27926 will be seen before the startkey of “carrots”, resulting in an empty
27927 list.
27928 Instead, you should reverse the values supplied to the startkey and
27930 endkey parameters to match the descending sorting applied to the keys.
27931 Changing the previous example to:
27932 GET /recipes/_design/recipes/_view/by_ingredient?descending=true&startkey=%22egg%22&endkey=%22carrots%22 HTTP/1.1
27934 Accept: application/json
27935 Host: localhost:5984
27936 Raw collation
27938 By default CouchDB uses an ICU driver for sorting view results. It’s
27939 possible use binary collation instead for faster view builds where Uni‐
27940 code collation is not important.
27941 To use raw collation add "collation": "raw" key-value pair to the
27943 design documents options object at the root level. After that, views
27944 will be regenerated and new order applied.
27945 SEE ALSO:
27947 views/collation
27948 Using Limits and Skipping Rows
27950 By default, views return all results. That’s ok when the number of
27951 results is small, but this may lead to problems when there are billions
27952 results, since the client may have to read them all and consume all
27953 available memory.
27954 But it’s possible to reduce output result rows by specifying limit
27956 query parameter. For example, retrieving the list of recipes using the
27957 by_title view and limited to 5 returns only 5 records, while there are
27958 total 2667 records in view:
27959 Request:
27961 GET /recipes/_design/recipes/_view/by_title?limit=5 HTTP/1.1
27963 Accept: application/json
27964 Host: localhost:5984
27965 Response:
27967 HTTP/1.1 200 OK
27969 Cache-Control: must-revalidate
27970 Content-Type: application/json
27971 Date: Wed, 21 Aug 2013 09:14:13 GMT
27972 ETag: "9Q6Q2GZKPH8D5F8L7PB6DBSS9"
27973 Server: CouchDB (Erlang/OTP)
27974 Transfer-Encoding: chunked
27975 {
27977 "offset" : 0,
27978 "rows" : [
27979 {
27980 "id" : "3-tiersalmonspinachandavocadoterrine",
27981 "key" : "3-tier salmon, spinach and avocado terrine",
27982 "value" : [
27983 null,
27984 "3-tier salmon, spinach and avocado terrine"
27985 ]
27986 },
27987 {
27988 "id" : "Aberffrawcake",
27989 "key" : "Aberffraw cake",
27990 "value" : [
27991 null,
27992 "Aberffraw cake"
27993 ]
27994 },
27995 {
27996 "id" : "Adukiandorangecasserole-microwave",
27997 "key" : "Aduki and orange casserole - microwave",
27998 "value" : [
27999 null,
28000 "Aduki and orange casserole - microwave"
28001 ]
28002 },
28003 {
28004 "id" : "Aioli-garlicmayonnaise",
28005 "key" : "Aioli - garlic mayonnaise",
28006 "value" : [
28007 null,
28008 "Aioli - garlic mayonnaise"
28009 ]
28010 },
28011 {
28012 "id" : "Alabamapeanutchicken",
28013 "key" : "Alabama peanut chicken",
28014 "value" : [
28015 null,
28016 "Alabama peanut chicken"
28017 ]
28018 }
28019 ],
28020 "total_rows" : 2667
28021 }
28022 To omit some records you may use skip query parameter:
28024 Request:
28026 GET /recipes/_design/recipes/_view/by_title?limit=3&skip=2 HTTP/1.1
28028 Accept: application/json
28029 Host: localhost:5984
28030 Response:
28032 HTTP/1.1 200 OK
28034 Cache-Control: must-revalidate
28035 Content-Type: application/json
28036 Date: Wed, 21 Aug 2013 09:14:13 GMT
28037 ETag: "H3G7YZSNIVRRHO5FXPE16NJHN"
28038 Server: CouchDB (Erlang/OTP)
28039 Transfer-Encoding: chunked
28040 {
28042 "offset" : 2,
28043 "rows" : [
28044 {
28045 "id" : "Adukiandorangecasserole-microwave",
28046 "key" : "Aduki and orange casserole - microwave",
28047 "value" : [
28048 null,
28049 "Aduki and orange casserole - microwave"
28050 ]
28051 },
28052 {
28053 "id" : "Aioli-garlicmayonnaise",
28054 "key" : "Aioli - garlic mayonnaise",
28055 "value" : [
28056 null,
28057 "Aioli - garlic mayonnaise"
28058 ]
28059 },
28060 {
28061 "id" : "Alabamapeanutchicken",
28062 "key" : "Alabama peanut chicken",
28063 "value" : [
28064 null,
28065 "Alabama peanut chicken"
28066 ]
28067 }
28068 ],
28069 "total_rows" : 2667
28070 }
28071 WARNING:
28073 Using limit and skip parameters is not recommended for results pagi‐
28074 nation. Read pagination recipe why it’s so and how to make it bet‐
28075 ter.
28076 Sending multiple queries to a view
28078 New in version 2.2.
28079 POST /{db}/_design/{ddoc}/_view/{view}/queries
28082 Executes multiple specified view queries against the view func‐
28083 tion from the specified design document.
28084 Parameters
28086 · db – Database name
28088 · ddoc – Design document name
28090 · view – View function name
28092 Request Headers
28094 · Content-Type – .INDENT 2.0
28096 · application/json
28098 · Accept – .INDENT 2.0
28101 · application/json
28103 Request JSON Object
28106 · queries – An array of query objects with fields for the param‐
28108 eters of each individual view query to be executed. The field
28109 names and their meaning are the same as the query parameters
28110 of a regular view request.
28111 Response Headers
28113 · Content-Type – .INDENT 2.0
28115 · application/json
28117 · ETag – Response signature
28120 · Transfer-Encoding – chunked
28122 Response JSON Object
28124 · results (array) – An array of result objects - one for each
28126 query. Each result object contains the same fields as the
28127 response to a regular view request.
28128 Status Codes
28130 · 200 OK – Request completed successfully
28132 · 400 Bad Request – Invalid request
28134 · 401 Unauthorized – Read permission required
28136 · 404 Not Found – Specified database, design document or view is
28138 missing
28139 · 500 Internal Server Error – View function execution error
28141
28143 POST /recipes/_design/recipes/_view/by_title/queries HTTP/1.1
28145 Content-Type: application/json
28146 Accept: application/json
28147 Host: localhost:5984
28148 {
28150 "queries": [
28151 {
28152 "keys": [
28153 "meatballs",
28154 "spaghetti"
28155 ]
28156 },
28157 {
28158 "limit": 3,
28159 "skip": 2
28160 }
28161 ]
28162 }
28163 Response:
28165 HTTP/1.1 200 OK
28167 Cache-Control: must-revalidate
28168 Content-Type: application/json
28169 Date: Wed, 20 Dec 2016 11:17:07 GMT
28170 ETag: "1H8RGBCK3ABY6ACDM7ZSC30QK"
28171 Server: CouchDB (Erlang/OTP)
28172 Transfer-Encoding: chunked
28173 {
28175 "results" : [
28176 {
28177 "offset": 0,
28178 "rows": [
28179 {
28180 "id": "SpaghettiWithMeatballs",
28181 "key": "meatballs",
28182 "value": 1
28183 },
28184 {
28185 "id": "SpaghettiWithMeatballs",
28186 "key": "spaghetti",
28187 "value": 1
28188 },
28189 {
28190 "id": "SpaghettiWithMeatballs",
28191 "key": "tomato sauce",
28192 "value": 1
28193 }
28194 ],
28195 "total_rows": 3
28196 },
28197 {
28198 "offset" : 2,
28199 "rows" : [
28200 {
28201 "id" : "Adukiandorangecasserole-microwave",
28202 "key" : "Aduki and orange casserole - microwave",
28203 "value" : [
28204 null,
28205 "Aduki and orange casserole - microwave"
28206 ]
28207 },
28208 {
28209 "id" : "Aioli-garlicmayonnaise",
28210 "key" : "Aioli - garlic mayonnaise",
28211 "value" : [
28212 null,
28213 "Aioli - garlic mayonnaise"
28214 ]
28215 },
28216 {
28217 "id" : "Alabamapeanutchicken",
28218 "key" : "Alabama peanut chicken",
28219 "value" : [
28220 null,
28221 "Alabama peanut chicken"
28222 ]
28223 }
28224 ],
28225 "total_rows" : 2667
28226 }
28227 ]
28228 }
28229 /db/_design/design-doc/_search/index-name
28231 WARNING:
28232 Search endpoints require a running search plugin connected to each
28233 cluster node. See Search Plugin Installation for details.
28234 New in version 3.0.
28236 GET /{db}/_design/{ddoc}/_search/{index}
28239 Executes a search request against the named index in the speci‐
28240 fied design document.
28241 Parameters
28243 · db – Database name
28245 · ddoc – Design document name
28247 · index – Search index name
28249 Request Headers
28251 · Accept – .INDENT 2.0
28253 · application/json
28255 · text/plain
28257 Query Parameters
28260 · bookmark (string) – A bookmark received from a previous
28262 search. This parameter enables paging through the results. If
28263 there are no more results after the bookmark, you get a
28264 response with an empty rows array and the same bookmark, con‐
28265 firming the end of the result list.
28266 · counts (json) – An array of names of string fields for which
28268 counts are requested. The response contains counts for each
28269 unique value of this field name among the documents that match
28270 the search query. Faceting must be enabled for this parameter
28271 to function.
28272 · drilldown (json) – This field can be used several times. Each
28274 use defines a pair with a field name and a value. The search
28275 matches only documents containing the value that was provided
28276 in the named field. It differs from using "fieldname:value" in
28277 the q parameter only in that the values are not analyzed.
28278 Faceting must be enabled for this parameter to function.
28279 · group_field (string) – Field by which to group search matches.
28281 :query number group_limit: Maximum group count. This field can
28282 be used only if group_field is specified.
28283 · group_sort (json) – This field defines the order of the groups
28285 in a search that uses group_field. The default sort order is
28286 relevance.
28287 · highlight_fields (json) – Specifies which fields to highlight.
28289 If specified, the result object contains a highlights field
28290 with an entry for each specified field.
28291 · highlight_pre_tag (string) – A string that is inserted before
28293 the highlighted word in the highlights output.
28294 · highlight_post_tag (string) – A string that is inserted after
28296 the highlighted word in the highlights output.
28297 · highlight_number (number) – Number of fragments that are
28299 returned in highlights. If the search term occurs less often
28300 than the number of fragments that are specified, longer frag‐
28301 ments are returned.
28302 · highlight_size (number) – Number of characters in each frag‐
28304 ment for highlights.
28305 · include_docs (boolean) – Include the full content of the docu‐
28307 ments in the response.
28308 · include_fields (json) – A JSON array of field names to include
28310 in search results. Any fields that are included must be
28311 indexed with the store:true option.
28312 · limit (number) – Limit the number of the returned documents to
28314 the specified number. For a grouped search, this parameter
28315 limits the number of documents per group.
28316 · q (string) – Alias for query.
28318 · query (string) – Required. The Lucene query string.
28320 · ranges (json) – This field defines ranges for faceted, numeric
28322 search fields. The value is a JSON object where the fields
28323 names are faceted numeric search fields, and the values of the
28324 fields are JSON objects. The field names of the JSON objects
28325 are names for ranges. The values are strings that describe the
28326 range, for example “[0 TO 10]”.
28327 · sort (json) – Specifies the sort order of the results. In a
28329 grouped search (when group_field is used), this parameter
28330 specifies the sort order within a group. The default sort
28331 order is relevance. A JSON string of the form "field‐
28332 name<type>" or -fieldname<type> for descending order, where
28333 fieldname is the name of a string or number field, and type is
28334 either a number, a string, or a JSON array of strings. The
28335 type part is optional, and defaults to number. Some examples
28336 are "foo", "-foo", "bar<string>", "-foo<number>" and
28337 ["-foo<number>", "bar<string>"]. String fields that are used
28338 for sorting must not be analyzed fields. Fields that are used
28339 for sorting must be indexed by the same indexer that is used
28340 for the search query.
28341 · stale (string) – Set to ok to allow the use of an out-of-date
28343 index.
28344 Response Headers
28346 · Content-Type – .INDENT 2.0
28348 · application/json
28350 · text/plain; charset=utf-8
28352 · ETag – Response signature
28355 · Transfer-Encoding – chunked
28357 Response JSON Object
28359 · rows (array) – Array of view row objects. By default the
28361 information returned contains only the document ID and revi‐
28362 sion.
28363 · total_rows (number) – Number of documents in the data‐
28365 base/view.
28366 · bookmark (string) – Opaque identifier to enable pagination.
28368 Status Codes
28370 · 200 OK – Request completed successfully
28372 · 400 Bad Request – Invalid request
28374 · 401 Unauthorized – Read permission required
28376 · 404 Not Found – Specified database, design document or view is
28378 missed
28379
28381 You must enable faceting before you can use the counts, drilldown,
28382 and ranges parameters.
28383 NOTE:
28385 Faceting and grouping are not supported on partitioned searches, so
28386 the following query parameters should not be used on those requests:
28387 counts, drilldown, ranges, and group_field, group_limit,
28388 group_sort``.
28389 NOTE:
28391 Do not combine the bookmark and stale options. These options con‐
28392 strain the choice of shard replicas to use for the response. When
28393 used together, the options might cause problems when contact is
28394 attempted with replicas that are slow or not available.
28395 SEE ALSO:
28397 For more information about how search works, see the Search User
28398 Guide.
28399 /db/_design/design-doc/_search_info/index-name
28401 WARNING:
28402 Search endpoints require a running search plugin connected to each
28403 cluster node. See Search Plugin Installation for details.
28404 New in version 3.0.
28406 GET /{db}/_design/{ddoc}/_search_info/{index}
28409 Parameters
28411 · db – Database name
28413 · ddoc – Design document name
28415 · index – Search index name
28417 Status Codes
28419 · 200 OK – Request completed successfully
28421 · 400 Bad Request – Request body is wrong (malformed or
28423 missing one of the mandatory fields)
28424 · 500 Internal Server Error – A server error (or other
28426 kind of error) occurred
28427 Request:
28429 GET /recipes/_design/cookbook/_search_info/ingredients HTTP/1.1
28431 Accept: application/json
28432 Host: localhost:5984
28433 Response:
28435 {
28437 "name": "_design/cookbook/ingredients",
28438 "search_index": {
28439 "pending_seq": 7125496,
28440 "doc_del_count": 129180,
28441 "doc_count": 1066173,
28442 "disk_size": 728305827,
28443 "committed_seq": 7125496
28444 }
28445 }
28446 /db/_design/design-doc/_show/show-name
28448 WARNING:
28449 Show functions are deprecated in CouchDB 3.0, and will be removed in
28450 CouchDB 4.0.
28451 GET /{db}/_design/{ddoc}/_show/{func}
28453 POST /{db}/_design/{ddoc}/_show/{func}
28455 Applies show function for null document.
28456 The request and response parameters are depended upon function
28458 implementation.
28459 Parameters
28461 · db – Database name
28463 · ddoc – Design document name
28465 · func – Show function name
28467 Response Headers
28469 · ETag – Response signature
28471 Query Parameters
28473 · format (string) – Format of the returned response.
28475 Used by provides() function
28476 Status Codes
28478 · 200 OK – Request completed successfully
28480 · 500 Internal Server Error – Query server error
28482 Function:
28484 function(doc, req) {
28486 if (!doc) {
28487 return {body: "no doc"}
28488 } else {
28489 return {body: doc.description}
28490 }
28491 }
28492 Request:
28494 GET /recipes/_design/recipe/_show/description HTTP/1.1
28496 Accept: application/json
28497 Host: localhost:5984
28498 Response:
28500 HTTP/1.1 200 OK
28502 Content-Length: 6
28503 Content-Type: text/html; charset=utf-8
28504 Date: Wed, 21 Aug 2013 12:34:07 GMT
28505 Etag: "7Z2TO7FPEMZ0F4GH0RJCRIOAU"
28506 Server: CouchDB (Erlang/OTP)
28507 Vary: Accept
28508 no doc
28510 /db/_design/design-doc/_show/show-name/doc-id
28512 WARNING:
28513 Show functions are deprecated in CouchDB 3.0, and will be removed in
28514 CouchDB 4.0.
28515 GET /{db}/_design/{ddoc}/_show/{func}/{docid}
28517 POST /{db}/_design/{ddoc}/_show/{func}/{docid}
28519 Applies show function for the specified document.
28520 The request and response parameters are depended upon function
28522 implementation.
28523 Parameters
28525 · db – Database name
28527 · ddoc – Design document name
28529 · func – Show function name
28531 · docid – Document ID
28533 Response Headers
28535 · ETag – Response signature
28537 Query Parameters
28539 · format (string) – Format of the returned response.
28541 Used by provides() function
28542 Status Codes
28544 · 200 OK – Request completed successfully
28546 · 500 Internal Server Error – Query server error
28548 Function:
28550 function(doc, req) {
28552 if (!doc) {
28553 return {body: "no doc"}
28554 } else {
28555 return {body: doc.description}
28556 }
28557 }
28558 Request:
28560 GET /recipes/_design/recipe/_show/description/SpaghettiWithMeatballs HTTP/1.1
28562 Accept: application/json
28563 Host: localhost:5984
28564 Response:
28566 HTTP/1.1 200 OK
28568 Content-Length: 88
28569 Content-Type: text/html; charset=utf-8
28570 Date: Wed, 21 Aug 2013 12:38:08 GMT
28571 Etag: "8IEBO8103EI98HDZL5Z4I1T0C"
28572 Server: CouchDB (Erlang/OTP)
28573 Vary: Accept
28574 An Italian-American dish that usually consists of spaghetti, tomato sauce and meatballs.
28576 /db/_design/design-doc/_list/list-name/view-name
28578 WARNING:
28579 List functions are deprecated in CouchDB 3.0, and will be removed in
28580 CouchDB 4.0.
28581 GET /{db}/_design/{ddoc}/_list/{func}/{view}
28583 POST /{db}/_design/{ddoc}/_list/{func}/{view}
28585 Applies list function for the view function from the same design
28586 document.
28587 The request and response parameters are depended upon function
28589 implementation.
28590 Parameters
28592 · db – Database name
28594 · ddoc – Design document name
28596 · func – List function name
28598 · view – View function name
28600 Response Headers
28602 · ETag – Response signature
28604 · Transfer-Encoding – chunked
28606 Query Parameters
28608 · format (string) – Format of the returned response.
28610 Used by provides() function
28611 Status Codes
28613 · 200 OK – Request completed successfully
28615 · 500 Internal Server Error – Query server error
28617 Function:
28619 function(head, req) {
28621 var row = getRow();
28622 if (!row){
28623 return 'no ingredients'
28624 }
28625 send(row.key);
28626 while(row=getRow()){
28627 send(', ' + row.key);
28628 }
28629 }
28630 Request:
28632 GET /recipes/_design/recipe/_list/ingredients/by_name HTTP/1.1
28634 Accept: text/plain
28635 Host: localhost:5984
28636 Response:
28638 HTTP/1.1 200 OK
28640 Content-Type: text/plain; charset=utf-8
28641 Date: Wed, 21 Aug 2013 12:49:15 GMT
28642 Etag: "D52L2M1TKQYDD1Y8MEYJR8C84"
28643 Server: CouchDB (Erlang/OTP)
28644 Transfer-Encoding: chunked
28645 Vary: Accept
28646 meatballs, spaghetti, tomato sauce
28648 /db/_design/design-doc/_list/list-name/other-ddoc/view-name
28650 WARNING:
28651 List functions are deprecated in CouchDB 3.0, and will be removed in
28652 CouchDB 4.0.
28653 GET /{db}/_design/{ddoc}/_list/{func}/{other-ddoc}/{view}
28655 POST /{db}/_design/{ddoc}/_list/{func}/{other-ddoc}/{view}
28657 Applies list function for the view function from the other
28658 design document.
28659 The request and response parameters are depended upon function
28661 implementation.
28662 Parameters
28664 · db – Database name
28666 · ddoc – Design document name
28668 · func – List function name
28670 · other-ddoc – Other design document name that holds view
28672 function
28673 · view – View function name
28675 Response Headers
28677 · ETag – Response signature
28679 · Transfer-Encoding – chunked
28681 Query Parameters
28683 · format (string) – Format of the returned response.
28685 Used by provides() function
28686 Status Codes
28688 · 200 OK – Request completed successfully
28690 · 500 Internal Server Error – Query server error
28692 Function:
28694 function(head, req) {
28696 var row = getRow();
28697 if (!row){
28698 return 'no ingredients'
28699 }
28700 send(row.key);
28701 while(row=getRow()){
28702 send(', ' + row.key);
28703 }
28704 }
28705 Request:
28707 GET /recipes/_design/ingredient/_list/ingredients/recipe/by_ingredient?key="spaghetti" HTTP/1.1
28709 Accept: text/plain
28710 Host: localhost:5984
28711 Response:
28713 HTTP/1.1 200 OK
28715 Content-Type: text/plain; charset=utf-8
28716 Date: Wed, 21 Aug 2013 12:49:15 GMT
28717 Etag: "5L0975X493R0FB5Z3043POZHD"
28718 Server: CouchDB (Erlang/OTP)
28719 Transfer-Encoding: chunked
28720 Vary: Accept
28721 spaghetti
28723 /db/_design/design-doc/_update/update-name
28725 POST /{db}/_design/{ddoc}/_update/{func}
28726 Executes update function on server side for null document.
28727 Parameters
28729 · db – Database name
28731 · ddoc – Design document name
28733 · func – Update function name
28735 Response Headers
28737 · X-Couch-Id – Created/updated document’s ID
28739 · X-Couch-Update-Newrev – Created/updated document’s
28741 revision
28742 Status Codes
28744 · 200 OK – No document was created or updated
28746 · 201 Created – Document was created or updated
28748 · 500 Internal Server Error – Query server error
28750 Function:
28752 function(doc, req) {
28754 if (!doc){
28755 return [null, {'code': 400,
28756 'json': {'error': 'missed',
28757 'reason': 'no document to update'}}]
28758 } else {
28759 doc.ingredients.push(req.body);
28760 return [doc, {'json': {'status': 'ok'}}];
28761 }
28762 }
28763 Request:
28765 POST /recipes/_design/recipe/_update/ingredients HTTP/1.1
28767 Accept: application/json
28768 Content-Length: 10
28769 Content-Type: application/json
28770 Host: localhost:5984
28771 "something"
28773 Response:
28775 HTTP/1.1 404 Object Not Found
28777 Cache-Control: must-revalidate
28778 Content-Length: 52
28779 Content-Type: application/json
28780 Date: Wed, 21 Aug 2013 14:00:58 GMT
28781 Server: CouchDB (Erlang/OTP)
28782 {
28784 "error": "missed",
28785 "reason": "no document to update"
28786 }
28787 /db/_design/design-doc/_update/update-name/doc-id
28789 PUT /{db}/_design/{ddoc}/_update/{func}/{docid}
28790 Executes update function on server side for the specified docu‐
28791 ment.
28792 Parameters
28794 · db – Database name
28796 · ddoc – Design document name
28798 · func – Update function name
28800 · docid – Document ID
28802 Response Headers
28804 · X-Couch-Id – Created/updated document’s ID
28806 · X-Couch-Update-Newrev – Created/updated document’s
28808 revision
28809 Status Codes
28811 · 200 OK – No document was created or updated
28813 · 201 Created – Document was created or updated
28815 · 500 Internal Server Error – Query server error
28817 Function:
28819 function(doc, req) {
28821 if (!doc){
28822 return [null, {'code': 400,
28823 'json': {'error': 'missed',
28824 'reason': 'no document to update'}}]
28825 } else {
28826 doc.ingredients.push(req.body);
28827 return [doc, {'json': {'status': 'ok'}}];
28828 }
28829 }
28830 Request:
28832 POST /recipes/_design/recipe/_update/ingredients/SpaghettiWithMeatballs HTTP/1.1
28834 Accept: application/json
28835 Content-Length: 5
28836 Content-Type: application/json
28837 Host: localhost:5984
28838 "love"
28840 Response:
28842 HTTP/1.1 201 Created
28844 Cache-Control: must-revalidate
28845 Content-Length: 16
28846 Content-Type: application/json
28847 Date: Wed, 21 Aug 2013 14:11:34 GMT
28848 Server: CouchDB (Erlang/OTP)
28849 X-Couch-Id: SpaghettiWithMeatballs
28850 X-Couch-Update-NewRev: 12-a5e099df5720988dae90c8b664496baf
28851 {
28853 "status": "ok"
28854 }
28855 /db/_design/design-doc/_rewrite/path
28857 WARNING:
28858 Rewrites are deprecated in CouchDB 3.0, and will be removed in
28859 CouchDB 4.0.
28860 ANY /{db}/_design/{ddoc}/_rewrite/{path}
28862 Rewrites the specified path by rules defined in the specified
28863 design document. The rewrite rules are defined by the rewrites
28864 field of the design document. The rewrites field can either be a
28865 string containing the a rewrite function or an array of rule
28866 definitions.
28867 Using a stringified function for rewrites
28869 New in version 2.0: When the rewrites field is a stringified function,
28870 the query server is used to pre-process and route requests.
28871 The function takes a request2_object.
28873 The return value of the function will cause the server to rewrite the
28875 request to a new location or immediately return a response.
28876 To rewrite the request, return an object containing the following prop‐
28878 erties:
28879 · path (string): Rewritten path.
28881 · query (array): Rewritten query. If omitted, the original query keys
28883 are used.
28884 · headers (object): Rewritten headers. If omitted, the original request
28886 headers are used.
28887 · method (string): HTTP method of rewritten request ("GET", "POST",
28889 etc). If omitted, the original request method is used.
28890 · body (string): Body for "POST"/"PUT" requests. If omitted, the origi‐
28892 nal request body is used.
28893 To immediately respond to the request, return an object containing the
28895 following properties:
28896 · code (number): Returned HTTP status code (200, 404, etc).
28898 · body (string): Body of the response to user.
28900 Example A. Restricting access.
28902 function(req2) {
28904 var path = req2.path.slice(4),
28905 isWrite = /^(put|post|delete)$/i.test(req2.method),
28906 isFinance = req2.userCtx.roles.indexOf("finance") > -1;
28907 if (path[0] == "finance" && isWrite && !isFinance) {
28908 // Deny writes to DB "finance" for users
28909 // having no "finance" role
28910 return {
28911 code: 403,
28912 body: JSON.stringify({
28913 error: "forbidden".
28914 reason: "You are not allowed to modify docs in this DB"
28915 })
28916 };
28917 }
28918 // Pass through all other requests
28919 return { path: "../../../" + path.join("/") };
28920 }
28921 Example B. Different replies for JSON and HTML requests.
28923 function(req2) {
28925 var path = req2.path.slice(4),
28926 h = headers,
28927 wantsJson = (h.Accept || "").indexOf("application/json") > -1,
28928 reply = {};
28929 if (!wantsJson) {
28930 // Here we should prepare reply object
28931 // for plain HTML pages
28932 } else {
28933 // Pass through JSON requests
28934 reply.path = "../../../"+path.join("/");
28935 }
28936 return reply;
28937 }
28938 Using an array of rules for rewrites
28941 When the rewrites field is an array of rule objects, the server will
28942 rewrite the request based on the first matching rule in the array.
28943 Each rule in the array is an object with the following fields:
28945 · method (string): HTTP request method to bind the request method to
28947 the rule. If omitted, uses "*", which matches all methods.
28948 · from (string): The pattern used to compare against the URL and
28950 define dynamic variables.
28951 · to (string): The path to rewrite the URL to. It can contain vari‐
28953 ables depending on binding variables discovered during pattern
28954 matching and query args (URL args and from the query member).
28955 · query (object): Query args passed to the rewritten URL. They may
28957 contain dynamic variables.
28958 The to and from paths may contains string patterns with leading : or
28960 * characters to define dynamic variables in the match.
28961 The first rule in the rewrites array that matches the incoming
28963 request is used to define the rewrite. To match the incoming
28964 request, the rule’s method must match the request’s HTTP method and
28965 the rule’s from must match the request’s path using the following
28966 pattern matching logic.
28967 · The from pattern and URL are first split on / to get a list of
28969 tokens. For example, if from field is /somepath/:var/* and the URL
28970 is /somepath/a/b/c, the tokens are somepath, :var, and * for the
28971 from pattern and somepath, a, b, and c for the URL.
28972 · Each token starting with : in the pattern will match the corre‐
28974 sponding token in the URL and define a new dynamic variable whose
28975 name is the remaining string after the : and value is the token
28976 from the URL. In this example, the :var token will match b and set
28977 var = a.
28978 · The star token * in the pattern will match any number of tokens in
28980 the URL and must be the last token in the pattern. It will define
28981 a dynamic variable with the remaining tokens. In this example, the
28982 * token will match the b and c tokens and set * = b/c.
28983 · The remaining tokens must match exactly for the pattern to be con‐
28985 sidered a match. In this example, somepath in the pattern matches
28986 somepath in the URL and all tokens in the URL have matched, caus‐
28987 ing this rule to be a match.
28988 Once a rule is found, the request URL is rewritten using the to and
28990 query fields. Dynamic variables are substituted into the : and *
28991 variables in these fields to produce the final URL.
28992 If no rule matches, a 404 Not Found response is returned.
28994 Examples:
28996 ┌────────────────────────────────┬──────────┬──────────────────┬────────┐
28998 │Rule │ URL │ Rewrite to │ Tokens │
28999 ├────────────────────────────────┼──────────┼──────────────────┼────────┤
29000 │ │ /a │ /some │ │
29001 │ {“from”: │ │ │ │
29002 │ “/a”, │ │ │ │
29003 │ “to”: │ │ │ │
29004 │ “/some”} │ │ │ │
29005 ├────────────────────────────────┼──────────┼──────────────────┼────────┤
29006 │ │ /a/b/c │ /some/b/c │ │
29007 │ {“from”: │ │ │ │
29008 │ “/a/*”, │ │ │ │
29009 │ “to”: │ │ │ │
29010 │ “/some/*} │ │ │ │
29011 ├────────────────────────────────┼──────────┼──────────────────┼────────┤
29012 │ │ /a/b?k=v │ /some?k=v │ k=v │
29013 │ {“from”: “/a/b”, │ │ │ │
29014 │ “to”: │ │ │ │
29015 │ “/some”} │ │ │ │
29016 ├────────────────────────────────┼──────────┼──────────────────┼────────┤
29017 │ │ /a/b │ /some/b?var=b │ var=b │
29018 │ {“from”: “/a/b”, │ │ │ │
29019 │ “to”: │ │ │ │
29020 │ “/some/:var”} │ │ │ │
29021 ├────────────────────────────────┼──────────┼──────────────────┼────────┤
29022 │ │ /a/b/c │ /some/b/c?foo=b │ foo=b │
29023 │ {“from”: “/a/:foo/”, │ │ │ │
29024 │ “to”: │ │ │ │
29025 │ “/some/:foo/”} │ │ │ │
29026 └────────────────────────────────┴──────────┴──────────────────┴────────┘
29027 │ │ /a/b │ /some/?k=b&foo=b │ foo=b │
29034 │ {“from”: “/a/:foo”, │ │ │ │
29035 │ “to”: “/some”, │ │ │ │
29036 │ “query”: { │ │ │ │
29037 │ “k”: “:foo” }} │ │ │ │
29038 ├────────────────────────────────┼──────────┼──────────────────┼────────┤
29039 │ │ /a?foo=b │ /some/?b&foo=b │ foo=b │
29040 │ {“from”: “/a”, │ │ │ │
29041 │ “to”: │ │ │ │
29042 │ “/some/:foo”} │ │ │ │
29043 └────────────────────────────────┴──────────┴──────────────────┴────────┘
29044 Request method, header, query parameters, request payload and
29046 response body are dependent on the endpoint to which the URL will be
29047 rewritten.
29048 param db
29050 Database name
29051 param ddoc
29053 Design document name
29054 param path
29056 URL path to rewrite
29057 Partitioned Databases
29059 Partitioned databases allow for data colocation in a cluster, which
29060 provides significant performance improvements for queries constrained
29061 to a single partition.
29062 See the guide for getting started with partitioned databases
29064 /db/_partition/partition
29066 GET /{db}/_partition/{partition}
29067 This endpoint returns information describing the provided parti‐
29068 tion. It includes document and deleted document counts along
29069 with external and active data sizes.
29070 Status Codes
29072 · 200 OK – Request completed successfully
29074 Request:
29076 GET /db/_partition/sensor-260 HTTP/1.1
29078 Accept: application/json
29079 Host: localhost:5984
29080 Response:
29082 HTTP/1.1 200 OK
29084 Cache-Control: must-revalidate
29085 Content-Length: 119
29086 Content-Type: application/json
29087 Date: Thu, 24 Jan 2019 17:19:59 GMT
29088 Server: CouchDB/2.3.0-a1e11cea9 (Erlang OTP/21)
29089 {
29091 "db_name": "my_new_db",
29092 "doc_count": 1,
29093 "doc_del_count": 0,
29094 "partition": "sensor-260",
29095 "sizes": {
29096 "active": 244,
29097 "external": 347
29098 }
29099 }
29100 /db/_partition/partition/_all_docs
29102 GET /{db}/_partition/{partition}/_all_docs
29103 Parameters
29105 · db – Database name
29107 · partition – Partition name
29109 This endpoint is a convenience endpoint for automatically set‐
29111 ting bounds on the provided partition range. Similar results can
29112 be had by using the global /db/_all_docs endpoint with appropri‐
29113 ately configured values for start_key and end_key.
29114 Refer to the view endpoint documentation for a complete descrip‐
29116 tion of the available query parameters and the format of the
29117 returned data.
29118 Request:
29120 GET /db/_partition/sensor-260/_all_docs HTTP/1.1
29122 Accept: application/json
29123 Host: localhost:5984
29124 Response:
29126 HTTP/1.1 200 OK
29128 Cache-Control: must-revalidate
29129 Content-Type: application/json
29130 Date: Sat, 10 Aug 2013 16:22:56 GMT
29131 ETag: "1W2DJUZFZSZD9K78UFA3GZWB4"
29132 Server: CouchDB (Erlang/OTP)
29133 Transfer-Encoding: chunked
29134 {
29136 "offset": 0,
29137 "rows": [
29138 {
29139 "id": "sensor-260:sensor-reading-ca33c748-2d2c-4ed1-8abf-1bca4d9d03cf",
29140 "key": "sensor-260:sensor-reading-ca33c748-2d2c-4ed1-8abf-1bca4d9d03cf",
29141 "value": {
29142 "rev": "1-05ed6f7abf84250e213fcb847387f6f5"
29143 }
29144 }
29145 ],
29146 "total_rows": 1
29147 }
29148 /db/_partition/partition/_design/design-doc/_view/view-name
29150 GET /{db}/_partition/{partition}/_design/{ddoc}/_view/{view}
29151 Parameters
29153 · db – Database name
29155 · partition – Partition name
29157 · ddoc – Design document id
29159 · view – View name
29161 This endpoint is responsible for executing a partitioned query.
29163 The returned view result will only contain rows with the speci‐
29164 fied partition name.
29165 Refer to the view endpoint documentation for a complete descrip‐
29167 tion of the available query parameters and the format of the
29168 returned data.
29169 GET /db/_partition/sensor-260/_design/sensor-readings/_view/by_sensor HTTP/1.1
29171 Accept: application/json
29172 Host: localhost:5984
29173 Response:
29175 HTTP/1.1 200 OK
29177 Cache-Control: must-revalidate
29178 Content-Type: application/json
29179 Date: Wed, 21 Aug 2013 09:12:06 GMT
29180 ETag: "2FOLSBSW4O6WB798XU4AQYA9B"
29181 Server: CouchDB (Erlang/OTP)
29182 Transfer-Encoding: chunked
29183 {
29185 "offset": 0,
29186 "rows": [
29187 {
29188 "id": "sensor-260:sensor-reading-ca33c748-2d2c-4ed1-8abf-1bca4d9d03cf",
29189 "key": [
29190 "sensor-260",
29191 "0"
29192 ],
29193 "value": null
29194 },
29195 {
29196 "id": "sensor-260:sensor-reading-ca33c748-2d2c-4ed1-8abf-1bca4d9d03cf",
29197 "key": [
29198 "sensor-260",
29199 "1"
29200 ],
29201 "value": null
29202 },
29203 {
29204 "id": "sensor-260:sensor-reading-ca33c748-2d2c-4ed1-8abf-1bca4d9d03cf",
29205 "key": [
29206 "sensor-260",
29207 "2"
29208 ],
29209 "value": null
29210 },
29211 {
29212 "id": "sensor-260:sensor-reading-ca33c748-2d2c-4ed1-8abf-1bca4d9d03cf",
29213 "key": [
29214 "sensor-260",
29215 "3"
29216 ],
29217 "value": null
29218 }
29219 ],
29220 "total_rows": 4
29221 }
29222 /db/_partition/partition_id/_find
29224 POST /{db}/_partition/{partition_id}/_find
29225 Parameters
29227 · db – Database name
29229 · id (partition) – Name of the partition to query
29231 This endpoint is responsible for finding a partition query by
29233 its ID. The returned view result will only contain rows with
29234 the specified partition id.
29235 Refer to the find endpoint documentation for a complete descrip‐
29237 tion of the available parameters and the format of the returned
29238 data.
29239 /db/_partition/partition_id/_explain
29241 POST /{db}/_partition/{partition_id}/_explain
29242 Parameters
29244 · db – Database name
29246 Partition id
29248 Name of the partition to query
29249 This endpoint shows which index is being used by the query.
29251 Refer to the explain endpoint documentation for a complete
29253 description of the available parameters and the format of the
29254 returned data.
29255 Local (non-replicating) Documents
29257 The Local (non-replicating) document interface allows you to create
29258 local documents that are not replicated to other databases. These docu‐
29259 ments can be used to hold configuration or other information that is
29260 required specifically on the local CouchDB instance.
29261 Local documents have the following limitations:
29263 · Local documents are not replicated to other databases.
29265 · Local documents are not output by views, or the api/db/all_docs view.
29267 From CouchDB 2.0, Local documents can be listed by using the
29269 /db/_local_docs endpoint.
29270 Local documents can be used when you want to store configuration or
29272 other information for the current (local) instance of a given database.
29273 A list of the available methods and URL paths are provided below:
29275 ┌──────────┬─────────────────┬─────────────────────┐
29277 │Method │ Path │ Description │
29278 ├──────────┼─────────────────┼─────────────────────┤
29279 │GET, POST │ /db/_local_docs │ Returns a list of │
29280 │ │ │ all the non-repli‐ │
29281 │ │ │ cated documents in │
29282 │ │ │ the database │
29283 ├──────────┼─────────────────┼─────────────────────┤
29284 │GET │ /db/_local/id │ Returns the latest │
29285 │ │ │ revision of the │
29286 │ │ │ non-replicated doc‐ │
29287 │ │ │ ument │
29288 ├──────────┼─────────────────┼─────────────────────┤
29289 │PUT │ /db/_local/id │ Inserts a new ver‐ │
29290 │ │ │ sion of the │
29291 │ │ │ non-replicated doc‐ │
29292 │ │ │ ument │
29293 └──────────┴─────────────────┴─────────────────────┘
29294 │DELETE │ /db/_local/id │ Deletes the │
29298 │ │ │ non-replicated doc‐ │
29299 │ │ │ ument │
29300 ├──────────┼─────────────────┼─────────────────────┤
29301 │COPY │ /db/_local/id │ Copies the │
29302 │ │ │ non-replicated doc‐ │
29303 │ │ │ ument │
29304 └──────────┴─────────────────┴─────────────────────┘
29305 /db/_local_docs
29307 GET /{db}/_local_docs
29308 Returns a JSON structure of all of the local documents in a
29309 given database. The information is returned as a JSON structure
29310 containing meta information about the return structure, includ‐
29311 ing a list of all local documents and basic contents, consisting
29312 the ID, revision and key. The key is the from the local docu‐
29313 ment’s _id.
29314 Parameters
29316 · db – Database name
29318 Request Headers
29320 · Accept – .INDENT 2.0
29322 · application/json
29324 · text/plain
29326 Query Parameters
29329 · conflicts (boolean) – Includes conflicts information in
29331 response. Ignored if include_docs isn’t true. Default is
29332 false.
29333 · descending (boolean) – Return the local documents in descend‐
29335 ing by key order. Default is false.
29336 · endkey (string) – Stop returning records when the specified
29338 key is reached. Optional.
29339 · end_key (string) – Alias for endkey param.
29341 · endkey_docid (string) – Stop returning records when the speci‐
29343 fied local document ID is reached. Optional.
29344 · end_key_doc_id (string) – Alias for endkey_docid param.
29346 · include_docs (boolean) – Include the full content of the local
29348 documents in the return. Default is false.
29349 · inclusive_end (boolean) – Specifies whether the specified end
29351 key should be included in the result. Default is true.
29352 · key (string) – Return only local documents that match the
29354 specified key. Optional.
29355 · keys (string) – Return only local documents that match the
29357 specified keys. Optional.
29358 · limit (number) – Limit the number of the returned local docu‐
29360 ments to the specified number. Optional.
29361 · skip (number) – Skip this number of records before starting to
29363 return the results. Default is 0.
29364 · startkey (string) – Return records starting with the specified
29366 key. Optional.
29367 · start_key (string) – Alias for startkey param.
29369 · startkey_docid (string) – Return records starting with the
29371 specified local document ID. Optional.
29372 · start_key_doc_id (string) – Alias for startkey_docid param.
29374 · update_seq (boolean) – Response includes an update_seq value
29376 indicating which sequence id of the underlying database the
29377 view reflects. Default is false.
29378 Response Headers
29380 · Content-Type – .INDENT 2.0
29382 · application/json
29384 · text/plain; charset=utf-8
29386 Response JSON Object
29389 · offset (number) – Offset where the local document list started
29391 · rows (array) – Array of view row objects. By default the
29393 information returned contains only the local document ID and
29394 revision.
29395 · total_rows (number) – Number of local documents in the data‐
29397 base. Note that this is not the number of rows returned in the
29398 actual query.
29399 · update_seq (number) – Current update sequence for the database
29401 Status Codes
29403 · 200 OK – Request completed successfully
29405
29407 GET /db/_local_docs HTTP/1.1
29409 Accept: application/json
29410 Host: localhost:5984
29411 Response:
29413 HTTP/1.1 200 OK
29415 Cache-Control: must-revalidate
29416 Content-Type: application/json
29417 Date: Sat, 23 Dec 2017 16:22:56 GMT
29418 Server: CouchDB (Erlang/OTP)
29419 Transfer-Encoding: chunked
29420 {
29422 "offset": null,
29423 "rows": [
29424 {
29425 "id": "_local/localdoc01",
29426 "key": "_local/localdoc01",
29427 "value": {
29428 "rev": "0-1"
29429 }
29430 },
29431 {
29432 "id": "_local/localdoc02",
29433 "key": "_local/localdoc02",
29434 "value": {
29435 "rev": "0-1"
29436 }
29437 },
29438 {
29439 "id": "_local/localdoc03",
29440 "key": "_local/localdoc03",
29441 "value": {
29442 "rev": "0-1"
29443 }
29444 },
29445 {
29446 "id": "_local/localdoc04",
29447 "key": "_local/localdoc04",
29448 "value": {
29449 "rev": "0-1"
29450 }
29451 },
29452 {
29453 "id": "_local/localdoc05",
29454 "key": "_local/localdoc05",
29455 "value": {
29456 "rev": "0-1"
29457 }
29458 }
29459 ],
29460 "total_rows": null
29461 }
29462 POST /{db}/_local_docs
29464 POST _local_docs functionality supports identical parameters and
29465 behavior as specified in the GET /{db}/_local_docs API but
29466 allows for the query string parameters to be supplied as keys in
29467 a JSON object in the body of the POST request.
29468 Request:
29470 POST /db/_local_docs HTTP/1.1
29472 Accept: application/json
29473 Content-Length: 70
29474 Content-Type: application/json
29475 Host: localhost:5984
29476 {
29478 "keys" : [
29479 "_local/localdoc02",
29480 "_local/localdoc05"
29481 ]
29482 }
29483 The returned JSON is the all documents structure, but with only
29485 the selected keys in the output:
29486 {
29488 "total_rows" : null,
29489 "rows" : [
29490 {
29491 "value" : {
29492 "rev" : "0-1"
29493 },
29494 "id" : "_local/localdoc02",
29495 "key" : "_local/localdoc02"
29496 },
29497 {
29498 "value" : {
29499 "rev" : "0-1"
29500 },
29501 "id" : "_local/localdoc05",
29502 "key" : "_local/localdoc05"
29503 }
29504 ],
29505 "offset" : null
29506 }
29507 /db/_local/id
29509 GET /{db}/_local/{docid}
29510 Gets the specified local document. The semantics are identical
29511 to accessing a standard document in the specified database,
29512 except that the document is not replicated. See GET
29513 /{db}/{docid}.
29514 PUT /{db}/_local/{docid}
29516 Stores the specified local document. The semantics are identical
29517 to storing a standard document in the specified database, except
29518 that the document is not replicated. See PUT /{db}/{docid}.
29519 DELETE /{db}/_local/{docid}
29521 Deletes the specified local document. The semantics are identi‐
29522 cal to deleting a standard document in the specified database,
29523 except that the document is not replicated. See DELETE
29524 /{db}/{docid}.
29525 COPY /{db}/_local/{docid}
29527 Copies the specified local document. The semantics are identical
29528 to copying a standard document in the specified database, except
29529 that the document is not replicated. See COPY /{db}/{docid}.
29530
29532 The following appendix provides a quick reference to all the JSON
29533 structures that you can supply to CouchDB, or get in return to
29534 requests.
29535 All Database Documents
29537 ┌──────────────────────┬────────────────────────────┐
29538 │Field │ Description │
29539 ├──────────────────────┼────────────────────────────┤
29540 │total_rows │ Number of documents in the │
29541 │ │ database/view │
29542 ├──────────────────────┼────────────────────────────┤
29543 │offset │ Offset where the document │
29544 │ │ list started │
29545 ├──────────────────────┼────────────────────────────┤
29546 │update_seq (optional) │ Current update sequence │
29547 │ │ for the database │
29548 ├──────────────────────┼────────────────────────────┤
29549 │rows [array] │ Array of document object │
29550 └──────────────────────┴────────────────────────────┘
29551 Bulk Document Response
29553 ┌─────────────┬────────────────────────────┐
29554 │Field │ Description │
29555 ├─────────────┼────────────────────────────┤
29556 │docs [array] │ Bulk Docs Returned Docu‐ │
29557 │ │ ments │
29558 └─────────────┴────────────────────────────┘
29559 │id │ Document ID │
29562 ├─────────────┼────────────────────────────┤
29563 │error │ Error type │
29564 ├─────────────┼────────────────────────────┤
29565 │reason │ Error string with extended │
29566 │ │ reason │
29567 └─────────────┴────────────────────────────┘
29568 Bulk Documents
29570 ┌────────────────────┬────────────────────────────┐
29571 │Field │ Description │
29572 ├────────────────────┼────────────────────────────┤
29573 │docs [array] │ Bulk Documents Document │
29574 ├────────────────────┼────────────────────────────┤
29575 │_id (optional) │ Document ID │
29576 ├────────────────────┼────────────────────────────┤
29577 │_rev (optional) │ Revision ID (when updating │
29578 │ │ an existing document) │
29579 ├────────────────────┼────────────────────────────┤
29580 │_deleted (optional) │ Whether the document │
29581 │ │ should be deleted │
29582 └────────────────────┴────────────────────────────┘
29583 Changes information for a database
29585 ┌────────────────┬────────────────────────────┐
29586 │Field │ Description │
29587 ├────────────────┼────────────────────────────┤
29588 │last_seq │ Last update sequence │
29589 ├────────────────┼────────────────────────────┤
29590 │pending │ Count of remaining items │
29591 │ │ in the feed │
29592 ├────────────────┼────────────────────────────┤
29593 │results [array] │ Changes made to a database │
29594 ├────────────────┼────────────────────────────┤
29595 │seq │ Update sequence │
29596 ├────────────────┼────────────────────────────┤
29597 │id │ Document ID │
29598 ├────────────────┼────────────────────────────┤
29599 │changes [array] │ List of changes, │
29600 │ │ field-by-field, for this │
29601 │ │ document │
29602 └────────────────┴────────────────────────────┘
29603 CouchDB Document
29605 ┌────────────────┬────────────────────────────┐
29606 │Field │ Description │
29607 ├────────────────┼────────────────────────────┤
29608 │_id (optional) │ Document ID │
29609 ├────────────────┼────────────────────────────┤
29610 │_rev (optional) │ Revision ID (when updating │
29611 │ │ an existing document) │
29612 └────────────────┴────────────────────────────┘
29613 CouchDB Error Status
29615 ┌───────┬────────────────────────────┐
29616 │Field │ Description │
29617 ├───────┼────────────────────────────┤
29618 │id │ Document ID │
29619 ├───────┼────────────────────────────┤
29620 │error │ Error type │
29621 ├───────┼────────────────────────────┤
29622 │reason │ Error string with extended │
29623 │ │ reason │
29624 └───────┴────────────────────────────┘
29625 CouchDB database information object
29627 ┌─────────────────────┬────────────────────────────┐
29628 │Field │ Description │
29629 ├─────────────────────┼────────────────────────────┤
29630 │db_name │ The name of the database. │
29631 ├─────────────────────┼────────────────────────────┤
29632 │committed_update_seq │ The number of committed │
29633 │ │ updates. │
29634 ├─────────────────────┼────────────────────────────┤
29635 │doc_count │ The number of documents in │
29636 │ │ the database. │
29637 ├─────────────────────┼────────────────────────────┤
29638 │doc_del_count │ The number of deleted doc‐ │
29639 │ │ uments. │
29640 ├─────────────────────┼────────────────────────────┤
29641 │compact_running │ Set to true if the data‐ │
29642 │ │ base compaction routine is │
29643 │ │ operating on this data‐ │
29644 │ │ base. │
29645 ├─────────────────────┼────────────────────────────┤
29646 │disk_format_version │ The version of the physi‐ │
29647 │ │ cal format used for the │
29648 │ │ data when it is stored on │
29649 │ │ hard disk. │
29650 ├─────────────────────┼────────────────────────────┤
29651 │disk_size │ Size in bytes of the data │
29652 │ │ as stored on disk. View │
29653 │ │ indexes are not included │
29654 │ │ in the calculation. │
29655 ├─────────────────────┼────────────────────────────┤
29656 │instance_start_time │ Timestamp indicating when │
29657 │ │ the database was opened, │
29658 │ │ expressed in microseconds │
29659 │ │ since the epoch. │
29660 ├─────────────────────┼────────────────────────────┤
29661 │purge_seq │ The number of purge opera‐ │
29662 │ │ tions on the database. │
29663 ├─────────────────────┼────────────────────────────┤
29664 │update_seq │ Current update sequence │
29665 │ │ for the database. │
29666 └─────────────────────┴────────────────────────────┘
29667 Design Document
29669 ┌──────────────────┬──────────────────────────┐
29670 │Field │ Description │
29671 ├──────────────────┼──────────────────────────┤
29672 │_id │ Design Document ID │
29673 ├──────────────────┼──────────────────────────┤
29674 │_rev │ Design Document Revision │
29675 ├──────────────────┼──────────────────────────┤
29676 │views │ View │
29677 ├──────────────────┼──────────────────────────┤
29678 │viewname │ View Definition │
29679 ├──────────────────┼──────────────────────────┤
29680 │map │ Map Function for View │
29681 ├──────────────────┼──────────────────────────┤
29682 │reduce (optional) │ Reduce Function for View │
29683 └──────────────────┴──────────────────────────┘
29684 Design Document Information
29686 ┌────────────────┬────────────────────────────┐
29687 │Field │ Description │
29688 ├────────────────┼────────────────────────────┤
29689 │name │ Name/ID of Design Document │
29690 └────────────────┴────────────────────────────┘
29691 │view_index │ View Index │
29694 ├────────────────┼────────────────────────────┤
29695 │compact_running │ Indicates whether a com‐ │
29696 │ │ paction routine is cur‐ │
29697 │ │ rently running on the view │
29698 ├────────────────┼────────────────────────────┤
29699 │disk_size │ Size in bytes of the view │
29700 │ │ as stored on disk │
29701 ├────────────────┼────────────────────────────┤
29702 │language │ Language for the defined │
29703 │ │ views │
29704 ├────────────────┼────────────────────────────┤
29705 │purge_seq │ The purge sequence that │
29706 │ │ has been processed │
29707 ├────────────────┼────────────────────────────┤
29708 │signature │ MD5 signature of the views │
29709 │ │ for the design document │
29710 ├────────────────┼────────────────────────────┤
29711 │update_seq │ The update sequence of the │
29712 │ │ corresponding database │
29713 │ │ that has been indexed │
29714 ├────────────────┼────────────────────────────┤
29715 │updater_running │ Indicates if the view is │
29716 │ │ currently being updated │
29717 ├────────────────┼────────────────────────────┤
29718 │waiting_clients │ Number of clients waiting │
29719 │ │ on views from this design │
29720 │ │ document │
29721 ├────────────────┼────────────────────────────┤
29722 │waiting_commit │ Indicates if there are │
29723 │ │ outstanding commits to the │
29724 │ │ underlying database that │
29725 │ │ need to processed │
29726 └────────────────┴────────────────────────────┘
29727 Document with Attachments
29729 ┌────────────────────────┬────────────────────────────┐
29730 │Field │ Description │
29731 ├────────────────────────┼────────────────────────────┤
29732 │_id (optional) │ Document ID │
29733 ├────────────────────────┼────────────────────────────┤
29734 │_rev (optional) │ Revision ID (when updating │
29735 │ │ an existing document) │
29736 ├────────────────────────┼────────────────────────────┤
29737 │_attachments (optional) │ Document Attachment │
29738 ├────────────────────────┼────────────────────────────┤
29739 │filename │ Attachment information │
29740 ├────────────────────────┼────────────────────────────┤
29741 │content_type │ MIME Content type string │
29742 ├────────────────────────┼────────────────────────────┤
29743 │data │ File attachment content, │
29744 │ │ Base64 encoded │
29745 └────────────────────────┴────────────────────────────┘
29746 List of Active Tasks
29748 ┌──────────────┬─────────────────────┐
29749 │Field │ Description │
29750 ├──────────────┼─────────────────────┤
29751 │tasks [array] │ Active Tasks │
29752 ├──────────────┼─────────────────────┤
29753 │pid │ Process ID │
29754 ├──────────────┼─────────────────────┤
29755 │status │ Task status message │
29756 ├──────────────┼─────────────────────┤
29757 │task │ Task name │
29758 ├──────────────┼─────────────────────┤
29759 │type │ Operation Type │
29760 └──────────────┴─────────────────────┘
29761 Replication Settings
29763 ┌───────────────────────────┬────────────────────────────┐
29764 │Field │ Description │
29765 ├───────────────────────────┼────────────────────────────┤
29766 │source │ Source database name or │
29767 │ │ URL. │
29768 ├───────────────────────────┼────────────────────────────┤
29769 │target │ Target database name or │
29770 │ │ URL. │
29771 ├───────────────────────────┼────────────────────────────┤
29772 │cancel (optional) │ Cancels the replication. │
29773 ├───────────────────────────┼────────────────────────────┤
29774 │checkpoint_interval │ Specifies the checkpoint │
29775 │(optional) │ interval in ms. │
29776 ├───────────────────────────┼────────────────────────────┤
29777 │continuous (optional) │ Configure the replication │
29778 │ │ to be continuous. │
29779 ├───────────────────────────┼────────────────────────────┤
29780 │create_target (optional) │ Creates the target data‐ │
29781 │ │ base. │
29782 ├───────────────────────────┼────────────────────────────┤
29783 │doc_ids (optional) │ Array of document IDs to │
29784 │ │ be synchronized. │
29785 ├───────────────────────────┼────────────────────────────┤
29786 │filter (optional) │ name of the filter func‐ │
29787 │ │ tion in the form of │
29788 │ │ ddoc/myfilter. │
29789 ├───────────────────────────┼────────────────────────────┤
29790 │source_proxy (optional) │ Address of a proxy server │
29791 │ │ through which replication │
29792 │ │ from the source should │
29793 │ │ occur. │
29794 ├───────────────────────────┼────────────────────────────┤
29795 │target_proxy (optional) │ Address of a proxy server │
29796 │ │ through which replication │
29797 │ │ to the target should │
29798 │ │ occur. │
29799 ├───────────────────────────┼────────────────────────────┤
29800 │query_params (optional) │ Query parameter that are │
29801 │ │ passed to the filter func‐ │
29802 │ │ tion; the value should be │
29803 │ │ a document containing │
29804 │ │ parameters as members. │
29805 ├───────────────────────────┼────────────────────────────┤
29806 │selector (optional) │ Select the documents │
29807 │ │ included in the replica‐ │
29808 │ │ tion. This option provides │
29809 │ │ performance benefits com‐ │
29810 │ │ pared with using the fil‐ │
29811 │ │ ter option. │
29812 ├───────────────────────────┼────────────────────────────┤
29813 │since_seq (optional) │ Sequence from which the │
29814 │ │ replication should start. │
29815 ├───────────────────────────┼────────────────────────────┤
29816 │use_checkpoints (optional) │ Whether to use replication │
29817 │ │ checkpoints or not. │
29818 └───────────────────────────┴────────────────────────────┘
29819 Replication Status
29821 ┌───────────────────┬────────────────────────────┐
29822 │Field │ Description │
29823 └───────────────────┴────────────────────────────┘
29824 │ok │ Replication status │
29826 ├───────────────────┼────────────────────────────┤
29827 │session_id │ Unique session ID │
29828 ├───────────────────┼────────────────────────────┤
29829 │source_last_seq │ Last sequence number read │
29830 │ │ from the source database │
29831 ├───────────────────┼────────────────────────────┤
29832 │history [array] │ Replication History │
29833 ├───────────────────┼────────────────────────────┤
29834 │session_id │ Session ID for this repli‐ │
29835 │ │ cation operation │
29836 ├───────────────────┼────────────────────────────┤
29837 │recorded_seq │ Last recorded sequence │
29838 │ │ number │
29839 ├───────────────────┼────────────────────────────┤
29840 │docs_read │ Number of documents read │
29841 ├───────────────────┼────────────────────────────┤
29842 │docs_written │ Number of documents writ‐ │
29843 │ │ ten to target │
29844 ├───────────────────┼────────────────────────────┤
29845 │doc_write_failures │ Number of document write │
29846 │ │ failures │
29847 ├───────────────────┼────────────────────────────┤
29848 │start_time │ Date/Time replication │
29849 │ │ operation started │
29850 ├───────────────────┼────────────────────────────┤
29851 │start_last_seq │ First sequence number in │
29852 │ │ changes stream │
29853 ├───────────────────┼────────────────────────────┤
29854 │end_time │ Date/Time replication │
29855 │ │ operation completed │
29856 ├───────────────────┼────────────────────────────┤
29857 │end_last_seq │ Last sequence number in │
29858 │ │ changes stream │
29859 ├───────────────────┼────────────────────────────┤
29860 │missing_checked │ Number of missing docu‐ │
29861 │ │ ments checked │
29862 ├───────────────────┼────────────────────────────┤
29863 │missing_found │ Number of missing docu‐ │
29864 │ │ ments found │
29865 └───────────────────┴────────────────────────────┘
29866 Request object
29868 ┌───────────────┬────────────────────────────┐
29869 │Field │ Description │
29870 ├───────────────┼────────────────────────────┤
29871 │body │ Request body data as │
29872 │ │ string. If the request │
29873 │ │ method is GET this field │
29874 │ │ contains the value "unde‐ │
29875 │ │ fined". If the method is │
29876 │ │ DELETE or HEAD the value │
29877 │ │ is "" (empty string). │
29878 ├───────────────┼────────────────────────────┤
29879 │cookie │ Cookies object. │
29880 ├───────────────┼────────────────────────────┤
29881 │form │ Form data object. Con‐ │
29882 │ │ tains the decoded body as │
29883 │ │ key-value pairs if the │
29884 │ │ Content-Type header was │
29885 │ │ applica‐ │
29886 │ │ tion/x-www-form-urlen‐ │
29887 │ │ coded. │
29888 ├───────────────┼────────────────────────────┤
29889 │headers │ Request headers object. │
29890 ├───────────────┼────────────────────────────┤
29891 │id │ Requested document id │
29892 │ │ string if it was specified │
29893 │ │ or null otherwise. │
29894 ├───────────────┼────────────────────────────┤
29895 │info │ Database information │
29896 ├───────────────┼────────────────────────────┤
29897 │method │ Request method as string │
29898 │ │ or array. String value is │
29899 │ │ a method as one of: HEAD, │
29900 │ │ GET, POST, PUT, DELETE, │
29901 │ │ OPTIONS, and TRACE. Other‐ │
29902 │ │ wise it will be repre‐ │
29903 │ │ sented as an array of char │
29904 │ │ codes. │
29905 ├───────────────┼────────────────────────────┤
29906 │path │ List of requested path │
29907 │ │ sections. │
29908 ├───────────────┼────────────────────────────┤
29909 │peer │ Request source IP address. │
29910 ├───────────────┼────────────────────────────┤
29911 │query │ URL query parameters │
29912 │ │ object. Note that multi‐ │
29913 │ │ ple keys are not supported │
29914 │ │ and the last key value │
29915 │ │ suppresses others. │
29916 ├───────────────┼────────────────────────────┤
29917 │requested_path │ List of actual requested │
29918 │ │ path section. │
29919 ├───────────────┼────────────────────────────┤
29920 │raw_path │ Raw requested path string. │
29921 ├───────────────┼────────────────────────────┤
29922 │secObj │ Security Object. │
29923 ├───────────────┼────────────────────────────┤
29924 │userCtx │ User Context Object. │
29925 ├───────────────┼────────────────────────────┤
29926 │uuid │ Generated UUID by a speci‐ │
29927 │ │ fied algorithm in the con‐ │
29928 │ │ fig file. │
29929 └───────────────┴────────────────────────────┘
29930 {
29932 "body": "undefined",
29933 "cookie": {
29934 "AuthSession": "cm9vdDo1MDZBRjQzRjrfcuikzPRfAn-EA37FmjyfM8G8Lw",
29935 "m": "3234"
29936 },
29937 "form": {},
29938 "headers": {
29939 "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
29940 "Accept-Charset": "ISO-8859-1,utf-8;q=0.7,*;q=0.3",
29941 "Accept-Encoding": "gzip,deflate,sdch",
29942 "Accept-Language": "en-US,en;q=0.8",
29943 "Connection": "keep-alive",
29944 "Cookie": "m=3234:t|3247:t|6493:t|6967:t|34e2:|18c3:t|2c69:t|5acb:t|ca3:t|c01:t|5e55:t|77cb:t|2a03:t|1d98:t|47ba:t|64b8:t|4a01:t; AuthSession=cm9vdDo1MDZBRjQzRjrfcuikzPRfAn-EA37FmjyfM8G8Lw",
29945 "Host": "127.0.0.1:5984",
29946 "User-Agent": "Mozilla/5.0 (Windows NT 5.2) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.75 Safari/535.7"
29947 },
29948 "id": "foo",
29949 "info": {
29950 "committed_update_seq": 2701412,
29951 "compact_running": false,
29952 "db_name": "mailbox",
29953 "disk_format_version": 6,
29954 "doc_count": 2262757,
29955 "doc_del_count": 560,
29956 "instance_start_time": "1347601025628957",
29957 "purge_seq": 0,
29958 "sizes": {
29959 "active": 7580843252,
29960 "disk": 14325313673,
29961 "external": 7803423459
29962 },
29963 "update_seq": 2701412
29964 },
29965 "method": "GET",
29966 "path": [
29967 "mailbox",
29968 "_design",
29969 "request",
29970 "_show",
29971 "dump",
29972 "foo"
29973 ],
29974 "peer": "127.0.0.1",
29975 "query": {},
29976 "raw_path": "/mailbox/_design/request/_show/dump/foo",
29977 "requested_path": [
29978 "mailbox",
29979 "_design",
29980 "request",
29981 "_show",
29982 "dump",
29983 "foo"
29984 ],
29985 "secObj": {
29986 "admins": {
29987 "names": [
29988 "Bob"
29989 ],
29990 "roles": []
29991 },
29992 "members": {
29993 "names": [
29994 "Mike",
29995 "Alice"
29996 ],
29997 "roles": []
29998 }
29999 },
30000 "userCtx": {
30001 "db": "mailbox",
30002 "name": "Mike",
30003 "roles": [
30004 "user"
30005 ]
30006 },
30007 "uuid": "3184f9d1ea934e1f81a24c71bde5c168"
30008 }
30009 Request2 object
30011 ┌───────────────┬────────────────────────────┐
30012 │Field │ Description │
30013 ├───────────────┼────────────────────────────┤
30014 │body │ Request body data as │
30015 │ │ string. If the request │
30016 │ │ method is GET this field │
30017 │ │ contains the value "unde‐ │
30018 │ │ fined". If the method is │
30019 │ │ DELETE or HEAD the value │
30020 │ │ is "" (empty string). │
30021 ├───────────────┼────────────────────────────┤
30022 │cookie │ Cookies object. │
30023 ├───────────────┼────────────────────────────┤
30024 │headers │ Request headers object. │
30025 ├───────────────┼────────────────────────────┤
30026 │method │ Request method as string │
30027 │ │ or array. String value is │
30028 │ │ a method as one of: HEAD, │
30029 │ │ GET, POST, PUT, DELETE, │
30030 │ │ OPTIONS, and TRACE. Other‐ │
30031 │ │ wise it will be repre‐ │
30032 │ │ sented as an array of char │
30033 │ │ codes. │
30034 ├───────────────┼────────────────────────────┤
30035 │path │ List of requested path │
30036 │ │ sections. │
30037 ├───────────────┼────────────────────────────┤
30038 │peer │ Request source IP address. │
30039 ├───────────────┼────────────────────────────┤
30040 │query │ URL query parameters │
30041 │ │ object. Note that multi‐ │
30042 │ │ ple keys are not supported │
30043 │ │ and the last key value │
30044 │ │ suppresses others. │
30045 ├───────────────┼────────────────────────────┤
30046 │requested_path │ List of actual requested │
30047 │ │ path section. │
30048 ├───────────────┼────────────────────────────┤
30049 │raw_path │ Raw requested path string. │
30050 ├───────────────┼────────────────────────────┤
30051 │secObj │ Security Object. │
30052 ├───────────────┼────────────────────────────┤
30053 │userCtx │ User Context Object. │
30054 └───────────────┴────────────────────────────┘
30055 Response object
30057 ┌────────┬────────────────────────────┐
30058 │Field │ Description │
30059 ├────────┼────────────────────────────┤
30060 │code │ HTTP status code number. │
30061 ├────────┼────────────────────────────┤
30062 │json │ JSON encodable object. │
30063 │ │ Implicitly sets Con‐ │
30064 │ │ tent-Type header as appli‐ │
30065 │ │ cation/json. │
30066 ├────────┼────────────────────────────┤
30067 │body │ Raw response text string. │
30068 │ │ Implicitly sets Con‐ │
30069 │ │ tent-Type header as │
30070 │ │ text/html; charset=utf-8. │
30071 ├────────┼────────────────────────────┤
30072 │base64 │ Base64 encoded string. │
30073 │ │ Implicitly sets Con‐ │
30074 │ │ tent-Type header as appli‐ │
30075 │ │ cation/binary. │
30076 ├────────┼────────────────────────────┤
30077 │headers │ Response headers object. │
30078 │ │ Content-Type header from │
30079 │ │ this object overrides any │
30080 │ │ implicitly assigned one. │
30081 ├────────┼────────────────────────────┤
30082 │stop │ boolean signal to stop │
30083 │ │ iteration over view result │
30084 │ │ rows (for list functions │
30085 │ │ only) │
30086 └────────┴────────────────────────────┘
30087 WARNING:
30089 The body, base64 and json object keys are overlapping each other
30090 where the last one wins. Since most realizations of key-value
30091 objects do not preserve the key order or if they are mixed, confus‐
30092 ing situations can occur. Try to use only one of them.
30093 NOTE:
30095 Any custom property makes CouchDB raise an internal exception. Fur‐
30096 thermore, the Response object could be a simple string value which
30097 would be implicitly wrapped into a {"body": ...} object.
30098 Returned CouchDB Document with Detailed Revision Info
30100 ┌───────────────────┬────────────────────────────┐
30101 │Field │ Description │
30102 ├───────────────────┼────────────────────────────┤
30103 │_id (optional) │ Document ID │
30104 ├───────────────────┼────────────────────────────┤
30105 │_rev (optional) │ Revision ID (when updating │
30106 │ │ an existing document) │
30107 ├───────────────────┼────────────────────────────┤
30108 │_revs_info [array] │ CouchDB document extended │
30109 │ │ revision info │
30110 ├───────────────────┼────────────────────────────┤
30111 │rev │ Full revision string │
30112 ├───────────────────┼────────────────────────────┤
30113 │status │ Status of the revision │
30114 └───────────────────┴────────────────────────────┘
30115 Returned CouchDB Document with Revision Info
30117 ┌────────────────┬────────────────────────────┐
30118 │Field │ Description │
30119 ├────────────────┼────────────────────────────┤
30120 │_id (optional) │ Document ID │
30121 ├────────────────┼────────────────────────────┤
30122 │_rev (optional) │ Revision ID (when updating │
30123 │ │ an existing document) │
30124 ├────────────────┼────────────────────────────┤
30125 │_revisions │ CouchDB document revisions │
30126 ├────────────────┼────────────────────────────┤
30127 │ids [array] │ Array of valid revision │
30128 │ │ IDs, in reverse order │
30129 │ │ (latest first) │
30130 ├────────────────┼────────────────────────────┤
30131 │start │ Prefix number for the lat‐ │
30132 │ │ est revision │
30133 └────────────────┴────────────────────────────┘
30134 Returned Document with Attachments
30136 ┌────────────────────────┬────────────────────────────┐
30137 │Field │ Description │
30138 ├────────────────────────┼────────────────────────────┤
30139 │_id (optional) │ Document ID │
30140 ├────────────────────────┼────────────────────────────┤
30141 │_rev (optional) │ Revision ID (when updating │
30142 │ │ an existing document) │
30143 ├────────────────────────┼────────────────────────────┤
30144 │_attachments (optional) │ Document attachment │
30145 ├────────────────────────┼────────────────────────────┤
30146 │filename │ Attachment │
30147 ├────────────────────────┼────────────────────────────┤
30148 │stub │ Indicates whether the │
30149 │ │ attachment is a stub │
30150 ├────────────────────────┼────────────────────────────┤
30151 │content_type │ MIME Content type string │
30152 ├────────────────────────┼────────────────────────────┤
30153 │length │ Length (bytes) of the │
30154 │ │ attachment data │
30155 ├────────────────────────┼────────────────────────────┤
30156 │revpos │ Revision where this │
30157 │ │ attachment exists │
30158 └────────────────────────┴────────────────────────────┘
30159 Security Object
30161 ┌──────────────┬────────────────────────────┐
30162 │Field │ Description │
30163 ├──────────────┼────────────────────────────┤
30164 │admins │ Roles/Users with admin │
30165 │ │ privileges │
30166 ├──────────────┼────────────────────────────┤
30167 │roles [array] │ List of roles with parent │
30168 │ │ privilege │
30169 ├──────────────┼────────────────────────────┤
30170 │names [array] │ List of users with parent │
30171 │ │ privilege │
30172 ├──────────────┼────────────────────────────┤
30173 │members │ Roles/Users with non-admin │
30174 │ │ privileges │
30175 ├──────────────┼────────────────────────────┤
30176 │roles [array] │ List of roles with parent │
30177 │ │ privilege │
30178 ├──────────────┼────────────────────────────┤
30179 │names [array] │ List of users with parent │
30180 │ │ privilege │
30181 └──────────────┴────────────────────────────┘
30182 {
30184 "admins": {
30185 "names": [
30186 "Bob"
30187 ],
30188 "roles": []
30189 },
30190 "members": {
30191 "names": [
30192 "Mike",
30193 "Alice"
30194 ],
30195 "roles": []
30196 }
30197 }
30198 User Context Object
30200 ┌──────┬────────────────────────────┐
30201 │Field │ Description │
30202 ├──────┼────────────────────────────┤
30203 │db │ Database name in the con‐ │
30204 │ │ text of the provided oper‐ │
30205 │ │ ation. │
30206 ├──────┼────────────────────────────┤
30207 │name │ User name. │
30208 ├──────┼────────────────────────────┤
30209 │roles │ List of user roles. │
30210 └──────┴────────────────────────────┘
30211 {
30213 "db": "mailbox",
30214 "name": null,
30215 "roles": [
30216 "_admin"
30217 ]
30218 }
30219 View Head Information
30221 ──────────────────────────────────────────
30222 Field Description
30223 ──────────────────────────────────────────
30224 total_rows Number of documents in the
30225 view
30226 ──────────────────────────────────────────
30227 offset Offset where the document
30228 list started
30229 ┌───────────┬────────────────────────────┐
30230 │ │ │
30231 { │ │ │
30232 "total_r│ows": 42, │ │
30233 "offset"│: 3 │ │
30234 } │ │ │
30235 │ │ │
30237 The Query serve│r is an exte│rnal process that communicate│s with CouchDB
30238 by JSON protoco│l through st│dio interface and processes a│ll design func‐
30239 tions calls, su│ch as JavaSc│ript views. │
30240 │ │ │
30241 The default que│ry server is│written in JavaScript, runni│ng via Mozilla
30242 SpiderMonkey. │You can u│se other languages by setting│a Query server
30243 key in the lang│uage propert│y of a design document or th│e Content-Type
30244 header of a tem│porary view.│Design documents that do not│specify a lan‐
30245 guage property │are assumed │to be of type javascript. │
30246 │ │ │
30247 Query Server Protoc│ol │ │
30248 A Query Server │is an extern│al process that communicates│ with CouchDB
30249 via a simple,│ custom JS│ON protocol over stdin/stdout│. It is used to
30250 processes all d│esign functi│ons calls: views, shows, l│ists, filters,
30251 updates and val│idate_doc_up│date. │
30252 │ │ │
30253 CouchDB communi│cates with t│he Query Server process throu│gh stdin/stdout
30254 with JSON messa│ges that are│terminated by a newline char│acter. Messages
30255 that are sent t│o the Query │Server are always array-typed│and follow the
30256 pattern [<comma│nd>, <*argum│ents>]\n. │
30257 │ │ │
30258 NOTE: │ │ │
30259 In the docum│entation exa│mples, we omit the trailing \│n for greater
30260 readability.│Also, examp│les contain formatted JSON va│lues while real
30261 data is tran│sferred in c│ompact mode without formattin│g spaces.
30262 │ │ │
30263 reset │ │ │
30264 Command │ │ │
30265 reset │ │ │
30266 │ │ │
30267 Arguments │ │ │
30268 Query se│rver state (│optional) │
30269 │ │ │
30270 Returns │ │ │
30271 true │ │ │
30272 │ │ │
30273 This resets the│state of th│e Query Server and makes it f│orget all pre‐
30274 vious input. │If applicab│le, this is the point to run │garbage collec‐
30275 tion. │ │ │
30276 │ │ │
30277 CouchDB sends: │ │ │
30278 │ │ │
30279 ["reset"] │ │ │
30280 │ │ │
30281 The Query Serve│r answers: │ │
30282 │ │ │
30283 true │ │ │
30284 │ │ │
30285 To set up new Q│uery Server │state, the second argument │is used with
30286 object data. │ │ │
30287 │ │ │
30288 CouchDB sends: │ │ │
30289 │ │ │
30290 ["reset", {"│reduce_limit│": true, "timeout": 5000}] │
30291 │ │ │
30292 The Query Serve│r answers: │ │
30293 │ │ │
30294 true │ │ │
30295 │ │ │
30296 add_lib │ │ │
30297 Command │ │ │
30298 add_lib │ │ │
30299 │ │ │
30300 Arguments │ │ │
30301 CommonJS│library obj│ect by views/lib path │
30302 │ │ │
30303 Returns │ │ │
30304 true │ │ │
30305 │ │ │
30306 Adds CommonJS │library to│Query Server state for furth│er usage in map
30307 functions. │ │ │
30308 │ │ │
30309 CouchDB sends: │ │ │
30310 │ │ │
30311 [ │ │ │
30312 "add_lib│", │ │
30313 { │ │ │
30314 "uti│ls": "export│s.MAGIC = 42;" │
30315 } │ │ │
30316 ] │ │ │
30317 │ │ │
30318 The Query Serve│r answers: │ │
30319 │ │ │
30320 true │ │ │
30321 │ │ │
30322 NOTE: │ │ │
30323 This library│shouldn’t h│ave any side effects nor trac│k its own state
30324 or you’ll │have a lot│ of happy debugging time if│something goes
30325 wrong. Reme│mber that a │complete index rebuild is a h│eavy operation
30326 and this is │the only way│to fix mistakes with shared │state.
30327 │ │ │
30328 add_fun │ │ │
30329 Command │ │ │
30330 add_fun │ │ │
30331 │ │ │
30332 Arguments │ │ │
30333 Map func│tion source │code. │
30334 │ │ │
30335 Returns │ │ │
30336 true │ │ │
30337 │ │ │
30338 When creating │or updating │a view, this is how the Query│Server is sent
30339 the view functi│on for evalu│ation. The Query Server shoul│d parse, com‐
30340 pile, and eval│uate the fun│ction it receives to make it │callable later.
30341 If this fails, │the Query Se│rver returns an error. Couc│hDB may store
30342 multiple functi│ons before s│ending any documents. │
30343 │ │ │
30344 CouchDB sends: │ │ │
30345 │ │ │
30346 [ │ │ │
30347 "add_fun│", │ │
30348 "functio│n(doc) { if(│doc.score > 50) emit(null, {'│player_name': doc.name}); }"
30349 ] │ │ │
30350 │ │ │
30351 The Query Serve│r answers: │ │
30352 │ │ │
30353 true │ │ │
30354 │ │ │
30355 map_doc │ │ │
30356 Command
30357 map_doc
30358 Arguments
30360 Document object
30361 Returns
30363 Array of key-value pairs per applied function
30364 When the view function is stored in the Query Server, CouchDB starts
30366 sending all the documents in the database, one at a time. The Query
30367 Server calls the previously stored functions one after another with a
30368 document and stores its result. When all functions have been called,
30369 the result is returned as a JSON string.
30370 CouchDB sends:
30372 [
30374 "map_doc",
30375 {
30376 "_id": "8877AFF9789988EE",
30377 "_rev": "3-235256484",
30378 "name": "John Smith",
30379 "score": 60
30380 }
30381 ]
30382 If the function above is the only function stored, the Query Server
30384 answers:
30385 [
30387 [
30388 [null, {"player_name": "John Smith"}]
30389 ]
30390 ]
30391 That is, an array with the result for every function for the given doc‐
30393 ument.
30394 If a document is to be excluded from the view, the array should be
30396 empty.
30397 CouchDB sends:
30399 [
30401 "map_doc",
30402 {
30403 "_id": "9590AEB4585637FE",
30404 "_rev": "1-674684684",
30405 "name": "Jane Parker",
30406 "score": 43
30407 }
30408 ]
30409 The Query Server answers:
30411 [[]]
30413 reduce
30415 Command
30416 reduce
30417 Arguments
30419 · Reduce function source
30421 · Array of map function results where each item represented in
30423 format [[key, id-of-doc], value]
30424 Returns
30426 Array with pair values: true and another array with reduced
30427 result
30428 If the view has a reduce function defined, CouchDB will enter into the
30430 reduce phase. The Query Server will receive a list of reduce functions
30431 and some map results on which it can apply them.
30432 CouchDB sends:
30434 [
30436 "reduce",
30437 [
30438 "function(k, v) { return sum(v); }"
30439 ],
30440 [
30441 [[1, "699b524273605d5d3e9d4fd0ff2cb272"], 10],
30442 [[2, "c081d0f69c13d2ce2050d684c7ba2843"], 20],
30443 [[null, "foobar"], 3]
30444 ]
30445 ]
30446 The Query Server answers:
30448 [
30450 true,
30451 [33]
30452 ]
30453 Note that even though the view server receives the map results in the
30455 form [[key, id-of-doc], value], the function may receive them in a dif‐
30456 ferent form. For example, the JavaScript Query Server applies functions
30457 on the list of keys and the list of values.
30458 rereduce
30460 Command
30461 rereduce
30462 Arguments
30464 · Reduce function source
30466 · List of values
30468 When building a view, CouchDB will apply the reduce step directly to
30470 the output of the map step and the rereduce step to the output of a
30471 previous reduce step.
30472 CouchDB will send a list of reduce functions and a list of values, with
30474 no keys or document ids to the rereduce step.
30475 CouchDB sends:
30477 [
30479 "rereduce",
30480 [
30481 "function(k, v, r) { return sum(v); }"
30482 ],
30483 [
30484 33,
30485 55,
30486 66
30487 ]
30488 ]
30489 The Query Server answers:
30491 [
30493 true,
30494 [154]
30495 ]
30496 ddoc
30498 Command
30499 ddoc
30500 Arguments
30502 Array of objects.
30503 · First phase (ddoc initialization):
30505 · "new"
30507 · Design document _id
30509 · Design document object
30511 · Second phase (design function execution):
30513 · Design document _id
30515 · Function path as an array of object keys
30517 · Array of function arguments
30519 Returns
30521 · First phase (ddoc initialization): true
30523 · Second phase (design function execution): custom object
30525 depending on executed function
30526 This command acts in two phases: ddoc registration and design function
30528 execution.
30529 In the first phase CouchDB sends a full design document content to the
30531 Query Server to let it cache it by _id value for further function exe‐
30532 cution.
30533 To do this, CouchDB sends:
30535 [
30537 "ddoc",
30538 "new",
30539 "_design/temp",
30540 {
30541 "_id": "_design/temp",
30542 "_rev": "8-d7379de23a751dc2a19e5638a7bbc5cc",
30543 "language": "javascript",
30544 "shows": {
30545 "request": "function(doc,req){ return {json: req}; }",
30546 "hello": "function(doc,req){ return {body: 'Hello, ' + (doc || {})._id + '!'}; }"
30547 }
30548 }
30549 ]
30550 The Query Server answers:
30552 true
30554 After this, the design document will be ready to serve subcommands in
30556 the second phase.
30557 NOTE:
30559 Each ddoc subcommand is the root design document key, so they are
30560 not actually subcommands, but first elements of the JSON path that
30561 may be handled and processed.
30562 The pattern for subcommand execution is common:
30564 ["ddoc", <design_doc_id>, [<subcommand>, <funcname>], [<argument1>,
30566 <argument2>, ...]]
30567 shows
30569 WARNING:
30570 Show functions are deprecated in CouchDB 3.0, and will be removed in
30571 CouchDB 4.0.
30572 Command
30574 ddoc
30575 SubCommand
30577 shows
30578 Arguments
30580 · Document object or null if document id isn’t specified in
30582 request
30583 · request_object
30585 Returns
30587 Array with two elements:
30588 · "resp"
30590 · response_object
30592 Executes show function.
30594 Couchdb sends:
30596 [
30598 "ddoc",
30599 "_design/temp",
30600 [
30601 "shows",
30602 "doc"
30603 ],
30604 [
30605 null,
30606 {
30607 "info": {
30608 "db_name": "test",
30609 "doc_count": 8,
30610 "doc_del_count": 0,
30611 "update_seq": 105,
30612 "purge_seq": 0,
30613 "compact_running": false,
30614 "sizes": {
30615 "active": 1535048,
30616 "disk": 15818856,
30617 "external": 15515850
30618 },
30619 "instance_start_time": "1359952188595857",
30620 "disk_format_version": 6,
30621 "committed_update_seq": 105
30622 },
30623 "id": null,
30624 "uuid": "169cb4cc82427cc7322cb4463d0021bb",
30625 "method": "GET",
30626 "requested_path": [
30627 "api",
30628 "_design",
30629 "temp",
30630 "_show",
30631 "request"
30632 ],
30633 "path": [
30634 "api",
30635 "_design",
30636 "temp",
30637 "_show",
30638 "request"
30639 ],
30640 "raw_path": "/api/_design/temp/_show/request",
30641 "query": {},
30642 "headers": {
30643 "Accept": "*/*",
30644 "Host": "localhost:5984",
30645 "User-Agent": "curl/7.26.0"
30646 },
30647 "body": "undefined",
30648 "peer": "127.0.0.1",
30649 "form": {},
30650 "cookie": {},
30651 "userCtx": {
30652 "db": "api",
30653 "name": null,
30654 "roles": [
30655 "_admin"
30656 ]
30657 },
30658 "secObj": {}
30659 }
30660 ]
30661 ]
30662 The Query Server sends:
30664 [
30666 "resp",
30667 {
30668 "body": "Hello, undefined!"
30669 }
30670 ]
30671 lists
30673 WARNING:
30674 List functions are deprecated in CouchDB 3.0, and will be removed in
30675 CouchDB 4.0.
30676 Command
30678 ddoc
30679 SubCommand
30681 lists
30682 Arguments
30684 · view_head_info_object:
30686 · request_object
30688 Returns
30690 Array. See below for details.
30691 Executes list function.
30693 The communication protocol for list functions is a bit complex so let’s
30695 use an example to illustrate.
30696 Assume we have view a function that emits id-rev pairs:
30698 function(doc) {
30700 emit(doc._id, doc._rev);
30701 }
30702 And we’d like to emulate _all_docs JSON response with list function.
30704 Our first version of the list functions looks like this:
30705 function(head, req){
30707 start({'headers': {'Content-Type': 'application/json'}});
30708 var resp = head;
30709 var rows = [];
30710 while(row=getRow()){
30711 rows.push(row);
30712 }
30713 resp.rows = rows;
30714 return toJSON(resp);
30715 }
30716 The whole communication session during list function execution could be
30718 divided on three parts:
30719 1. Initialization
30721 The first returned object from the list function is an array with
30723 the following structure:
30724 ["start", <chunks>, <headers>]
30726 Where <chunks> is an array of text chunks that will be sent to the
30728 client and <headers> is an object with response HTTP headers.
30729 This message is sent from the Query Server to CouchDB on the start()
30731 call which initializes the HTTP response to the client:
30732 [
30734 "start",
30735 [],
30736 {
30737 "headers": {
30738 "Content-Type": "application/json"
30739 }
30740 }
30741 ]
30742 After this, the list function may start to process view rows.
30744 2. View Processing
30746 Since view results can be extremely large, it is not wise to pass
30748 all its rows in a single command. Instead, CouchDB can send view
30749 rows one by one to the Query Server allowing view processing and
30750 output generation to be processed as a stream.
30751 CouchDB sends a special array that carries view row data:
30753 [
30755 "list_row",
30756 {
30757 "id": "0cb42c267fe32d4b56b3500bc503e030",
30758 "key": "0cb42c267fe32d4b56b3500bc503e030",
30759 "value": "1-967a00dff5e02add41819138abb3284d"
30760 }
30761 ]
30762 If the Query Server has something to return on this, it returns an
30764 array with a "chunks" item in the head and an array of data in the
30765 tail. For this example it has nothing to return, so the response
30766 will be:
30767 [
30769 "chunks",
30770 []
30771 ]
30772 When there are no more view rows to process, CouchDB sends a
30774 list_end message to signify there is no more data to send:
30775 ["list_end"]
30777 3. Finalization
30779 The last stage of the communication process is the returning list
30781 tail: the last data chunk. After this, processing of the list func‐
30782 tion will be complete and the client will receive a complete
30783 response.
30784 For our example the last message is:
30786 [
30788 "end",
30789 [
30790 "{\"total_rows\":2,\"offset\":0,\"rows\":[{\"id\":\"0cb42c267fe32d4b56b3500bc503e030\",\"key\":\"0cb42c267fe32d4b56b3500bc503e030\",\"value\":\"1-967a00dff5e02add41819138abb3284d\"},{\"id\":\"431926a69504bde41851eb3c18a27b1f\",\"key\":\"431926a69504bde41851eb3c18a27b1f\",\"value\":\"1-967a00dff5e02add41819138abb3284d\"}]}"
30791 ]
30792 ]
30793 In this example, we have returned our result in a single message from
30795 the Query Server. This is okay for small numbers of rows, but for large
30796 data sets, perhaps with millions of documents or millions of view rows,
30797 this would not be acceptable.
30798 Let’s fix our list function and see the changes in communication:
30800 function(head, req){
30802 start({'headers': {'Content-Type': 'application/json'}});
30803 send('{');
30804 send('"total_rows":' + toJSON(head.total_rows) + ',');
30805 send('"offset":' + toJSON(head.offset) + ',');
30806 send('"rows":[');
30807 if (row=getRow()){
30808 send(toJSON(row));
30809 }
30810 while(row=getRow()){
30811 send(',' + toJSON(row));
30812 }
30813 send(']');
30814 return '}';
30815 }
30816 “Wait, what?” - you’d like to ask. Yes, we’d build JSON response manu‐
30818 ally by string chunks, but let’s take a look on logs:
30819 [Wed, 24 Jul 2013 05:45:30 GMT] [debug] [<0.19191.1>] OS Process #Port<0.4444> Output :: ["start",["{","\"total_rows\":2,","\"offset\":0,","\"rows\":["],{"headers":{"Content-Type":"application/json"}}]
30821 [Wed, 24 Jul 2013 05:45:30 GMT] [info] [<0.18963.1>] 127.0.0.1 - - GET /blog/_design/post/_list/index/all_docs 200
30822 [Wed, 24 Jul 2013 05:45:30 GMT] [debug] [<0.19191.1>] OS Process #Port<0.4444> Input :: ["list_row",{"id":"0cb42c267fe32d4b56b3500bc503e030","key":"0cb42c267fe32d4b56b3500bc503e030","value":"1-967a00dff5e02add41819138abb3284d"}]
30823 [Wed, 24 Jul 2013 05:45:30 GMT] [debug] [<0.19191.1>] OS Process #Port<0.4444> Output :: ["chunks",["{\"id\":\"0cb42c267fe32d4b56b3500bc503e030\",\"key\":\"0cb42c267fe32d4b56b3500bc503e030\",\"value\":\"1-967a00dff5e02add41819138abb3284d\"}"]]
30824 [Wed, 24 Jul 2013 05:45:30 GMT] [debug] [<0.19191.1>] OS Process #Port<0.4444> Input :: ["list_row",{"id":"431926a69504bde41851eb3c18a27b1f","key":"431926a69504bde41851eb3c18a27b1f","value":"1-967a00dff5e02add41819138abb3284d"}]
30825 [Wed, 24 Jul 2013 05:45:30 GMT] [debug] [<0.19191.1>] OS Process #Port<0.4444> Output :: ["chunks",[",{\"id\":\"431926a69504bde41851eb3c18a27b1f\",\"key\":\"431926a69504bde41851eb3c18a27b1f\",\"value\":\"1-967a00dff5e02add41819138abb3284d\"}"]]
30826 [Wed, 24 Jul 2013 05:45:30 GMT] [debug] [<0.19191.1>] OS Process #Port<0.4444> Input :: ["list_end"]
30827 [Wed, 24 Jul 2013 05:45:30 GMT] [debug] [<0.19191.1>] OS Process #Port<0.4444> Output :: ["end",["]","}"]]
30828 Note, that now the Query Server sends response by lightweight chunks
30830 and if our communication process was extremely slow, the client will
30831 see how response data appears on their screen. Chunk by chunk, without
30832 waiting for the complete result, like they have for our previous list
30833 function.
30834 updates
30836 Command
30837 ddoc
30838 SubCommand
30840 updates
30841 Arguments
30843 · Document object or null if document id wasn’t specified in
30845 request
30846 · request_object
30848 Returns
30850 Array with there elements:
30851 · "up"
30853 · Document object or null if nothing should be stored
30855 · response_object
30857 Executes update function.
30859 CouchDB sends:
30861 [
30863 "ddoc",
30864 "_design/id",
30865 [
30866 "updates",
30867 "nothing"
30868 ],
30869 [
30870 null,
30871 {
30872 "info": {
30873 "db_name": "test",
30874 "doc_count": 5,
30875 "doc_del_count": 0,
30876 "update_seq": 16,
30877 "purge_seq": 0,
30878 "compact_running": false,
30879 "sizes": {
30880 "active": 7979745,
30881 "disk": 8056936,
30882 "external": 8024930
30883 },
30884 "instance_start_time": "1374612186131612",
30885 "disk_format_version": 6,
30886 "committed_update_seq": 16
30887 },
30888 "id": null,
30889 "uuid": "7b695cb34a03df0316c15ab529002e69",
30890 "method": "POST",
30891 "requested_path": [
30892 "test",
30893 "_design",
30894 "1139",
30895 "_update",
30896 "nothing"
30897 ],
30898 "path": [
30899 "test",
30900 "_design",
30901 "1139",
30902 "_update",
30903 "nothing"
30904 ],
30905 "raw_path": "/test/_design/1139/_update/nothing",
30906 "query": {},
30907 "headers": {
30908 "Accept": "*/*",
30909 "Accept-Encoding": "identity, gzip, deflate, compress",
30910 "Content-Length": "0",
30911 "Host": "localhost:5984"
30912 },
30913 "body": "",
30914 "peer": "127.0.0.1",
30915 "form": {},
30916 "cookie": {},
30917 "userCtx": {
30918 "db": "test",
30919 "name": null,
30920 "roles": [
30921 "_admin"
30922 ]
30923 },
30924 "secObj": {}
30925 }
30926 ]
30927 ]
30928 The Query Server answers:
30930 [
30932 "up",
30933 null,
30934 {"body": "document id wasn't provided"}
30935 ]
30936 or in case of successful update:
30938 [
30940 "up",
30941 {
30942 "_id": "7b695cb34a03df0316c15ab529002e69",
30943 "hello": "world!"
30944 },
30945 {"body": "document was updated"}
30946 ]
30947 filters
30949 Command
30950 ddoc
30951 SubCommand
30953 filters
30954 Arguments
30956 · Array of document objects
30958 · request_object
30960 Returns
30962 Array of two elements:
30963 · true
30965 · Array of booleans in the same order of input documents.
30967 Executes filter function.
30969 CouchDB sends:
30971 [
30973 "ddoc",
30974 "_design/test",
30975 [
30976 "filters",
30977 "random"
30978 ],
30979 [
30980 [
30981 {
30982 "_id": "431926a69504bde41851eb3c18a27b1f",
30983 "_rev": "1-967a00dff5e02add41819138abb3284d",
30984 "_revisions": {
30985 "start": 1,
30986 "ids": [
30987 "967a00dff5e02add41819138abb3284d"
30988 ]
30989 }
30990 },
30991 {
30992 "_id": "0cb42c267fe32d4b56b3500bc503e030",
30993 "_rev": "1-967a00dff5e02add41819138abb3284d",
30994 "_revisions": {
30995 "start": 1,
30996 "ids": [
30997 "967a00dff5e02add41819138abb3284d"
30998 ]
30999 }
31000 }
31001 ],
31002 {
31003 "info": {
31004 "db_name": "test",
31005 "doc_count": 5,
31006 "doc_del_count": 0,
31007 "update_seq": 19,
31008 "purge_seq": 0,
31009 "compact_running": false,
31010 "sizes": {
31011 "active": 7979745,
31012 "disk": 8056936,
31013 "external": 8024930
31014 },
31015 "instance_start_time": "1374612186131612",
31016 "disk_format_version": 6,
31017 "committed_update_seq": 19
31018 },
31019 "id": null,
31020 "uuid": "7b695cb34a03df0316c15ab529023a81",
31021 "method": "GET",
31022 "requested_path": [
31023 "test",
31024 "_changes?filter=test",
31025 "random"
31026 ],
31027 "path": [
31028 "test",
31029 "_changes"
31030 ],
31031 "raw_path": "/test/_changes?filter=test/random",
31032 "query": {
31033 "filter": "test/random"
31034 },
31035 "headers": {
31036 "Accept": "application/json",
31037 "Accept-Encoding": "identity, gzip, deflate, compress",
31038 "Content-Length": "0",
31039 "Content-Type": "application/json; charset=utf-8",
31040 "Host": "localhost:5984"
31041 },
31042 "body": "",
31043 "peer": "127.0.0.1",
31044 "form": {},
31045 "cookie": {},
31046 "userCtx": {
31047 "db": "test",
31048 "name": null,
31049 "roles": [
31050 "_admin"
31051 ]
31052 },
31053 "secObj": {}
31054 }
31055 ]
31056 ]
31057 The Query Server answers:
31059 [
31061 true,
31062 [
31063 true,
31064 false
31065 ]
31066 ]
31067 views
31069 Command
31070 ddoc
31071 SubCommand
31073 views
31074 Arguments
31076 Array of document objects
31077 Returns
31079 Array of two elements:
31080 · true
31082 · Array of booleans in the same order of input documents.
31084 New in version 1.2.
31086 Executes view function in place of the filter.
31089 Acts in the same way as filters command.
31091 validate_doc_update
31093 Command
31094 ddoc
31095 SubCommand
31097 validate_doc_update
31098 Arguments
31100 · Document object that will be stored
31102 · Document object that will be replaced
31104 · userctx_object
31106 · security_object
31108 Returns
31110 1
31111 Executes validation function.
31113 CouchDB send:
31115 [
31117 "ddoc",
31118 "_design/id",
31119 ["validate_doc_update"],
31120 [
31121 {
31122 "_id": "docid",
31123 "_rev": "2-e0165f450f6c89dc6b071c075dde3c4d",
31124 "score": 10
31125 },
31126 {
31127 "_id": "docid",
31128 "_rev": "1-9f798c6ad72a406afdbf470b9eea8375",
31129 "score": 4
31130 },
31131 {
31132 "name": "Mike",
31133 "roles": ["player"]
31134 },
31135 {
31136 "admins": {},
31137 "members": []
31138 }
31139 ]
31140 ]
31141 The Query Server answers:
31143 1
31145 NOTE:
31147 While the only valid response for this command is true, to prevent
31148 the document from being saved, the Query Server needs to raise an
31149 error: forbidden or unauthorized; these errors will be turned into
31150 correct HTTP 403 and HTTP 401 responses respectively.
31151 rewrites
31153 Command
31154 ddoc
31155 SubCommand
31157 rewrites
31158 Arguments
31160 · request2_object
31162 Returns
31164 1
31165 Executes rewrite function.
31167 CouchDB send:
31169 [
31171 "ddoc",
31172 "_design/id",
31173 ["rewrites"],
31174 [
31175 {
31176 "method": "POST",
31177 "requested_path": [
31178 "test",
31179 "_design",
31180 "1139",
31181 "_update",
31182 "nothing"
31183 ],
31184 "path": [
31185 "test",
31186 "_design",
31187 "1139",
31188 "_update",
31189 "nothing"
31190 ],
31191 "raw_path": "/test/_design/1139/_update/nothing",
31192 "query": {},
31193 "headers": {
31194 "Accept": "*/*",
31195 "Accept-Encoding": "identity, gzip, deflate, compress",
31196 "Content-Length": "0",
31197 "Host": "localhost:5984"
31198 },
31199 "body": "",
31200 "peer": "127.0.0.1",
31201 "cookie": {},
31202 "userCtx": {
31203 "db": "test",
31204 "name": null,
31205 "roles": [
31206 "_admin"
31207 ]
31208 },
31209 "secObj": {}
31210 }
31211 ]
31212 ]
31213 The Query Server answers:
31215 [
31217 "ok",
31218 {
31219 "path": "some/path",
31220 "query": {"key1": "value1", "key2": "value2"},
31221 "method": "METHOD",
31222 "headers": {"Header1": "value1", "Header2": "value2"},
31223 "body": ""
31224 }
31225 ]
31226 or in case of direct response:
31228 [
31230 "ok",
31231 {
31232 "headers": {"Content-Type": "text/plain"},
31233 "body": "Welcome!",
31234 "code": 200
31235 }
31236 ]
31237 or for immediate redirect:
31239 [
31241 "ok",
31242 {
31243 "headers": {"Location": "http://example.com/path/"},
31244 "code": 302
31245 }
31246 ]
31247 Returning errors
31249 When something goes wrong, the Query Server can inform CouchDB by send‐
31250 ing a special message in response to the received command.
31251 Error messages prevent further command execution and return an error
31253 description to CouchDB. Errors are logically divided into two groups:
31254 · Common errors. These errors only break the current Query Server com‐
31256 mand and return the error info to the CouchDB instance without termi‐
31257 nating the Query Server process.
31258 · Fatal errors. Fatal errors signal a condition that cannot be recov‐
31260 ered. For instance, if your a design function is unable to import a
31261 third party module, it’s better to count such error as fatal and ter‐
31262 minate whole process.
31263 error
31265 To raise an error, the Query Server should respond with:
31266 ["error", "error_name", "reason why"]
31268 The "error_name" helps to classify problems by their type e.g. if it’s
31270 "value_error" to indicate improper data, "not_found" to indicate a
31271 missing resource and "type_error" to indicate an improper data type.
31272 The "reason why" explains in human-readable terms what went wrong, and
31274 possibly how to resolve it.
31275 For example, calling updatefun against a non-existent document could
31277 produce the error message:
31278 ["error", "not_found", "Update function requires existent document"]
31280 forbidden
31282 The forbidden error is widely used by vdufun to stop further function
31283 processing and prevent storage of the new document revision. Since this
31284 is not actually an error, but an assertion against user actions,
31285 CouchDB doesn’t log it at “error” level, but returns HTTP 403 Forbidden
31286 response with error information object.
31287 To raise this error, the Query Server should respond with:
31289 {"forbidden": "reason why"}
31291 unauthorized
31293 The unauthorized error mostly acts like forbidden one, but with the
31294 meaning of please authorize first. This small difference helps end
31295 users to understand what they can do to solve the problem. Similar to
31296 forbidden, CouchDB doesn’t log it at “error” level, but returns a HTTP
31297 401 Unauthorized response with an error information object.
31298 To raise this error, the Query Server should respond with:
31300 {"unauthorized": "reason why"}
31302 Logging
31304 At any time, the Query Server may send some information that will be
31305 saved in CouchDB’s log file. This is done by sending a special log
31306 object with a single argument, on a separate line:
31307 ["log", "some message"]
31309 CouchDB does not respond, but writes the received message to the log
31311 file:
31312 [Sun, 13 Feb 2009 23:31:30 GMT] [info] [<0.72.0>] Query Server Log Message: some message
31314 These messages are only logged at info level.
31316 JavaScript
31318 NOTE:
31319 While every design function has access to all JavaScript objects,
31320 the table below describes appropriate usage cases. For example, you
31321 may use emit() in mapfun, but getRow() is not permitted during map‐
31322 fun.
31323 ┌───────────────┬────────────────────────────┐
31325 │JS Function │ Reasonable to use in │
31326 │ │ design doc functions │
31327 ├───────────────┼────────────────────────────┤
31328 │emit() │ mapfun │
31329 ├───────────────┼────────────────────────────┤
31330 │getRow() │ listfun │
31331 ├───────────────┼────────────────────────────┤
31332 │JSON │ any │
31333 ├───────────────┼────────────────────────────┤
31334 │isArray() │ any │
31335 ├───────────────┼────────────────────────────┤
31336 │log() │ any │
31337 ├───────────────┼────────────────────────────┤
31338 │provides() │ showfun, listfun │
31339 ├───────────────┼────────────────────────────┤
31340 │registerType() │ showfun, listfun │
31341 ├───────────────┼────────────────────────────┤
31342 │require() │ any, except reducefun │
31343 ├───────────────┼────────────────────────────┤
31344 │send() │ listfun │
31345 ├───────────────┼────────────────────────────┤
31346 │start() │ listfun │
31347 ├───────────────┼────────────────────────────┤
31348 │sum() │ any │
31349 ├───────────────┼────────────────────────────┤
31350 │toJSON() │ any │
31351 └───────────────┴────────────────────────────┘
31352 Design functions context
31354 Each design function executes in a special context of predefined
31355 objects, modules and functions:
31356 emit(key, value)
31358 Emits a key-value pair for further processing by CouchDB after
31359 the map function is done.
31360 Arguments
31362 · key – The view key
31364 · value – The key’s associated value
31366 function(doc){
31368 emit(doc._id, doc._rev);
31369 }
31370 getRow()
31372 Extracts the next row from a related view result.
31373 Returns
31375 View result row
31376 Return type
31378 object
31379 function(head, req){
31381 send('[');
31382 row = getRow();
31383 if (row){
31384 send(toJSON(row));
31385 while(row = getRow()){
31386 send(',');
31387 send(toJSON(row));
31388 }
31389 }
31390 return ']';
31391 }
31392 JSON JSON2 object.
31394 isArray(obj)
31396 A helper function to check if the provided value is an Array.
31397 Arguments
31399 · obj – Any JavaScript value
31401 Returns
31403 true if obj is Array-typed, false otherwise
31404 Return type
31406 boolean
31407 log(message)
31409 Log a message to the CouchDB log (at the INFO level).
31410 Arguments
31412 · message – Message to be logged
31414 function(doc){
31416 log('Procesing doc ' + doc['_id']);
31417 emit(doc['_id'], null);
31418 }
31419 After the map function has run, the following line can be found
31421 in CouchDB logs (e.g. at /var/log/couchdb/couch.log):
31422 [Sat, 03 Nov 2012 17:38:02 GMT] [info] [<0.7543.0>] OS Process #Port<0.3289> Log :: Processing doc 8d300b86622d67953d102165dbe99467
31424 provides(key, func)
31426 Registers callable handler for specified MIME key.
31427 Arguments
31429 · key – MIME key previously defined by registerType()
31431 · func – MIME type handler
31433 registerType(key, *mimes)
31435 Registers list of MIME types by associated key.
31436 Arguments
31438 · key – MIME types
31440 · mimes – MIME types enumeration
31442 Predefined mappings (key-array):
31444 · all: */*
31446 · text: text/plain; charset=utf-8, txt
31448 · html: text/html; charset=utf-8
31450 · xhtml: application/xhtml+xml, xhtml
31452 · xml: application/xml, text/xml, application/x-xml
31454 · js: text/javascript, application/javascript, applica‐
31456 tion/x-javascript
31457 · css: text/css
31459 · ics: text/calendar
31461 · csv: text/csv
31463 · rss: application/rss+xml
31465 · atom: application/atom+xml
31467 · yaml: application/x-yaml, text/yaml
31469 · multipart_form: multipart/form-data
31471 · url_encoded_form: application/x-www-form-urlencoded
31473 · json: application/json, text/x-json
31475 require(path)
31477 Loads CommonJS module by a specified path. The path should not
31478 start with a slash.
31479 Arguments
31481 · path – A CommonJS module path started from design docu‐
31483 ment root
31484 Returns
31486 Exported statements
31487 send(chunk)
31489 Sends a single string chunk in response.
31490 Arguments
31492 · chunk – Text chunk
31494 function(head, req){
31496 send('Hello,');
31497 send(' ');
31498 send('Couch');
31499 return ;
31500 }
31501 start(init_resp)
31503 Initiates chunked response. As an option, a custom response
31504 object may be sent at this point. For list-functions only!
31505 NOTE:
31507 list functions may set the HTTP response code and headers by
31508 calling this function. This function must be called before
31509 send(), getRow() or a return statement; otherwise, the query
31510 server will implicitly call this function with the empty
31511 object ({}).
31512 function(head, req){
31514 start({
31515 "code": 302,
31516 "headers": {
31517 "Location": "http://couchdb.apache.org"
31518 }
31519 });
31520 return "Relax!";
31521 }
31522 sum(arr)
31524 Sum arr’s items.
31525 Arguments
31527 · arr – Array of numbers
31529 Return type
31531 number
31532 toJSON(obj)
31534 Encodes obj to JSON string. This is an alias for the
31535 JSON.stringify method.
31536 Arguments
31538 · obj – JSON-encodable object
31540 Returns
31542 JSON string
31543 CommonJS Modules
31545 Support for CommonJS Modules (introduced in CouchDB 0.11.0) allows you
31546 to create modular design functions without the need for duplication of
31547 functionality.
31548 Here’s a CommonJS module that checks user permissions:
31550 function user_context(userctx, secobj) {
31552 var is_admin = function() {
31553 return userctx.indexOf('_admin') != -1;
31554 }
31555 return {'is_admin': is_admin}
31556 }
31557 exports['user'] = user_context
31559 Each module has access to additional global variables:
31561 · module (object): Contains information about the stored module
31563 · id (string): The module id; a JSON path in ddoc context
31565 · current (code): Compiled module code object
31567 · parent (object): Parent frame
31569 · exports (object): Export statements
31571 · exports (object): Shortcut to the module.exports object
31573 The CommonJS module can be added to a design document, like so:
31575 {
31577 "views": {
31578 "lib": {
31579 "security": "function user_context(userctx, secobj) { ... }"
31580 }
31581 },
31582 "validate_doc_update": "function(newdoc, olddoc, userctx, secobj) {
31583 user = require('views/lib/security').user_context(userctx, secobj);
31584 return user.is_admin();
31585 }"
31586 "_id": "_design/test"
31587 }
31588 Modules paths are relative to the design document’s views object, but
31590 modules can only be loaded from the object referenced via lib. The lib
31591 structure can still be used for view functions as well, by simply stor‐
31592 ing view functions at e.g. views.lib.map, views.lib.reduce, etc.
31593 Erlang
31595 NOTE:
31596 The Erlang query server is disabled by default. Read configuration
31597 guide about reasons why and how to enable it.
31598 Emit(Id, Value)
31600 Emits key-value pairs to view indexer process.
31601 fun({Doc}) ->
31603 <<K,_/binary>> = proplists:get_value(<<"_rev">>, Doc, null),
31604 V = proplists:get_value(<<"_id">>, Doc, null),
31605 Emit(<<K>>, V)
31606 end.
31607 FoldRows(Fun, Acc)
31609 Helper to iterate over all rows in a list function.
31610 Arguments
31612 · Fun – Function object.
31614 · Acc – The value previously returned by Fun.
31616 fun(Head, {Req}) ->
31618 Fun = fun({Row}, Acc) ->
31619 Id = couch_util:get_value(<<"id">>, Row),
31620 Send(list_to_binary(io_lib:format("Previous doc id: ~p~n", [Acc]))),
31621 Send(list_to_binary(io_lib:format("Current doc id: ~p~n", [Id]))),
31622 {ok, Id}
31623 end,
31624 FoldRows(Fun, nil),
31625 ""
31626 end.
31627 GetRow()
31629 Retrieves the next row from a related view result.
31630 %% FoldRows background implementation.
31632 %% https://git-wip-us.apache.org/repos/asf?p=couchdb.git;a=blob;f=src/couchdb/couch_native_process.erl;hb=HEAD#l368
31633 %%
31634 foldrows(GetRow, ProcRow, Acc) ->
31635 case GetRow() of
31636 nil ->
31637 {ok, Acc};
31638 Row ->
31639 case (catch ProcRow(Row, Acc)) of
31640 {ok, Acc2} ->
31641 foldrows(GetRow, ProcRow, Acc2);
31642 {stop, Acc2} ->
31643 {ok, Acc2}
31644 end
31645 end.
31646 Log(Msg)
31648 Arguments
31650 · Msg – Log a message at the INFO level.
31652 fun({Doc}) ->
31654 <<K,_/binary>> = proplists:get_value(<<"_rev">>, Doc, null),
31655 V = proplists:get_value(<<"_id">>, Doc, null),
31656 Log(lists:flatten(io_lib:format("Hello from ~s doc!", [V]))),
31657 Emit(<<K>>, V)
31658 end.
31659 After the map function has run, the following line can be found
31661 in CouchDB logs (e.g. at /var/log/couchdb/couch.log):
31662 [Sun, 04 Nov 2012 11:33:58 GMT] [info] [<0.9144.2>] Hello from 8d300b86622d67953d102165dbe99467 doc!
31664 Send(Chunk)
31666 Sends a single string Chunk in response.
31667 fun(Head, {Req}) ->
31669 Send("Hello,"),
31670 Send(" "),
31671 Send("Couch"),
31672 "!"
31673 end.
31674 The function above produces the following response:
31676 Hello, Couch!
31678 Start(Headers)
31680 Arguments
31682 · Headers – Proplist of response object.
31684 Initialize listfun response. At this point, response code and
31686 headers may be defined. For example, this function redirects to
31687 the CouchDB web site:
31688 fun(Head, {Req}) ->
31690 Start({[{<<"code">>, 302},
31691 {<<"headers">>, {[
31692 {<<"Location">>, <<"http://couchdb.apache.org">>}]
31693 }}
31694 ]}),
31695 "Relax!"
31696 end.
31697
31699 A partitioned database forms documents into logical partitions by using
31700 a partition key. All documents are assigned to a partition, and many
31701 documents are typically given the same partition key. The benefit of
31702 partitioned databases is that secondary indices can be significantly
31703 more efficient when locating matching documents since their entries are
31704 contained within their partition. This means a given secondary index
31705 read will only scan a single partition range instead of having to read
31706 from a copy of every shard.
31707 As a means to introducing partitioned databases, we’ll consider a moti‐
31709 vating use case to describe the benefits of this feature. For this
31710 example, we’ll consider a database that stores readings from a large
31711 network of soil moisture sensors.
31712 NOTE:
31714 Before reading this document you should be familiar with the theory
31715 of sharding in CouchDB.
31716 Traditionally, a document in this database may have something like the
31718 following structure:
31719 {
31721 "_id": "sensor-reading-ca33c748-2d2c-4ed1-8abf-1bca4d9d03cf",
31722 "_rev":"1-14e8f3262b42498dbd5c672c9d461ff0",
31723 "sensor_id": "sensor-260",
31724 "location": [41.6171031, -93.7705674],
31725 "field_name": "Bob's Corn Field #5",
31726 "readings": [
31727 ["2019-01-21T00:00:00", 0.15],
31728 ["2019-01-21T06:00:00", 0.14],
31729 ["2019-01-21T12:00:00", 0.16],
31730 ["2019-01-21T18:00:00", 0.11]
31731 ]
31732 }
31733 NOTE:
31735 While this example uses IoT sensors, the main thing to consider is
31736 that there is a logical grouping of documents. Similar use cases
31737 might be documents grouped by user or scientific data grouped by
31738 experiment.
31739 So we’ve got a bunch of sensors, all grouped by the field they monitor
31741 along with their readouts for a given day (or other appropriate time
31742 period).
31743 Along with our documents, we might expect to have two secondary indexes
31745 for querying our database that might look something like:
31746 function(doc) {
31748 if(doc._id.indexOf("sensor-reading-") != 0) {
31749 return;
31750 }
31751 for(var r in doc.readings) {
31752 emit([doc.sensor_id, r[0]], r[1])
31753 }
31754 }
31755 and:
31757 function(doc) {
31759 if(doc._id.indexOf("sensor-reading-") != 0) {
31760 return;
31761 }
31762 emit(doc.field_name, doc.sensor_id)
31763 }
31764 With these two indexes defined, we can easily find all readings for a
31766 given sensor, or list all sensors in a given field.
31767 Unfortunately, in CouchDB, when we read from either of these indexes,
31769 it requires finding a copy of every shard and asking for any documents
31770 related to the particular sensor or field. This means that as our data‐
31771 base scales up the number of shards, every index request must perform
31772 more work, which is unnecessary since we are only interested in a small
31773 number of documents. Fortunately for you, dear reader, partitioned
31774 databases were created to solve this precise problem.
31775 What is a partition?
31777 In the previous section, we introduced a hypothetical database that
31778 contains sensor readings from an IoT field monitoring service. In this
31779 particular use case, it’s quite logical to group all documents by their
31780 sensor_id field. In this case, we would call the sensor_id the parti‐
31781 tion key.
31782 A good partition has two basic properties. First, it should have a high
31784 cardinality. That is, a large partitioned database should have many
31785 more partitions than documents in any single partition. A database that
31786 has a single partition would be an anti-pattern for this feature. Sec‐
31787 ondly, the amount of data per partition should be “small”. The general
31788 recommendation is to limit individual partitions to less than ten giga‐
31789 bytes (10 GB) of data. Which, for the example of sensor documents,
31790 equates to roughly 60,000 years of data.
31791 Why use partitions?
31793 The primary benefit of using partitioned databases is for the perfor‐
31794 mance of partitioned queries. Large databases with lots of documents
31795 often have a similar pattern where there are groups of related docu‐
31796 ments that are queried together.
31797 By using partitions, we can execute queries against these individual
31799 groups of documents more efficiently by placing the entire group within
31800 a specific shard on disk. Thus, the view engine only has to consult one
31801 copy of the given shard range when executing a query instead of execut‐
31802 ing the query across all q shards in the database. This mean that you
31803 do not have to wait for all q shards to respond, which is both effi‐
31804 cient and faster.
31805 Partitions By Example
31807 To create a partitioned database, we simply need to pass a query string
31808 parameter:
31809 shell> curl -X PUT http://127.0.0.1:5984/my_new_db?partitioned=true
31811 {"ok":true}
31812 To see that our database is partitioned, we can look at the database
31814 information:
31815 shell> curl http://127.0.0.1:5984/my_new_db
31817 {
31818 "cluster": {
31819 "n": 3,
31820 "q": 8,
31821 "r": 2,
31822 "w": 2
31823 },
31824 "compact_running": false,
31825 "db_name": "my_new_db",
31826 "disk_format_version": 7,
31827 "doc_count": 0,
31828 "doc_del_count": 0,
31829 "instance_start_time": "0",
31830 "props": {
31831 "partitioned": true
31832 },
31833 "purge_seq": "0-g1AAAAFDeJzLYWBg4M...",
31834 "sizes": {
31835 "active": 0,
31836 "external": 0,
31837 "file": 66784
31838 },
31839 "update_seq": "0-g1AAAAFDeJzLYWBg4M..."
31840 }
31841 You’ll now see that the "props" member contains "partitioned": true.
31843 NOTE:
31845 Every document in a partitioned database (except _design and _local
31846 documents) must have the format “partition:docid”. More specifi‐
31847 cally, the partition for a given document is everything before the
31848 first colon. The document id is everything after the first colon,
31849 which may include more colons.
31850 NOTE:
31852 System databases (such as _users) are not allowed to be partitioned.
31853 This is due to system databases already having their own incompati‐
31854 ble requirements on document ids.
31855 Now that we’ve created a partitioned database, it’s time to add some
31857 documents. Using our earlier example, we could do this as such:
31858 shell> cat doc.json
31860 {
31861 "_id": "sensor-260:sensor-reading-ca33c748-2d2c-4ed1-8abf-1bca4d9d03cf",
31862 "sensor_id": "sensor-260",
31863 "location": [41.6171031, -93.7705674],
31864 "field_name": "Bob's Corn Field #5",
31865 "readings": [
31866 ["2019-01-21T00:00:00", 0.15],
31867 ["2019-01-21T06:00:00", 0.14],
31868 ["2019-01-21T12:00:00", 0.16],
31869 ["2019-01-21T18:00:00", 0.11]
31870 ]
31871 }
31872 shell> $ curl -X POST -H "Content-Type: application/json" \
31873 http://127.0.0.1:5984/my_new_db -d @doc.json
31874 {
31875 "ok": true,
31876 "id": "sensor-260:sensor-reading-ca33c748-2d2c-4ed1-8abf-1bca4d9d03cf",
31877 "rev": "1-05ed6f7abf84250e213fcb847387f6f5"
31878 }
31879 The only change required to the first example document is that we are
31881 now including the partition name in the document id by prepending it to
31882 the old id separated by a colon.
31883 NOTE:
31885 The partition name in the document id is not magical. Internally,
31886 the database is simply using only the partition for hashing the doc‐
31887 ument to a given shard, instead of the entire document id.
31888 Working with documents in a partitioned database is no different than a
31890 non-partitioned database. All APIs are available, and existing client
31891 code will all work seamlessly.
31892 Now that we have created a document, we can get some info about the
31894 partition containing the document:
31895 shell> curl http://127.0.0.1:5984/my_new_db/_partition/sensor-260
31897 {
31898 "db_name": "my_new_db",
31899 "doc_count": 1,
31900 "doc_del_count": 0,
31901 "partition": "sensor-260",
31902 "sizes": {
31903 "active": 244,
31904 "external": 347
31905 }
31906 }
31907 And we can also list all documents in a partition:
31909 shell> curl http://127.0.0.1:5984/my_new_db/_partition/sensor-260/_all_docs
31911 {"total_rows": 1, "offset": 0, "rows":[
31912 {
31913 "id":"sensor-260:sensor-reading-ca33c748-2d2c-4ed1-8abf-1bca4d9d03cf",
31914 "key":"sensor-260:sensor-reading-ca33c748-2d2c-4ed1-8abf-1bca4d9d03cf",
31915 "value": {"rev": "1-05ed6f7abf84250e213fcb847387f6f5"}
31916 }
31917 ]}
31918 Note that we can use all of the normal bells and whistles available to
31920 _all_docs requests. Accessing _all_docs through the /dbname/_parti‐
31921 tion/name/_all_docs endpoint is mostly a convenience so that requests
31922 are guaranteed to be scoped to a given partition. Users are free to use
31923 the normal /dbname/_all_docs to read documents from multiple parti‐
31924 tions. Both query styles have the same performance.
31925 Next, we’ll create a design document containing our index for getting
31927 all readings from a given sensor. The map function is similar to our
31928 earlier example except we’ve accounted for the change in the document
31929 id.
31930 function(doc) {
31932 if(doc._id.indexOf(":sensor-reading-") < 0) {
31933 return;
31934 }
31935 for(var r in doc.readings) {
31936 emit([doc.sensor_id, r[0]], r[1])
31937 }
31938 }
31939 After uploading our design document, we can try out a partitioned
31941 query:
31942 shell> cat ddoc.json
31944 {
31945 "_id": "_design/sensor-readings",
31946 "views": {
31947 "by_sensor": {
31948 "map": "function(doc) { ... }"
31949 }
31950 }
31951 }
31952 shell> $ curl -X POST -H "Content-Type: application/json" http://127.0.0.1:5984/my_new_db -d @ddoc2.json
31953 {
31954 "ok": true,
31955 "id": "_design/all_sensors",
31956 "rev": "1-4a8188d80fab277fccf57bdd7154dec1"
31957 }
31958 shell> curl http://127.0.0.1:5984/my_new_db/_partition/sensor-260/_design/sensor-readings/_view/by_sensor
31959 {"total_rows":4,"offset":0,"rows":[
31960 {"id":"sensor-260:sensor-reading-ca33c748-2d2c-4ed1-8abf-1bca4d9d03cf","key":["sensor-260","0"],"value":null},
31961 {"id":"sensor-260:sensor-reading-ca33c748-2d2c-4ed1-8abf-1bca4d9d03cf","key":["sensor-260","1"],"value":null},
31962 {"id":"sensor-260:sensor-reading-ca33c748-2d2c-4ed1-8abf-1bca4d9d03cf","key":["sensor-260","2"],"value":null},
31963 {"id":"sensor-260:sensor-reading-ca33c748-2d2c-4ed1-8abf-1bca4d9d03cf","key":["sensor-260","3"],"value":null}
31964 ]}
31965 Hooray! Our first partitioned query. For experienced users, that may
31967 not be the most exciting development, given that the only things that
31968 have changed are a slight tweak to the document id, and accessing views
31969 with a slightly different path. However, for anyone who likes perfor‐
31970 mance improvements, it’s actually a big deal. By knowing that the view
31971 results are all located within the provided partition name, our parti‐
31972 tioned queries now perform nearly as fast as document lookups!
31973 The last thing we’ll look at is how to query data across multiple par‐
31975 titions. For that, we’ll implement the example sensors by field query
31976 from our initial example. The map function will use the same update to
31977 account for the new document id format, but is otherwise identical to
31978 the previous version:
31979 function(doc) {
31981 if(doc._id.indexOf(":sensor-reading-") < 0) {
31982 return;
31983 }
31984 emit(doc.field_name, doc.sensor_id)
31985 }
31986 Next, we’ll create a new design doc with this function. Be sure to
31988 notice that the "options" member contains "partitioned": false.
31989 shell> cat ddoc2.json
31991 {
31992 "_id": "_design/all_sensors",
31993 "options": {
31994 "partitioned": false
31995 },
31996 "views": {
31997 "by_field": {
31998 "map": "function(doc) { ... }"
31999 }
32000 }
32001 }
32002 shell> $ curl -X POST -H "Content-Type: application/json" http://127.0.0.1:5984/my_new_db -d @ddoc2.json
32003 {
32004 "ok": true,
32005 "id": "_design/all_sensors",
32006 "rev": "1-4a8188d80fab277fccf57bdd7154dec1"
32007 }
32008 NOTE:
32010 Design documents in a partitioned database default to being parti‐
32011 tioned. Design documents that contain views for queries across mul‐
32012 tiple partitions must contain the "partitioned": false member in the
32013 "options" object.
32014 NOTE:
32016 Design documents are either partitioned or global. They cannot con‐
32017 tain a mix of partitioned and global indexes.
32018 And to see a request showing us all sensors in a field, we would use a
32020 request like:
32021 shell> curl -u adm:pass http://127.0.0.1:15984/my_new_db/_design/all_sensors/_view/by_field
32023 {"total_rows":1,"offset":0,"rows":[
32024 {"id":"sensor-260:sensor-reading-ca33c748-2d2c-4ed1-8abf-1bca4d9d03cf","key":"Bob's Corn Field #5","value":"sensor-260"}
32025 ]}
32026 Notice that we’re not using the /dbname/_partition/... path for global
32028 queries. This is because global queries, by definition, do not cover
32029 individual partitions. Other than having the “partitioned”: false
32030 parameter in the design document, global design documents and queries
32031 are identical in behavior to design documents on non-partitioned data‐
32032 bases.
32033 WARNING:
32035 To be clear, this means that global queries perform identically to
32036 queries on non-partitioned databases. Only partitioned queries on a
32037 partitioned database benefit from the performance improvements.
32038
32040 3.1.x Branch
32041 · Version 3.1.1
32042 · Version 3.1.0
32044 Version 3.1.1
32046 Features and Enhancements
32047 · #3102, #1600, #2877, #2041: When a client disconnects unexpectedly,
32048 CouchDB will no longer log a “normal : unknown” error. Bring forth
32049 the rainbows.
32050 [image: The Gravity Falls gnome pukes some rainbows for us.] [image]
32051 · #3109: Drilldown parameters for text index searches may now be speci‐
32053 fied as a list of lists, to avoid having to define this redundantly
32054 in a single query. (Some languages don’t have this facility.)
32055 · #3132: The new [chttpd] buffer_response option can be enabled to
32057 delay the start of a response until the end has been calculated. This
32058 increases memory usage, but simplifies client error handling as it
32059 eliminates the possibility that a response may be deliberately termi‐
32060 nated midway through, due to a timeout. This config value may be
32061 changed at runtime, without impacting any in-flight responses.
32062 Performance
32064 Bugfixes
32065 · #2935: The replicator now correctly picks jobs to restart during
32066 rescheduling, where previously with high load it may have failed to
32067 try to restart crashed jobs.
32068 · #2981: When handling extremely large documents (≥50MB), CouchDB can
32070 no longer time out on a gen_server:call if bypassing the IOQ.
32071 · #2941: CouchDB will no longer fail to compact databases if it finds
32073 files from a 2.x compaction process (prior to an upgrade) on disk.
32074 · #2955 CouchDB now sends the correct CSP header to ensure Fauxton
32076 operates correctly with newer browsers.
32077 · #3061, #3080: The couch_index server won’t crash and log errors if a
32079 design document is deleted while that index is building, or when a
32080 ddoc is added immediately after database creation.
32081 · #3078: CouchDB now checks for and complains correctly about invalid
32083 parameters on database creation.
32084 · #3090: CouchDB now correctly encodes URLs correctly when encoding the
32086 atts_since query string.
32087 · #2953: Some parameters not allowed for text-index queries on parti‐
32089 tioned database are now properly validated and rejected.
32090 · #3118: Text-based search indexes may now be cleaned up correctly,
32092 even if the design document is now invalid.
32093 · #3121: fips is now only reported in the welcome message if FIPS mode
32095 was enabled at boot (such as in vm.args).
32096 · #3128: Using COPY to copy a document will no longer return a JSON
32098 result with two ok fields.
32099 · #3138: Malformed URLs in replication requests or documents will no
32101 longer throw an error.
32102 Other
32104 · JS tests skip faster now.
32105 · More JS tests ported into elixir: reader_acl, reduce_builtin,
32107 reduce_false, rev_stemming, update_documents, view_collation_raw,
32108 view_compaction, all the view_multi_key tests, view_sandboxing,
32109 view_update_seq.
32110 Version 3.1.0
32112 Features and Enhancements
32113 · #2648: Authentication via JSON Web Token (JWT). Full documentation is
32114 at the friendly link.
32115 · #2770: CouchDB now supports linking against SpiderMonkey 68, the cur‐
32117 rent Mozilla SpiderMonkey ESR release. This provides direct support
32118 for packaging on the latest operating system variants, including
32119 Ubuntu 20.04 “Focal Fossa.”
32120 ·
32122 A new Fauxton release is included, with updated dependencies, and a
32124 new optional
32125 CouchDB news page.
32126 Performance
32128 · #2754: Optimized compactor performance, resulting in a 40% speed
32129 improvement when document revisions approach the revs_limit. The
32130 fixes also include additional metrics on size tracking during the
32131 sort and copy phases, accessible via the GET /_active_tasks
32132 </active_tasks> endpoint.
32133 · A big bowl of candy! OK, no, not really. If you got this far…thank
32135 you for reading.
32136 3.0.x Branch
32138 · Upgrade Notes
32139 · Version 3.0.1
32141 · Version 3.0.0
32143 Upgrade Notes
32145 · #2228: The default maximum document size has been reduced to 8MB.
32146 This means that databases with larger documents will not be able to
32147 replicate into CouchDB 3.0 correctly without modification. This
32148 change has been made in preparation for anticipated hard upper limits
32149 on document size imposed by CouchDB 4.0. For 3.x, the max document
32150 size setting can be relaxed via the [couchdb] max_document_size con‐
32151 fig setting.
32152 · #2228: The default database sharding factor q has been reduced to 2
32154 by default. This, combined with automated database resharding (see
32155 below), is a better starting place for new CouchDB databases. As in
32156 CouchDB 2.x, specify ?q=# to change the value upon database creation
32157 if desired. The default can be changed via the config [cluster] q
32158 setting.
32159 · #1523, #2092, #2336, #2475: The “node-local” HTTP interface, by
32161 default exposed on port 5986, has been removed. All functionality
32162 previously available at that port is now available on the main, clus‐
32163 tered interface (by default, port 5984). Examples:
32164 GET /_node/{nodename}/_stats
32166 GET /_node/{nodename}/_system
32167 GET /_node/{nodename}/_all_dbs
32168 GET /_node/{nodename}/_uuids
32169 GET /_node/{nodename}/_config
32170 GET /_node/{nodename}/_config/couchdb/uuid
32171 POST /_node/{nodename}_config/_reload
32172 GET /_node/{nodename}/_nodes/_changes?include_docs=true
32173 PUT /_node/{nodename}/_dbs/{dbname}
32174 POST /_node/{nodename}/_restart
32175 GET /_node/{nodename}/{db-shard}
32176 GET /_node/{nodename}/{db-shard}/{doc}
32177 GET /_node/{nodename}/{db-shard}/{ddoc}/_info
32178 …and so on. Documentation has been updated to reflect this change.
32180 · #2389: CouchDB 3.0 now requires a server admin user to be defined at
32182 startup, or will print an error message and exit. If you do not have
32183 one, be sure to create an admin user. (The Admin Party is now over.)
32184 [image: Dizzy the cat with a Santa hat.] [image] CC-BY-NC 2.0:
32185 hehaden @ Flickr.UNINDENT
32186 · #2576: CouchDB 3.0 now requires admin-level access for the /_all_dbs
32188 endpoint.
32189 · #2339: All databases are now created by default as admin-only. That
32191 is, the default new database _security object is now:
32192 {
32194 "members" : { "roles" : [ "_admin" ] },
32195 "admins" : { "roles" : [ "_admin" ] }
32196 }
32197 This can be changed after database creation.
32199 · Due to code changes in #2324, it is not possible to upgrade transpar‐
32201 ently from CouchDB 1.x to 3.x. In addition, the couchup utility has
32202 been removed from CouchDB 3.0 by #2399. If you are upgrading from
32203 CouchDB 1.x, you must first upgrade to CouchDB 2.3.1 to convert your
32204 database and indexes, using couchup if desired. You can then upgrade
32205 to CouchDB 3.0. Or, you can start a new CouchDB 3.0 installation and
32206 replicate directly from 1.x to 3.0.
32207 · #1833, #2358, #1871, #1857: CouchDB 3.0 supports running only under
32209 the following Erlang/OTP versions:
32210 · 19.x - “soft” support only. No longer tested, but should work.
32212 · 20.x - must be newer than 20.3.8.11 (20.0, 20.1, 20.2 versions all
32214 invalid)
32215 · 21.x - for 21.2, must be newer than 21.2.3
32217 · 22.x - for 22.0, must be newer than 22.0.5
32219 · #1804: By default, views are limited to return a maximum of 2**28
32221 (268435456) results. This limit can be configured separately for
32222 views and partitioned views via the query_limit and parti‐
32223 tion_query_limit values in the ini file [query_server_config] sec‐
32224 tion.
32225 · After upgrading all nodes in a cluster to 3.0, add [rexi]
32227 use_kill_all = true to local.ini to save some intra-cluster network
32228 bandwidth.
32229 Deprecated feature removal
32231 The following features, deprecated in CouchDB 2.x, have been removed or
32232 replaced in CouchDB 3.0:
32233 · #2089, #2128, #2251: Local endpoints for replication targets, which
32235 never functioned as expected in CouchDB 2.x, have been completely
32236 removed. When replicating databases, always specify a full URL for
32237 the source and target. In addition, the node local _replicator data‐
32238 base is no longer automatically created.
32239 · #2163: The disk_size and data_size fields have been retired from the
32241 database info object returned by GET /{db}/. These were deprecated in
32242 CouchDB 2.x and replaced by the sizes object, which contains the
32243 improved file, active and external size metrics. Fauxton has been
32244 updated to match.
32245 · #2173: The ability to submit multiple queries against a view using
32247 the POST to /{db}/_design/{ddoc}/_view/{view} with the ?queries=
32248 option has been replaced by the new queries endpoint. The same is
32249 true of the _all_docs, _design_docs, and _local_docs endpoints.
32250 Specify a keys object when POST-ing to these endpoints.
32251 · #2248: CouchDB externals (_external/) have been removed entirely.
32253 · #2208: CouchDB no longer supports the delayed_commits option in the
32255 configuration file. All writes are now full commits. The
32256 /_ensure_full_commit API endpoint has been retained (as a no-op) for
32257 backwards compatibility with old CouchDB replicators.
32258 · #2395: The security object in the _users database cannot be edited by
32260 default. A setting exists in the configuration file to revert this
32261 behaviour. The ability to override the disable setting is expected to
32262 be removed in CouchDB 4.0.
32263 Deprecated feature warnings
32265 The following features are deprecated in CouchDB 3.0 and will be
32266 removed in CouchDB 4.0:
32267 · Show functions (/{db}/{ddoc}/_show)
32269 · List functions (/{db}/{ddoc}/_list)
32271 · Update functions (/{db}/{ddoc}/_update)
32273 · Virtual hosts and ini-file rewrites
32275 · Rewrite functions (/{db}/{ddoc}/_rewrite)
32277 Version 3.0.1
32279 Features and Enhancements
32280 · Fauxton was updated to version v1.2.3.
32281 Bugfixes
32283 · #2441: A memory leak when encoding large binary content was patched.
32284 This should resolve a long-standing gradual memory increase bug in
32285 CouchDB.
32286 · #2613: Simultaneous attempts to create the same new database should
32288 no longer result in a :code 500: error.
32289 · #2678: Defaults for the smoosh compaction daemon are now consistent
32291 with the shipped default.ini file.
32292 · #2680: The Windows CouchDB startup batch file will no longer fail to
32294 start CouchDB if incompatible versions of OpenSSL are on the PATH.
32295 · #2741: A small performance improvement in the couch_server process
32297 was made.
32298 · #2745: The require_valid_user exception logic was corrected.
32300 · #2643: The users_db_security_editable setting is now in the correct
32302 section of the default.ini file.
32303 · #2654: Filtered changes feeds that need to rewind partially should no
32305 longer rewind all the way to the beginning of the feed.
32306 · #2655: When deleting a session cookie, CouchDB should now respect the
32308 operator-specified cookie domain, if set.
32309 · #2690: Nodes that re-enter a cluster after a database was created
32311 (while the node was offline or in maintenance mode) should more cor‐
32312 rectly handle creating local replicas of that database.
32313 · #2805: Mango operators more correctly handle being passed empty
32315 arrays.
32316 · #2716, #2738: The remsh utility will now try and guess the node name
32318 and Erlang cookie of the local installation. It will also respect the
32319 COUCHDB_ARGS_FILE environment variable.
32320 · #2797: The cluster setup workflow now uses the correct logging mod‐
32322 ule.
32323 · #2818: Mango now uses a safer method of bookmark creation that pre‐
32325 vents unexpectedly creating new Erlang atoms.
32326 · #2756: SpiderMonkey 60+ will no longer corrupt UTF-8 strings when
32328 various JS functions are applied to them.
32329 · Multiple test case improvements, including more ports of JS tests to
32331 Elixir.
32332 Version 3.0.0
32334 Features and Enhancements
32335 · #1789: User-defined partitioned databases.
32336 These special databases support user-driven placement of documents
32338 into the same shard range. JavaScript views and Mango indexes have
32339 specific optimizations for partitioned databases as well.
32340 Two tweakable configuration parameters exist:
32342 · #1842: Partition size limits. By default, each partition is limited
32344 to 10 GiB.
32345 · #1684: Partitioned database support can be disabled via feature
32347 flag in default.ini.
32348 · #1972, #2012: Automated shard splitting. Databases can now be
32350 re-sharded while online to increase the q factor to a larger number.
32351 This can be configured to require specific node and range parameters
32352 upon execution.
32353 · #1910: Automatic background indexing, internally known as ken. This
32355 subsystem ensures secondary indexes (such as JavaScript, Mango, and
32356 text search) are kept up to date, without requiring an external query
32357 to trigger building them. Many configuration parameters are avail‐
32358 able.
32359 · #1904: Completely rewritten automatic compaction daemon, internally
32361 known as smoosh. This subsystem automatically triggers background
32362 compaction jobs for both databases and views, based on configurable
32363 thresholds.
32364 · #1889, #2408: New IO Queue subsystem implementation. This is highly
32366 configurable and well-documented.
32367 · #2436, #2455: CouchDB now regression tests against, and officially
32369 supports, running on the arm64v8 (aarch64) and ppc64le (ppc64el)
32370 machine architectures. Convenience binaries are generated on these
32371 architectures for Debian 10.x (“buster”) packages, and for the Docker
32372 containers.
32373 · #1875, #2437, #2423: CouchDB now supports linking against SpiderMon‐
32375 key 60 or SpiderMonkey 1.8.5. SpiderMonkey 60 provides enhanced sup‐
32376 port for ES5, ES6, and ES2016+. Full compatibility information is
32377 available at the ECMAScript compatibility table: click on “Show obso‐
32378 lete platforms,” then look for “FF 60 ESR” in the list of engine
32379 types.
32380 However, it was discovered that on some ARM 64-bit distributions, SM
32382 60 segfaults frequently, including the SM 60 packages on CentOS 8 and
32383 Debian 10.
32384 As a result, CouchDB’s convenience binaries only link against SM 60
32386 on the ``x86_64`` and ``ppc64le`` architectures. This includes the
32387 Docker image for these architectures.
32388 At present, CouchDB ships with SM 60 linked in on the following
32390 binary distributions:
32391 · Debian buster (10.x)
32393 · CentOS / RedHat 8.x
32395 · macOS (10.10+)
32397 · Windows (7+)
32399 · Docker (3.0.0)
32401 · FreeBSD (CURRENT)
32403 We expect to add SM 60 support to Ubuntu with Focal Fossa (20.04 LTS)
32405 when it ships in April 2020.
32406 It is unlikely we will backport SM 60 packages to older versions of
32408 Debian, CentOS, RedHat, or Ubuntu.
32409 · The Windows installer has many improvements, including:
32411 · Prompts for an admin user/password as CouchDB 3.0 requires * Will
32413 not overwrite existing credentials if in place
32414 · No longer remove user-modified config files, closing #1989 * Also
32416 will not overwrite them on install.
32417 · Checkbox to disable installation of the Windows service
32419 · Silent install support.
32421 · Friendly link to these online release notes in the exit dialog
32423 · Higher resolution icon for HiDPI (500x500)
32425 WARNING:
32427 Windows 8, 8.1, and 10 require the .NET Framework v3.5 to be
32428 installed.
32429 · #2037: Dreyfus, the CouchDB side of the Lucene-powered search solu‐
32431 tion, is now shipped with CouchDB. When one or more Clouseau Java
32432 nodes are joined to the cluster, text-based indexes can be enabled in
32433 CouchDB. It is recommended to have as many Clouseau nodes as you have
32434 CouchDB nodes. Search is advertised in the feature list present at
32435 GET / if configured correctly (#2206). Configuration and installa‐
32436 tion documentation is available.
32437 · #2411: The /_up endpoint no longer requires authentication, even when
32439 require_valid_user is true.
32440 · #2392: A new _metrics role can be given to a user. This allows that
32442 user access only to the /_node/{node}/_stats and /_node/{node}/_sys‐
32443 tem endpoints.
32444 · #1912: A new alternative systemd-journald logging backend has been
32446 added, and can be enabled through the ini file. The new backend does
32447 not include CouchDB’s microsecond-accurate timestamps, and uses the
32448 sd-daemon(3) logging levels.
32449 · #2296, #1977: If the configuration file setting [couchdb] single_node
32451 is set to true, CouchDB will automatically create the system data‐
32452 bases on startup if they are not present.
32453 · #2338, #2343: POST request to CouchDB views and the /{db}/_all_docs,
32455 /{db}/_local_docs and /{db}/_design_docs endpoints now support the
32456 same functionality as GET. Parameters are passed in the body as a
32457 JSON object, rather than in the URL when using POST.
32458 · #2292: The _scheduler/docs and _scheduler/info endpoints now return
32460 detailed replication stats for running and pending jobs.
32461 · #2282, #2272, #2290: CouchDB now supports specifying separate proxies
32463 for both the source and target in a replication via source_proxy and
32464 target_proxy keys. The API documentation has been updated.
32465 · #2240: Headers are now returned from the /{db}/_changes feed immedi‐
32467 ately, even when there are no changes available. This avoids client
32468 blocking.
32469 · #2005, #2006: The name of any node can now be retrieved through the
32471 new API endpoint GET /_node/{node-name}.
32472 · #1766: Timeouts for requests, all_docs, attachments, views, and par‐
32474 titioned view requests can all be specified separately in the ini
32475 file under the [fabric] section. See default.ini for more detail.
32476 · #1963: Metrics are now kept on the number of partition and global
32478 view queries, along with the number of timeouts that occur.
32479 · #2452, #2221: A new configuration field [couch_httpd_auth] same_site
32481 has been added to set the value of the CouchDB auth cookie’s SameSite
32482 attribute. It may be necessary to set this to strict for compatibil‐
32483 ity with future versions of Google Chrome. If CouchDB CORS support is
32484 enabled, set this to None.
32485 Performance
32487 · #2277: The couch_server process has been highly optimized, supporting
32488 significantly more load than before.
32489 · #2360: It is now possible to make the rexi interface’s unacked mes‐
32491 sage limit configurable. A new, more optimized default (5, lowered
32492 from 10) has been set. This results in a ~50% improvement on view
32493 queries on large clusters with q ≥ 8.
32494 · #2280: Connection sharing for replication now functions correctly
32496 when replicating through a forward proxy. Closes #2271.
32497 · #2195, #2207: Metrics aggregation now supports CouchDB systems that
32499 sleep or hibernate, ensuring that on wakeup does not trigger thou‐
32500 sands of unnecessary function calls.
32501 · #1795: Avoid calling fabric:update_docs with empty doc lists.
32503 · #2497: The setup wizard no longer automatically creates the
32505 _global_changes database, as the majority of users do not need this
32506 functionality. This reduces overall CouchDB load.
32507 Bugfixes
32509 · #1752, #2398, #1803: The cluster setup wizard now ensures a consis‐
32510 tent UUID and http secret across all nodes in a cluster. CouchDB
32511 admin passwords are also synced when the cluster setup wizard is
32512 used. This prevents being logged out when using Fauxton as a server
32513 admin user through a load balancer.
32514 · #2388: A compatibility change has been made to support replication
32516 with future databases containing per-document access control fields.
32517 · #2379: Any replicator error messages will provide an object in the
32519 response, or null, but never a string.
32520 · #2244, #2310: CouchDB will no longer send more data than is requested
32522 when retrieving partial attachment data blocks.
32523 · #2138: Manual operator updates to a database’s shard map will not
32525 corrupt additional database properties, such as partitioning values.
32526 · #1877: The _purge and _purged_infos_limit endpoints are now correctly
32528 restricted to server admin only.
32529 · #1794: The minimum purge sequence value for a database is now gath‐
32531 ered without a clustered _all_docs lookup.
32532 · #2351: A timeout case clause in fabric_db_info has been normalised to
32534 match other case clauses.
32535 · #1897: The /{db}/_bulk_docs endpoint now correctly catches invalid
32537 (i.e., non-hexadecimal) _rev_ values and responds with a :code 400:
32538 error.
32539 · #2321: CouchDB no longer requires Basic auth credentials to reach the
32541 /_session endpoint for login, even when require_valid_user is
32542 enabled.
32543 · #2295: CouchDB no longer marks a job as failed permanently if the
32545 internal doc processor crashes.
32546 · #2178: View compaction files are now removed on view cleanup.
32548 · #2179: The error message logged when CouchDB does not have a _users
32550 database is now less scary.
32551 · #2153: CouchDB no longer may return a badmatch error when querying
32553 all_docs with a passed keys array.
32554 · #2137: If search is not available, return a :code 400: instead of a
32556 :code 500: status code.
32557 · #2077: Any failed fsync(2) calls are now correctly raised to avoid
32559 data corruption arising from retry attempts.
32560 · #2027: Handle epoch mismatch when duplicate UUIDs are created through
32562 invalid operator intervention.
32563 · #2019: If a database is deleted and re-created while internal cluster
32565 replication is still active, CouchDB will no longer retry to delete
32566 it continuously.
32567 · #2003, #2438: CouchDB will no longer automatically reset an index
32569 file if any attempt to read its header fails (such as when the
32570 couch_file process terminates unexpectedly). CouchDB now also han‐
32571 dles the case when a view file lacks a proper header.
32572 · #1983: Improve database “external” size calcuation to be more pre‐
32574 cise.
32575 · #1971: Correctly compare ETags using weak comparison methods to sup‐
32577 port W/ prefix added by some load balancer configurations.
32578 · #1901: Invalid revision specified for a document update will no
32580 longer result in a badarg crash.
32581 · #1845: The end_time field in /_replicate now correctly converts time
32583 to UTC.
32584 · #1824: rexi stream workers are now cleaned up when the coordinator
32586 process is killed, such as when the ddoc cache is refreshed.
32587 · #1770: Invalid database _security objects no longer return a func‐
32589 tion_clause error and stack trace.
32590 · #2412: Mango execution stats now correctly count documents read which
32592 weren’t followed by a match within a given shard.
32593 · #2393, #2143: It is now possible to override the query server envi‐
32595 ronment variables COUCHDB_QUERY_SERVER_JAVASCRIPT and
32596 COUCHDB_QUERY_SERVER_COFFEESCRIPT without overwriting the
32597 couchdb/couchdb.cmd startup scripts.
32598 · #2426, #2415: The replicator now better handles the situation where
32600 design document writes to the target fail when replicating with
32601 non-admin credentials.
32602 · #2444, #2413: Replicator error messages are now significantly
32604 improved, reducing function_clause responses.
32605 · #2454: The replication auth session plugin now ignores other cookies
32607 it may receive without logging an error.
32608 · #2458: Partitioned queries and dreyfus search functions no longer
32610 fail if there is a single failed node or rexi worker error.
32611 · #1783: Mango text indexes no longer error when given an empty selec‐
32613 tor or operators with empty arrays.
32614 · #2466: Mango text indexes no longer error if the indexed document
32616 revision no longer exists in the primary index.
32617 · #2486: The $lt, $lte, $gt, and $gte Mango operators are correctly
32619 quoted internally when used in conjunction with a text index search.
32620 · #2493: The couch_auth_cache no longer has a runaway condition in
32622 which it creates millions of monitors on the _users database.
32623 Other
32625 The 3.0.0 release also includes the following minor improvements:
32626 · #2472: CouchDB now logs the correct, clustered URI at startup (by
32628 default: port 5984.)
32629 · #2034, #2416: The path to the Fauxton installation can now be speci‐
32631 fied via the COUCHDB_FAUXTON_DOCROOT environment variable.
32632 · #2447: Replication stats are both persisted when jobs are re-created,
32634 as well as properly handled when bulk document batches are split.
32635 · #2410, #2390, #1913: Many metrics were added for Mango use, including
32637 counts of unindexed queries, invalid index queries, docs examined
32638 that do and don’t meet cluster quorum, query time, etc.
32639 · #2152, #2504: CouchDB can now be started via a symlink to the binary
32641 on UNIX-based platforms.
32642 · #1844: A new internal API has been added to write custom Erlang
32644 request-level metrics reporting plugins.
32645 · #2293, #1095: The -args_file, -config and -couch_ini parameters may
32647 now be overridden via the COUCHDB_INI_FILES environment variable on
32648 UNIX-based systems.
32649 · #2352: The remsh utility now searches for the Erlang cookie in
32651 ERL_FLAGS as well as vm.args.
32652 · #2324: All traces of the (never fully functional) view-based _changes
32654 feed have been expunged from the code base.
32655 · #2337: The md5 shim (introduced to support FIPS-compliance) is now
32657 used consistently throughout the code base.
32658 · #2270: Negative and non-integer heartbeat values now return :code
32660 400: Bad Request.
32661 · #2268: When rescheduling jobs, CouchDB now stops sufficient running
32663 jobs to make room for the pending jobs.
32664 · #2186: CouchDB plugin writers have a new field in which endpoint cre‐
32666 dentials may be stashed for later use.
32667 · #2183: dev/run now supports an --extra-args flag to modify the Erlang
32669 runtime environment during development.
32670 · #2105: dev/run no longer fails on unexpected remote end connection
32672 close during cluster setup.
32673 · #2118: Improve couch_epi process replacement mechanism using map
32675 childspecs functionality in modern Erlang.
32676 · #2111: When more than MaxJobs replication jobs are defined, CouchDB
32678 now correctly handles job rotation when some jobs crash.
32679 · #2020: Fix full ring assertion in fabric stream shard replacements
32681 · #1925: Support list for docid when using couch_db:purge_docs/3.
32683 · #1642: io_priority is now set properly on view update and compaction
32685 processes.
32686 · #1865: Purge now supports >100 document IDs in a single request.
32688 · #1861: The vm.args file has improved commentary.
32690 · #1808: Pass document update type for additional checks in
32692 before_doc_update.
32693 · #1835: Module lists are no longer hardcoded in .app files.
32695 · #1798, #1933: Multiple compilation warnings were eliminated.
32697 · #1826: The couch_replicator_manager shim has been fully removed.
32699 · #1820: After restarting CouchDB, JS and Elixir tests now wait up to
32701 30s for it to be ready before timing out.
32702 · #1800: make elixir supports specifying individual tests to run with
32704 tests=.
32705 · #1805: dev/run supports --with-haproxy again.
32707 · #1774: dev/run now supports more than 3 nodes.
32709 · #1779: Refactor Elixir test suite initialization.
32711 · #1769: The Elixir test suite uses Credo for static analysis.
32713 · #1776: All Python code is now formatted using Python black.
32715 · #1786: dev/run: do not create needless dev/data/ directory.
32717 · #2482: A redundant get_ring_opts call has been removed from drey‐
32719 fus_fabric_search.
32720 · #2506: CouchDB’s release candidates no longer propagate the RC tags
32722 into each Erlang application’s version string.
32723 · #2511: recon, the Erlang diagnostic toolkit, has been added to
32725 CouchDB’s build process and ships in the release + convenience bina‐
32726 ries.
32727 · Fauxton updated to v1.2.3, which includes:
32729 · Support multiple server-generated warnings when running queries
32731 · Partitioned database support
32733 · Search index support
32735 · Remove references to deprecated dbinfo fields
32737 · Improve accessibility for screen readers
32739 · Numerous CSS fixes
32741 · Improved test cases:
32743 · Many, many test race conditions and bugs have been removed (PR list
32745 too long to include here!)
32746 · More test cases were ported to Elixir, including:
32748 · Cluster with and without quorum tests (#1812)
32750 · delayed_commits (#1796)
32752 · multiple_rows (#1958)
32754 · invalid_docids (#1968)
32756 · replication (#2090)
32758 · All attachment_* tests (#1999)
32760 · copy_doc (#2000)
32762 · attachments (#1953)
32764 · erlang_views (#2237)
32766 · auth_cache, cookie_auth, lorem*, multiple_rows, users_db, utf8 (‐
32768 #2394)
32769 · etags_head (#2464, #2469)
32771 · #2431: chttpd_purge_tests have been improved in light of CI fail‐
32773 ures.
32774 · #2432: Address flaky test failure on t_invalid_view/1.
32776 · #2363: Elixir tests now run against a single node cluster, in line
32778 with the original design of the JavaScript test suite. This is a
32779 permanent change.
32780 · #1893: Add “w:3” for lots of doc tests.
32782 · #1939, #1931: Multiple fixes to improve support in constrained CI
32784 environments.
32785 · #2346: Big-endian support for the couch_compress tests.
32787 · #2314: Do not auto-index when testing update=false in Mango.
32789 · #2141: Fix couch_views encoding test.
32791 · #2123: Timeout added for fold_docs-with_different_keys test.
32793 · #2114: EUnit tests now correctly inherit necessary environment
32795 variables.
32796 · #2122: :meck.unload() is now called automatically after every test.
32798 · #2098: Fix cpse_test_purge_replication eunit test.
32800 · #2085, #2086: Fix a flaky mem3_sync_event_listener test.
32802 · #2084: Increase timeouts on two slow btree tests.
32804 · #1960, #1961: Fix for chttpd_socket_buffer_size_test.
32806 · #1922: Tests added for shard splitting functionality.
32808 · #1869: New test added for doc reads with etag If-None-Match header.
32810 · #1831: Re-introduced cpse_test_purge_seqs test.
32812 · #1790: Reorganise couch_flag_config_tests into a proper suite.
32814 · #1785: Use devclean on elixir target for consistency of Makefile.
32816 · #2476: For testing, Triq has been replaced with PropEr as an
32818 optional dependency.
32819 · External dependency updates:
32821 · #1870: Mochiweb has been updated to 2.19.0.
32823 · #1938: Folsom has been updated to 0.8.3.
32825 · #2001: ibrowse has been updated to 4.0.1-1.
32827 · #2400: jiffy has been updated to 1.0.1.
32829 · A llama! OK, no, not really. If you got this far…thank you for read‐
32831 ing.
32832 2.3.x Branch
32834 · Upgrade Notes
32835 · Version 2.3.1
32837 · Version 2.3.0
32839 Upgrade Notes
32841 · #1602: To improve security, there have been major changes in the con‐
32842 figuration of query servers, SSL support, and HTTP global handlers:
32843 1. Query servers
32845 Query servers are NO LONGER DEFINED in the .ini files, and can no
32847 longer be altered at run-time.
32848 The JavaScript and CoffeeScript query servers continue to be
32850 enabled by default. Setup differences have been moved from
32851 default.ini to the couchdb and couchdb.cmd start scripts respec‐
32852 tively.
32853 Additional query servers can now be configured using environment
32855 variables:
32856 export COUCHDB_QUERY_SERVER_PYTHON="/path/to/python/query/server.py with args"
32858 couchdb
32859 where the last segment in the environment variable (_PYTHON)
32861 matches the usual lowercase(!) query language in the design doc
32862 language field (here, python.)
32863 Multiple query servers can be configured by using more environment
32865 variables.
32866 You can also override the default servers if you need to set com‐
32868 mand- line options (such as couchjs stack size):
32869 export COUCHDB_QUERY_SERVER_JAVASCRIPT="/path/to/couchjs /path/to/main.js -S <STACKSIZE>"
32871 couchdb
32872 2. Native Query Servers
32874 The mango query server continues to be enabled by default. The
32876 Erlang query server continues to be disabled by default. This
32877 change adds a [native_query_servers] enable_erlang_query_server =
32878 BOOL setting (defaults to false) to enable the Erlang query
32879 server.
32880 If the legacy configuration for enabling the query server is
32882 detected, that is counted as a true setting as well, so existing
32883 configurations continue to work just fine.
32884 3. SSL Support
32886 Enabling SSL support in the ini file is now easier:
32888 [ssl]
32890 enable = true
32891 If the legacy httpsd configuration is found in your ini file, this
32893 will still enable SSL support, so existing configurations do not
32894 need to be changed.
32895 4. HTTP global handlers
32897 These are no longer defined in the default.ini file, but have been
32899 moved to the couch.app context. If you need to customize your han‐
32900 dlers, you can modify the app context using a couchdb.config file
32901 as usual.
32902 · #1602: Also to improve security, the deprecated os_daemons and
32904 couch_httpd_proxy functionality has been completely removed ahead of
32905 the planned CouchDB 3.0 release. We recommend the use of OS-level
32906 daemons such as runit, sysvinit, systemd, upstart, etc. to launch and
32907 maintain OS daemons instead, and the use of a reverse proxy server in
32908 front of CouchDB (such as haproxy) to proxy access to other services
32909 or domains alongside CouchDB.
32910 · #1543: The node-local (default port 5986) /_restart endpoint has been
32912 replaced by the clustered (default port 5984) endpoint
32913 /_node/$node/_restart and /_node/_local/_restart endpoints. The
32914 node-local endpoint has been removed.
32915 · #1764: All python scripts shipped with CouchDB, including couchup and
32917 the dev/run development cluster script, now specify and require
32918 Python 3.x.
32919 · #1396: CouchDB is now compatible with Erlang 21.x.
32921 · #1680: The embedded version of rebar used to build CouchDB has been
32923 updated to the last version of rebar2 available. This assists in
32924 building on non-x86 platforms.
32925 · #1857: Refuse building with known bad versions of Erlang.
32927 Version 2.3.1
32929 Features
32930 · #1811: Add new /{db}/_sync_shards endpoint (admin-only).
32931 · #1870: Update to mochiweb 2.19.0. See also #1875.
32933 · #1857: Refuse building with known bad versions of Erlang.
32935 · #1880: Compaction: Add snooze_period_ms for finer tuning.
32937 Bugfixes
32939 · #1795: Filter out empty missing_revs results in mem3_rep.
32940 · #1384: Fix function_clause error on invalid DB _security objects.
32942 · #1841: Fix end_time field in /_replicate response.
32944 · #1860: Fix read repair in a mixed cluster environment.
32946 · #1862: Fix fabric_open_doc_revs.
32948 · #1865: Support purge requests with more than 100 doc ids.
32950 · #1867: Fix timeout in chttpd_purge_tests.
32952 · #1766: Add default fabric request timeouts.
32954 · #1810: Requests return 400 Bad Request when URL length exceeds 1460
32956 characters. See #1870 for details.
32957 · #1799: Restrict _purge to server admin.
32959 · #1874: This fixes inability to set keys with regex symbols in them.
32961 · #1901: Fix badarg crash on invalid rev for individual doc update.
32963 · #1897: Fix from_json_obj_validate crash when provided rev isn’t a
32965 valid hex.
32966 · #1803: Use the same salt for admin passwords on cluster setup.
32968 · #1053: Fix python2 compatibility for couchup.
32970 · #1905: Fix python3 compatibility for couchup.
32972 Version 2.3.0
32974 Features
32975 · (Multiple) Clustered purge is now available. This feature restores
32976 the CouchDB 1.x ability to completely remove any record of a document
32977 from a database. Conditions apply; to use the feature safely, and for
32978 full details, read the complete cluster/purging documentation.
32979 · #1658: A new config setting is available, allowing an administrator
32981 to configure an initial list of nodes that should be contacted when a
32982 node boots up. Nodes in the seedlist that are successfully reached
32983 will be added to that node’s _nodes database automatically, trigger‐
32984 ing a distributed Erlang connection and replication of the internal
32985 system databases to the new node. This can be used instead of manual
32986 config or the cluster setup wizard to bootstrap a cluster. The
32987 progress of the initial seeding of new nodes is exposed at the GET
32988 /_up endpoint.
32989 · Replication supports ipv6-only peers after updating ibrowse depen‐
32991 dency.
32992 · #1708: The UUID of the server/cluster is once again exposed in the
32994 GET / response. This was a regression from CouchDB 1.x.
32995 · #1722: Stats counts between job runs of the replicator are no longer
32997 reset on job restart.
32998 · #1195, #1742: CouchDB’s _bulk_get implementation now supports the
33000 multipart/mixed and multipart/related content types if requested,
33001 extending compatibility with third-party replication clients.
33002 Performance
33004 · #1409: CouchDB no longer forces the TCP receive buffer to a fixed
33005 size of 256KB, allowing the operating system to dynamically adjust
33006 the buffer size. This can lead to siginificantly improved network
33007 performance when transferring large attachments.
33008 · #1423: Mango selector matching now occurs at the shard level, reduc‐
33010 ing the network traffic within a cluster for a mango query.
33011 · #1423: Long running operations at the node level could exceed the
33013 inter-node timeout, leading to a fabric timeout error in the logfile
33014 and a cancellation of the task. Nodes can now ping to stop that from
33015 happening.
33016 · #1560: An optimization to how external data sizes of attachments were
33018 recorded was made.
33019 · #1586: When cleaning up outdated secondary index files, the search is
33021 limited to the index directory of a specific database.
33022 · #1593: The couch_server ETS table now has the read_concurrency option
33024 set, improving access to the global list of open database handles.
33025 · #1593: Messages to update the least-recently used (LRU) cache are not
33027 sent when the [couchdb] update_lru_on_read setting is disabled.
33028 · #1625: All nodes in a cluster now run their own rexi server.
33030 Bugfixes
33032 · #1484: _stats now correctly handles the case where a map function
33033 emits an array of integers. This bug was introduced in 2.2.0.
33034 · #1544: Certain list functions could return a render_error error
33036 intermittently.
33037 · #1550: Replicator _session support was incompatible with CouchDB
33039 installations using the require_valid_user = true setting.
33040 · #1571: Under very heavy load, it was possible that rexi_server could
33042 die in such a way that it’s never restarted, leaving a cluster with‐
33043 out the ability to issue RPC calls - effectively rendering the clus‐
33044 ter useless.
33045 · #1574: The built-in _sum reduce function has been improved to check
33047 if the objects being summed are not overflowing the view storage.
33048 Previously, there was no protection for _sum-introduced overflows.
33049 · #1582: Database creation parameters now have improved validation,
33051 giving a more readable error on invalid input.
33052 · #1588: A missing security check has been restored for the noop
33054 /db/_ensure_full_commit call to restore database validation checks.
33055 · #1591: CouchDB now creates missing shard files when accessing a data‐
33057 base if necessary. This handles the situation when, on database cre‐
33058 ation, no nodes were capable of creating any of the shard files
33059 required for that database.
33060 · #1568: CouchDB now logs a warning if a changes feed is rewound to 0.
33062 This can help diagnose problems in busy or malfunctioning clusters.
33063 · #1596: It is no longer possible that a busy couch_server, under a
33065 specific ordering and timing of events, will incorrectly track
33066 open_async messages in its mailbox.
33067 · #1601, #1654: CouchDB now logs better when an error causes it to read
33069 past the EOF of a database shard. The check for whether CouchDB is
33070 trying to read too many bytes has been correctly separated out from
33071 the error indicating it has attempted to read past the EOF.
33072 · #1613: Local nodes are now filtered out during read repair opera‐
33074 tions.
33075 · #1636: A memory leak when replicating over HTTPS and a problem occurs
33077 has been squashed.
33078 · #1635: /_replicate jobs are no longer restarted if parameters haven’t
33080 changed.
33081 · #1612: JavaScript rewrite functions now send the body of the request
33083 to the rewritten endpoint.
33084 · #1631: The replicator no longer crashes if the user has placed an
33086 invalid VDU function into one of the _replicator databases.
33087 · #1644, #1647: It is no longer possible to create illegally-named
33089 databases within the reserved system space (_ prefix.)
33090 · #1650: _bulk_get is once again operational for system databases such
33092 as _users.
33093 · #1652: Access to /_active_tasks is once again restricted to server
33095 admins only.
33096 · #1662: The couch_log application no longer crashes when new, addi‐
33098 tional information is supplied by a crashing application, or when any
33099 of its own children are restarted.
33100 · #1666: Mango could return an error that would crash the
33102 couch_query_servers application. This is no longer the case.
33103 · #1655: Configuration of ets_lru in chttpd now performs proper error
33105 checking of the specified config value.
33106 · #1667: The snappy dependency has been updated to fix a memory alloca‐
33108 tion error.
33109 · #1683: Attempting to create a local document with an invalid revision
33111 no longer throws a badarg exception. Also, when setting new_edits to
33112 false and performing a bulk write operation, local documents are no
33113 longer written into the wrong btree. Finally, it is no longer possi‐
33114 ble to create a document with an empty ID during a bulk operation
33115 with new_edits set to false.
33116 · #1721: The couchup convenience script for upgrading from CouchDB 1.x
33118 now also copies a database’s _security object on migration.
33119 · #1672: When checking the status of a view compaction immediately
33121 after starting it, the total_changes and changes_done fields are now
33122 immediately populated with valid values.
33123 · #1717: If the .ini config file is read only, an attempt to update the
33125 config through the HTTP API will now result in a proper eacces error
33126 response.
33127 · #1603: CouchDB now returns the correct total_rows result when query‐
33129 ing /{db}/_design_docs.
33130 · #1629: Internal load validation functions no longer incorrectly hold
33132 open a deleted database or its host process.
33133 · #1746: Server admins defined in the ini file accessing via HTTP API
33135 no longer result in the auth cache logging the access as a miss in
33136 the statistics.
33137 · #1607: The replicator no longer fails to re-authenticate to open a
33139 remote database when its session cookie times out due to a VDU func‐
33140 tion forbidding writes or a non-standard cookie expiration duration.
33141 · #1579: The compaction daemon no longer incorrectly only compacts a
33143 single view shard for databases with a q value greater than 1.
33144 · #1737: CouchDB 2.x now performs as well as 1.x when using a _doc_ids
33146 or _design_docs filter on a changes feed.
33147 Mango
33149 Other
33150 The 2.3.0 release also includes the following minor improvements:
33151 · Improved test cases:
33153 · The Elixir test suite has been merged. These test cases are
33155 intended to replace the aging, unmaintainable JavaScript test
33156 suite, and help reduce our dependency on Mozilla Spidermonkey
33157 1.8.5. The test suite does not yet cover all of the tests that the
33158 JS test suite does. Once it achieves full coverage, the JS test
33159 suite will be removed.
33160 · Many racy test cases improved for reliable CI runs.
33162 · The Makefile targets for list-eunit-* now work correctly on macOS.
33164 · #1732, #1733, #1736: All of the test suites run and pass on the
33166 Windows platform once again.
33167 · #1597: Off-heap messages, a new feature in Erlang 19+, can now be
33169 disabled per module if desired.
33170 · #1682: A new [feature_flags] config section exists for the purpose of
33172 enabling or disabling experimental features by CouchDB developers.
33173 · A narwhal! OK, no, not really. If you got this far…thank you for
33175 reading.
33176 2.2.x Branch
33178 · Upgrade Notes
33179 · Version 2.2.0
33181 Upgrade Notes
33183 · The minimum supported version of Erlang is now 17, not R16B03. Sup‐
33184 port for Erlang 21 is still ongoing and will be provided in a future
33185 release.
33186 · The CouchDB replication client can now use the /_session endpoint
33188 when authenticating against remote CouchDB instances, improving per‐
33189 formance since re-authorization does not have to be performed with
33190 every request. Because of this performance improvement, it is recom‐
33191 mended to increase the PBKDF2 work factor beyond the default 10 to a
33192 modern default such as 10000. This is done via the local ini file
33193 setting [couch_httpd_auth] iterations = 10000.
33194 Do not do this if an older version of CouchDB is replicating TO this
33196 instance or cluster regularly, since CouchDB < 2.2.0 must perform
33197 authentication on every request and replication performance will suf‐
33198 fer.
33199 A future version will make this increased number of iterations a
33201 default.
33202 · #820, #1032: Multiple queries can now be made at the POST
33204 /{db}/_all_docs/queries, POST /{db}/_design_docs/queries and POST
33205 /{db}/_local_docs/queries endpoints. Also, a new endpoint POST
33206 /{db}/_design/{ddoc}/_view/{view}/queries has been introduced to
33207 replace the ?queries parameter formerly provided for making multiple
33208 queries to a view. The old ?queries parameter is now deprecated and
33209 will be removed in a future release of CouchDB.
33210 · The maximum http request limit, which had been lowered in 2.1.0, has
33212 been re-raised to a 4GB limit for now. (#1446). Ongoing discussion
33213 about the path forward for future releases is available in #1200 and
33214 #1253.
33215 · #1118: The least recently used (LRU) cache of databases is now only
33217 updated on database write, not read. This has lead to significant
33218 performance enhancements on very busy clusters. To restore the previ‐
33219 ous behaviour, your local ini file can contain the block [couchdb]
33220 update_lru_on_read = true.
33221 · #1153: The CouchDB replicator can now make use of the /_session end‐
33223 point rather than relying entirely on HTTP basic authentication head‐
33224 ers. This can greatly improve replication performance. We encourage
33225 you to upgrade any nodes or clusters that regularly act as replica‐
33226 tion clients to use this new feature, which is enabled by default (‐
33227 #1462).
33228 · #1283: The [couchdb] enable_database_recovery feature, which only
33230 soft-deletes databases in response to a DELETE /{db} call, is now
33231 documented in default.ini.
33232 · #1330: CouchDB externals and OS daemons are now officially deprecated
33234 and no longer documented. Support for these features will be com‐
33235 pletely removed in a future release of CouchDB (probably 3.0.0).
33236 · #1436: CouchDB proxy authentication now uses a proper chttpd_auth
33238 module, simplifying configuration in local ini files. While this is
33239 not a backward- compatible breaking change, it is best to update your
33240 local ini files to reference the new {chttpd_auth, proxy_authentica‐
33241 tion_handler} handler rather than the couch_httpd_auth version, as
33242 couch_httpd is in the process of being deprecated completely.
33243 · #1476, #1477: The obsolete update_notification feature, which was
33245 replaced by /{db}/_changes feeds c. CouchDB 1.2, has been completely
33246 removed. This feature never worked in 2.0 for databases, only for
33247 shards, making it effectively useless.
33248 Version 2.2.0
33250 Features
33251 · Much improved documentation. Highlights include:
33252 · A complete rewrite of the sharding documentation.
33254 · Developer installation notes (INSTALL.*.rst)
33256 · Much of the content of the original CouchDB Wiki has been imported
33258 into the official docs. (The old CouchDB Wiki is in the process of
33259 being deprecated.)
33260 · Much improved Fauxton functionality. Highlights include:
33262 · Search support in the code editor
33264 · Support for relative Fauxton URLs (i.e., not always at /_utils)
33266 · Replication setup enhancements for various authentication mecha‐
33268 nisms
33269 · Fixes for IE10, IE11, and Edge (we hope…)
33271 · Resolving conflicts of design documents is now allowed
33273 · #496, COUCHDB-3287: New pluggable storage engine framework has landed
33275 in CouchDB. This internal refactor makes it possible for CouchDB to
33276 use different backends for storing the base database file itself. The
33277 refactor included a full migration of the existing “legacy” storage
33278 engine into the new framework.
33279 · #603: When creating a new database on a cluster without quorum,
33281 CouchDB will now return a 202 Accepted code if possible, indicating
33282 that at least one node has written the database record to disk, and
33283 that other nodes will be updated as they return to an online state.
33284 This replaces the former 500 internal error.
33285 · #1136, #1139: When deleting a database in a cluster without quorum,
33287 CouchDB will no longer throw a 500 error status, but a 202 as long as
33288 at least one node records the deletion, or a 200 when all nodes
33289 respond. This fix parallels the one made for #603.
33290 · #745: CouchDB no longer fails to complete replicating databases with
33292 large attachments. The fix for this issue included several related
33293 changes:
33294 · The maximum http request limit, which had been lowered in 2.1.0,
33296 has been re-raised to a 4GB limit for now. (#1446). Ongoing discus‐
33297 sion about the path forward for future releases is available in
33298 #1200 and #1253.
33299 · An update to the replicator http client that improves active socket
33301 accounting, without which CouchDB can cease to be responsive over
33302 the main http interface (#1117)
33303 · The replicator’s http client no longer performs unconditional
33305 retries on failure (#1177)
33306 · A path by which CouchDB could lose track of their RPC workers dur‐
33308 ing multipart attachment processing was removed. (#1178)
33309 · When CouchDB transmits a 413 Payload Too Large response on attach‐
33311 ment upload, it now correctly flushes the receive socket before
33312 closing the connection to avoid a TCP reset, and to give the client
33313 a better chance of parsing the 413 response. In tandem, the repli‐
33314 cator http client correctly closes its own socket after processing
33315 any 413 response. (#1234)
33316 · A fabric process to receive unchunked attachments can no longer
33318 orphan processes that leave unprocessed binaries in memory until
33319 all available memory is exhausted. (#1264).
33320 · When using CouchDB’s native SSL responder (port 6984 by default),
33322 sessions are now timed out by default after 300s. This is to work
33323 around RAM explosion in the BEAM VM when using the Erlang-native
33324 SSL libraries. (#1321
33325 · #822: A new end point api/server/dbs_info has been added to return
33327 information about a list of specified databases. This endpoint can
33328 take the place of multiple queries to /{db}.
33329 · #875, #1030: couch_peruser installations can now specify a default q
33331 value for each peruser-created database that is different from the
33332 cluster’s q value. Set this in your local ini file, under
33333 [couch_peruser] q.
33334 · #876, #1068: The couch_peruser database prefix is now configurable
33336 through your local ini file, under [couch_peruser] database_prefix.
33337 · #887: Replicator documents can now include parameters for target
33339 database creation, such as "create_target_params": {"q": "1"}. This
33340 can assist in database resharding or placement.
33341 · #977: When using COPY to copy a document, CouchDB no longer fails if
33343 the new ID includes Unicode characters.
33344 · #1095: Recognize the environment variables ARGS_FILE, SYSCONFIG_FILE,
33346 COUCHDB_ARGS_FILE and COUCHDB_SYSCONFIG_FILE to overrride where
33347 CouchDB looks for the vm.args and sys.config files at startup.
33348 · #1101, #1425: Mango can now be used to find conflicted documents in a
33350 database by adding conflicts: true to a mango selector.
33351 · #1126: When queried back after saving, replication documents no
33353 longer contain sensitive credential information (such as basic
33354 authenticataion headers).
33355 · #1203:
33357 · The compaction daemon now has a snooze period, during which it
33359 waits to start the next compaction after finishing the previous
33360 one. This value is useful in setups with many databases (e.g.
33361 with couch_peruser) or many design docs, which can cause a CPU
33362 spike every check_interval seconds. The setting can be adjusted
33363 in your local ini file via [compaction_daemon] snooze_period.
33364 The current default is a 3 second pause.
33365 · The check_interval has been raised from 300 seconds to 3600 sec‐
33367 onds.
33368 · A notice-level log about closing view indexes has been demoted
33370 to the debug level. In a sceario with many design docs, this
33371 would createsignficant load on the logging subsystem every [com‐
33372 paction_daemon] check_interval for no discernible benefit.
33373 · #1309, #1435: CouchDB now reports the git sha at the time of build in
33375 the top-level GET / version string, in a new git_sha key. This can be
33376 used to help ensure an unmodified version of CouchDB has been built
33377 and is running on any given machine.
33378 · COUCHDB-2971, #1346: CouchDB now includes a new builtin reduce func‐
33380 tion _approx_count_distinct, that uses a HyperLogLog algorithm to
33381 estimate the number of distinct keys in the view index. The precision
33382 is currently fixed to 2^11 observables, and therefore uses approxi‐
33383 mately 1.5KB of memory.
33384 · #1377: CouchDB finalization of view reduces now occurs at the coordi‐
33386 nator node. This simplified the built-in _stats function.
33387 · #1392: When running CouchDB under Erlang 19.0 or newer, messages can
33389 now be stored off the process heap. This is extremely useful for
33390 Erlang processes that can have huge number of messages in their mail‐
33391 box, and is now enabled for couch_server, couch_log_server,
33392 ddoc_cache, mem3_shards, and rexi_server whenever possible.
33393 · #1424: The CouchDB native SSL/TLS server httpsd now accepts
33395 socket-level configuration options through the [httpsd]
33396 server_options ini file setting.
33397 · #1440: CouchDB can now be configured to prevent non-admins from
33399 accessing the GET /_all_dbs method by specifying [chttpd]
33400 admin_only_all_dbs = true in your local ini file(s). The true setting
33401 will become default in future versions.
33402 · #1171, #1445: CouchDB can now be configured to use the internal
33404 Erlang MD5 hash function when not available in the external environ‐
33405 ment (e.g. FIPS enabled CentOS) at compile time with the configure
33406 flag --enable-md5. Because this implementation is slower, it is not
33407 recommended in the general case.
33408 Performance
33410 · #958: The revision stemming algorithm was optimized down from O(N^2)
33411 to O(N) via a depth-first search approach, and then further improved
33412 by calling the stemming operation only when necessary. This new algo‐
33413 rithm can be disabled by setting the option [couchdb] stem_interac‐
33414 tive_updates = false if necessary.
33415 · #1246: CouchDB now checks for request authorization only once per
33417 each database request, improving the performance of any request that
33418 requires authorization.
33419 Bugfixes
33421 · #832, #1064: Tracking of Couch logging stats has been added back into
33422 the per-node /_node/<node-name>/_stats endpoint.
33423 · #953, #973: Return 404 Not Found on GET /_scheduler, not 405 Method
33425 Not Allowed.
33426 · #955: The /{db}/_bulk_docs endpoint now correctly responds with a 400
33428 Bad Request error if the new_edits parameter is not a boolean.
33429 · #969: CouchDB now returns offset and update_seq values when keys are
33431 provided to the GET or POST /{db}/_all_docs?update_seq=true end‐
33432 points. This was affecting PouchDB compatibility.
33433 · #984, #1434: CouchDB views now retain their update_seq after com‐
33435 paction, preventing potentially expensive client-side view rewinds
33436 after compaction.
33437 · #1012: Address a theoretical race condition the replication scheduler
33439 could encounter when trying to determine if the cluster is “stable”
33440 enough to resume handling replication-introduced document updates.
33441 · #1051: Return a user-friendly error message when attempting to create
33443 a CouchDB user with an invalid password field (non-string).
33444 · #1059: DB-specific compaction configurations were not working cor‐
33446 rectly. The syntax now also supports shard-level custom compaction
33447 configuration if desired (which it probably isn’t.)
33448 · #1097: Compaction daemon will not crash out when trying to check spe‐
33450 cific file system mounts that are not “real” file systems (like /run
33451 on Linux).
33452 · #1198: Fauxton is no longer available on the node-local port (5986,
33454 by default). The node-local port is only to be used for specific
33455 administrative tasks; removing the Fauxton interface prevents mistak‐
33456 ing the node-local port as the correct CouchDB port (5984, by
33457 default).
33458 · #1165: validate_doc_update view functions can once again be imple‐
33460 mented directly in Erlang (after enabling the optional Erlang view
33461 server).
33462 · #1223: The couch_config application now correctly handles non-persis‐
33464 tent integer and boolean-valued configuration changes.
33465 · #1242: couch_os_daemons may now reside in directories with spaces.
33467 · #1258: CouchDB will now successfully login users, even if password
33469 encryption is very slow.
33470 · #1276: The replication scheduler status for a repeatedly erroring job
33472 now correctly reflects the crashing state in more scenarios.
33473 · #1375: If CouchDB fails authorization but passes authentication, it
33475 no longer drops the user_ctx out of the request.
33476 · #1390: The active size of views (as returned in a database info
33478 response) no longer is incorrectly calculated in such a way that it
33479 could occasionally be larger than the actual on-disk file size.
33480 · #1401: CouchDB Erlang views no longer crash in the couch_native
33482 process with an unexpected function_clause error.
33483 · #1419: When deleting a file, CouchDB now properly ignores the config‐
33485 uration flag enable_database_recovery when set when compacting data‐
33486 bases, rather than always retaining the old, renamed, uncompacted
33487 database file.
33488 · #1439: The CouchDB setup wizard now correctly validates
33490 bind_addresses. It also no longer logs credentials by moving logging
33491 of internal wizard setup steps to the debug level from the notice
33492 level.
33493 Mango
33495 · #816, #962, #1038: If a user specifies a value for use_index that is
33496 not valid for the selector (does not meet coverage requirements or
33497 proper sort fields), attempt to fall back to a valid index or full DB
33498 scan rather than returning a 400. If we fall back, populate a warn‐
33499 ing field in the response. Mango also tries to use indexes where $or
33500 may select a field only when certain values are present.
33501 · #849: When {"seq_indexed": true} is specified, a badmatch error was
33503 returned. This is now fixed.
33504 · #927, #1310: Error messages when attempting to sort incorrectly are
33506 now actually useful.
33507 · #951: When using GET /{db}/_index, only use a partial filter selector
33509 for an index if it is set to something other than the default.
33510 · #961: Do not prefix _design/ to a Mango index name whose user-speci‐
33512 fied name already starts with _design/.
33513 · #988, #989: When specifying a use_index value with an invalid index,
33515 correctly return a 400 Bad Request showing that the requested index
33516 is invalid for the request specified.
33517 · #998: The fix for CVE 2017-12635 presented a breaking change to
33519 Mango’s /{db}/_find, which would evaluate all instances of all JSON
33520 fields in a selector. Mango is now tested to ensure it only considers
33521 the last instance of a field, silently ignoring those that appear
33522 before it.
33523 · #1014: Correctly deduce list of indexed fields in a selector when
33525 nested $and operators are specified.
33526 · #1023: Fix an unexpected 500 error if startkey and endkey in a Mango
33528 selector were reversed.
33529 · #1067: Prevent an invalid_cast crash when the couch_proc_manager soft
33531 limit for processes is reached and mango idle processes are stopped.
33532 · #1336: The built-in fields _id and rev will always be covered by any
33534 index, and Mango now correctly ignores their presence in any index
33535 that explicitly includes them for selector matching purposes.
33536 · #1376: Mango now appropriately selects some indexes as usable for
33538 queries, even if not all columns for an index are added to the
33539 query’s sort field list.
33540 · Multiple fixes related to using Mango as a front-end for full text
33542 indexing (a feature not shipped with couch, but for which support is
33543 in place as a compile-time addon).
33544 Other
33546 The 2.2.0 release also includes the following minor improvements:
33547 · Developers can, at build time, enable curl libraries & disable Faux‐
33549 ton and documentation builds by specifying the new --dev option to
33550 the configure script.
33551 · The mochiweb dependency was bumped to version 2.17.0, in part to
33553 address the difficult #745 issue.
33554 · Improved compatibility with newer versions of Erlang (20.x)
33556 · Improved release process for CouchDB maintainers and PMC members.
33558 · Multiple test suite improvements, focused on increased coverage,
33560 speed, and reliability.
33561 · Improvements to the Travis CI and Jenkins CI setups, focused on
33563 improved long-term project maintenance and automatability.
33564 · Related improvements to the CouchDB deb/rpm packaging and Docker
33566 repositories to make deployment even easier.
33567 · #1007: Move etc/default.ini entries back into [replicator] section
33569 (incorrectly moved to [couch_peruser] section)
33570 · #1245: Increased debug-level logging for shard open errors is now
33572 available.
33573 · #1296: CouchDB by default now always invokes the SMP-enabled BEAM VM,
33575 even on single-processor machines. A future release of Erlang will
33576 remove the non-SMP BEAM VM entirely.
33577 · A pony! OK, no, not really. If you got this far…thank you for read‐
33579 ing.
33580 2.1.x Branch
33582 · Upgrade Notes
33583 · Version 2.1.2
33585 · Version 2.1.1
33587 · Version 2.1.0
33589 · Fixed Issues
33591 Upgrade Notes
33593 · When upgrading from 2.x to 2.1.1, if you have not customized your
33594 node name in vm.args, be sure to retain your original vm.args file.
33595 The default node name has changed from couchdb@localhost to
33596 couchdb@127.0.0.1, which can prevent CouchDB from accessing existing
33597 databases on the system. You may also change the name option back to
33598 the old value by setting -name couchdb@localhost in etc/vm.args by
33599 hand. The default has changed to meet new guidelines and to provide
33600 additional functionality in the future.
33601 If you receive errors in the logfile, such as internal_server_error :
33603 No DB shards could be opened. or in Fauxton, such as This database
33604 failed to load. you need to make this change.
33605 · The deprecated (and broken) OAuth 1.0 implementation has been
33607 removed.
33608 · If user code reads or manipulates replicator document states, con‐
33610 sider using the [replicator] update_docs = true compatibility parame‐
33611 ter. In that case the replicator will continue updating documents
33612 with transient replication states. However, that will incur a perfor‐
33613 mance cost. Consider instead using the _scheduler/docs HTTP endpoint.
33614 · The stale parameter for views and _find has been deprecated in favour
33616 of two new parameters: stable and update. The old stale=ok behaviour
33617 is equivalent to stable=true&update=false, and the old
33618 stale=update_after behaviour is equivalent to sta‐
33619 ble=true&update=lazy. The deprecated stale parameter will be removed
33620 in CouchDB 3.0.
33621 · The new httpd/max_http_request_size configuration parameter was
33623 added. This has the same behavior as the old couchdb/max_docu‐
33624 ment_size configuration parameter, which had been unfortunately mis‐
33625 named, and has now been updated to behave as the name would suggest.
33626 Both are documented in the shipped default.ini file.
33627 Note that the default for this new parameter is 64MB instead of 4GB.
33629 If you get errors when trying to PUT or POST and see HTTP 413 return
33630 codes in couchdb logs, this could be the culprit. This can affect
33631 couchup in-place upgrades as well.
33632 · #914: Certain critical config sections are blacklisted from being
33634 modified through the HTTP API. These sections can still be modified
33635 through the standard local.ini or local.d/*.ini files.
33636 · #916: couchjs now disables eval() and the Function() constructor by
33638 default. To restore the original behaviour, add the --eval flag to
33639 the definition of the javascript query server in your local.ini file.
33640 Version 2.1.2
33642 Security
33643 · CVE 2018-8007
33644 Version 2.1.1
33646 Security
33647 · CVE 2017-12635
33648 · CVE 2017-12636
33650 General
33652 · #617: CouchDB now supports compilation and running under Erlang/OTP
33653 20.x.
33654 · #756: The couch_peruser functionality is now really fixed. Really.
33656 · #827: The cookie domain for AuthSession cookies, used in a proxy
33658 authentication configuration, can now be customized via the ini file.
33659 · #858: It is now possible to modify shard maps for system databases.
33661 · #732: Due to an Erlang bug (ERL-343), invalid paths can be returned
33663 if volumes are mounted containing whitespace in their name. This
33664 problem surfaced primarily on macOS (Time Machine volumes). CouchDB
33665 now works around this bug in unpatched versions of Erlang by skipping
33666 the free space check performed by the compaction daemon. Erlang
33667 itself will correctly perform free space checks in version 21.0.
33668 · #824: The current node’s local interface can now be accessed at
33670 /_node/_local/{endpoint} as well as at /_node/<nodename>@<host‐
33671 name>/{endpoint}.
33672 · The Dockerfile in the source repository has been retired. For a cur‐
33674 rent Dockerfile, see the couchdb-docker repository.
33675 · Fauxton now uses a version of React with a BSD license.
33677 Performance
33679 · #835: CouchDB now no longer decompresses documents just to determine
33680 their uncompressed size. In tests, this has lead to improvements
33681 between 10-40% in both CPU and wall-clock time for database com‐
33682 paction.
33683 · The design document cache (ddoc_cache) has been rewritten to improve
33685 performance.
33686 Mango
33688 · #808: Mango now supports partial indexes. Partial indexes allow docu‐
33689 ments to be filtered at indexing time, potentially offering signifi‐
33690 cant performance improvements for query selectors that don’t map
33691 cleanly to a range query on an index.
33692 · #740: Mango queries can now be paginated. Each query response
33694 includes a bookmark. The bookmark can be provided on a subsequent
33695 query to continue from a specific key.
33696 · #768: Mango _find accepts an execution_stats parameter. If present, a
33698 new object is included in the response which contains information
33699 about the query executed. The object contains the count of total keys
33700 examined (0 for json indexes), total documents examined (when
33701 include_docs=true is used), and the total quorum documents examined
33702 (when fabric doc lookups are used).
33703 · #816 and #866: Mango now requires that all of the fields in a candi‐
33705 date index must exist in a query’s selector. Previously, this check
33706 was incorrect, and indexes that might only contain a subset of valid
33707 documents might be selected by the query planner if no explicit index
33708 was specified at query time. Further, if a sort field is specified at
33709 query time, that field needs to exist (but could be null) in the
33710 results returned.
33711 Other
33713 The 2.1.1 release also includes the following minor improvements:
33714 · #635: Stop couch_index processes on ddoc update
33716 · #721: Save migrated replicator checkpoint documents immediately
33718 · #688: Reuse http-based replication checkpoints when upgrading to
33720 https
33721 · #729: Recommend the use only of -name and not -sname in vm.args
33723 for compatibility.
33724 · #738: Allow replicator application to always update replicator
33726 docs.
33727 · #605: Add Prefer: return=minimal header options from RFC7240 to
33729 reduce the number of headers in the response.
33730 · #744: Allow a 503 response to be returned to clients (with metric
33732 support)
33733 · #746: Log additional information on crashes from rexi
33735 · #752: Allow Mango $in queries without requiring the index to use
33737 an array
33738 · (multiple) Additional debugging utilities have been added.
33740 · (multiple) Hot code upgrades from 2.0 -> 2.1.1 are now possible.
33742 · (multiple) Improvements to the test suite have been made.
33744 · #765: Mango _explain now includes view parameters as requested by
33746 the user.
33747 · #653: _show and _list should now work for admin-only databases
33749 such as _users.
33750 · #807: Mango index selection should occur only once.
33752 · #804: Unhandled Mango errors are now logged.
33754 · #659: Improve accuracy of the max_document_size check.
33756 · #817: Invalid Base64 in inline attachments is now caught.
33758 · #825: Replication IDs no longer need to be URL encoded when using
33760 the _scheduler/jobs/<job_id> endpoint.
33761 · #838: Do not buffer rexi messages to disconnected nodes.
33763 · #830: The stats collection interval is now configurable in an ini
33765 file, not in the application context. The default value is 10, and
33766 the setting is reloaded every 600 seconds.
33767 · #812: The /{db} endpoint now includes a cluster block with the
33769 database’s q, n, and default w and r values. This supplements the
33770 existing /{db}/_shards and /{db}/_shards/{id} detailed information
33771 on sharding and quorum.
33772 · #810: The replicator scheduler crashed counter gauge more reliably
33774 detects replication crashes by reducing the default number of
33775 retries from 10 to 5 (reducing the duration from 4 mins to 8
33776 secs).
33777 · COUCHDB-3288: Tolerate mixed clusters for the upcoming pluggable
33779 storage engine work.
33780 · #839: Mango python tests now support Python 3 as well as 2.
33782 · #845: A convenience remsh script has been added to support live
33784 debugging of running systems.
33785 · #846: Replicator logging is now less verbose and more informative
33787 when replication terminates unexpectedly.
33788 · #797: Reduce overflow errors are now returned to the client,
33790 allowing views with a single bad reduce to build while not
33791 exhausting the server’s RAM usage.
33792 · #881: Mango now allows match on documents where the indexed value
33794 is an object if a range query is issued. Previously, query results
33795 might change in the presence of an index, and operators/selectors
33796 which explicitly depend on a full index scan (such as $exists)
33797 would not return a complete result set.
33798 · #883: Erlang time module compatibility has been improved for
33800 releases of Erlang newer than 18.0.
33801 · #933: 410 is now returned when attempting to make a temporary view
33803 request.
33804 · #934: The replicator now has a configurable delay before retrying
33806 to retrieve a document after receiving a missing_doc error.
33807 · #936: jiffy now deduplicates JSON keys.
33809 Version 2.1.0
33811 · The Mango _find endpoint supports a new combination operator, $all‐
33812 Match, which matches and returns all documents that contain an array
33813 field with all its elements matching all the specified query crite‐
33814 ria.
33815 · New scheduling replicator. The core of the new replicator is a sched‐
33817 uler which allows running a large number of replication jobs by
33818 switching between them, stopping some and starting others periodi‐
33819 cally. Jobs which fail are backed off exponentially. There is also an
33820 improved inspection and querying API: _scheduler/jobs and _sched‐
33821 uler/docs:
33822 · _scheduler/jobs : This endpoint shows active replication jobs.
33824 These are jobs managed by the scheduler. Some of them might be run‐
33825 ning, some might be waiting to run, or backed off (penalized)
33826 because they crashed too many times. Semantically this is somewhat
33827 equivalent to _active_tasks but focuses only on replications. Jobs
33828 which have completed or which were never created because of mal‐
33829 formed replication documents will not be shown here as they are not
33830 managed by the scheduler. _replicate replications, started form
33831 _replicate endpoint not from a document in a _replicator db, will
33832 also show up here.
33833 · _scheduler/docs : This endpoint is an improvement on having to go
33835 back and read replication documents to query their state. It repre‐
33836 sents the state of all the replications started from documents in
33837 _replicator db. Unlike _scheduler/jobs it will also show jobs which
33838 have failed or have completed.
33839 By default, scheduling replicator will not update documents with
33841 transient states like triggered or error anymore, instead _sched‐
33842 uler/docs API should be used to query replication document states.
33843 Other scheduling replicator improvements
33845 · Network resource usage and performance was improved by implement‐
33846 ing a shared connection pool. This should help in cases of a large
33847 number of connections to the same sources or target. Previously
33848 connection pools were shared only withing a single replication
33849 job.
33850 · Improved request rate limit handling. Replicator requests will
33852 auto-discover rate limit capacity on targets and sources based on
33853 a proven Additive Increase / Multiplicative Decrease feedback con‐
33854 trol algorithm.
33855 · Improved performance by having exponential backoff for all repli‐
33857 cation jobs failures. Previously there were some scenarios were
33858 failure led to continuous repeated retries, consuming CPU and disk
33859 resources in the process.
33860 · Improved recovery from long but temporary network failure. Cur‐
33862 rently if replications jobs fail to start 10 times in a row, they
33863 will not be retried anymore. This is sometimes desirable, but in
33864 some cases, for example, after a sustained DNS failure which even‐
33865 tually recovers, replications reach their retry limit, stop retry‐
33866 ing and never recover. Previously it required user intervention to
33867 continue. Scheduling replicator will never give up retrying a
33868 valid scheduled replication job and so it should recover automati‐
33869 cally.
33870 · Better handling of filtered replications. Failing user filter code
33872 fetches from the source will not block replicator manager and
33873 stall other replications. Failing filter fetches will also be
33874 backed off exponentially. Another improvement is when filter code
33875 changes on the source, a running replication will detect that and
33876 restart itself with a new replication ID automatically.
33877 The 2.1.0 release also includes the following minor improvements:
33879 · COUCHDB-1946: Hibernate couch_stream after each write (up to 70%
33881 reduction in memory usage during replication of DBs with large
33882 attachments)
33883 · COUCHDB-2964: Investigate switching replicator manager change
33885 feeds to using “normal” instead of “longpoll”
33886 · COUCHDB-2988: (mango) Allow query selector as changes and replica‐
33888 tion filter
33889 · COUCHDB-2992: Add additional support for document size
33891 · COUCHDB-3046: Improve reduce function overflow protection
33893 · COUCHDB-3061: Use vectored reads to search for buried headers in
33895 .couch files. “On a modern linux system with SSD, we see improve‐
33896 ments up to 15x.”
33897 · COUCHDB-3063: “stale=ok” option replaced with new “stable” and
33899 “update” options.
33900 · COUCHDB-3180: Add features list in the welcome message
33902 · COUCHDB-3203: Make auth handlers configurable (in ini files)
33904 · COUCHDB-3234: Track open shard timeouts with a counter instead of
33906 logging
33907 · COUCHDB-3242: Make get view group info timeout in couch_indexer
33909 configurable
33910 · COUCHDB-3249: Add config to disable index all fields (text
33912 indexes)
33913 · COUCHDB-3251: Remove hot loop usage of filename:rootname/1
33915 · COUCHDB-3284: 8Kb read-ahead in couch_file causes extra IO and
33917 binary memory usage
33918 · COUCHDB-3298: Optimize writing btree nodes
33920 · COUCHDB-3302: (Improve) Attachment replication over low bandwidth
33922 network connections
33923 · COUCHDB-3307: Limit calls to maybe_add_sys_db_callbacks to once
33925 per db open
33926 · COUCHDB-3318: bypass couch_httpd_vhost if there are none
33928 · COUCHDB-3323: Idle dbs cause excessive overhead
33930 · COUCHDB-3324: Introduce couch_replicator_scheduler
33932 · COUCHDB-3337: End-point _local_docs doesn’t conform to query
33934 params of _all_docs
33935 · COUCHDB-3358: (mango) Use efficient set storage for field names
33937 · COUCHDB-3425: Make _doc_ids _changes filter fast-path limit con‐
33939 figurable
33940 · #457: TeX/LaTeX/texinfo removed from default docs build chain
33942 · #469: (mango) Choose index based on fields match
33944 · #483: couchup database migration tool
33946 · #582: Add X-Frame-Options support to help protect against click‐
33948 jacking
33949 · #593: Allow bind address of 127.0.0.1 in _cluster_setup for single
33951 nodes
33952 · #624: Enable compaction daemon by default
33954 · #626: Allow enable node decom using string “true”
33956 · (mango) Configurable default limit, defaults to 25.
33958 · (mango) _design documents ignored when querying _all_docs
33960 · (mango) add $allMatch selector
33962 · Add local.d/default.d directories by default and document
33964 · Improved INSTALL.* text files
33966 Fixed Issues
33968 The 2.1.0 release includes fixes for the following issues:
33969 · COUCHDB-1447: X-Couch-Update-NewRev header is missed if custom head‐
33971 ers are specified in response of _update handler (missed in 2.0
33972 merge)
33973 · COUCHDB-2731: Authentication DB was not considered a system DB
33975 · COUCHDB-3010: (Superceded fix for replication exponential backoff)
33977 · COUCHDB-3090: Error when handling empty “Access-Control-Request-Head‐
33979 ers” header
33980 · COUCHDB-3100: Fix documentation on require_valid_user
33982 · COUCHDB-3109: 500 when include_docs=true for linked documents
33984 · COUCHDB-3113: fabric:open_revs can return {ok, []}
33986 · COUCHDB-3149: Exception written to the log if db deleted while there
33988 is a change feed running
33989 · COUCHDB-3150: Update all shards with stale=update_after
33991 · COUCHDB-3158: Fix a crash when connection closes for _update
33993 · COUCHDB-3162: Default ssl settings cause a crash
33995 · COUCHDB-3164: Request fails when using
33997 _changes?feed=eventsource&heartbeat=30000
33998 · COUCHDB-3168: Replicator doesn’t handle well writing documents to a
34000 target db which has a small max_document_size
34001 · COUCHDB-3173: Views return corrupt data for text fields containing
34003 non-BMP characters
34004 · COUCHDB-3174: max_document_size setting can by bypassed by issuing
34006 multipart/related requests
34007 · COUCHDB-3178: Fabric does not send message when filtering lots of
34009 documents
34010 · COUCHDB-3181: function_clause error when adding attachment to doc in
34012 _users db
34013 · COUCHDB-3184: couch_mrview_compactor:recompact/1 does not handle
34015 errors in spawned process
34016 · COUCHDB-3193: fabric:open_revs returns multiple results when one of
34018 the shards has stem_interactive_updates=false
34019 · COUCHDB-3199: Replicator VDU function doesn’t acount for an already
34021 malformed document in replicator db
34022 · COUCHDB-3202: (mango) do not allow empty field names
34024 · COUCHDB-3220: Handle timeout in _revs_diff
34026 · COUCHDB-3222: (Fix) HTTP code 500 instead of 400 for invalid key dur‐
34028 ing document creation
34029 · COUCHDB-3231: Allow fixing users’ documents (type and roles)
34031 · COUCHDB-3232: user context not passed down in fabric_view_all_docs
34033 · COUCHDB-3238: os_process_limit documentation wrong
34035 · COUCHDB-3241: race condition in couch_server if delete msg for a db
34037 is received before open_result msg
34038 · COUCHDB-3245: Make couchjs -S option take effect again
34040 · COUCHDB-3252: Include main-coffee.js in release artifact (broken Cof‐
34042 feeScript view server)
34043 · COUCHDB-3255: Conflicts introduced by recreating docs with attach‐
34045 ments
34046 · COUCHDB-3259: Don’t trap exits in couch_file
34048 · COUCHDB-3264: POST to _all_docs does not respect conflicts=true
34050 · COUCHDB-3269: view response can ‘hang’ with filter and limit speci‐
34052 fied
34053 · COUCHDB-3271: Replications crash with ‘kaboom’ exit
34055 · COUCHDB-3274: eof in couch_file can be incorrect after error
34057 · COUCHDB-3277: Replication manager crashes when it finds _replicator
34059 db shards which are not part of a mem3 db
34060 · COUCHDB-3286: Validation function throwing unexpected json crashes
34062 with function_clause
34063 · COUCHDB-3289: handle error clause when calling fabric:open_revs
34065 · COUCHDB-3291: Excessively long document IDs prevent replicator from
34067 making progress
34068 · COUCHDB-3293: Allow limiting length of document ID (for CouchDB
34070 proper)
34071 · COUCHDB-3305: (mango) don’t crash with invalid input to built in
34073 reducer function
34074 · COUCHDB-3362: DELETE attachment on non-existing document creates the
34076 document, rather than returning 404
34077 · COUCHDB-3364: Don’t crash compactor when compacting process fails.
34079 · COUCHDB-3367: Require server admin user for db/_compact and
34081 db_view_cleanup endpoints
34082 · COUCHDB-3376: Fix mem3_shards under load
34084 · COUCHDB-3378: Fix mango full text detection
34086 · COUCHDB-3379: Fix couch_auth_cache reinitialization logic
34088 · COUCHDB-3400: Notify couch_index_processes on all shards when ddoc
34090 updated
34091 · COUCHDB-3402: race condition in mem3 startup
34093 · #511: (mango) Return false for empty list
34095 · #595: Return 409 to PUT attachment with non-existent rev
34097 · #623: Ensure replicator _active_tasks entry reports recent pending
34099 changes value
34100 · #627: Pass UserCtx to fabric’s all_docs from mango query
34102 · #631: fix couchdb_os_proc_pool eunit timeouts
34104 · #644: Make couch_event_sup:stop/1 synchronous
34106 · #645: Pass db open options to fabric_view_map for _view and _list
34108 queries on _users DB
34109 · #648: Fix couch_replicator_changes_reader:process_change
34111 · #649: Avoid a race when restarting an index updater
34113 · #667: Prevent a terrible race condition
34115 · #677: Make replication filter fetch error for _replicate return a 404
34117 · Fix CORS max_age configuration parameter via Access-Control-Max-Age
34119 · Chunk missing revisions before attempting to save on target (improves
34121 replication for very conflicted, very deep revision tree documents)
34122 · Allow w parameter for attachments
34124 · Return “Bad Request” when count in /_uuids exceeds max
34126 · Fix crashes when replicator db is deleted
34128 · Skip internal replication if changes already replicated
34130 · Fix encoding issues on _update/../doc_id and PUT attachments
34132 2.0.x Branch
34134 · Version 2.0.0
34135 · Upgrade Notes
34137 · Known Issues
34139 · Breaking Changes
34141 Version 2.0.0
34143 · Native clustering is now supported. Rather than use CouchDB replica‐
34144 tion between multiple, distinct CouchDB servers, configure a cluster
34145 of CouchDB nodes. These nodes will use an optimized Erlang-driven
34146 ‘internal replication’ to ensure data durability and accessibility.
34147 Combine a clustered CouchDB with a load balancer (such as haproxy) to
34148 scale CouchDB out horizontally. More details of the clustering fea‐
34149 ture are available in the cluster.
34150 · Futon replaced by brand-new, completely re-engineered Fauxton inter‐
34152 face. URL remains the same.
34153 · The new Mango Query Server provides a simple JSON-based way to per‐
34155 form CouchDB queries without JavaScript or MapReduce. Mango Queries
34156 have a similar indexing speed advantage over JavaScript Queries than
34157 the Erlang Queries have (2x-10x faster indexing depending on doc size
34158 and system configuration). We recommend all new apps start using
34159 Mango as a default. Further details are available in the _find,
34160 _index and _explain API.
34161 · Mango selectors can be used in _changes feeds instead of JavaScript
34163 MapReduce filters. Mango has been tested to be up to an order of mag‐
34164 nitude (10x) faster than JavaScript in this application.
34165 · Rewrite rules for URLs can be performed using JavaScript functions.
34167 · Multiple queries can be made of a view with a single HTTP request.
34169 · Views can be queried with sorting turned off ( sorted=false) for a
34171 performance boost.
34172 · The global changes feed has been enhanced. It is now resumable and
34174 persistent.
34175 · New endpoints added (documentation forthcoming):
34177 · api/server/membership shows all nodes in a cluster
34179 · /_bulk_get speeds up the replication protocol over low-latency con‐
34181 nections
34182 · /_node/ api to access individual nodes’ configuration and com‐
34184 paction features
34185 · /_cluster_setup api to set up a cluster from scratch.
34187 · /_up api to signal health of a node to a load-balancer
34189 · /db/_local_docs and /db/_design_docs (similar to /db/_all_docs)
34191 · The /_log endpoint was removed.
34193 · “Backend” interface on port 5986 used for specific cluster admin
34195 tasks. Of interest are the _nodes and _dbs databases visible only
34196 through this interface.
34197 · Support added for Erlang/OTP 17.x, 18.x and 19
34199 · New streamlined build system written for Unix-like systems and Micro‐
34201 soft Windows
34202 · Configuration has moved from /_config to /_node/{node-name}/_config
34204 · instance_start_time now always reports "0".
34206 Upgrade Notes
34208 · The update sequences returned by the api/db/changes feed are no
34209 longer integers. They can be any JSON value. Applications should
34210 treat them as opaque values and return them to CouchDB as-is.
34211 · Temporary views are no longer supported.
34213 · It is possible to have multiple replicator databases. replicator/db
34215 config option has been removed. Instead _replicator and any database
34216 names ending with the /_replicator suffix will be recognized as
34217 replicator databases by the system.
34218 · Note that the semantics of some API calls have changed due to the
34220 introduction of the clustering feature. Specifically, make note of
34221 the difference between receiving a 201 and a 202 when storing a docu‐
34222 ment.
34223 · all_or_nothing is no longer supported by the bulk_docs API
34225 · After updating a design document containing a show, an immediate GET
34227 to that same show function may still return results from the previous
34228 definition. This is due to design document caching, which may take a
34229 few seconds to fully evict, or longer (up to ~30s) for a clustered
34230 installation.
34231 Known Issues
34233 All known issues filed against the 2.0 release are contained within the
34234 official CouchDB JIRA instance or CouchDB GitHub Issues.
34235 The following are some highlights of known issues for which fixes did
34237 not land in time for the 2.0.0 release:
34238 · COUCHDB-2980: The replicator (whether invoked via _replicate or a
34240 document stored in the _replicator database) understands two kinds of
34241 source and target:
34242 1. A URL (e.g., https://foo:bar@foo.com/db1), called a “remote”
34244 source or target
34245 2. A database name (e.g., db1), called a “local” source or target.
34247 Whenever the latter type is used, this refers to a local unclustered
34249 database, not a clustered one.
34250 In a future release we hope to support “local” source or target specs
34252 to clustered databases. For now, we recommend always using the URL
34253 format for both source and target specifications.
34254 · COUCHDB-3034: CouchDB will occasionally return 500 errors when multi‐
34256 ple clients attempt to PUT or DELETE the same database concurrently.
34257 · COUCHDB-3119: Adding nodes to a cluster fails if the Erlang node name
34259 is not couchdb (of the form couchdb@hostname.)
34260 · COUCHDB-3050: Occasionally the dev/run script used for development
34262 purposes to start a local 3-node cluster will fail to start one or
34263 more nodes.
34264 · COUCHDB-2817: The compaction daemon will only compact views for
34266 shards that contain the design document.
34267 · COUCHDB-2804: The fast_view optimization is not enabled on the clus‐
34269 tered interface.
34270 · #656: The OAuth 1.0 support is broken and deprecated. It will be
34272 removed in a future version of CouchDB.
34273 Breaking Changes
34275 The following changes in 2.0 represent a significant deviation from
34276 CouchDB 1.x and may alter behaviour of systems designed to work with
34277 older versions of CouchDB:
34278 · #620: POST /dbname no longer returns an ETag response header, in com‐
34280 pliance with RFC 7231, Section 7.2.
34281 1.7.x Branch
34283 · Version 1.7.2
34284 · Version 1.7.1
34286 · Version 1.7.0
34288 Version 1.7.2
34290 Security
34291 · CVE 2018-8007
34292 Version 1.7.1
34294 Bug Fix
34295 · #974: Fix access to /db/_all_docs for database members.
34296 Version 1.7.0
34298 Security
34299 · CVE 2017-12635
34300 · CVE 2017-12636
34302 API Changes
34304 · COUCHDB-1356: Return username on POST /_session.
34305 · COUCHDB-1876: Fix duplicated Content-Type for show/update functions.
34307 · COUCHDB-2310: Implement POST /{db}/_bulk_get.
34309 · COUCHDB-2375: 400 Bad Request returned when invalid revision speci‐
34311 fied.
34312 · COUCHDB-2845: 400 Bad Request returned when revs is not a list.
34314 Build
34316 · COUCHDB-1964: Replace etap test suite with EUnit.
34317 · COUCHDB-2225: Enforce that shared libraries can be built by the sys‐
34319 tem.
34320 · COUCHDB-2761: Support glibc >= 2.20.
34322 · COUCHDB-2747: Support Erlang 18.
34324 · #5b9742c: Support Erlang 19.
34326 · #1545bf4: Remove broken benchmarks.
34328 Database Core
34330 · COUCHDB-2534: Improve checks for db admin/member.
34331 · COUCHDB-2735: Duplicate document _ids created under high edit load.
34333 Documentation
34335 · #c3c9588: Improve documentation of cacert_file ssl option.
34336 · #3266f23: Clarify the purpose of tombstones.
34338 · #75887d9: Improve CouchDB Replication Protocol definition.
34340 · #3b1dc0f: Remove mention of group_level=exact.
34342 · #2a11daa: Remove mention of “Test Suite” in Futon.
34344 · #01c60f1: Clarify type of key, startkey and endkey params.
34346 Futon
34348 · COUCHDB-241: Support document copying.
34349 · COUCHDB-1011: Run replication filtered by document ids from Futon.
34351 · COUCHDB-1275: Unescape database names in Futon recently used list.
34353 · #f18f82a: Update jquery.ui to 1.10.4 with fixes of potential XSS
34355 issues.
34356 HTTP Server
34358 · COUCHDB-2430: Disable Nagle’s algorithm by default.
34359 · COUCHDB-2583: Don’t drop connection by the endpoints which doesn’t
34361 require any payload.
34362 · COUCHDB-2673: Properly escape Location: HTTP header.
34364 · COUCHDB-2677: Wrong Expires header weekday.
34366 · COUCHDB-2783: Bind both to IPv4 and IPv6.
34368 · #f30f3dd: Support for user configurable SSL ciphers.
34370 Query Server
34372 · COUCHDB-1447: Custom response headers from design functions get
34373 merged with default ones.
34374 · #7779c11: Upgrade Coffeescript to version 1.10.
34376 jquery.couch.js
34378 · #f9095e7: Fix document copying.
34379 1.6.x Branch
34381 · Upgrade Notes
34382 · Version 1.6.0
34384 Upgrade Notes
34386 The Proxy Authentication handler was renamed to proxy_authentica‐
34387 tion_handler to follow the *_authentication_handler form of all other
34388 handlers. The old proxy_authentification_handler name is marked as dep‐
34389 recated and will be removed in future releases. It’s strongly recom‐
34390 mended to update httpd/authentication_handlers option with new value in
34391 case if you had used such handler.
34392 Version 1.6.0
34394 · COUCHDB-2200: support Erlang/OTP 17.0 #35e16032
34395 · Fauxton: many improvements in our experimental new user interface,
34397 including switching the code editor from CodeMirror to Ace as well as
34398 better support for various browsers.
34399 · Add the max_count option (config/uuids) to allow rate-limiting the
34401 amount of UUIDs that can be requested from the api/server/uuids han‐
34402 dler in a single request (CVE 2014-2668).
34403 · COUCHDB-1986: increase socket buffer size to improve replication
34405 speed for large documents and attachments, and fix tests on BSD-like
34406 systems. #9a0e561b
34407 · COUCHDB-1953: improve performance of multipart/related requests.
34409 #ce3e89dc
34410 · COUCHDB-2221: verify that authentication-related configuration set‐
34412 tings are well-formed. #dbe769c6
34413 · COUCHDB-1922: fix CORS exposed headers. #4f619833
34415 · Rename proxy_authentification_handler to proxy_authentication_han‐
34417 dler. #c66ac4a8
34418 · COUCHDB-1795: ensure the startup script clears the pid file on termi‐
34420 nation. #818ef4f9
34421 · COUCHDB-1962: replication can now be performed without having write
34423 access to the source database (#1d5fe2aa), the replication checkpoint
34424 interval is now configurable (#0693f98e).
34425 · COUCHDB-2025: add support for SOCKS5 proxies for replication.
34427 #fcd76c9
34428 · COUCHDB-1930: redirect to the correct page after submitting a new
34430 document with a different ID than the one suggested by Futon.
34431 #4906b591
34432 · COUCHDB-1923: add support for attachments and att_encoding_info
34434 options (formerly only available on the documents API) to the view
34435 API. #ca41964b
34436 · COUCHDB-1647: for failed replications originating from a document in
34438 the _replicator database, store the failure reason in the document.
34439 #08cac68b
34440 · A number of improvements for the documentation.
34442 1.5.x Branch
34444 · Version 1.5.1
34445 · Version 1.5.0
34447 WARNING:
34449 Version 1.5.1 contains important security fixes. Previous 1.5.x
34450 releases are not recommended for regular usage.
34451 Version 1.5.1
34453 · Add the max_count option (config/uuids) to allow rate-limiting the
34454 amount of UUIDs that can be requested from the api/server/uuids han‐
34455 dler in a single request (CVE 2014-2668).
34456 Version 1.5.0
34458 · COUCHDB-1781: The official documentation has been overhauled. A lot
34459 of content from other sources have been merged, and the index page
34460 has been rebuilt to make the docs much more accessible. #54813a7
34461 · A new administration UI, codenamed Fauxton, has been included as an
34463 experimental preview. It can be accessed at /_utils/fauxton/. There
34464 are too many improvements here to list them all. We are looking for
34465 feedback from the community on this preview release.
34466 · COUCHDB-1888: Fixed an issue where admin users would be restricted by
34468 the public_fields feature.
34469 · Fixed an issue with the JavaScript CLI test runner. #be76882,
34471 #54813a7
34472 · COUCHDB-1867: An experimental plugin feature has been added. See
34474 src/couch_plugin/README.md for details. We invite the community to
34475 test and report any findings.
34476 · COUCHDB-1894: An experimental Node.js-based query server runtime has
34478 been added. See experimental for details. We invite the community to
34479 test and report any findings.
34480 · COUCHDB-1901: Better retry mechanism for transferring attachments
34482 during replication. #4ca2cec
34483 1.4.x Branch
34485 · Upgrade Notes
34486 · Version 1.4.0
34488 WARNING:
34490 1.4.x Branch is affected by the issue described in cve/2014-2668.
34491 Upgrading to a more recent release is strongly recommended.
34492 Upgrade Notes
34494 We now support Erlang/OTP R16B and R16B01; the minimum required version
34495 is R14B.
34496 User document role values must now be strings. Other types of values
34498 will be refused when saving the user document.
34499 Version 1.4.0
34501 · COUCHDB-1139: it’s possible to apply list functions to _all_docs
34502 view. #54fd258e
34503 · COUCHDB-1632: Ignore epilogues in multipart/related MIME attachments.
34505 #2b4ab67a
34506 · COUCHDB-1634: Reduce PBKDF2 work factor. #f726bc4d
34508 · COUCHDB-1684: Support for server-wide changes feed reporting on cre‐
34510 ation, updates and deletion of databases. #917d8988
34511 · COUCHDB-1772: Prevent invalid JSON output when using all_or_nothing
34513 of bulk API. #dfd39d57
34514 · Add a configurable whitelist of user document properties. #8d7ab8b1
34516 · COUCHDB-1852: Support Last-Event-ID header in EventSource changes
34518 feeds. #dfd2199a
34519 · Allow storing pre-hashed admin passwords via config API. #c98ba561
34521 · Automatic loading of CouchDB plugins. #3fab6bb5
34523 · Much improved documentation, including an expanded description of
34525 validate_doc_update functions (commit:ef9ac469) and a description of
34526 how CouchDB handles JSON number values (#bbd93f77).
34527 · Split up replicator_db tests into multiple independent tests.
34529 1.3.x Branch
34531 · Upgrade Notes
34532 · Version 1.3.1
34534 · Version 1.3.0
34536 WARNING:
34538 1.3.x Branch is affected by the issue described in cve/2014-2668.
34539 Upgrading to a more recent release is strongly recommended.
34540 Upgrade Notes
34542 You can upgrade your existing CouchDB 1.0.x installation to 1.3.0 with‐
34543 out any specific steps or migration. When you run CouchDB, the existing
34544 data and index files will be opened and used as normal.
34545 The first time you run a compaction routine on your database within
34547 1.3.0, the data structure and indexes will be updated to the new ver‐
34548 sion of the CouchDB database format that can only be read by CouchDB
34549 1.3.0 and later. This step is not reversible. Once the data files have
34550 been updated and migrated to the new version the data files will no
34551 longer work with a CouchDB 1.0.x release.
34552 WARNING:
34554 If you want to retain support for opening the data files in CouchDB
34555 1.0.x you must back up your data files before performing the upgrade
34556 and compaction process.
34557 Version 1.3.1
34559 Replicator
34560 · COUCHDB-1788: Tolerate missing source and target fields in _replica‐
34561 tor docs. #869f42e2
34562 Log System
34564 · COUCHDB-1794: Fix bug in WARN level logging from 1.3.0.
34565 · Don’t log about missing .compact files. #06f1a8dc
34567 View Server
34569 · COUCHDB-1792: Fix the -S option to couchjs to increase memory limits.
34570 #cfaa66cd
34571 Miscellaneous
34573 · COUCHDB-1784: Improvements to test suite and VPATH build system.
34574 #01afaa4f
34575 · Improve documentation: better structure, improve language, less
34577 duplication.
34578 Version 1.3.0
34580 Database core
34581 · COUCHDB-1512: Validate bind address before assignment. #09ead8a0
34582 · Restore max_document_size protection. #bf1eb135
34584 Documentation
34586 · COUCHDB-1523: Import CouchBase documentation and convert them into
34587 Sphinx docs
34588 Futon
34590 · COUCHDB-509: Added view request duration to Futon. #2d2c7d1e
34591 · COUCHDB-627: Support all timezones. #b1a049bb
34593 · COUCHDB-1383: Futon view editor won’t allow you to save original view
34595 after saving a revision. #ce48342
34596 · COUCHDB-1470: Futon raises pop-up on attempt to navigate to
34598 missed/deleted document. #5da40eef
34599 · COUCHDB-1473, COUCHDB-1472: Disable buttons for actions that the user
34601 doesn’t have permissions to. #7156254d
34602 HTTP Interface
34604 · COUCHDB-431: Introduce experimental CORS support. #b90e4021
34605 · COUCHDB-764, COUCHDB-514, COUCHDB-430: Fix sending HTTP headers from
34607 _list function, #2a74f88375
34608 · COUCHDB-887: Fix bytes and offset parameters semantic for _log
34610 resource (explanation) #ad700014
34611 · COUCHDB-986: Added Server-Sent Events protocol to db changes API.
34613 See http://www.w3.org/TR/eventsource/ for details. #093d2aa6
34614 · COUCHDB-1026: Database names are encoded with respect of special
34616 characters in the rewriter now. #272d6415
34617 · COUCHDB-1097: Allow OPTIONS request to shows and lists functions.
34619 #9f53704a
34620 · COUCHDB-1210: Files starting with underscore can be attached and
34622 updated now. #05858792
34623 · COUCHDB-1277: Better query parameter support and code clarity:
34625 #7e3c69ba
34626 · Responses to documents created/modified via form data POST to
34628 /db/doc or copied with COPY should now include Location header.
34629 · Form data POST to /db/doc now includes an ETag response header.
34631 · ?batch=ok is now supported for COPY and POST /db/doc updates.
34633 · ?new_edits=false is now supported for more operations.
34635 · COUCHDB-1285: Allow configuration of vendor and modules version in
34637 CouchDB welcome message. #3c24a94d
34638 · COUCHDB-1321: Variables in rewrite rules breaks OAuth authentication.
34640 #c307ba95
34641 · COUCHDB-1337: Use MD5 for attachment ETag header value. #6d912c9f
34643 · COUCHDB-1381: Add jquery.couch support for Windows 8 Metro apps.
34645 #dfc5d37c
34646 · COUCHDB-1441: Limit recursion depth in the URL rewriter. Defaults to
34648 a maximum of 100 invocations but is configurable. #d076976c
34649 · COUCHDB-1442: No longer rewrites the X-CouchDB-Requested-Path during
34651 recursive calls to the rewriter. #56744f2f
34652 · COUCHDB-1501: Changes feed now can take special parameter since=now
34654 to emit changes since current point of time. #3bbb2612
34655 · COUCHDB-1502: Allow users to delete own _users doc. #f0d6f19bc8
34657 · COUCHDB-1511: CouchDB checks roles field for _users database docu‐
34659 ments with more care. #41205000
34660 · COUCHDB-1537: Include user name in show/list ETags. #ac320479
34662 · Send a 202 response for _restart. #b213e16f
34664 · Make password hashing synchronous when using the /_config/admins API.
34666 #08071a80
34667 · Add support to serve single file with CouchDB, #2774531ff2
34669 · Allow any 2xx code to indicate success, #0d50103cfd
34671 · Fix _session for IE7.
34673 · Restore 400 error for empty PUT, #2057b895
34675 · Return X-Couch-Id header if doc is created, #98515bf0b9
34677 · Support auth cookies with : characters, #d9566c831d
34679 Log System
34681 · COUCHDB-1380: Minor fixes for logrotate support.
34682 · Improve file I/O error logging and handling, #4b6475da
34684 · Module Level Logging, #b58f069167
34686 · Log 5xx responses at error level, #e896b0b7
34688 · Log problems opening database at ERROR level except for auto-created
34690 system dbs, #41667642f7
34691 Replicator
34693 · COUCHDB-1248: HTTP 500 error now doesn’t occurs when replicating with
34694 ?doc_ids=null. #bea76dbf
34695 · COUCHDB-1259: Stabilize replication id, #c6252d6d7f
34697 · COUCHDB-1323: Replicator now acts as standalone application.
34699 #f913ca6e
34700 · COUCHDB-1363: Fix rarely occurred, but still race condition in
34702 changes feed if a quick burst of changes happens while replication is
34703 starting the replication can go stale. #573a7bb9
34704 · COUCHDB-1557: Upgrade some code to use BIFs bring good improvements
34706 for replication.
34707 Security
34709 · COUCHDB-1060: Passwords are now hashed using the PBKDF2 algorithm
34710 with a configurable work factor. #7d418134
34711 Source Repository
34713 · The source repository was migrated from SVN to Git.
34714 Storage System
34716 · Fixed unnecessary conflict when deleting and creating a document in
34717 the same batch.
34718 Test Suite
34720 · COUCHDB-1321: Moved the JS test suite to the CLI.
34721 · COUCHDB-1338: Start CouchDB with port=0. While CouchDB might be
34723 already running on the default port 5984, port number 0 let the TCP
34724 stack figure out a free port to run. #127cbe3
34725 · COUCHDB-1339: Use shell trap to catch dying beam processes during
34727 test runs. #2921c78
34728 · COUCHDB-1389: Improved tracebacks printed by the JS CLI tests.
34730 · COUCHDB-1563: Ensures urlPrefix is set in all ajax requests.
34732 #07a6af222
34733 · Fix race condition for test running on faster hardware.
34735 · Improved the reliability of a number of tests.
34737 URL Rewriter & Vhosts
34739 · COUCHDB-1026: Database name is encoded during rewriting (allowing
34740 embedded /’s, etc). #272d6415
34741 UUID Algorithms
34743 · COUCHDB-1373: Added the utc_id algorithm #5ab712a2
34744 Query and View Server
34746 · COUCHDB-111: Improve the errors reported by the JavaScript view
34747 server to provide a more friendly error report when something goes
34748 wrong. #0c619ed
34749 · COUCHDB-410: More graceful error handling for JavaScript vali‐
34751 date_doc_update functions.
34752 · COUCHDB-1372: _stats built-in reduce function no longer produces
34754 error for empty view result.
34755 · COUCHDB-1444: Fix missed_named_view error that occurs on existed
34757 design documents and views. #b59ac98b
34758 · COUCHDB-1445: CouchDB tries no more to delete view file if it
34760 couldn’t open it, even if the error is emfile.
34761 · COUCHDB-1483: Update handlers requires valid doc ids. #72ea7e38
34763 · COUCHDB-1491: Clean up view tables. #c37204b7
34765 · Deprecate E4X support, #cdfdda2314
34767 Windows
34769 · COUCHDB-1482: Use correct linker flag to build snappy_nif.dll on Win‐
34770 dows. #a6eaf9f1
34771 · Allows building cleanly on Windows without cURL, #fb670f5712
34773 1.2.x Branch
34775 · Upgrade Notes
34776 · Version 1.2.2
34778 · Version 1.2.1
34780 · Version 1.2.0
34782 Upgrade Notes
34784 WARNING:
34785 This version drops support for the database format that was intro‐
34786 duced in version 0.9.0. Compact your older databases (that have not
34787 been compacted for a long time) before upgrading, or they will
34788 become inaccessible.
34789 WARNING:
34791 Version 1.2.1 contains important security fixes. Previous 1.2.x
34792 releases are not recommended for regular usage.
34793 Security changes
34795 The interface to the _users and _replicator databases have been changed
34796 so that non-administrator users can see less information:
34797 · In the _users database:
34799 · User documents can now only be read by the respective users, as
34801 well as administrators. Other users cannot read these documents.
34802 · Views can only be defined and queried by administrator users.
34804 · The _changes feed can only be queried by administrator users.
34806 · In the _replicator database:
34808 · Documents now have a forced owner field that corresponds to the
34810 authenticated user that created them.
34811 · Non-owner users will not see confidential information like pass‐
34813 words or OAuth tokens in replication documents; they can still see
34814 the other contents of those documents. Administrators can see
34815 everything.
34816 · Views can only be defined and queried by administrators.
34818 Database Compression
34820 The new optional (but enabled by default) compression of disk files
34821 requires an upgrade of the on-disk format (5 -> 6) which occurs on cre‐
34822 ation for new databases and views, and on compaction for existing
34823 files. This format is not supported in previous releases, so rollback
34824 would require replication to the previous CouchDB release or restoring
34825 from backup.
34826 Compression can be disabled by setting compression = none in your
34828 local.ini [couchdb] section, but the on-disk format will still be
34829 upgraded.
34830 Version 1.2.2
34832 Build System
34833 · Fixed issue in couchdb script where stopped status returns before
34834 process exits.
34835 HTTP Interface
34837 · Reset rewrite counter on new request, avoiding unnecessary request
34838 failures due to bogus rewrite limit reports.
34839 Version 1.2.1
34841 Build System
34842 · Fix couchdb start script.
34843 · Win: fix linker invocations.
34845 Futon
34847 · Disable buttons that aren’t available for the logged-in user.
34848 HTTP Interface
34850 · No longer rewrites the X-CouchDB-Requested-Path during recursive
34851 calls to the rewriter.
34852 · Limit recursion depth in the URL rewriter. Defaults to a maximum of
34854 100 invocations but is configurable.
34855 Security
34857 · Fixed cve/2012-5641
34858 · Fixed cve/2012-5649
34860 · Fixed cve/2012-5650
34862 Replication
34864 · Fix potential timeouts.
34865 View Server
34867 · Change use of signals to avoid broken view groups.
34868 Version 1.2.0
34870 Authentication
34871 · Fix use of OAuth with VHosts and URL rewriting.
34872 · OAuth secrets can now be stored in the users system database as an
34874 alternative to key value pairs in the .ini configuration. By default
34875 this is disabled (secrets are stored in the .ini) but can be enabled
34876 via the .ini configuration key use_users_db in the couch_httpd_oauth
34877 section.
34878 · Documents in the _users database are no longer publicly readable.
34880 · Confidential information in the _replication database is no longer
34882 publicly readable.
34883 · Password hashes are now calculated by CouchDB. Clients are no longer
34885 required to do this manually.
34886 · Cookies used for authentication can be made persistent by enabling
34888 the .ini configuration key allow_persistent_cookies in the
34889 couch_httpd_auth section.
34890 Build System
34892 · cURL is no longer required to build CouchDB as it is only used by the
34893 command line JS test runner. If cURL is available when building
34894 CouchJS you can enable the HTTP bindings by passing -H on the command
34895 line.
34896 · Temporarily made make check pass with R15B. A more thorough fix is in
34898 the works (COUCHDB-1424).
34899 · Fixed –with-js-include and –with-js-lib options.
34901 · Added –with-js-lib-name option.
34903 Futon
34905 · The Status screen (active tasks) now displays two new task status
34906 fields: Started on and Updated on.
34907 · Futon remembers view code every time it is saved, allowing to save an
34909 edit that amounts to a revert.
34910 HTTP Interface
34912 · Added a native JSON parser.
34913 · The _active_tasks API now offers more granular fields. Each task type
34915 is now able to expose different properties.
34916 · Added built-in changes feed filter _view.
34918 · Fixes to the _changes feed heartbeat option which caused heartbeats
34920 to be missed when used with a filter. This caused timeouts of contin‐
34921 uous pull replications with a filter.
34922 · Properly restart the SSL socket on configuration changes.
34924 OAuth
34926 · Updated bundled erlang_oauth library to the latest version.
34927 Replicator
34929 · A new replicator implementation. It offers more performance and con‐
34930 figuration options.
34931 · Passing non-string values to query_params is now a 400 bad request.
34933 This is to reduce the surprise that all parameters are converted to
34934 strings internally.
34935 · Added optional field since_seq to replication objects/documents. It
34937 allows to bootstrap a replication from a specific source sequence
34938 number.
34939 · Simpler replication cancellation. In addition to the current method,
34941 replications can now be canceled by specifying the replication ID
34942 instead of the original replication object/document.
34943 Storage System
34945 · Added optional database and view index file compression (using
34946 Google’s snappy or zlib’s deflate). This feature is enabled by
34947 default, but it can be disabled by adapting local.ini accordingly.
34948 The on-disk format is upgraded on compaction and new DB/view creation
34949 to support this.
34950 · Several performance improvements, most notably regarding database
34952 writes and view indexing.
34953 · Computation of the size of the latest MVCC snapshot data and all its
34955 supporting metadata, both for database and view index files. This
34956 information is exposed as the data_size attribute in the database and
34957 view group information URIs.
34958 · The size of the buffers used for database and view compaction is now
34960 configurable.
34961 · Added support for automatic database and view compaction. This fea‐
34963 ture is disabled by default, but it can be enabled via the .ini con‐
34964 figuration.
34965 · Performance improvements for the built-in changes feed filters
34967 _doc_ids and _design.
34968 View Server
34970 · Add CoffeeScript (http://coffeescript.org/) as a first class view
34971 server language.
34972 · Fixed old index file descriptor leaks after a view cleanup.
34974 · The requested_path property keeps the pre-rewrite path even when no
34976 VHost configuration is matched.
34977 · Fixed incorrect reduce query results when using pagination parame‐
34979 ters.
34980 · Made icu_driver work with Erlang R15B and later.
34982 1.1.x Branch
34984 · Upgrade Notes
34985 · Version 1.1.2
34987 · Version 1.1.1
34989 · Version 1.1.0
34991 Upgrade Notes
34993 WARNING:
34994 Version 1.1.2 contains important security fixes. Previous 1.1.x
34995 releases are not recommended for regular usage.
34996 Version 1.1.2
34998 Build System
34999 · Don’t ln the couchjs install target on Windows
35000 · Remove ICU version dependency on Windows.
35002 · Improve SpiderMonkey version detection.
35004 HTTP Interface
35006 · ETag of attachment changes only when the attachment changes, not the
35007 document.
35008 · Fix retrieval of headers larger than 4k.
35010 · Allow OPTIONS HTTP method for list requests.
35012 · Don’t attempt to encode invalid json.
35014 Log System
35016 · Improvements to log messages for file-related errors.
35017 Replicator
35019 · Fix pull replication of documents with many revisions.
35020 · Fix replication from an HTTP source to an HTTP target.
35022 Security
35024 · Fixed cve/2012-5641
35025 · Fixed cve/2012-5649
35027 · Fixed cve/2012-5650
35029 View Server
35031 · Avoid invalidating view indexes when running out of file descriptors.
35032 Version 1.1.1
35034 · Support SpiderMonkey 1.8.5
35035 · Add configurable maximum to the number of bytes returned by _log.
35037 · Allow CommonJS modules to be an empty string.
35039 · Bump minimum Erlang version to R13B02.
35041 · Do not run deleted validate_doc_update functions.
35043 · ETags for views include current sequence if include_docs=true.
35045 · Fix bug where duplicates can appear in _changes feed.
35047 · Fix bug where update handlers break after conflict resolution.
35049 · Fix bug with _replicator where include “filter” could crash couch.
35051 · Fix crashes when compacting large views.
35053 · Fix file descriptor leak in _log
35055 · Fix missing revisions in _changes?style=all_docs.
35057 · Improve handling of compaction at max_dbs_open limit.
35059 · JSONP responses now send “text/javascript” for Content-Type.
35061 · Link to ICU 4.2 on Windows.
35063 · Permit forward slashes in path to update functions.
35065 · Reap couchjs processes that hit reduce_overflow error.
35067 · Status code can be specified in update handlers.
35069 · Support provides() in show functions.
35071 · _view_cleanup when ddoc has no views now removes all index files.
35073 · max_replication_retry_count now supports “infinity”.
35075 · Fix replication crash when source database has a document with empty
35077 ID.
35078 · Fix deadlock when assigning couchjs processes to serve requests.
35080 · Fixes to the document multipart PUT API.
35082 · Fixes regarding file descriptor leaks for databases with views.
35084 Version 1.1.0
35086 NOTE:
35087 All CHANGES for 1.0.2 and 1.0.3 also apply to 1.1.0.
35088 Externals
35090 · Added OS Process module to manage daemons outside of CouchDB.
35091 · Added HTTP Proxy handler for more scalable externals.
35093 Futon
35095 · Added a “change password”-feature to Futon.
35096 HTTP Interface
35098 · Native SSL support.
35099 · Added support for HTTP range requests for attachments.
35101 · Added built-in filters for _changes: _doc_ids and _design.
35103 · Added configuration option for TCP_NODELAY aka “Nagle”.
35105 · Allow POSTing arguments to _changes.
35107 · Allow keys parameter for GET requests to views.
35109 · Allow wildcards in vhosts definitions.
35111 · More granular ETag support for views.
35113 · More flexible URL rewriter.
35115 · Added support for recognizing “Q values” and media parameters in HTTP
35117 Accept headers.
35118 · Validate doc ids that come from a PUT to a URL.
35120 Replicator
35122 · Added _replicator database to manage replications.
35123 · Fixed issues when an endpoint is a remote database accessible via
35125 SSL.
35126 · Added support for continuous by-doc-IDs replication.
35128 · Fix issue where revision info was omitted when replicating attach‐
35130 ments.
35131 · Integrity of attachment replication is now verified by MD5.
35133 Storage System
35135 · Multiple micro-optimizations when reading data.
35136 URL Rewriter & Vhosts
35138 · Fix for variable substitution
35139 View Server
35141 · Added CommonJS support to map functions.
35142 · Added stale=update_after query option that triggers a view update
35144 after returning a stale=ok response.
35145 · Warn about empty result caused by startkey and endkey limiting.
35147 · Built-in reduce function _sum now accepts lists of integers as input.
35149 · Added view query aliases start_key, end_key, start_key_doc_id and
35151 end_key_doc_id.
35152 1.0.x Branch
35154 · Upgrade Notes
35155 · Version 1.0.4
35157 · Version 1.0.3
35159 · Version 1.0.2
35161 · Version 1.0.1
35163 · Version 1.0.0
35165 Upgrade Notes
35167 Note, to replicate with a 1.0 CouchDB instance you must first upgrade
35168 in-place your current CouchDB to 1.0 or 0.11.1 – backporting so that
35169 0.10.x can replicate to 1.0 wouldn’t be that hard. All that is required
35170 is patching the replicator to use the application/json content type.
35171 · _log and _temp_views are now admin-only resources.
35173 · _bulk_docs now requires a valid Content-Type header of applica‐
35175 tion/json.
35176 · JSONP is disabled by default. An .ini option was added to selectively
35178 enable it.
35179 · The key, startkey and endkey properties of the request object passed
35181 to list and show functions now contain JSON objects representing the
35182 URL encoded string values in the query string. Previously, these
35183 properties contained strings which needed to be converted to JSON
35184 before using.
35185 WARNING:
35187 Version 1.0.4 contains important security fixes. Previous 1.0.x
35188 releases are not recommended for regular usage.
35189 Version 1.0.4
35191 HTTP Interface
35192 · Fix missing revisions in _changes?style=all_docs.
35193 · Fix validation of attachment names.
35195 Log System
35197 · Fix file descriptor leak in _log.
35198 Replicator
35200 · Fix a race condition where replications can go stale
35201 Security
35203 · Fixed cve/2012-5641
35204 · Fixed cve/2012-5649
35206 · Fixed cve/2012-5650
35208 View System
35210 · Avoid invalidating view indexes when running out of file descriptors.
35211 Version 1.0.3
35213 General
35214 · Fixed compatibility issues with Erlang R14B02.
35215 Etap Test Suite
35217 · Etap tests no longer require use of port 5984. They now use a ran‐
35218 domly selected port so they won’t clash with a running CouchDB.
35219 Futon
35221 · Made compatible with jQuery 1.5.x.
35222 HTTP Interface
35224 · Fix bug that allows invalid UTF-8 after valid escapes.
35225 · The query parameter include_docs now honors the parameter conflicts.
35227 This applies to queries against map views, _all_docs and _changes.
35228 · Added support for inclusive_end with reduce views.
35230 Replicator
35232 · Enabled replication over IPv6.
35233 · Fixed for crashes in continuous and filtered changes feeds.
35235 · Fixed error when restarting replications in OTP R14B02.
35237 · Upgrade ibrowse to version 2.2.0.
35239 · Fixed bug when using a filter and a limit of 1.
35241 Security
35243 · Fixed OAuth signature computation in OTP R14B02.
35244 · Handle passwords with : in them.
35246 Storage System
35248 · More performant queries against _changes and _all_docs when using the
35249 include_docs parameter.
35250 Windows
35252 · Windows builds now require ICU >= 4.4.0 and Erlang >= R14B03. See
35253 COUCHDB-1152, and COUCHDB-963 + OTP-9139 for more information.
35254 Version 1.0.2
35256 Futon
35257 · Make test suite work with Safari and Chrome.
35258 · Fixed animated progress spinner.
35260 · Fix raw view document link due to overzealous URI encoding.
35262 · Spell javascript correctly in loadScript(uri).
35264 HTTP Interface
35266 · Allow reduce=false parameter in map-only views.
35267 · Fix parsing of Accept headers.
35269 · Fix for multipart GET APIs when an attachment was created during a
35271 local-local replication. See COUCHDB-1022 for details.
35272 Log System
35274 · Reduce lengthy stack traces.
35275 · Allow logging of native <xml> types.
35277 Replicator
35279 · Updated ibrowse library to 2.1.2 fixing numerous replication issues.
35280 · Make sure that the replicator respects HTTP settings defined in the
35282 config.
35283 · Fix error when the ibrowse connection closes unexpectedly.
35285 · Fix authenticated replication (with HTTP basic auth) of design docu‐
35287 ments with attachments.
35288 · Various fixes to make replication more resilient for edge-cases.
35290 Storage System
35292 · Fix leaking file handles after compacting databases and views.
35293 · Fix databases forgetting their validation function after compaction.
35295 · Fix occasional timeout errors after successfully compacting large
35297 databases.
35298 · Fix occasional error when writing to a database that has just been
35300 compacted.
35301 · Fix occasional timeout errors on systems with slow or heavily loaded
35303 IO.
35304 · Fix for OOME when compactions include documents with many conflicts.
35306 · Fix for missing attachment compression when MIME types included
35308 parameters.
35309 · Preserve purge metadata during compaction to avoid spurious view
35311 rebuilds.
35312 · Fix spurious conflicts introduced when uploading an attachment after
35314 a doc has been in a conflict. See COUCHDB-902 for details.
35315 · Fix for frequently edited documents in multi-master deployments being
35317 duplicated in _changes and _all_docs. See COUCHDB-968 for details on
35318 how to repair.
35319 · Significantly higher read and write throughput against database and
35321 view index files.
35322 View Server
35324 · Don’t trigger view updates when requesting _design/doc/_info.
35325 · Fix for circular references in CommonJS requires.
35327 · Made isArray() function available to functions executed in the query
35329 server.
35330 · Documents are now sealed before being passed to map functions.
35332 · Force view compaction failure when duplicated document data exists.
35334 When this error is seen in the logs users should rebuild their views
35335 from scratch to fix the issue. See COUCHDB-999 for details.
35336 Version 1.0.1
35338 Authentication
35339 ·
35340 Enable basic-auth popup when required to access the server, to pre‐
35342 vent
35343 people from getting locked out.
35344 Build and System Integration
35346 · Included additional source files for distribution.
35347 Futon
35349 · User interface element for querying stale (cached) views.
35350 HTTP Interface
35352 · Expose committed_update_seq for monitoring purposes.
35353 · Show fields saved along with _deleted=true. Allows for auditing of
35355 deletes.
35356 · More robust Accept-header detection.
35358 Replicator
35360 · Added support for replication via an HTTP/HTTPS proxy.
35361 · Fix pull replication of attachments from 0.11 to 1.0.x.
35363 · Make the _changes feed work with non-integer seqnums.
35365 Storage System
35367 · Fix data corruption bug COUCHDB-844. Please see
35368 http://couchdb.apache.org/notice/1.0.1.html for details.
35369 Version 1.0.0
35371 Security
35372 · Added authentication caching, to avoid repeated opening and closing
35373 of the users database for each request requiring authentication.
35374 Storage System
35376 · Small optimization for reordering result lists.
35377 · More efficient header commits.
35379 · Use O_APPEND to save lseeks.
35381 · Faster implementation of pread_iolist(). Further improves performance
35383 on concurrent reads.
35384 View Server
35386 · Faster default view collation.
35387 · Added option to include update_seq in view responses.
35389 0.11.x Branch
35391 · Upgrade Notes
35392 · Version 0.11.2
35394 · Version 0.11.1
35396 · Version 0.11.0
35398 Upgrade Notes
35400 WARNING:
35401 Version 0.11.2 contains important security fixes. Previous 0.11.x
35402 releases are not recommended for regular usage.
35403 Changes Between 0.11.0 and 0.11.1
35405 · _log and _temp_views are now admin-only resources.
35406 · _bulk_docs now requires a valid Content-Type header of applica‐
35408 tion/json.
35409 · JSONP is disabled by default. An .ini option was added to selectively
35411 enable it.
35412 · The key, startkey and endkey properties of the request object passed
35414 to list and show functions now contain JSON objects representing the
35415 URL encoded string values in the query string. Previously, these
35416 properties contained strings which needed to be converted to JSON
35417 before using.
35418 Changes Between 0.10.x and 0.11.0
35420 show, list, update and validation functions
35421 The req argument to show, list, update and validation functions now
35422 contains the member method with the specified HTTP method of the cur‐
35423 rent request. Previously, this member was called verb. method is fol‐
35424 lowing RFC 2616 (HTTP 1.1) closer.
35425 _admins -> _security
35427 The /db/_admins handler has been removed and replaced with a /db/_secu‐
35428 rity object. Any existing _admins will be dropped and need to be added
35429 to the security object again. The reason for this is that the old sys‐
35430 tem made no distinction between names and roles, while the new one
35431 does, so there is no way to automatically upgrade the old admins list.
35432 The security object has 2 special fields, admins and readers, which
35434 contain lists of names and roles which are admins or readers on that
35435 database. Anything else may be stored in other fields on the security
35436 object. The entire object is made available to validation functions.
35437 json2.js
35439 JSON handling in the query server has been upgraded to use json2.js.
35440 This allows us to use faster native JSON serialization when it is
35441 available.
35442 In previous versions, attempts to serialize undefined would throw an
35444 exception, causing the doc that emitted undefined to be dropped from
35445 the view index. The new behavior is to serialize undefined as null.
35446 Applications depending on the old behavior will need to explicitly
35447 check for undefined.
35448 Another change is that E4X’s XML objects will not automatically be
35450 stringified. XML users will need to call my_xml_object.toXMLString() to
35451 return a string value. #8d3b7ab3
35452 WWW-Authenticate
35454 The default configuration has been changed to avoid causing basic-auth
35455 popups which result from sending the WWW-Authenticate header. To enable
35456 basic-auth popups, uncomment the config option httpd/WWW-Authenticate
35457 line in local.ini.
35458 Query server line protocol
35460 The query server line protocol has changed for all functions except
35461 map, reduce, and rereduce. This allows us to cache the entire design
35462 document in the query server process, which results in faster perfor‐
35463 mance for common operations. It also gives more flexibility to query
35464 server implementators and shouldn’t require major changes in the future
35465 when adding new query server features.
35466 UTF8 JSON
35468 JSON request bodies are validated for proper UTF-8 before saving,
35469 instead of waiting to fail on subsequent read requests.
35470 _changes line format
35472 Continuous changes are now newline delimited, instead of having each
35473 line followed by a comma.
35474 Version 0.11.2
35476 Authentication
35477 · User documents can now be deleted by admins or the user.
35478 Futon
35480 · Add some Futon files that were missing from the Makefile.
35481 HTTP Interface
35483 · Better error messages on invalid URL requests.
35484 Replicator
35486 · Fix bug when pushing design docs by non-admins, which was hanging the
35487 replicator for no good reason.
35488 · Fix bug when pulling design documents from a source that requires
35490 basic-auth.
35491 Security
35493 · Avoid potential DOS attack by guarding all creation of atoms.
35494 · Fixed cve/2010-2234
35496 Version 0.11.1
35498 Build and System Integration
35499 · Output of couchdb –help has been improved.
35500 · Fixed compatibility with the Erlang R14 series.
35502 · Fixed warnings on Linux builds.
35504 · Fixed build error when aclocal needs to be called during the build.
35506 · Require ICU 4.3.1.
35508 · Fixed compatibility with Solaris.
35510 Configuration System
35512 · Fixed timeout with large .ini files.
35513 Futon
35515 · Use “expando links” for over-long document values in Futon.
35516 · Added continuous replication option.
35518 · Added option to replicating test results anonymously to a community
35520 CouchDB instance.
35521 · Allow creation and deletion of config entries.
35523 · Fixed display issues with doc ids that have escaped characters.
35525 · Fixed various UI issues.
35527 HTTP Interface
35529 · Mask passwords in active tasks and logging.
35530 · Update mochijson2 to allow output of BigNums not in float form.
35532 · Added support for X-HTTP-METHOD-OVERRIDE.
35534 · Better error message for database names.
35536 · Disable jsonp by default.
35538 · Accept gzip encoded standalone attachments.
35540 · Made max_concurrent_connections configurable.
35542 · Made changes API more robust.
35544 · Send newly generated document rev to callers of an update function.
35546 JavaScript Clients
35548 · Added tests for couch.js and jquery.couch.js
35549 · Added changes handler to jquery.couch.js.
35551 · Added cache busting to jquery.couch.js if the user agent is msie.
35553 · Added support for multi-document-fetch (via _all_docs) to
35555 jquery.couch.js.
35556 · Added attachment versioning to jquery.couch.js.
35558 · Added option to control ensure_full_commit to jquery.couch.js.
35560 · Added list functionality to jquery.couch.js.
35562 · Fixed issues where bulkSave() wasn’t sending a POST body.
35564 Log System
35566 · Log HEAD requests as HEAD, not GET.
35567 · Keep massive JSON blobs out of the error log.
35569 · Fixed a timeout issue.
35571 Replication System
35573 · Refactored various internal APIs related to attachment streaming.
35574 · Fixed hanging replication.
35576 · Fixed keepalive issue.
35578 Security
35580 · Added authentication redirect URL to log in clients.
35581 · Fixed query parameter encoding issue in oauth.js.
35583 · Made authentication timeout configurable.
35585 · Temporary views are now admin-only resources.
35587 Storage System
35589 · Don’t require a revpos for attachment stubs.
35590 · Added checking to ensure when a revpos is sent with an attachment
35592 stub, it’s correct.
35593 · Make file deletions async to avoid pauses during compaction and db
35595 deletion.
35596 · Fixed for wrong offset when writing headers and converting them to
35598 blocks, only triggered when header is larger than 4k.
35599 · Preserve _revs_limit and instance_start_time after compaction.
35601 Test Suite
35603 · Made the test suite overall more reliable.
35604 View Server
35606 · Provide a UUID to update functions (and all other functions) that
35607 they can use to create new docs.
35608 · Upgrade CommonJS modules support to 1.1.1.
35610 · Fixed erlang filter funs and normalize filter fun API.
35612 · Fixed hang in view shutdown.
35614 URL Rewriter & Vhosts
35616 · Allow more complex keys in rewriter.
35617 · Allow global rewrites so system defaults are available in vhosts.
35619 · Allow isolation of databases with vhosts.
35621 · Fix issue with passing variables to query parameters.
35623 Version 0.11.0
35625 Build and System Integration
35626 · Updated and improved source documentation.
35627 · Fixed distribution preparation for building on Mac OS X.
35629 · Added support for building a Windows installer as part of ‘make
35631 dist’.
35632 · Bug fix for building couch.app’s module list.
35634 · ETap tests are now run during make distcheck. This included a number
35636 of updates to the build system to properly support VPATH builds.
35637 · Gavin McDonald set up a build-bot instance. More info can be found at
35639 http://ci.apache.org/buildbot.html
35640 Futon
35642 · Added a button for view compaction.
35643 · JSON strings are now displayed as-is in the document view, without
35645 the escaping of new-lines and quotes. That dramatically improves
35646 readability of multi-line strings.
35647 · Same goes for editing of JSON string values. When a change to a field
35649 value is submitted, and the value is not valid JSON it is assumed to
35650 be a string. This improves editing of multi-line strings a lot.
35651 · Hitting tab in textareas no longer moves focus to the next form
35653 field, but simply inserts a tab character at the current caret posi‐
35654 tion.
35655 · Fixed some font declarations.
35657 HTTP Interface
35659 · Provide Content-MD5 header support for attachments.
35660 · Added URL Rewriter handler.
35662 · Added virtual host handling.
35664 Replication
35666 · Added option to implicitly create replication target databases.
35667 · Avoid leaking file descriptors on automatic replication restarts.
35669 · Added option to replicate a list of documents by id.
35671 · Allow continuous replication to be cancelled.
35673 Runtime Statistics
35675 · Statistics are now calculated for a moving window instead of
35676 non-overlapping timeframes.
35677 · Fixed a problem with statistics timers and system sleep.
35679 · Moved statistic names to a term file in the priv directory.
35681 Security
35683 · Fixed CVE-2010-0009: Apache CouchDB Timing Attack Vulnerability.
35684 · Added default cookie-authentication and users database.
35686 · Added Futon user interface for user signup and login.
35688 · Added per-database reader access control lists.
35690 · Added per-database security object for configuration data in valida‐
35692 tion functions.
35693 · Added proxy authentication handler
35695 Storage System
35697 · Adds batching of multiple updating requests, to improve throughput
35698 with many writers. Removed the now redundant couch_batch_save module.
35699 · Adds configurable compression of attachments.
35701 View Server
35703 · Added optional ‘raw’ binary collation for faster view builds where
35704 Unicode collation is not important.
35705 · Improved view index build time by reducing ICU collation callouts.
35707 · Improved view information objects.
35709 · Bug fix for partial updates during view builds.
35711 · Move query server to a design-doc based protocol.
35713 · Use json2.js for JSON serialization for compatiblity with native
35715 JSON.
35716 · Major refactoring of couchjs to lay the groundwork for disabling cURL
35718 support. The new HTTP interaction acts like a synchronous XHR. Exam‐
35719 ple usage of the new system is in the JavaScript CLI test runner.
35720 0.10.x Branch
35722 · Upgrade Notes
35723 · Version 0.10.2
35725 · Version 0.10.1
35727 · Version 0.10.0
35729 Upgrade Notes
35731 WARNING:
35732 Version 0.10.2 contains important security fixes. Previous 0.10.x
35733 releases are not recommended for regular usage.
35734 Modular Configuration Directories
35736 CouchDB now loads configuration from the following places (glob(7) syn‐
35737 tax) in order:
35738 · PREFIX/default.ini
35740 · PREFIX/default.d/*
35742 · PREFIX/local.ini
35744 · PREFIX/local.d/*
35746 The configuration options for couchdb script have changed to:
35748 -a FILE add configuration FILE to chain
35750 -A DIR add configuration DIR to chain
35751 -n reset configuration file chain (including system default)
35752 -c print configuration file chain and exit
35753 Show and List API change
35755 Show and List functions must have a new structure in 0.10. See
35756 Formatting_with_Show_and_List for details.
35757 Stricter enforcing of reduciness in reduce-functions
35759 Reduce functions are now required to reduce the number of values for a
35760 key.
35761 View query reduce parameter strictness
35763 CouchDB now considers the parameter reduce=false to be an error for
35764 queries of map-only views, and responds with status code 400.
35765 Version 0.10.2
35767 Build and System Integration
35768 · Fixed distribution preparation for building on Mac OS X.
35769 Security
35771 · Fixed cve/2010-0009
35772 Replicator
35774 · Avoid leaking file descriptors on automatic replication restarts.
35775 Version 0.10.1
35777 Build and System Integration
35778 · Test suite now works with the distcheck target.
35779 Replicator
35781 · Stability enhancements regarding redirects, timeouts, OAuth.
35782 Query Server
35784 · Avoid process leaks
35785 · Allow list and view to span languages
35787 Stats
35789 · Eliminate new process flood on system wake
35790 Version 0.10.0
35792 Build and System Integration
35793 · Changed couchdb script configuration options.
35794 · Added default.d and local.d configuration directories to load
35796 sequence.
35797 HTTP Interface
35799 · Added optional cookie-based authentication handler.
35800 · Added optional two-legged OAuth authentication handler.
35802 Storage Format
35804 · Add move headers with checksums to the end of database files for
35805 extra robust storage and faster storage.
35806 View Server
35808 · Added native Erlang views for high-performance applications.
35809 0.9.x Branch
35811 · Upgrade Notes
35812 · Version 0.9.2
35814 · Version 0.9.1
35816 · Version 0.9.0
35818 Upgrade Notes
35820 Response to Bulk Creation/Updates
35821 The response to a bulk creation / update now looks like this
35822 [
35824 {"id": "0", "rev": "3682408536"},
35825 {"id": "1", "rev": "3206753266"},
35826 {"id": "2", "error": "conflict", "reason": "Document update conflict."}
35827 ]
35828 Database File Format
35830 The database file format has changed. CouchDB itself does yet not pro‐
35831 vide any tools for migrating your data. In the meantime, you can use
35832 third-party scripts to deal with the migration, such as the dump/load
35833 tools that come with the development version (trunk) of couchdb-python.
35834 Renamed “count” to “limit”
35836 The view query API has been changed: count has become limit. This is a
35837 better description of what the parameter does, and should be a simple
35838 update in any client code.
35839 Moved View URLs
35841 The view URLs have been moved to design document resources. This means
35842 that paths that used to be like:
35843 http://hostname:5984/mydb/_view/designname/viewname?limit=10
35845 will now look like:
35847 http://hostname:5984/mydb/_design/designname/_view/viewname?limit=10.
35849 See the REST, Hypermedia, and CouchApps thread on dev for details.
35851 Attachments
35853 Names of attachments are no longer allowed to start with an underscore.
35854 Error Codes
35856 Some refinements have been made to error handling. CouchDB will send
35857 400 instead of 500 on invalid query parameters. Most notably, document
35858 update conflicts now respond with 409 Conflict instead of 412 Precondi‐
35859 tion Failed. The error code for when attempting to create a database
35860 that already exists is now 412 instead of 409.
35861 ini file format
35863 CouchDB 0.9 changes sections and configuration variable names in con‐
35864 figuration files. Old .ini files won’t work. Also note that CouchDB now
35865 ships with two .ini files where 0.8 used couch.ini there are now
35866 default.ini and local.ini. default.ini contains CouchDB’s standard
35867 configuration values. local.ini is meant for local changes. local.ini
35868 is not overwritten on CouchDB updates, so your edits are safe. In addi‐
35869 tion, the new runtime configuration system persists changes to the con‐
35870 figuration in local.ini.
35871 Version 0.9.2
35873 Build and System Integration
35874 · Remove branch callbacks to allow building couchjs against newer ver‐
35875 sions of Spidermonkey.
35876 Replication
35878 · Fix replication with 0.10 servers initiated by an 0.9 server (‐
35879 COUCHDB-559).
35880 Version 0.9.1
35882 Build and System Integration
35883 · PID file directory is now created by the SysV/BSD daemon scripts.
35884 · Fixed the environment variables shown by the configure script.
35886 · Fixed the build instructions shown by the configure script.
35888 · Updated ownership and permission advice in README for better secu‐
35890 rity.
35891 Configuration and stats system
35893 · Corrected missing configuration file error message.
35894 · Fixed incorrect recording of request time.
35896 Database Core
35898 · Document validation for underscore prefixed variables.
35899 · Made attachment storage less sparse.
35901 · Fixed problems when a database with delayed commits pending is con‐
35903 sidered idle, and subject to losing changes when shutdown. (‐
35904 COUCHDB-334)
35905 External Handlers
35907 · Fix POST requests.
35908 Futon
35910 · Redirect when loading a deleted view URI from the cookie.
35911 HTTP Interface
35913 · Attachment requests respect the “rev” query-string parameter.
35914 JavaScript View Server
35916 · Useful JavaScript Error messages.
35917 Replication
35919 · Added support for Unicode characters transmitted as UTF-16 surrogate
35920 pairs.
35921 · URL-encode attachment names when necessary.
35923 · Pull specific revisions of an attachment, instead of just the latest
35925 one.
35926 · Work around a rare chunk-merging problem in ibrowse.
35928 · Work with documents containing Unicode characters outside the Basic
35930 Multilingual Plane.
35931 Version 0.9.0
35933 Build and System Integration
35934 · The couchdb script now supports system chainable configuration files.
35935 · The Mac OS X daemon script now redirects STDOUT and STDERR like
35937 SysV/BSD.
35938 · The build and system integration have been improved for portability.
35940 · Added COUCHDB_OPTIONS to etc/default/couchdb file.
35942 · Remove COUCHDB_INI_FILE and COUCHDB_PID_FILE from etc/default/couchdb
35944 file.
35945 · Updated configure.ac to manually link libm for portability.
35947 · Updated configure.ac to extended default library paths.
35949 · Removed inets configuration files.
35951 · Added command line test runner.
35953 · Created dev target for make.
35955 Configuration and stats system
35957 · Separate default and local configuration files.
35958 · HTTP interface for configuration changes.
35960 · Statistics framework with HTTP query API.
35962 Database Core
35964 · Faster B-tree implementation.
35965 · Changed internal JSON term format.
35967 · Improvements to Erlang VM interactions under heavy load.
35969 · User context and administrator role.
35971 · Update validations with design document validation functions.
35973 · Document purge functionality.
35975 · Ref-counting for database file handles.
35977 Design Document Resource Paths
35979 · Added httpd_design_handlers config section.
35980 · Moved _view to httpd_design_handlers.
35982 · Added ability to render documents as non-JSON content-types with
35984 _show and _list functions, which are also httpd_design_handlers.
35985 Futon Utility Client
35987 · Added pagination to the database listing page.
35988 · Implemented attachment uploading from the document page.
35990 · Added page that shows the current configuration, and allows modifica‐
35992 tion of option values.
35993 · Added a JSON “source view” for document display.
35995 · JSON data in view rows is now syntax highlighted.
35997 · Removed the use of an iframe for better integration with browser his‐
35999 tory and bookmarking.
36000 · Full database listing in the sidebar has been replaced by a short
36002 list of recent databases.
36003 · The view editor now allows selection of the view language if there is
36005 more than one configured.
36006 · Added links to go to the raw view or document URI.
36008 · Added status page to display currently running tasks in CouchDB.
36010 · JavaScript test suite split into multiple files.
36012 · Pagination for reduce views.
36014 HTTP Interface
36016 · Added client side UUIDs for idempotent document creation
36017 · HTTP COPY for documents
36019 · Streaming of chunked attachment PUTs to disk
36021 · Remove negative count feature
36023 · Add include_docs option for view queries
36025 · Add multi-key view post for views
36027 · Query parameter validation
36029 · Use stale=ok to request potentially cached view index
36031 · External query handler module for full-text or other indexers.
36033 · Etags for attachments, views, shows and lists
36035 · Show and list functions for rendering documents and views as devel‐
36037 oper controlled content-types.
36038 · Attachment names may use slashes to allow uploading of nested direc‐
36040 tories (useful for static web hosting).
36041 · Option for a view to run over design documents.
36043 · Added newline to JSON responses. Closes bike-shed.
36045 Replication
36047 · Using ibrowse.
36048 · Checkpoint replications so failures are less expensive.
36050 · Automatically retry of failed replications.
36052 · Stream attachments in pull-replication.
36054 0.8.x Branch
36056 · Version 0.8.1-incubating
36057 · Version 0.8.0-incubating
36059 Version 0.8.1-incubating
36061 Build and System Integration
36062 · The couchdb script no longer uses awk for configuration checks as
36063 this was causing portability problems.
36064 · Updated sudo example in README to use the -i option, this fixes prob‐
36066 lems when invoking from a directory the couchdb user cannot access.
36067 Database Core
36069 · Fix for replication problems where the write queues can get backed up
36070 if the writes aren’t happening fast enough to keep up with the reads.
36071 For a large replication, this can exhaust memory and crash, or slow
36072 down the machine dramatically. The fix keeps only one document in the
36073 write queue at a time.
36074 · Fix for databases sometimes incorrectly reporting that they contain 0
36076 documents after compaction.
36077 · CouchDB now uses ibrowse instead of inets for its internal HTTP
36079 client implementation. This means better replication stability.
36080 Futon
36082 · The view selector dropdown should now work in Opera and Internet
36083 Explorer even when it includes optgroups for design documents. (‐
36084 COUCHDB-81)
36085 JavaScript View Server
36087 · Sealing of documents has been disabled due to an incompatibility with
36088 SpiderMonkey 1.9.
36089 · Improve error handling for undefined values emitted by map functions.
36091 (COUCHDB-83)
36092 HTTP Interface
36094 · Fix for chunked responses where chunks were always being split into
36095 multiple TCP packets, which caused problems with the test suite under
36096 Safari, and in some other cases.
36097 · Fix for an invalid JSON response body being returned for some kinds
36099 of views. (COUCHDB-84)
36100 · Fix for connections not getting closed after rejecting a chunked
36102 request. (COUCHDB-55)
36103 · CouchDB can now be bound to IPv6 addresses.
36105 · The HTTP Server header now contains the versions of CouchDB and
36107 Erlang.
36108 Version 0.8.0-incubating
36110 Build and System Integration
36111 · CouchDB can automatically respawn following a server crash.
36112 · Database server no longer refuses to start with a stale PID file.
36114 · System logrotate configuration provided.
36116 · Improved handling of ICU shared libraries.
36118 · The couchdb script now automatically enables SMP support in Erlang.
36120 · The couchdb and couchjs scripts have been improved for portability.
36122 · The build and system integration have been improved for portability.
36124 Database Core
36126 · The view engine has been completely decoupled from the storage
36127 engine. Index data is now stored in separate files, and the format of
36128 the main database file has changed.
36129 · Databases can now be compacted to reclaim space used for deleted doc‐
36131 uments and old document revisions.
36132 · Support for incremental map/reduce views has been added.
36134 · To support map/reduce, the structure of design documents has changed.
36136 View values are now JSON objects containing at least a map member,
36137 and optionally a reduce member.
36138 · View servers are now identified by name (for example javascript)
36140 instead of by media type.
36141 · Automatically generated document IDs are now based on proper UUID
36143 generation using the crypto module.
36144 · The field content-type in the JSON representation of attachments has
36146 been renamed to content_type (underscore).
36147 Futon
36149 · When adding a field to a document, Futon now just adds a field with
36150 an autogenerated name instead of prompting for the name with a dia‐
36151 log. The name is automatically put into edit mode so that it can be
36152 changed immediately.
36153 · Fields are now sorted alphabetically by name when a document is dis‐
36155 played.
36156 · Futon can be used to create and update permanent views.
36158 · The maximum number of rows to display per page on the database page
36160 can now be adjusted.
36161 · Futon now uses the XMLHTTPRequest API asynchronously to communicate
36163 with the CouchDB HTTP server, so that most operations no longer block
36164 the browser.
36165 · View results sorting can now be switched between ascending and
36167 descending by clicking on the Key column header.
36168 · Fixed a bug where documents that contained a @ character could not be
36170 viewed. (COUCHDB-12)
36171 · The database page now provides a Compact button to trigger database
36173 compaction. (COUCHDB-38)
36174 · Fixed portential double encoding of document IDs and other URI seg‐
36176 ments in many instances. (COUCHDB-39)
36177 · Improved display of attachments.
36179 · The JavaScript Shell has been removed due to unresolved licensing
36181 issues.
36182 JavaScript View Server
36184 · SpiderMonkey is no longer included with CouchDB, but rather treated
36185 as a normal external dependency. A simple C program (_couchjs) is
36186 provided that links against an existing SpiderMonkey installation and
36187 uses the interpreter embedding API.
36188 · View functions using the default JavaScript view server can now do
36190 logging using the global log(message) function. Log messages are
36191 directed into the CouchDB log at INFO level. (COUCHDB-59)
36192 · The global map(key, value) function made available to view code has
36194 been renamed to emit(key, value).
36195 · Fixed handling of exceptions raised by view functions.
36197 HTTP Interface
36199 · CouchDB now uses MochiWeb instead of inets for the HTTP server imple‐
36200 mentation. Among other things, this means that the extra configura‐
36201 tion files needed for inets (such as couch_httpd.conf) are no longer
36202 used.
36203 · The HTTP interface now completely supports the HEAD method. (‐
36205 COUCHDB-3)
36206 · Improved compliance of Etag handling with the HTTP specification. (‐
36208 COUCHDB-13)
36209 · Etags are no longer included in responses to document GET requests
36211 that include query string parameters causing the JSON response to
36212 change without the revision or the URI having changed.
36213 · The bulk document update API has changed slightly on both the request
36215 and the response side. In addition, bulk updates are now atomic.
36216 · CouchDB now uses TCP_NODELAY to fix performance problems with persis‐
36218 tent connections on some platforms due to nagling.
36219 · Including a ?descending=false query string parameter in requests to
36221 views no longer raises an error.
36222 · Requests to unknown top-level reserved URLs (anything with a leading
36224 underscore) now return a unknown_private_path error instead of the
36225 confusing illegal_database_name.
36226 · The Temporary view handling now expects a JSON request body, where
36228 the JSON is an object with at least a map member, and optional reduce
36229 and language members.
36230 · Temporary views no longer determine the view server based on the Con‐
36232 tent-Type header of the POST request, but rather by looking for a
36233 language member in the JSON body of the request.
36234 · The status code of responses to DELETE requests is now 200 to reflect
36236 that that the deletion is performed synchronously.
36237
36239 CVE-2010-0009: Apache CouchDB Timing Attack Vulnerability
36240 Date 31.03.2010
36241 Affected
36243 Apache CouchDB 0.8.0 to 0.10.1
36244 Severity
36246 Important
36247 Vendor The Apache Software Foundation
36249 Description
36251 Apache CouchDB versions prior to version 0.11.0 are vulnerable to tim‐
36252 ing attacks, also known as side-channel information leakage, due to
36253 using simple break-on-inequality string comparisons when verifying
36254 hashes and passwords.
36255 Mitigation
36257 All users should upgrade to CouchDB 0.11.0. Upgrades from the 0.10.x
36258 series should be seamless. Users on earlier versions should consult
36259 with upgrade notes.
36260 Example
36262 A canonical description of the attack can be found in
36263 http://codahale.com/a-lesson-in-timing-attacks/
36264 Credit
36266 This issue was discovered by Jason Davies of the Apache CouchDB devel‐
36267 opment team.
36268 CVE-2010-2234: Apache CouchDB Cross Site Request Forgery Attack
36270 Date 21.02.2010
36271 Affected
36273 Apache CouchDB 0.8.0 to 0.11.1
36274 Severity
36276 Important
36277 Vendor The Apache Software Foundation
36279 Description
36281 Apache CouchDB versions prior to version 0.11.1 are vulnerable to Cross
36282 Site Request Forgery (CSRF) attacks.
36283 Mitigation
36285 All users should upgrade to CouchDB 0.11.2 or 1.0.1.
36286 Upgrades from the 0.11.x and 0.10.x series should be seamless.
36288 Users on earlier versions should consult with upgrade notes.
36290 Example
36292 A malicious website can POST arbitrary JavaScript code to well known
36293 CouchDB installation URLs (like http://localhost:5984/) and make the
36294 browser execute the injected JavaScript in the security context of
36295 CouchDB’s admin interface Futon.
36296 Unrelated, but in addition the JSONP API has been turned off by default
36298 to avoid potential information leakage.
36299 Credit
36301 This CSRF issue was discovered by a source that wishes to stay anony‐
36302 mous.
36303 CVE-2010-3854: Apache CouchDB Cross Site Scripting Issue
36305 Date 28.01.2011
36306 Affected
36308 Apache CouchDB 0.8.0 to 1.0.1
36309 Severity
36311 Important
36312 Vendor The Apache Software Foundation
36314 Description
36316 Apache CouchDB versions prior to version 1.0.2 are vulnerable to Cross
36317 Site Scripting (XSS) attacks.
36318 Mitigation
36320 All users should upgrade to CouchDB 1.0.2.
36321 Upgrades from the 0.11.x and 0.10.x series should be seamless.
36323 Users on earlier versions should consult with upgrade notes.
36325 Example
36327 Due to inadequate validation of request parameters and cookie data in
36328 Futon, CouchDB’s web-based administration UI, a malicious site can exe‐
36329 cute arbitrary code in the context of a user’s browsing session.
36330 Credit
36332 This XSS issue was discovered by a source that wishes to stay anony‐
36333 mous.
36334 CVE-2012-5641: Information disclosure via unescaped backslashes in URLs on
36336 Windows
36337 Date 14.01.2013
36338 Affected
36340 All Windows-based releases of Apache CouchDB, up to and includ‐
36341 ing 1.0.3, 1.1.1, and 1.2.0 are vulnerable.
36342 Severity
36344 Moderate
36345 Vendor The Apache Software Foundation
36347 Description
36349 A specially crafted request could be used to access content directly
36350 that would otherwise be protected by inbuilt CouchDB security mecha‐
36351 nisms. This request could retrieve in binary form any CouchDB database,
36352 including the _users or _replication databases, or any other file that
36353 the user account used to run CouchDB might have read access to on the
36354 local filesystem. This exploit is due to a vulnerability in the
36355 included MochiWeb HTTP library.
36356 Mitigation
36358 Upgrade to a supported CouchDB release that includes this fix, such as:
36359 · 1.0.4
36361 · 1.1.2
36363 · 1.2.1
36365 · 1.3.x
36367 All listed releases have included a specific fix for the MochiWeb com‐
36369 ponent.
36370 Work-Around
36372 Users may simply exclude any file-based web serving components directly
36373 within their configuration file, typically in local.ini. On a default
36374 CouchDB installation, this requires amending the httpd_global_han‐
36375 dlers/favicon.ico and httpd_global_handlers/_utils lines within
36376 httpd_global_handlers:
36377 [httpd_global_handlers]
36379 favicon.ico = {couch_httpd_misc_handlers, handle_welcome_req, <<"Forbidden">>}
36380 _utils = {couch_httpd_misc_handlers, handle_welcome_req, <<"Forbidden">>}
36381 If additional handlers have been added, such as to support Adobe’s
36383 Flash crossdomain.xml files, these would also need to be excluded.
36384 Acknowledgement
36386 The issue was found and reported by Sriram Melkote to the upstream
36387 MochiWeb project.
36388 References
36390 · https://github.com/melkote/mochiweb/commit/ac2bf
36391 CVE-2012-5649: JSONP arbitrary code execution with Adobe Flash
36393 Date 14.01.2013
36394 Affected
36396 Releases up to and including 1.0.3, 1.1.1, and 1.2.0 are vulner‐
36397 able, if administrators have enabled JSONP.
36398 Severity
36400 Moderate
36401 Vendor The Apache Software Foundation
36403 Description
36405 A hand-crafted JSONP callback and response can be used to run arbitrary
36406 code inside client-side browsers via Adobe Flash.
36407 Mitigation
36409 Upgrade to a supported CouchDB release that includes this fix, such as:
36410 · 1.0.4
36412 · 1.1.2
36414 · 1.2.1
36416 · 1.3.x
36418 All listed releases have included a specific fix.
36420 Work-Around
36422 Disable JSONP or don’t enable it since it’s disabled by default.
36423 CVE-2012-5650: DOM based Cross-Site Scripting via Futon UI
36425 Date 14.01.2013
36426 Affected
36428 Apache CouchDB releases up to and including 1.0.3, 1.1.1, and
36429 1.2.0 are vulnerable.
36430 Severity
36432 Moderate
36433 Vendor The Apache Software Foundation
36435 Description
36437 Query parameters passed into the browser-based test suite are not sani‐
36438 tised, and can be used to load external resources. An attacker may exe‐
36439 cute JavaScript code in the browser, using the context of the remote
36440 user.
36441 Mitigation
36443 Upgrade to a supported CouchDB release that includes this fix, such as:
36444 · 1.0.4
36446 · 1.1.2
36448 · 1.2.1
36450 · 1.3.x
36452 All listed releases have included a specific fix.
36454 Work-Around
36456 Disable the Futon user interface completely, by adapting local.ini and
36457 restarting CouchDB:
36458 [httpd_global_handlers]
36460 _utils = {couch_httpd_misc_handlers, handle_welcome_req, <<"Forbidden">>}
36461 Or by removing the UI test suite components:
36463 · share/www/verify_install.html
36465 · share/www/couch_tests.html
36467 · share/www/custom_test.html
36469 Acknowledgement
36471 This vulnerability was discovered & reported to the Apache Software
36472 Foundation by Frederik Braun.
36473 CVE-2014-2668: DoS (CPU and memory consumption) via the count parameter to
36475 /_uuids
36476 Date 26.03.2014
36477 Affected
36479 Apache CouchDB releases up to and including 1.3.1, 1.4.0, and
36480 1.5.0 are vulnerable.
36481 Severity
36483 Moderate
36484 Vendor The Apache Software Foundation
36486 Description
36488 The api/server/uuids resource’s count query parameter is able to take
36489 unreasonable huge numeric value which leads to exhaustion of server
36490 resources (CPU and memory) and to DoS as the result.
36491 Mitigation
36493 Upgrade to a supported CouchDB release that includes this fix, such as:
36494 · 1.5.1
36496 · 1.6.0
36498 All listed releases have included a specific fix to
36500 Work-Around
36502 Disable the api/server/uuids handler completely, by adapting local.ini
36503 and restarting CouchDB:
36504 [httpd_global_handlers]
36506 _uuids =
36507 CVE-2017-12635: Apache CouchDB Remote Privilege Escalation
36509 Date 14.11.2017
36510 Affected
36512 All Versions of Apache CouchDB
36513 Severity
36515 Critical
36516 Vendor The Apache Software Foundation
36518 Description
36520 Due to differences in CouchDB’s Erlang-based JSON parser and
36521 JavaScript-based JSON parser, it is possible to submit _users documents
36522 with duplicate keys for roles used for access control within the data‐
36523 base, including the special case _admin role, that denotes administra‐
36524 tive users. In combination with CVE-2017-12636 (Remote Code Execution),
36525 this can be used to give non-admin users access to arbitrary shell com‐
36526 mands on the server as the database system user.
36527 Mitigation
36529 All users should upgrade to CouchDB 1.7.1 or 2.1.1.
36530 Upgrades from previous 1.x and 2.x versions in the same series should
36532 be seamless.
36533 Users on earlier versions, or users upgrading from 1.x to 2.x should
36535 consult with upgrade notes.
36536 Example
36538 The JSON parser differences result in behaviour that if two roles keys
36539 are available in the JSON, the second one will be used for authorising
36540 the document write, but the first roles key is used for subsequent
36541 authorisation for the newly created user. By design, users can not
36542 assign themselves roles. The vulnerability allows non-admin users to
36543 give themselves admin privileges.
36544 We addressed this issue by updating the way CouchDB parses JSON in
36546 Erlang, mimicking the JavaScript behaviour of picking the last key, if
36547 duplicates exist.
36548 Credit
36550 This issue was discovered by Max Justicz.
36551 CVE-2017-12636: Apache CouchDB Remote Code Execution
36553 Date 14.11.2017
36554 Affected
36556 All Versions of Apache CouchDB
36557 Severity
36559 Critical
36560 Vendor The Apache Software Foundation
36562 Description
36564 CouchDB administrative users can configure the database server via
36565 HTTP(S). Some of the configuration options include paths for operating
36566 system-level binaries that are subsequently launched by CouchDB. This
36567 allows a CouchDB admin user to execute arbitrary shell commands as the
36568 CouchDB user, including downloading and executing scripts from the pub‐
36569 lic internet.
36570 Mitigation
36572 All users should upgrade to CouchDB 1.7.1 or 2.1.1.
36573 Upgrades from previous 1.x and 2.x versions in the same series should
36575 be seamless.
36576 Users on earlier versions, or users upgrading from 1.x to 2.x should
36578 consult with upgrade notes.
36579 Credit
36581 This issue was discovered by Joan Touzet of the CouchDB Security team
36582 during the investigation of CVE-2017-12635.
36583 CVE-2018-11769: Apache CouchDB Remote Code Execution
36585 Date 08.08.2018
36586 Affected
36588 Apache CouchDB 1.x and ≤2.1.2
36589 Severity
36591 Low
36592 Vendor The Apache Software Foundation
36594 Description
36596 CouchDB administrative users can configure the database server via
36597 HTTP(S). Due to insufficient validation of administrator-supplied con‐
36598 figuration settings via the HTTP API, it is possible for a CouchDB
36599 administrator user to escalate their privileges to that of the operat‐
36600 ing system’s user under which CouchDB runs, by bypassing the blacklist
36601 of configuration settings that are not allowed to be modified via the
36602 HTTP API.
36603 This privilege escalation effectively allows a CouchDB admin user to
36605 gain arbitrary remote code execution, bypassing mitigations for
36606 CVE-2017-12636 and CVE-2018-8007.
36607 Mitigation
36609 All users should upgrade to CouchDB 2.2.0.
36610 Upgrades from previous 2.x versions in the same series should be seam‐
36612 less.
36613 Users still on CouchDB 1.x should be advised that the Apache CouchDB
36615 team no longer support 1.x.
36616 In-place mitigation (on any 1.x release, or 2.x prior to 2.2.0) is pos‐
36618 sible by removing the _config route from the default.ini file, as fol‐
36619 lows:
36620 [httpd_global_handlers]
36622 ;_config = {couch_httpd_misc_handlers, handle_config_req}
36623 or by blocking access to the /_config (1.x) or /_node/*/_config routes
36625 at a reverse proxy in front of the service.
36626 CVE-2018-17188: Apache CouchDB Remote Privilege Escalations
36628 Date 17.12.2018
36629 Affected
36631 All Versions of Apache CouchDB
36632 Severity
36634 Medium
36635 Vendor The Apache Software Foundation
36637 Description
36639 Prior to CouchDB version 2.3.0, CouchDB allowed for runtime-configura‐
36640 tion of key components of the database. In some cases, this lead to
36641 vulnerabilities where CouchDB admin users could access the underlying
36642 operating system as the CouchDB user. Together with other vulnerabili‐
36643 ties, it allowed full system entry for unauthenticated users.
36644 These vulnerabilities were fixed and disclosed in the following CVE
36646 reports:
36647 · CVE-2018-11769: Apache CouchDB Remote Code Execution
36649 · CVE-2018-8007: Apache CouchDB Remote Code Execution
36651 · CVE-2017-12636: Apache CouchDB Remote Code Execution
36653 · CVE-2017-12635: Apache CouchDB Remote Privilege Escalation
36655 Rather than waiting for new vulnerabilities to be discovered, and fix‐
36657 ing them as they come up, the CouchDB development team decided to make
36658 changes to avoid this entire class of vulnerabilities.
36659 With CouchDB version 2.3.0, CouchDB no longer can configure key compo‐
36661 nents at runtime. While some flexibility is needed for speciality con‐
36662 figurations of CouchDB, the configuration was changed from being avail‐
36663 able at runtime to start-up time. And as such now requires shell access
36664 to the CouchDB server.
36665 This closes all future paths for vulnerabilities of this type.
36667 Mitigation
36669 All users should upgrade to CouchDB 2.3.0.
36670 Upgrades from previous 2.x versions in the same series should be seam‐
36672 less.
36673 Users on earlier versions should consult with upgrade notes.
36675 Credit
36677 This issue was discovered by the Apple Information Security team.
36678 CVE-2018-8007: Apache CouchDB Remote Code Execution
36680 Date 30.04.2018
36681 Affected
36683 All Versions of Apache CouchDB
36684 Severity
36686 Low
36687 Vendor The Apache Software Foundation
36689 Description
36691 CouchDB administrative users can configure the database server via
36692 HTTP(S). Due to insufficient validation of administrator-supplied con‐
36693 figuration settings via the HTTP API, it is possible for a CouchDB
36694 administrator user to escalate their privileges to that of the operat‐
36695 ing system’s user that CouchDB runs under, by bypassing the backlist of
36696 configuration settings that are not allowed to be modified via the HTTP
36697 API.
36698 This privilege escalation effectively allows a CouchDB admin user to
36700 gain arbitrary remote code execution, bypassing CVE-2017-12636
36701 Mitigation
36703 All users should upgrade to CouchDB 1.7.2 or 2.1.2.
36704 Upgrades from previous 1.x and 2.x versions in the same series should
36706 be seamless.
36707 Users on earlier versions, or users upgrading from 1.x to 2.x should
36709 consult with upgrade notes.
36710 Credit
36712 This issue was discovered by Francesco Oddo of MDSec Labs.
36713 CVE-2020-1955: Apache CouchDB Remote Privilege Escalation
36715 Date 19.05.2020
36716 Affected
36718 3.0.0
36719 Severity
36721 Medium
36722 Vendor The Apache Software Foundation
36724 Description
36726 CouchDB version 3.0.0 shipped with a new configuration setting that
36727 governs access control to the entire database server called
36728 require_valid_user_except_for_up. It was meant as an extension to the
36729 long-standing setting require_valid_user, which in turn requires that
36730 any and all requests to CouchDB will have to be made with valid creden‐
36731 tials, effectively forbidding any anonymous requests.
36732 The new require_valid_user_except_for_up is an off-by-default setting
36734 that was meant to allow requiring valid credentials for all endpoints
36735 except for the /_up endpoint.
36736 However, the implementation of this made an error that lead to not
36738 enforcing credentials on any endpoint, when enabled.
36739 CouchDB versions 3.0.1 and 3.1.0 fix this issue.
36741 Mitigation
36743 Users who have not enabled require_valid_user_except_for_up are not
36744 affected.
36745 Users who have it enabled can either disable it again, or upgrade to
36747 CouchDB versions 3.0.1 and 3.1.0
36748 Credit
36750 This issue was discovered by Stefan Klein.
36751
36753 The Apache Software Foundation takes a very active stance in eliminat‐
36754 ing security problems and denial of service attacks against Apache
36755 CouchDB.
36756 We strongly encourage folks to report such problems to our private
36758 security mailing list first, before disclosing them in a public forum.
36759 Please note that the security mailing list should only be used for
36761 reporting undisclosed security vulnerabilities in Apache CouchDB and
36762 managing the process of fixing such vulnerabilities. We cannot accept
36763 regular bug reports or other queries at this address. All mail sent to
36764 this address that does not relate to an undisclosed security problem in
36765 the Apache CouchDB source code will be ignored.
36766 If you need to report a bug that isn’t an undisclosed security vulnera‐
36768 bility, please use the bug reporting page.
36769 Questions about:
36771 · How to configure CouchDB securely
36773 · If a vulnerability applies to your particular application
36775 · Obtaining further information on a published vulnerability
36777 · Availability of patches and/or new releases
36779 should be address to the users mailing list. Please see the mailing
36781 lists page for details of how to subscribe.
36782 The private security mailing address is: security@couchdb.apache.org
36784 Please read how the Apache Software Foundation handles security reports
36786 to know what to expect.
36787 Note that all networked servers are subject to denial of service
36789 attacks, and we cannot promise magic workarounds to generic problems
36790 (such as a client streaming lots of data to your server, or re-request‐
36791 ing the same URL repeatedly). In general our philosophy is to avoid any
36792 attacks which can cause the server to consume resources in a non-linear
36793 relationship to the size of inputs.
36794
36796 License
36797 Apache License
36798 Version 2.0, January 2004
36799 http://www.apache.org/licenses/
36800 TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
36802 1. Definitions.
36804 "License" shall mean the terms and conditions for use, reproduction,
36806 and distribution as defined by Sections 1 through 9 of this document.
36807 "Licensor" shall mean the copyright owner or entity authorized by
36809 the copyright owner that is granting the License.
36810 "Legal Entity" shall mean the union of the acting entity and all
36812 other entities that control, are controlled by, or are under common
36813 control with that entity. For the purposes of this definition,
36814 "control" means (i) the power, direct or indirect, to cause the
36815 direction or management of such entity, whether by contract or
36816 otherwise, or (ii) ownership of fifty percent (50%) or more of the
36817 outstanding shares, or (iii) beneficial ownership of such entity.
36818 "You" (or "Your") shall mean an individual or Legal Entity
36820 exercising permissions granted by this License.
36821 "Source" form shall mean the preferred form for making modifications,
36823 including but not limited to software source code, documentation
36824 source, and configuration files.
36825 "Object" form shall mean any form resulting from mechanical
36827 transformation or translation of a Source form, including but
36828 not limited to compiled object code, generated documentation,
36829 and conversions to other media types.
36830 "Work" shall mean the work of authorship, whether in Source or
36832 Object form, made available under the License, as indicated by a
36833 copyright notice that is included in or attached to the work
36834 (an example is provided in the Appendix below).
36835 "Derivative Works" shall mean any work, whether in Source or Object
36837 form, that is based on (or derived from) the Work and for which the
36838 editorial revisions, annotations, elaborations, or other modifications
36839 represent, as a whole, an original work of authorship. For the purposes
36840 of this License, Derivative Works shall not include works that remain
36841 separable from, or merely link (or bind by name) to the interfaces of,
36842 the Work and Derivative Works thereof.
36843 "Contribution" shall mean any work of authorship, including
36845 the original version of the Work and any modifications or additions
36846 to that Work or Derivative Works thereof, that is intentionally
36847 submitted to Licensor for inclusion in the Work by the copyright owner
36848 or by an individual or Legal Entity authorized to submit on behalf of
36849 the copyright owner. For the purposes of this definition, "submitted"
36850 means any form of electronic, verbal, or written communication sent
36851 to the Licensor or its representatives, including but not limited to
36852 communication on electronic mailing lists, source code control systems,
36853 and issue tracking systems that are managed by, or on behalf of, the
36854 Licensor for the purpose of discussing and improving the Work, but
36855 excluding communication that is conspicuously marked or otherwise
36856 designated in writing by the copyright owner as "Not a Contribution."
36857 "Contributor" shall mean Licensor and any individual or Legal Entity
36859 on behalf of whom a Contribution has been received by Licensor and
36860 subsequently incorporated within the Work.
36861 2. Grant of Copyright License. Subject to the terms and conditions of
36863 this License, each Contributor hereby grants to You a perpetual,
36864 worldwide, non-exclusive, no-charge, royalty-free, irrevocable
36865 copyright license to reproduce, prepare Derivative Works of,
36866 publicly display, publicly perform, sublicense, and distribute the
36867 Work and such Derivative Works in Source or Object form.
36868 3. Grant of Patent License. Subject to the terms and conditions of
36870 this License, each Contributor hereby grants to You a perpetual,
36871 worldwide, non-exclusive, no-charge, royalty-free, irrevocable
36872 (except as stated in this section) patent license to make, have made,
36873 use, offer to sell, sell, import, and otherwise transfer the Work,
36874 where such license applies only to those patent claims licensable
36875 by such Contributor that are necessarily infringed by their
36876 Contribution(s) alone or by combination of their Contribution(s)
36877 with the Work to which such Contribution(s) was submitted. If You
36878 institute patent litigation against any entity (including a
36879 cross-claim or counterclaim in a lawsuit) alleging that the Work
36880 or a Contribution incorporated within the Work constitutes direct
36881 or contributory patent infringement, then any patent licenses
36882 granted to You under this License for that Work shall terminate
36883 as of the date such litigation is filed.
36884 4. Redistribution. You may reproduce and distribute copies of the
36886 Work or Derivative Works thereof in any medium, with or without
36887 modifications, and in Source or Object form, provided that You
36888 meet the following conditions:
36889 (a) You must give any other recipients of the Work or
36891 Derivative Works a copy of this License; and
36892 (b) You must cause any modified files to carry prominent notices
36894 stating that You changed the files; and
36895 (c) You must retain, in the Source form of any Derivative Works
36897 that You distribute, all copyright, patent, trademark, and
36898 attribution notices from the Source form of the Work,
36899 excluding those notices that do not pertain to any part of
36900 the Derivative Works; and
36901 (d) If the Work includes a "NOTICE" text file as part of its
36903 distribution, then any Derivative Works that You distribute must
36904 include a readable copy of the attribution notices contained
36905 within such NOTICE file, excluding those notices that do not
36906 pertain to any part of the Derivative Works, in at least one
36907 of the following places: within a NOTICE text file distributed
36908 as part of the Derivative Works; within the Source form or
36909 documentation, if provided along with the Derivative Works; or,
36910 within a display generated by the Derivative Works, if and
36911 wherever such third-party notices normally appear. The contents
36912 of the NOTICE file are for informational purposes only and
36913 do not modify the License. You may add Your own attribution
36914 notices within Derivative Works that You distribute, alongside
36915 or as an addendum to the NOTICE text from the Work, provided
36916 that such additional attribution notices cannot be construed
36917 as modifying the License.
36918 You may add Your own copyright statement to Your modifications and
36920 may provide additional or different license terms and conditions
36921 for use, reproduction, or distribution of Your modifications, or
36922 for any such Derivative Works as a whole, provided Your use,
36923 reproduction, and distribution of the Work otherwise complies with
36924 the conditions stated in this License.
36925 5. Submission of Contributions. Unless You explicitly state otherwise,
36927 any Contribution intentionally submitted for inclusion in the Work
36928 by You to the Licensor shall be under the terms and conditions of
36929 this License, without any additional terms or conditions.
36930 Notwithstanding the above, nothing herein shall supersede or modify
36931 the terms of any separate license agreement you may have executed
36932 with Licensor regarding such Contributions.
36933 6. Trademarks. This License does not grant permission to use the trade
36935 names, trademarks, service marks, or product names of the Licensor,
36936 except as required for reasonable and customary use in describing the
36937 origin of the Work and reproducing the content of the NOTICE file.
36938 7. Disclaimer of Warranty. Unless required by applicable law or
36940 agreed to in writing, Licensor provides the Work (and each
36941 Contributor provides its Contributions) on an "AS IS" BASIS,
36942 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
36943 implied, including, without limitation, any warranties or conditions
36944 of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
36945 PARTICULAR PURPOSE. You are solely responsible for determining the
36946 appropriateness of using or redistributing the Work and assume any
36947 risks associated with Your exercise of permissions under this License.
36948 8. Limitation of Liability. In no event and under no legal theory,
36950 whether in tort (including negligence), contract, or otherwise,
36951 unless required by applicable law (such as deliberate and grossly
36952 negligent acts) or agreed to in writing, shall any Contributor be
36953 liable to You for damages, including any direct, indirect, special,
36954 incidental, or consequential damages of any character arising as a
36955 result of this License or out of the use or inability to use the
36956 Work (including but not limited to damages for loss of goodwill,
36957 work stoppage, computer failure or malfunction, or any and all
36958 other commercial damages or losses), even if such Contributor
36959 has been advised of the possibility of such damages.
36960 9. Accepting Warranty or Additional Liability. While redistributing
36962 the Work or Derivative Works thereof, You may choose to offer,
36963 and charge a fee for, acceptance of support, warranty, indemnity,
36964 or other liability obligations and/or rights consistent with this
36965 License. However, in accepting such obligations, You may act only
36966 on Your own behalf and on Your sole responsibility, not on behalf
36967 of any other Contributor, and only if You agree to indemnify,
36968 defend, and hold each Contributor harmless for any liability
36969 incurred by, or claims asserted against, such Contributor by reason
36970 of your accepting any such warranty or additional liability.
36971 END OF TERMS AND CONDITIONS
36973 APPENDIX: How to apply the Apache License to your work.
36975 To apply the Apache License to your work, attach the following
36977 boilerplate notice, with the fields enclosed by brackets "[]"
36978 replaced with your own identifying information. (Don't include
36979 the brackets!) The text should be enclosed in the appropriate
36980 comment syntax for the file format. We also recommend that a
36981 file or class name and description of purpose be included on the
36982 same "printed page" as the copyright notice for easier
36983 identification within third-party archives.
36984 Copyright [yyyy] [name of copyright owner]
36986 Licensed under the Apache License, Version 2.0 (the "License");
36988 you may not use this file except in compliance with the License.
36989 You may obtain a copy of the License at
36990 http://www.apache.org/licenses/LICENSE-2.0
36992 Unless required by applicable law or agreed to in writing, software
36994 distributed under the License is distributed on an "AS IS" BASIS,
36995 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
36996 See the License for the specific language governing permissions and
36997 limitations under the License.
36998
37001 The documentation lives in its own source tree. We’ll start by forking
37002 and cloning the CouchDB documentation GitHub mirror. That will allow us
37003 to send the contribution to CouchDB with a pull request.
37004 If you don’t have a GitHub account yet, it is a good time to get one,
37006 they are free. If you don’t want to use GitHub, there are alternate
37007 ways to contributing back, that we’ll cover next time.
37008 Go to https://github.com/apache/couchdb-documentation and click the
37010 “fork” button in the top right. This will create a fork of CouchDB in
37011 your GitHub account. If your account is username, your fork lives at
37012 https://github.com/username/couchdb-documentation. In the header, it
37013 tells me my “GitHub Clone URL”. We need to copy that and start a termi‐
37014 nal:
37015 $ git clone https://github.com/username/couchdb-documentation.git
37017 $ cd couchdb-documentation
37018 $ subl .
37019 I’m opening the whole CouchDB documentation source tree in my favourite
37021 editor. It gives me the usual directory listing:
37022 ebin/
37024 ext/
37025 .git/
37026 .gitignore
37027 images/
37028 LICENSE
37029 make.bat
37030 Makefile
37031 NOTICE
37032 rebar.config
37033 src/
37034 static/
37035 templates/
37036 themes/
37037 .travis.yml
37038 The documentation sources live in src, you can safely ignore all the
37040 other files and directories.
37041 First we should determine where we want to document this inside the
37043 documentation. We can look through http://docs.couchdb.org/en/latest/
37044 for inspiration. The JSON Structure Reference looks like a fine place
37045 to write this up.
37046 The current state includes mostly tables describing the JSON structure
37048 (after all, that’s the title of this chapter), but some prose about the
37049 number representation can’t hurt. For future reference, since the topic
37050 in the thread includes views and different encoding in views (as
37051 opposed to the storage engine), we should remember to make a note in
37052 the views documentation as well, but we’ll leave this for later.
37053 Let’s try and find the source file that builds the file
37055 http://docs.couchdb.org/en/latest/json-structure.html – we are in luck,
37056 under share/doc/src we find the file json-structure.rst. That looks
37057 promising. .rst stands for ReStructured Text (see
37058 http://thomas-cokelaer.info/tutorials/sphinx/rest_syntax.html for a
37059 markup reference), which is an ASCII format for writing documents, doc‐
37060 umentation in this case. Let’s have a look and open it.
37061 We see ASCII tables with some additional formatting, all looking like
37063 the final HTML. So far so easy. For now, let’s just add to the bottom
37064 of this. We can worry about organising this better later.
37065 We start by adding a new headline:
37067 Number Handling
37069 ===============
37070 Now we paste in the rest of the main email of the thread. It is mostly
37072 text, but it includes some code listings. Let’s mark them up. We’ll
37073 turn:
37074 ejson:encode(ejson:decode(<<"1.1">>)).
37076 <<"1.1000000000000000888">>
37077 Into:
37079 .. code-block:: erlang
37081 ejson:encode(ejson:decode(<<"1.1">>)).
37083 <<"1.1000000000000000888">>
37084 And we follow along with the other code samples. We turn:
37086 Spidermonkey
37088 $ js -h 2>&1 | head -n 1
37090 JavaScript-C 1.8.5 2011-03-31
37091 $ js
37092 js> JSON.stringify(JSON.parse("1.01234567890123456789012345678901234567890"))
37093 "1.0123456789012346"
37094 js> var f = JSON.stringify(JSON.parse("1.01234567890123456789012345678901234567890"))
37095 js> JSON.stringify(JSON.parse(f))
37096 "1.0123456789012346"
37097 into:
37099 Spidermonkey::
37101 $ js -h 2>&1 | head -n 1
37103 JavaScript-C 1.8.5 2011-03-31
37104 $ js
37105 js> JSON.stringify(JSON.parse("1.01234567890123456789012345678901234567890"))
37106 "1.0123456789012346"
37107 js> var f = JSON.stringify(JSON.parse("1.01234567890123456789012345678901234567890"))
37108 js> JSON.stringify(JSON.parse(f))
37109 "1.0123456789012346"
37110 And then follow all the other ones.
37112 I cleaned up the text a little but to make it sound more like a docu‐
37114 mentation entry as opposed to a post on a mailing list.
37115 The next step would be to validate that we got all the markup right.
37117 I’ll leave this for later. For now we’ll contribute our change back to
37118 CouchDB.
37119 First, we commit our changes:
37121 $ > git commit -am 'document number encoding'
37123 [master a84b2cf] document number encoding
37124 1 file changed, 199 insertions(+)
37125 Then we push the commit to our CouchDB fork:
37127 $ git push origin master
37129 Next, we go back to our GitHub page
37131 https://github.com/username/couchdb-documentation and click the “Pull
37132 Request” button. Fill in the description with something useful and hit
37133 the “Send Pull Request” button.
37134 And we’re done!
37136 Style Guidelines for this Documentation
37138 When you make a change to the documentation, you should make sure that
37139 you follow the style. Look through some files and you will see that the
37140 style is quite straightforward. If you do not know if your formating is
37141 in compliance with the style, ask yourself the following question:
37142 Is it needed for correct syntax?
37144 If the answer is No. then it is probably not.
37146 These guidelines strive be simple, without contradictions and excep‐
37148 tions. The best style is the one that is followed because it seems to
37149 be the natural way of doing it.
37150 The guidelines
37152 The guidelines are in descending priority.
37153 1. Syntax
37155 · Correct syntax is always more important than style. This includes
37157 configuration files, HTML responses, etc.
37158 2. Encoding
37160 · All files are UTF-8.
37162 3. Line ending
37164 · All lines end with \n.
37166 · No trailing whitespace.
37168 4. Line length
37170 · The maximum line length is 80 characters.
37172 5. Links
37174 · All internal links are relative.
37176 6. Indentation
37178 · 4 spaces.
37180 7. Titles
37182 · The highest level titles in a file is over and underlined with =.
37184 · Lower level titles are underlined with the following characters in
37186 descending order:
37187 = - ^ * + # ` : . " ~ _
37189 · Over and underline match the title length.
37191 8. Empty lines
37193 · No empty line at the end of the file.
37195 · Lists may separated each item with an empty line.
37197
37199 unknown
37200
37202 2020, Apache Software Foundation. CouchDB® is a registered trademark of
37203 the Apache Software Foundation
372043.1 Sep 11, 2020 APACHECOUCHDB(1)